Sequence and structure representations for peptide property prediction

Therapeutic peptides are short amino-acid chains. Whether one can be a drug depends on properties that are slow and costly to measure: does it enter a cell (permeability), dissolve (solubility), avoid sticking to the wrong things (non-fouling), and leave red blood cells alone (hemolysis). Predicting these from sequence would let a candidate library be triaged before any bench work. This was my I-CUREUS internship in the CUBIC Lab: 150 hours over the summer of 2026, supervised by Prof. James Green, alongside postdoc Dr. François Charih and Prof. Kyle Biggar's biochemistry group.

The question

A peptide can be read two ways: as a sequence, by a protein language model, or as a chemical structure, by a chemical language model. PeptiVerse (2026) is the state of the art on all four properties and publishes its data and splits, so every result here is directly comparable. I tested two hypotheses.

• Fusion. Encoding sequence and structure together beats either alone.
• Multitask. One model shared across the four properties beats four single-task models.

Method

ESM-2 (3B) embeds the sequence and PeptideCLM embeds the structure. Both encoders stay frozen; each run trains only the fusion block, a shared body, and the task heads. Four fusion designs differ in where the two token streams meet: concatenated at the input, joined before attention, crossed midway with bidirectional cross-attention (each stream attending to the other through the usual $\mathrm{softmax}(QK^{\top}/\sqrt{d_k})\,V$), or kept in separate transformers and merged at the end. Training ran on a Cloud TPU v4-32 pod from the Google TPU Research Cloud, with a JAX and Grain pipeline over ArrayRecord files and sixteen experiments running at once. The benchmark has 44,033 labelled examples over 33,949 peptides, and the classes are uneven: solubility is 42% of labels, non-fouling 39%, hemolysis 14%, permeability 5%.

Results

One model per task reaches test F1 of 0.893 on permeability, 0.734 on solubility, 0.726 on non-fouling, and 0.557 on hemolysis. PeptiVerse reports 0.918, 0.745, 0.734, and 0.584. Neither hypothesis held.

• The structure encoder made little difference either way. The sequence encoder alone was the stronger single view on most properties.
• One shared model never beat four separate ones. It came close on permeability and non-fouling and fell clearly behind on solubility and hemolysis.
• Where the encoders join did not matter. The four designs fell within 0.020 validation F1 of each other on every property.
• Every model did better on peptides it had trained on than on held-out ones. The gap varied by property: 0.11 on solubility, 0.46 on hemolysis.

What it means

The likeliest explanation is that the two encoders say the same thing. Both describe the same molecule, and both were pretrained on overlapping peptide collections, so the second view adds little. Because the encoders were frozen, the result bounds the pretrained representations, not the architectures. Adapting the encoders looks more promising than adding views or sharing heads. The evaluation is one public benchmark; transfer to other data and to the lab is untested.

Knowing which choices make no measurable difference is the useful output. It narrows where to search before anyone spends more compute.