NOBBLE

Neosubstrate and Oligopeptide Binder Benchmarking for Ligase Engagement

running
jobs
76/100
pairs
6/5 br
RMSD
17.14
valid
1/3
Pinned E3 none GLUE none TARGET none PEPTIDE none

What NOBBLE measures

NOBBLE generates and scores peptide poses against curated targeted-protein-degradation machinery using Boltz-2 ternary co-folding, and asks one question of every pair: does the presence of the glue or degrader change the predicted interface, and does that change separate experimentally degraded substrates from experimentally non-degraded ones assayed in the same screen?

delta_iptm = iptm_min_ternary - iptm_min_binary

Measured on the receptor-to-peptide chain pair, not the global ipTM: on a ternary complex the global figure averages in the receptor-to-ligand interface, which is the easy one, so a glue sitting perfectly in its pocket while the degron fails to dock would raise it.

The confidence caveat, stated plainly

The Thera-SAbDab benchmark of Boltz-2 across 519 therapeutic antibodies paired with cognate and non-cognate antigens found that its confidence metrics gave only weak, though above-chance, discrimination: ROC-AUC between 0.5 and 0.60, with ipTM-min carrying the strongest signal and multivariate modelling adding essentially nothing. Peptide-protein should fare better than antibody-antigen, but this application does not present ipTM as a verdict. It is reported as enrichment, alongside the geometry.

That is not a theoretical caution. On the benchmark panel's first pair, a pose that genuinely bridged receptor to peptide scored ipTM 0.885 and one that did not scored 0.877; the highest ipTM in the whole ternary set, 0.916, did not bridge at all. The bridge percentile separated them; ipTM did not.

Scoring signals and how far each is trusted

SignalUnitsSourceTrustNote
Bridge percentile percentile measured against 285,996 observed bridges primary Buried surface of the glue against both chains, as a percentile of the experimentally observed distribution. Compares a prediction to real interfaces rather than to the model's own confidence.
Delta-ipTM ipTM difference model-derived headline Ternary minus binary interface confidence on the receptor-to-peptide chain pair, minimum across seeds. A collapsed seed can dominate it: the median is shown alongside.
Pose convergence 0 to 1, from mean pairwise RMSD measured from the predictions high Pairwise peptide backbone RMSD across seeds after superposing on the receptor. Tracks reality better than confidence for peptides.
PLIP fingerprint match fraction of canonical contacts measured from the predictions high Fraction of the canonical degron contacts for that E3 class that the prediction reproduces.
Lysine geometry not validated measured, but not a validated predictor not validated BINMAN's reach window is deliberately unfitted: Cb-Cb AUC 0.42 to 0.48 on 369 assayed lysines, at or below chance and slightly inverted. Reported as a measurement only.
ProtCID packing vetoflag unavailable not computed ProtCID publishes through a JavaScript front end with no bulk export, so it was never acquired. Every pair reports furniture: null with status not_computed, never false: recording it as false would assert a check that was never run.

Lysine reach: why there is no score

The specification asks for BINMAN's reach-window fit, described as already calibrated against 976 observed diGly sites. It is not calibrated. BINMAN attempted the fit, measured it and declined to ship it, and these are the numbers behind that decision:

Cb-Cb distance 0.4223AUC
NZ to centroid 0.477AUC
NZ relative SASA 0.4595AUC
Assayed lysines 369in 43 ternaries

All at or below chance, and the Cb-Cb direction is slightly inverted: ubiquitylated lysines sit farther from the ligand (18.73 A median) than unmodified ones (14.89 A). The explanation is mechanistic rather than technical, which is why refitting is unlikely to help: the ligand site is where a substrate is recruited, not where ubiquitin is transferred, and the E2 that transfers it sits elsewhere on the cullin scaffold and is not in the trimmed construct at all. NOBBLE therefore reports lysine geometry as a measurement and emits no reach score.

Receptor constructs and the token budget

Every construct is defined by a named InterPro entry, never by a residue range, and its boundaries are resolved at build time. Two are declared exceptions to the 400-token tier-1 cap, both because the receptor does not fold or function without its partners.

ConstructE3ResiduesTokensVerdict
CRBN_CULT CRBN 109 154 within cap
VHL_VCB VHL 370 415 declared exception
DCAF15_DDB1_DDA1 DCAF15 1184 1229 declared exception
DDB1_BPC DDB1 310 355 within cap
KEAP1_KELCH KEAP1 316 361 within cap
FEM1B_ANK FEM1B 297 342 within cap

Data sources

NOBBLE re-acquires nothing. Every source below is read from what BINMAN already curated and pinned. Licence terms carry over unchanged: PROTAC-DB is internal-use only and never redistributed, UbiBrowser ships as derived counts only, and BioLiP2 annotations are not redistributed. The published repository bundles no third-party dataset.

SourceStatus
sievers_zf_screen resolved
mgtbind_ternary resolved
mgtbind_compounds resolved
mgdb_glues resolved
molgluedb_glues resolved
ubibrowser_literature_e3 resolved
ubibrowser_predicted_e3 resolved
protacdb_protacs resolved
digly_sites resolved
biolip2_annotations resolved
biolip2_artefacts resolved
assayed_ubiquitylation resolved
mgtbind_complexes resolved
mgtbind_compounds_raw resolved
degronopedia unavailable
protcid_interfaces unavailable

Reference bridge distribution: 285,996 experimentally observed bridges, which is what every bridge percentile on these pages is measured against.

The composite, and why it may be empty

Not fitted. not_fitted: 6 positive(s) and 0 negative(s) scored, below the floor of 8 labelled pairs. Spec 4.8 fits on the benchmark panel and the Sievers matched set; the Sievers arm runs after the panel. A composite fitted on fewer pairs than it has components would be fitting noise

Spec 4.8 fits the composite weights on the benchmark panel and the Sievers matched set and never hand-sets them, so until that fit has data every pair's nobble_score is null and the components are reported individually. A ranked column looks authoritative whatever its provenance, which is the reason for refusing rather than defaulting. The sliders on the Campaign tab explore a weighting; they do not publish one.

Validation

Three layers, all gating. A campaign that fails any of them publishes its failure rather than its hits. A poor ROC is not a gate failure; reporting no ROC, or reporting one without the matched negatives, is.

1. Pose recovery

Poses compared 45 across 5 of 7 pairs
Median RMSD 17.14 A
Within 1.0 A 0.0%
Within 2.0 A 0.0%
Within 3.0 A 0.0%
Within 5.0 A 2.2%
PairPosesBestMedianWorst
bench01_GSPT1_CRBN 10 24.543 A 29.221 A 72.196 A
bench02_BRD4_VHL 10 15.311 A 19.235 A 26.814 A
bench03_BRD4_CRBN 5 17.99 A 20.399 A 40.681 A
bench04_IKZF1_CRBN 10 4.992 A 5.357 A 5.399 A
bench05_CSNK1A1_CRBN 10 11.806 A 12.24 A 12.362 A

Superposed on the receptor, peptide backbone atoms only. Every record carries its own anchor size and anchor RMSD, because a pose RMSD means nothing without evidence that the frame it was measured in was sound.
2 benchmark pair(s) have no recovery number yet: their ternary jobs have not completed

2. Time split

not computed not_computed: no benchmark deposition was released after the 2021-09-30 training cutoff, so this campaign has no blind set. The retrospective number stands alone and must not be read as prospective performance

The seven panel members are all canonical, long-published ternaries, which is exactly why they are good positive controls and exactly why they cannot estimate prospective performance. A blind number needs post-cutoff depositions from mgtbind_ternary that the campaign does not currently run.

3. Matched-control ROC

not computed not_computed: 5 positive(s) and 0 matched negative(s) scored. The Sievers arm runs after the benchmark panel; a curve drawn from benchmark pairs alone would have no matched negatives and spec 5 forbids reporting one

Measured throughput

The campaign sizes itself from this measurement, not from the specification's estimate. Note that the figure is dominated by whichever jobs have run so far: pairformer cost is superlinear in sequence length, so a rate measured on small constructs is optimistic for large ones.

Median s/token0.4557
Fastest0.2074
Slowest0.6863
Jobs timed75
Affordable pairs175in 48.0 h
Devicemps

Provenance

The repository records every judgement made during the build, with the alternatives, the reason and how to reverse it, including the places where the build specification turned out to be wrong about its own data: DECISIONS.md for decisions, FINDINGS.md for results, BUILD_LOG.md for the event log and BUILD_SPEC.md for the specification itself.

github.com/bellcheddar/NOBBLE · built on BoltzMaker and BINMAN