AlphaFold3 is no longer the best model.
Over the past year, we've seen many (mostly companies) claim that their newly released models matched or outperformed AlphaFold3 in some important structure prediction task. A substantive amount of these also underperformed their self-professed benchmarks under independent scrutiny.
This is why I was initially skeptical on the results from the ESMFold2, Protenix-v2, and OpenDDE teams' respective releases. Within a few weeks of each others, these groups each released benchmarks showing meaningful outperformance over AF3.
Surprisingly enough, in our independent benchmarks on Tamarind Bio, these results are shown to be correct!
On the FoldBench and Elofsson/Fromm benchmarks, we ran these models with the same inputs and setup, and each showed compared substantive improvements over previously published AF3 results:
On FoldBench v1 (172 antibody–antigen interfaces across 113 complexes), previously published AF3 lands at 47.9% DockQ success. Running the new models on identical inputs and setup, ESMFold2 hits 71.1%, Protenix-v2 67.1%, and OpenDDE v1 76.1%
The Elofsson/Fromm split is the harder, more honest test — out-of-distribution AbAg with a 2021-09-30 cutoff, so recent structures can't leak into training and inflate the numbers. AF3 comes in higher here (60.9%), and as you'd expect on a stricter split the margins compress, but the ordering largely holds: OpenDDE abag 68.2%, ESMFold2 65.5%, and OpenDDE v1 64.5% all clear AF3. Protenix-v2 (58.2%) is the one exception.
What does this mean for actual discovery? The obvious part is we get better model structures for historically difficult tasks like AbAg complexes. What is most interesting to me, is the added accuracy to downstream tasks.
Most cutting edge de novo design protocols use structure predictors as oracles to evaluate the quality and iterate on a given design, better evaluators mean better designs with higher hit rates.
Similarly, optimization and affinity maturation tools like the ProteinMPNN family also get a large benefit since the base structures the structure-based optimizer is built on have higher accuracy.
Evidently, the first breakthrough result doesn't stay at the frontier for long in molecular or scientific AI, and open model efforts eventually match, then surpass the state of the art.
I think we will see the rising tide lifting all boats here very soon. In the meantime try out the models described above on Tamarind Bio today!