BEND 2.0 Tests Whether DNA Foundation Models Can Guide Design

Introduction

Genomic foundation models are increasingly being used to propose DNA sequences and edits. Yet a model that performs well on a standard prediction benchmark is not automatically a useful design system. Practical design requires predicting what a change will do, determining whether that effect depends on cellular or experimental context, and using the prediction to choose candidates worth testing.

BEND 2.0 was created to measure these capabilities more directly. The benchmark combines eight tasks across five experimental datasets. It evaluates both the prediction of functional consequences caused by DNA changes and the ability to use model predictions to search for sequences with a desired response.

Key findings

• Internal representations can outperform native outputs. Linear readouts trained on frozen model representations can match or exceed the models’ own output heads. This suggests that useful functional information may already be present inside the representation, even when the default prediction mechanism does not extract it effectively.
• Functional post-training helps. Additional training targeted at functional objectives substantially improves capability, showing that broad pretraining alone does not guarantee strong performance on a particular design problem.
• Transfer across assays and genes is encouraging. Linear readouts transfer well across experimental assays and genes, indicating that the models may capture patterns that extend beyond the dataset used to fit an individual predictor.
• Average effects can hide conditional weakness. A model may predict the average impact of a change accurately while struggling to determine how that impact varies across contexts.
• Combined effects are not the same as interactions. Predicting the outcome of multiple mutations together does not demonstrate that the model has learned the biological interaction between those mutations.
• Guided search is useful, but incomplete. Model-based selection can reduce the number of candidates needed to find a target response. However, there remains considerable room for better choices, particularly when the objective is context-specific or must be optimized across multiple cell types.

Why it matters

The main contribution of BEND 2.0 is not simply another leaderboard. It separates accurate prediction from useful design. The distinction matters because a design model must remain helpful under changes in context and must handle combinations of edits, rather than only reproduce average trends seen in its training data.

The results also argue for more careful interpretation of benchmark scores. An improvement on an aggregate metric may not mean that a model has learned transferable biological rules. Future evaluations should examine cross-assay and cross-gene transfer, context-dependent effects, mutation interactions, and whether model recommendations actually reduce experimental search costs.

Overall, BEND 2.0 finds signs of general utility in current genomic models, but also shows that they are far from saturating even relatively simple tasks. For now, these systems look more like promising design assistants than autonomous DNA designers.

Source: Hugging Face Daily Papers