Use EvoProtGrad Online

EvoProtGrad: Gradient-Based Directed Evolution of Proteins with a Product of Experts

EvoProtGrad implements plug-and-play directed evolution of proteins with gradient-based discrete Markov chain Monte Carlo (MCMC), a sampling framework the paper calls PPDE. It was developed by Patrick Emami, Aidan Perreault, Jeffrey Law, David Biagioni and Peter St. John at the National Renewable Energy Laboratory, Stanford University and Maplewell Energy, and published open access in Machine Learning: Science and Technology in 2023.

Directed evolution searches for mutations that improve a known wild-type protein, but the search space is huge and beneficial mutations are rare. Brute-force search is practical only for variants with one or two mutations, and the paper notes that a 400-residue protein has roughly 10^19 ways to make five single substitutions. Classic approaches also depend on high-throughput wet-lab screening.

EvoProtGrad evolves proteins in silico without fine-tuning or retraining any model. It composes unsupervised models, such as protein language models, with supervised models that predict function from sequence into a single product of experts distribution, then samples from it with a fast gradient-guided MCMC sampler instead of random or brute-force search.

Why Gradient-Guided Protein Directed Evolution Matters

A product of experts places probability mass where all of its models agree, so a variant must look evolutionarily plausible and also score well on predicted function. Using gradients means the sampler does not have to enumerate every possible mutation near the current sequence, which makes searching several mutations away from the wild type practical.

How EvoProtGrad Works: Product of Experts and Gradient-Based MCMC

1. Choose experts: evolutionary density experts such as Potts models or ESM2 language models are combined with supervised activity predictors, with a scale factor weighting the supervised part.

2. Propose mutations with gradients: a first-order Taylor approximation of the combined score ranks single-residue changes around the current variant, giving a tempered softmax proposal.

3. Accept or reject: a Metropolis-Hastings step keeps the chain faithful to the target distribution. Each step needs one forward and two backward passes.

4. Escape local optima: a path proposal applies several single changes in sequence, controlled by a maximum path length.

5. Run in parallel: a population of chains is evolved at once; the paper used populations of 128 variants.

EvoProtGrad Results: Diversity, Fitness and Evolutionary Density

The authors compared PPDE with random search, simulated annealing, a continuous-relaxation sampler (MALA-approx) and CMA-ES on three benchmark proteins. PPDE with a Potts expert found variants with higher log fitness and more mutations (about 2 to 4 or more) than random search, simulated annealing and MALA-approx, whose variants averaged about one mutation. The authors report that PPDE-based samplers achieved the best population diversity across the three proteins, measured as the percentage of unique variants.

CMA-ES reached high fitness on two proteins but with diversity of only 0.8 percent on PABP and 3.1 percent on UBE4B, and it struggled on the larger proteins. Combining a Potts expert with ESM2 produced better top-percentile fitness and density than either alone, and larger ESM2 models tended to find variants with higher predicted activity.

What is Tamarind Bio?

Tamarind Bio is a no-code bioinformatics platform built to give life scientists and researchers access to powerful computational tools. Many cutting-edge machine learning models are hard to deploy and use. Tamarind provides an intuitive, web-based environment that removes the complexity of high-performance computing, software dependencies and command-line interfaces.

The platform is designed for biologists, chemists and other researchers who may not have a background in programming or cloud infrastructure but want to run models on their own data. Key features include:

• A user-friendly graphical interface for setting up and launching experiments

• A robust API for integration into existing research pipelines

• An automated system for managing and scaling computational resources

Tamarind treats information and data security as a top priority, as detailed in its Trust Center and Terms of Service.

Accelerating Discovery with EvoProtGrad on Tamarind Bio

• Proposing variants of a wild type: generate candidates several mutations from a known protein for screening.

• Combining knowledge sources: mix evolutionary language-model scores with a supervised fitness predictor in one objective.

• Exploring diverse optima: sample a population of distinct variants rather than one local optimum.

• Constrained design: the authors describe fixing positions by zeroing their mutation probabilities, though they did not demonstrate it.

How to Use EvoProtGrad on Tamarind Bio

1. Log in: sign in at tamarind.bio and open the EvoProtGrad tool.

2. Provide the wild-type sequence: enter the protein you want to evolve.

3. Select experts: choose the protein language model expert, if exposed. Supervised experts need training data and may not be offered.

4. Set sampling parameters: choose the number of chains, MCMC steps and maximum mutations, if exposed.

5. Run the job: submit it and wait for sampling to finish.

6. Download outputs: retrieve the sampled variants and their scores.

7. Interpret results: rank variants by score and check how many mutations each carries.

8. Use downstream: send top variants to structure prediction or experimental testing.

Things to Keep in Mind

• Large language model experts are costly because each step needs two backward passes through the model, though chains can be run in parallel.

• Experts that are slow to evaluate and differentiate, such as variational autoencoders, are difficult to use.

• Insertions and deletions are not supported, only substitutions.

• Results in the paper are in silico, scored by oracle models; wet-lab validation is still needed.

Source: Emami P, Perreault A, Law J, Biagioni D, St. John P. Plug and play directed evolution of proteins with gradient-based discrete MCMC. Machine Learning: Science and Technology 2023;4:025014. DOI 10.1088/2632-2153/accacd. Code: github.com/pemami4911/ppde