Use mRNABERT Online
Commercially Available mRNABERT No-Code Web Server
mRNABERT: Universal Language Model for Full-Length mRNA Design and Property Prediction
mRNABERT is a pre-trained mRNA language model for designing and analysing complete mRNA sequences, from the 5′UTR through the coding sequence (CDS) to the 3′UTR. It was developed by Ying Xiong, Aowen Wang, Yu Kang, Chao Shen, Chang-Yu Hsieh, Tingjun Hou and colleagues, with corresponding author Tingjun Hou at the College of Pharmaceutical Sciences, Zhejiang University, and was published open access in Nature Communications in 2025.
Therapeutic mRNA performance depends on how the UTRs and the CDS work together, yet most earlier sequence models handle only an isolated UTR or CDS region. That leaves the synergy across a full transcript unmodelled. mRNABERT addresses this with a single foundation model trained on whole mRNA sequences.
The authors report state-of-the-art results on 5′UTR and CDS design tasks, RNA-binding protein (RBP) site prediction and full-length mRNA property prediction, and say it also outperforms large protein language models on several protein-level tasks.
Feature | Detail |
|---|---|
Architecture | 12 bidirectional Transformer encoder blocks (BERT-style) with ALiBi positional bias and FlashAttention |
Pre-training data | More than 18 million non-redundant mRNA sequences curated by the authors |
Tokenization | Dual scheme: single nucleotides for UTRs, codons for the CDS |
Second training stage | Cross-modality contrastive learning on about 500,000 CDS and their amino acid sequences |
Reported tasks | 5′UTR ribosome load, CDS properties, RBP binding, m6A, splice sites, polyadenylation, stability and translation efficiency |
How mRNABERT Models Full-Length mRNA
The model is trained with masked language modelling on a custom tokenizer, then refined with a contrastive stage that ties each CDS to its protein sequence.
Dual tokenization: UTRs are read nucleotide by nucleotide, while the CDS is read codon by codon, so each region is represented in its natural unit.
Masked language modelling: The encoder learns sequence context from more than 18 million non-redundant mRNAs, with regions identified using NCBI ORF Finder.
Cross-modality contrastive learning: Separate encoders for CDS and amino acid sequences are projected into a shared space. The authors report that this raised the adjusted Rand index for amino-acid property clustering from 0.166 to 0.498.
Long-sequence handling: ALiBi and FlashAttention let the model be tested at maximum lengths of 1,022, 2,044 and 3,066 tokens.
Benchmark Results for mRNA Property Prediction
Selected results reported by the authors
Task | Reported result |
|---|---|
5′UTR mean ribosome load, largest libraries | Spearman R of 0.962 (U1) and 0.924 (U2), top performance |
3′UTR RBP binding (22 RBPs, 5-fold CV) | Average ACC 0.786, F1 0.751, MCC 0.501; best on 13 of 22 RBPs |
Protein melting point | R2 rose from 0.60 to 0.77 after contrastive learning |
Protein solubility | R2 of 0.63 (CaLM 0.61) |
Ultra-long mRNA translation efficiency | Mean R2 of 0.66 across cell types versus 0.42 for the best prior RNA model |
On m6A site prediction in nine cell lines the model ranked second, behind 3UTRBERT, and on splice sites it ranked second behind ERNIE-RNA. On a full-length set of 233 mRNAs (112 UTRs, 121 CDSs) it is reported to outperform other models on two stability and four translation-efficiency labels. The ultra-long translation efficiency data came from ribosome profiling across more than 140 human and mouse cell types.
What is Tamarind Bio?
Tamarind Bio is a no-code bioinformatics platform built to give life scientists and researchers access to powerful computational tools. Many cutting-edge machine learning models are hard to deploy and use. Tamarind provides an intuitive, web-based environment that removes the complexity of high-performance computing, software dependencies and command-line interfaces.
The platform is designed for biologists, chemists and other researchers who may not have a background in programming or cloud infrastructure but want to run models on their own data. Key features include:
A user-friendly graphical interface for setting up and launching experiments
A robust API for integration into existing research pipelines
An automated system for managing and scaling computational resources
Tamarind treats information and data security as a top priority, as detailed in its Trust Center and Terms of Service.
Accelerating Discovery with mRNABERT on Tamarind Bio
5′UTR design: Score candidate 5′UTRs for expected mean ribosome load to prioritise sequences for expression.
CDS optimisation: Evaluate coding sequences for properties such as degradation behaviour, as in the SARS-CoV-2 vaccine degradation benchmark.
Regulatory site prediction: Predict RBP binding, m6A sites, splice sites and alternative polyadenylation from sequence.
Full-length mRNA screening: Compare whole transcripts for stability and translation efficiency before synthesis.
How to Use mRNABERT on Tamarind Bio
Log in: Sign in at tamarind.bio and open the mRNABERT tool.
Provide sequences: Enter or upload mRNA nucleotide sequences, with UTR and CDS regions as the model expects.
Choose the task: Select the prediction or embedding output you need, if exposed.
Set parameters: Adjust options such as maximum sequence length if exposed, noting the authors tested up to 3,066 tokens.
Run: Submit the job and wait for it to finish.
Download outputs: Download the predictions or embeddings.
Interpret: Compare scores across candidates and weigh them against the benchmark performance above.
Use downstream: Rank designs for experimental testing or feed embeddings into your own models.
Things to Keep in Mind
mRNABERT did not beat CaLM on protein melting point and was second, not first, on m6A and splice-site prediction.
Transcript abundance gains over CaLM held in five of seven species, not in E. coli or Haloferax volcanii.
Several full-length and supplementary results are reported only in the paper supplement.
Predictions are computational and should be validated experimentally.
Source: Xiong Y, Wang A, Kang Y, Shen C, Hsieh C-Y, Hou T, et al. mRNABERT: advancing mRNA sequence design with a universal language model and comprehensive dataset. Nature Communications 16, 10371 (2025). DOI: 10.1038/s41467-025-65340-8. Code archive cited by the article: yyly6/mRNABERT v1.0 (Zenodo).