Models for bioinformatics

This page is for bioinformaticians who bring a pretrained model, such as ESM-2, or their own PyTorch code. The upload works as described in Bring your model: install tracebloc[training], pass the model to user.upload_model() with the use case’s dataset key, confirm its licence, then link and train.

Upload ESM-2, step by step

ESM-2 is a Hugging Face model, so you upload its folder. The smallest ESM-2 makes a quick first experiment. In a fresh virtual environment, on Python 3.11, 3.12 or 3.13:

pip install "tracebloc[training]>=1.2.82"
hf download facebook/esm2_t6_8M_UR50D --local-dir esm2_t6_8M

Then, in your notebook:

from tracebloc import User

user = User()  # prompts for your email and password

user.upload_model("esm2_t6_8M", dataset_id="<dataset key>")

training_plan = user.link_model_dataset("<dataset key>")
training_plan.experiment_name("ESM-2 8M")
training_plan.start()

The SDK builds the model the use case’s task needs from the ESM-2 backbone and, for a task with classes, starts a new head for the use case’s classes. It turns ESM-2’s vocab.txt into a tokenizer.json and checks it gives the same token ids as ESM-2’s own tokenizer. The licence it reads is MIT. The converted model reads up to 1,024 tokens, ESM-2’s own limit, or the dataset’s sequence length when that is shorter. The same steps work for the larger ESM-2 models up to 650M parameters. ESM-2 650M’s weights are over 2 GiB, so the SDK ships them in bf16, and it trains with LoRA by default. The 3B and 15B ESM-2 models are over the 705M parameters federated averaging can merge, so they can’t be trained.

Which models convert

These models convert with the released SDK and pass the acceptance check:

Domain	Model	Bring	Tasks
Protein	ESM-2, 8M to 650M parameters	Hugging Face folder	Text classification (a label per sequence), Token classification (a label per residue), Masked language modeling
Protein	ProtBERT	Hugging Face folder	Text classification. At 420M parameters it trains with LoRA by default.
Molecule	ChemBERTa, MoLFormer	Hugging Face folder, SMILES strings as text	Text classification. MoLFormer ships its own code, which the SDK converts with it.
Pathology	Phikon, Phikon-v2	Hugging Face folder	Image classification
Pathology	torchvision and timm models	Checkpoint	Image classification, with the model’s own image normalisation
Gene expression	scikit-learn models such as a random forest, XGBoost, LightGBM, CatBoost	Pickled fitted model	Tabular classification, Tabular regression
Survival	Cox proportional hazards in lifelines or scikit-survival	Pickled fitted model	Survival analysis
Any	Your own PyTorch model	Your own code, with its checkpoint	The task it is built for

ESM-2 can’t be converted for Embeddings: its checkpoint carries no pooler, which an embeddings model needs.

Models that can’t be converted

The SDK stops these models before anything uploads:

Model	Why
Nucleotide Transformer v2, DNABERT-2	Their own code does not run under the training versions of `transformers`.
HyenaDNA	Its code imports `json`, which model code may not use.
Caduceus	Its code imports `inspect`, which model code may not use, and it needs compiled GPU code, which training cannot run.
GENA-LM	Its code builds no classification model.
Geneformer	Its gene vocabulary is a pickled dictionary, not a tokenizer.
CTransPath	Its own stem replaces the Swin patch embedding, and its folder carries no code for that stem.
UNI, Virchow2	They are licensed CC-BY-NC-ND-4.0, which forbids redistributing the weights.

H-optimus-0 is licensed Apache-2.0, but its 1.1B parameters are over the 705M limit, so it can’t be trained. If you need a model this page doesn’t list, reach out to us at [email protected]. UNI, Virchow2 and H-optimus-0 are gated on the Hugging Face Hub: you accept their terms there before you can download them.

Lay out your data

The data owner ingests the data into their secure environment with the tracebloc CLI. Each task has a dataset template:

Your data	Task	Layout
One label per protein, DNA sequence or molecule	Text classification	One `.txt` file per sequence in `texts/`, and `labels.csv` with `filename,label`
One label per residue	Token classification	One `.txt` file per sequence with its residues separated by spaces, such as `M K T A Y`, and one tag per residue in the `label` column: `O`, `B-<TYPE>` or `I-<TYPE>`
Sequences without labels	Masked language modeling	One `.txt` file per sequence in `sequences/`, listed in `labels.csv`
Pathology tiles	Image classification	JPG or PNG tiles in `images/`, and `labels.csv` with `filename,label`
Expression values	Tabular classification or Tabular regression	One CSV, one row per sample and one column per gene

A residue-level label cell holds at most 255 characters, so per-residue labels fit short sequences only. A table holds at most 1,006 feature columns: for expression data, select the genes you train on, such as the most variable ones, before you ingest.

From FASTA to text files

The data ingestor reads one .txt file per sequence. To ingest sequences from a FASTA file, write each record to its own file named by its id, and list it in labels.csv. This script reads proteins.fasta and the labels in classes.csv (columns id,label), and writes the folder proteins/ for Text classification:

import csv
from pathlib import Path

out = Path("proteins")
(out / "texts").mkdir(parents=True, exist_ok=True)

with open("classes.csv", newline="") as f:
    label_of = {row["id"]: row["label"] for row in csv.DictReader(f)}

records, name, seq = [], None, []
for line in Path("proteins.fasta").read_text().splitlines():
    if line.startswith(">"):
        if name:
            records.append((name, "".join(seq)))
        name, seq = line[1:].split()[0], []
    elif line.strip():
        seq.append(line.strip())
if name:
    records.append((name, "".join(seq)))

missing = [name for name, _ in records if name not in label_of]
if missing:
    raise SystemExit(f"classes.csv has no label for: {', '.join(missing)}")

with open(out / "labels.csv", "w", newline="") as f:
    writer = csv.writer(f)
    writer.writerow(["filename", "label"])
    for name, seq in records:
        (out / "texts" / f"{name}.txt").write_text(seq + "\n")
        writer.writerow([name, label_of[name]])

Then ingest proteins/ as described in the Text classification dataset template.

Overview

Create a Use Case

Join a Use Case

Environment Setup

Tools & Help

Releases

Models for bioinformatics

Upload ESM-2, step by step

Which models convert

Models that can’t be converted

Lay out your data

From FASTA to text files

​Upload ESM-2, step by step

​Which models convert

​Models that can’t be converted

​Lay out your data

​From FASTA to text files

Upload ESM-2, step by step

Which models convert

Models that can’t be converted

Lay out your data

From FASTA to text files