Masked language modeling

Predict masked-out words. This task is self-supervised: there is no label — the training side masks tokens on the fly. Each sample is one .txt file holding a space-separated token sequence, listed in a manifest CSV.

Folder layout

/data/shared/primekg-mlm/
├── labels.csv
└── sequences/
    ├── seq_0000001.txt
    ├── seq_0000002.txt
    └── ...

The folder is named sequences (not texts) — the ingestor reserves sequences/ for pre-tokenized data — and sits next to the manifest CSV.
One sequence per file, tokens separated by spaces, UTF-8 encoded, one extension across the dataset (.txt by default). Example from the shipped sample: Lepirudin indication Huntington phenotype_present Chorea associated_with Dystonia.
The shipped sample also contains a tokenizer.json next to the CSV. The current ingestor neither reads nor copies it; it is part of the sample only.

Manifest CSV

filename
seq_0000001
seq_0000002
seq_0000003

Column	Required	Meaning
`filename`	yes, exactly this name	The sequence file, with or without extension.
`extension`	no	Present in the shipped sample (`'.txt'`); not read by the ingestor.

There is no label column. Setting label: in ingest.yaml is rejected.

ingest.yaml

apiVersion: tracebloc.io/v1
kind: IngestConfig
category: masked_language_modeling
table: primekg_mlm_train
intent: train
csv: /data/shared/primekg-mlm/labels.csv
sequences: /data/shared/primekg-mlm/sequences/

Field	Required	Meaning
`csv`	yes	Path to the manifest CSV.
`sequences`	yes	The `sequences/` folder. Its parent becomes the dataset root.
`label`	must not be set	Self-supervised task.
`schema`	no	Extra typed columns only. Never `filename`.
`spec.file_options.extension`	no	`.txt` (default) or `.text`.
`language`, `normalization`	no	Dataset language and applied normalization.

What the ingestor checks

Check	Rejects
File type	Files under `sequences/` with a different extension than configured, or mixed extensions.
Text content	NUL bytes or invalid UTF-8; empty files produce a warning.
Data types	With a `schema`: values that do not match their declared type.

Plus the checks every ingest runs, except label diversity (no label). For text tasks the ingestor also records a data-derived text profile — Unicode-script mix and document-length distribution, never the text itself — so the platform can warn when a model’s tokenizer is a poor fit for the dataset.

Sample dataset

The template ships five knowledge-graph random-walk sequences and a five-row manifest. The ingest.yaml above ingests it with no overrides.

Next steps

Stage the data and run the ingest: Prepare Data
Shared rules for every template: Dataset templates

Overview

Create a Use Case

Join a Use Case

Environment Setup

Tools & Help

Masked language modeling

Folder layout

Manifest CSV

ingest.yaml

What the ingestor checks

Sample dataset

Next steps

​Folder layout

​Manifest CSV

​ingest.yaml

​What the ingestor checks

​Sample dataset

​Next steps

Folder layout

Manifest CSV

ingest.yaml

What the ingestor checks

Sample dataset

Next steps