> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tracebloc.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Masked language modeling

> Dataset template for masked language modeling: one token sequence per .txt under sequences/, a manifest CSV without labels, ingest.yaml and the checks the data ingestor runs.

Predict masked-out words. This task is **self-supervised**: there is no label — the training side masks tokens on the fly. Each sample is one `.txt` file holding a space-separated token sequence, listed in a manifest CSV.

## Folder layout

```text theme={null}
/data/shared/primekg-mlm/
├── labels.csv
└── sequences/
    ├── seq_0000001.txt
    ├── seq_0000002.txt
    └── ...
```

* The folder is named `sequences` (not `texts`) — the ingestor reserves `sequences/` for pre-tokenized data — and sits next to the manifest CSV.
* One sequence per file, tokens separated by spaces, UTF-8 encoded, one extension across the dataset (`.txt` by default). Example from the shipped sample: `Lepirudin indication Huntington phenotype_present Chorea associated_with Dystonia`.
* The shipped sample also contains a `tokenizer.json` next to the CSV. The current ingestor neither reads nor copies it; it is part of the sample only.

## Manifest CSV

```csv theme={null}
filename
seq_0000001
seq_0000002
seq_0000003
```

| Column      | Required               | Meaning                                                             |
| ----------- | ---------------------- | ------------------------------------------------------------------- |
| `filename`  | yes, exactly this name | The sequence file, with or without extension.                       |
| `extension` | no                     | Present in the shipped sample (`'.txt'`); not read by the ingestor. |

There is no label column. Setting `label:` in `ingest.yaml` is rejected.

## ingest.yaml

```yaml theme={null}
apiVersion: tracebloc.io/v1
kind: IngestConfig
category: masked_language_modeling
table: primekg_mlm_train
intent: train
csv: /data/shared/primekg-mlm/labels.csv
sequences: /data/shared/primekg-mlm/sequences/
```

| Field                         | Required            | Meaning                                                       |
| ----------------------------- | ------------------- | ------------------------------------------------------------- |
| `csv`                         | yes                 | Path to the manifest CSV.                                     |
| `sequences`                   | yes                 | The `sequences/` folder. Its parent becomes the dataset root. |
| `label`                       | **must not be set** | Self-supervised task.                                         |
| `schema`                      | no                  | Extra typed columns only. Never `filename`.                   |
| `spec.file_options.extension` | no                  | `.txt` (default) or `.text`.                                  |
| `language`, `normalization`   | no                  | Dataset language and applied normalization.                   |

## What the ingestor checks

| Check        | Rejects                                                                                   |
| ------------ | ----------------------------------------------------------------------------------------- |
| File type    | Files under `sequences/` with a different extension than configured, or mixed extensions. |
| Text content | NUL bytes or invalid UTF-8; empty files produce a warning.                                |
| Data types   | With a `schema`: values that do not match their declared type.                            |

Plus the [checks every ingest runs](/create-use-case/templates#checks-every-ingest-runs), except label diversity (no label). For text tasks the ingestor also records a data-derived text profile — Unicode-script mix and document-length distribution, never the text itself — so the platform can warn when a model's tokenizer is a poor fit for the dataset.

## Sample dataset

The template ships five knowledge-graph random-walk sequences and a five-row manifest. The `ingest.yaml` above ingests it with no overrides.

## Next steps

* Stage the data and run the ingest: [Prepare Data](/create-use-case/prepare-dataset)
* Shared rules for every template: [Dataset templates](/create-use-case/templates)