> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tracebloc.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Causal language modeling

> Dataset template for causal language modeling: raw text or prompt/completion pairs, one per .txt, a manifest CSV without labels, ingest.yaml and the checks the data ingestor runs.

Predict the next word. This task is **self-supervised**: there is no label. Each sample is one `.txt` file of raw text in one of two shapes:

* **Pretraining** — the whole file is plain text.
* **Instruction tuning (SFT)** — one line of the form `prompt<TAB>completion`. Everything before the first tab is the prompt, everything after is the completion.

A dataset may mix both shapes. The ingestor does not enforce either shape; it only checks that files are valid text.

## Folder layout

```text theme={null}
/data/shared/dolly-clm/
├── labels.csv
└── texts/
    ├── clm_0000001.txt      # plain text
    ├── clm_0000002.txt      # plain text
    ├── clm_0000004.txt      # prompt<TAB>completion
    └── ...
```

* The folder is named `texts` (raw text) and sits next to the manifest CSV.
* UTF-8 encoded; one extension across the dataset (`.txt` by default).

## Manifest CSV

```csv theme={null}
filename
clm_0000001
clm_0000002
clm_0000003
```

| Column      | Required               | Meaning                                                             |
| ----------- | ---------------------- | ------------------------------------------------------------------- |
| `filename`  | yes, exactly this name | The text file, with or without extension.                           |
| `extension` | no                     | Present in the shipped sample (`'.txt'`); not read by the ingestor. |

There is no label column. Setting `label:` in `ingest.yaml` is rejected.

## ingest.yaml

```yaml theme={null}
apiVersion: tracebloc.io/v1
kind: IngestConfig
category: causal_language_modeling
table: dolly_clm_train
intent: train
csv: /data/shared/dolly-clm/labels.csv
texts: /data/shared/dolly-clm/texts/
```

| Field                         | Required            | Meaning                                     |
| ----------------------------- | ------------------- | ------------------------------------------- |
| `csv`                         | yes                 | Path to the manifest CSV.                   |
| `texts`                       | yes                 | The `texts/` folder.                        |
| `label`                       | **must not be set** | Self-supervised task.                       |
| `schema`                      | no                  | Extra typed columns only. Never `filename`. |
| `spec.file_options.extension` | no                  | `.txt` (default) or `.text`.                |
| `language`, `normalization`   | no                  | Dataset language and applied normalization. |

## What the ingestor checks

| Check        | Rejects                                                                               |
| ------------ | ------------------------------------------------------------------------------------- |
| File type    | Files under `texts/` with a different extension than configured, or mixed extensions. |
| Text content | NUL bytes or invalid UTF-8; empty files produce a warning.                            |
| Data types   | With a `schema`: values that do not match their declared type.                        |

Plus the [checks every ingest runs](/create-use-case/templates#checks-every-ingest-runs), except label diversity (no label). No tokenizer is needed at ingest; the ingestor records a data-derived text profile (script mix and length distribution, never the text) for the platform's tokenizer-fit warning.

## Sample dataset

The template ships five files: three plain-text passages and two `prompt<TAB>completion` pairs, with a five-row manifest. The `ingest.yaml` above ingests it with no overrides.

## Next steps

* Stage the data and run the ingest: [Prepare Data](/create-use-case/prepare-dataset)
* Shared rules for every template: [Dataset templates](/create-use-case/templates)