> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tracebloc.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Embeddings

> Dataset template for contrastive embedding training: anchor/positive pairs or anchor/positive/negative triplets, one per .txt, ingest.yaml and the checks the data ingestor runs.

Learn vector representations from text pairs. This task is **self-supervised** (contrastive): there is no label column — the pairing is the supervision. Each sample is one `.txt` file with one tab-separated record:

* a **pair** `anchor<TAB>positive` — two texts that should embed close together, or
* a **triplet** `anchor<TAB>positive<TAB>negative` — with a hard negative that should embed far from the anchor.

Pairs and triplets may be mixed in one dataset.

## Folder layout

```text theme={null}
/data/shared/stsb-embeddings/
├── labels.csv
└── texts/
    ├── emb_0000001.txt      # anchor<TAB>positive
    ├── emb_0000004.txt      # anchor<TAB>positive<TAB>negative
    └── ...
```

Example pair:

```text theme={null}
How do I reset my password?	What are the steps to recover my account login?
```

* The folder is named `texts` and sits next to the manifest CSV.
* Exactly one tab between fields, all fields non-empty, one record per file on a single line. Unlike the other self-supervised text tasks, this structure **is enforced**.
* UTF-8 encoded; one extension across the dataset (`.txt` by default).

## Manifest CSV

```csv theme={null}
filename
emb_0000001
emb_0000002
emb_0000003
```

| Column      | Required               | Meaning                                                             |
| ----------- | ---------------------- | ------------------------------------------------------------------- |
| `filename`  | yes, exactly this name | The text file, with or without extension.                           |
| `extension` | no                     | Present in the shipped sample (`'.txt'`); not read by the ingestor. |

There is no label column. Setting `label:` in `ingest.yaml` is rejected.

## ingest.yaml

```yaml theme={null}
apiVersion: tracebloc.io/v1
kind: IngestConfig
category: embeddings
table: stsb_embeddings_train
intent: train
csv: /data/shared/stsb-embeddings/labels.csv
texts: /data/shared/stsb-embeddings/texts/
```

| Field                         | Required            | Meaning                                                                                                                                 |
| ----------------------------- | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `csv`                         | yes                 | Path to the manifest CSV.                                                                                                               |
| `texts`                       | yes                 | The `texts/` folder.                                                                                                                    |
| `label`                       | **must not be set** | No label column.                                                                                                                        |
| `positive_definition`         | no                  | Free text describing what makes a pair positive (for example `paraphrase`), recorded for consistency checks when datasets are combined. |
| `schema`                      | no                  | Extra typed columns only. Never `filename`.                                                                                             |
| `spec.file_options.extension` | no                  | `.txt` (default) or `.text`.                                                                                                            |
| `language`, `normalization`   | no                  | Dataset language and applied normalization.                                                                                             |

## What the ingestor checks

| Check             | Rejects                                                                                                             |
| ----------------- | ------------------------------------------------------------------------------------------------------------------- |
| File type         | Files under `texts/` with a different extension than configured, or mixed extensions.                               |
| Text content      | NUL bytes or invalid UTF-8; empty files produce a warning.                                                          |
| Contrastive pairs | A referenced file that is missing, spans several lines, has a field count other than 2 or 3, or has an empty field. |
| Data types        | With a `schema`: values that do not match their declared type.                                                      |

Plus the [checks every ingest runs](/create-use-case/templates#checks-every-ingest-runs), except label diversity (no label).

## Sample dataset

The template ships three pairs and two triplets with a five-row manifest. The `ingest.yaml` above ingests it with no overrides.

## Next steps

* Stage the data and run the ingest: [Prepare Data](/create-use-case/prepare-dataset)
* Shared rules for every template: [Dataset templates](/create-use-case/templates)