Sequence-to-sequence

Map an input sequence to an output sequence — translation, summarization, paraphrase. This task is self-supervised from the ingestor’s point of view: there is no label column, because the target side of each pair is the supervision. Each sample is one .txt file with one line of the form source<TAB>target.

Folder layout

/data/shared/wmt-seq2seq/
├── labels.csv
└── texts/
    ├── s2s_0000001.txt
    ├── s2s_0000002.txt
    └── ...

Each file holds one pair, for example:

Translate to French: Good morning.	Bonjour.

The folder is named texts and sits next to the manifest CSV.
Put exactly one tab between source and target: everything before the first tab is the source, everything after is the target. The ingestor does not enforce this structure — it only checks that files are valid text — so a malformed file surfaces at training time, not at ingest.
UTF-8 encoded; one extension across the dataset (.txt by default).

Manifest CSV

filename
s2s_0000001
s2s_0000002
s2s_0000003

Column	Required	Meaning
`filename`	yes, exactly this name	The text file, with or without extension.
`extension`	no	Present in the shipped sample (`'.txt'`); not read by the ingestor.

There is no label column. Setting label: in ingest.yaml is rejected.

ingest.yaml

apiVersion: tracebloc.io/v1
kind: IngestConfig
category: seq2seq
table: wmt_seq2seq_train
intent: train
csv: /data/shared/wmt-seq2seq/labels.csv
texts: /data/shared/wmt-seq2seq/texts/

Field	Required	Meaning
`csv`	yes	Path to the manifest CSV.
`texts`	yes	The `texts/` folder.
`label`	must not be set	No label column.
`schema`	no	Extra typed columns only. Never `filename`.
`spec.file_options.extension`	no	`.txt` (default) or `.text`.
`language`, `normalization`	no	Dataset language and applied normalization.

What the ingestor checks

Check	Rejects
File type	Files under `texts/` with a different extension than configured, or mixed extensions.
Text content	NUL bytes or invalid UTF-8; empty files produce a warning.
Data types	With a `schema`: values that do not match their declared type.

Plus the checks every ingest runs, except label diversity (no label).

Sample dataset

The template ships five source<TAB>target pairs (translation, summarization and paraphrase examples) with a five-row manifest. The ingest.yaml above ingests it with no overrides.

Next steps

Stage the data and run the ingest: Prepare Data
Shared rules for every template: Dataset templates

Overview

Create a Use Case

Join a Use Case

Environment Setup

Tools & Help

Sequence-to-sequence

Folder layout

Manifest CSV

ingest.yaml

What the ingestor checks

Sample dataset

Next steps

​Folder layout

​Manifest CSV

​ingest.yaml

​What the ingestor checks

​Sample dataset

​Next steps

Folder layout

Manifest CSV

ingest.yaml

What the ingestor checks

Sample dataset

Next steps