Ingest with the CLI
Lay out each split as shown in Folder layout, then ingest the training data and the test data:--holds open_data marks the dataset as open data. If your data is about patients, pass --holds patient_data; if it was generated, pass --holds synthetic_data. Leave --holds out on a terminal and the CLI asks; in a script, the dataset is marked as patient data. The mark decides which use case types you can create on the dataset.
--label-column names the class column in labels.csv. Images sorted into one folder per class, with no labels.csv, ingest too: each image’s label is its folder’s name, the CLI writes labels.csv in the copy, and you leave out --label-column. On a terminal, tracebloc data ingest ./cats-dogs-train works the task out from the folder and asks only what the files can’t say; see Guided ingest. --target-size WxH, with the letter x, for example 256x256, is the size every image has to be; without it, the CLI takes the most common size among your images. See Image resolution. The [256, 256] default in the ingest.yaml table below applies only to Helm ingests.
Folder layout
- With the CLI, the dataset folder is on your machine (
./cats-dogs-train). With Helm, it sits on the shared volume (/data/shared/...). - The image folder must be named
imagesand sit next to the labels CSV, which the CLI expects to be namedlabels.csv. - Every image in the dataset must have the same extension and the same width and height. Mixed extensions or mixed resolutions fail the ingest.
- Supported extensions:
.jpg,.jpeg,.png. The CLI reads the extension from your files; with Helm, the default is.jpeg— override it withspec.file_options.extension. - The ingestor copies images as they are. It does not resize them — bring them to the target size before you ingest.
Labels CSV
Extra columns are ignored unless you declare them in
schema.
ingest.yaml
To use images of another size, set
target_size (or spec.file_options.target_size) to their exact dimensions:
What the ingestor checks
Plus the checks every ingest runs.
Size limit
Onetracebloc data ingest takes at most 1 GiB in total and 500 MiB per file. If your dataset is larger:
- Save the images as JPEG instead of PNG.
- Lower the resolution: resize every image to one smaller size.
- Use fewer images.
Sample dataset
The template ships six 256×256 RGB JPEG images (three cats, three dogs) and a labels CSV with the two classescat and dog. The ingest.yaml above ingests it with no overrides.
Next steps
- Ingest the data: Prepare Data
- Shared rules for every template: Dataset templates