Ingest with the CLI
Lay out each split as shown in Folder layout, then ingest the training set and the test set:--label-column: the classes come from the XML files. In a guided run, the CLI asks for the image resolution: press Enter to read it from the first image, or type it as WxH with the letter x, for example 1920x1080. Every image is checked against it. Pass --target-size WxH to set it without the question. To read an image’s size on macOS, run sips -g pixelWidth -g pixelHeight <file> or use the file’s Get Info; see Image resolution. The [1920, 1080] default in the ingest.yaml table below applies only to Helm ingests.
Folder layout
- With the CLI, the dataset folder is on your machine (
./visdrone-train). With Helm, it sits on the shared volume (/data/shared/...). - Both folders must have exactly these names and sit side by side.
- Images and annotations pair by file stem:
images/frame01.jpgbelongs toannotations/frame01.xml. Every image needs its XML and every XML needs its image. - All images share one extension (
.jpg,.jpegor.png; default.jpg) and one resolution. The ingestor copies them unchanged and does not resize.
Annotation format (Pascal VOC)
An XML file with no
<object> is accepted with a warning, but its image is skipped: it is not ingested and does not act as a background (negative) example.
ingest.yaml
How records are stored
One record per image, not per box. The stored label of an image is its class histogram (for examplecar:3 motor:1). It must fit in 255 characters; an image with so many distinct classes that it does not fails the ingest. Because records are per image, the dataset summary reports class counts in boxes while the record count is the number of images. Data ids are content hashes, so re-running a failed ingest re-uses its rows.
What the ingestor checks
Plus the checks every ingest runs. A record is copied only when both its image and its XML copy succeed.
Training and test sets
Ingest the training set and the test set separately (--intent train and --intent test). The classes in the test set must be a subset of the classes in the training set. If the test set has a class the training set lacks, it does not appear in the test dataset list when you define the use case. The ingest does not catch this. Each split must also have at least two classes on its own.
Size limit
Onetracebloc data ingest takes at most 1 GiB in total and 500 MiB per file. If your dataset is larger:
- Save the images as JPEG instead of PNG.
- Lower the resolution: resize every image to one smaller size, and scale the box coordinates in the XML files to match.
- Use fewer images.
Sample dataset
The template ships one 1920×1080 aerial traffic frame from the public VisDrone dataset with its VOC annotation (classes such ascar and motor). The ingest.yaml above ingests it with no overrides.
Next steps
- Ingest the data: Prepare Data
- Shared rules for every template: Dataset templates