> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tracebloc.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Define Use Case

> Create and publish an AI use case on tracebloc in two steps, and pick the score that ranks submissions.

This guide walks you through creating and publishing an AI use case on the tracebloc platform. The **Create Use Case** panel has two steps. Make sure you have a [secure environment](https://docs.tracebloc.io/environment-setup/setup-guide.md) running and your training and test data is ingested. Go to the [use cases section](https://ai.tracebloc.io/my-use-cases), click **Create a Use Case** and follow along.

## Step 1: Define your use case

* **Use case title**: Up to 82 characters.
* **What task should contributors work on?**: The task, for example **Object Detection**. It filters the training datasets offered in step 2. See the full list of [supported data types and tasks](https://docs.tracebloc.io/create-use-case/prerequisites.md#supported-data-types-and-tasks). If your use case is not yet supported, reach out to us at [support@tracebloc.io](mailto:support@tracebloc.io).

Click **Next**.

## Step 2: Data & Evaluation

A note at the top of this step says what the panel sets for you: each peer gets a 50 PF compute budget, and you set an end date after publishing.

* **Select your training dataset**: The training datasets you ingested in the [Prepare Data](https://docs.tracebloc.io/create-use-case/prepare-dataset.md) step, for the task you picked.
* **Select your test dataset**: The test and score lists appear after you pick the training dataset. The test list shows only test datasets compatible with it. When the task has classes, the test data's classes must be a subset of the training data's classes. Under it, a list names each of your other test datasets for the task and why it isn't offered, for example "Not offered: class `bicycle` isn't in the training data".
* **Select your score**: The metric that ranks submissions on the leaderboard. The list holds the metrics in the [tables below](#supported-metrics-per-data-type-and-task) for your task. If your metric is not supported, reach out to us at [support@tracebloc.io](mailto:support@tracebloc.io).
* **What does this use case give?**: The use case's **output**.
  * **Report**: the results as a document, with exact numbers ready to cite in a paper.
  * **Weights download**: each peer can download the weights files of their own experiments.
* **Who can join this use case?**: **Only you** or **Peers you invite**. Peers join through your invitation; accepting it is accepting the use case's contract.
* **Purpose**: **Research** (non-commercial) or **Commercial**.
* **Upload EDA File** (optional): A `.ipynb` or `.json` file, up to 25 MB, to help peers understand the data. Explore [template use cases](https://ai.tracebloc.io/explore) for inspiration.

The output, who can join and the purpose together are the use case's **type**. All three are required, and the panel offers only what your data allows. A choice your data rules out stays visible but disabled, with the note "Only what your data allows is offered". What your data allows follows the mark each dataset got when you ingested it: **Patient data**, **Open data** or **Synthetic data**. Every dataset of the use case must allow the type:

| Dataset mark | What the panel offers |
| - | - |
| **Open data** or **Synthetic data** | Every output, who can join and purpose. |
| **Patient data**, or a dataset with no mark | **Research** only. **Weights download** only with **Only you**. |

Click **Create Use Case**. The use case is published at once; there is no draft. The panel sets these values for you:

* **Description**: The title. Edit it on the use case's **Overview** tab with **Edit as admin**.
* **Compute budget**: 50 PF per peer. Change it on the **Admin** tab under **Compute usage** with **Adjust limit**.
* **End date**: None, so the use case runs until you end it. Set one on the **Admin** tab under **Duration**.

To invite peers, open the use case and click the **Invite team members** icon next to the team avatars. For how peers join, see the [join a use case guide](https://docs.tracebloc.io/join-use-case/join-use-case.md).

### Supported metrics per data type and task

The tables below list the metrics you can pick as the score for each task today. Each metric uses either **higher is better** or **lower is better** sort order on the leaderboard. Some tasks report more metrics during training than you can pick as the score; the notes under those tables name them.

#### Image Classification

| Metric | Description | Sort Order |
| - | - | - |
| Accuracy | Proportion of predictions that exactly match the ground truth. Can be misleading on imbalanced datasets. | Higher is better |
| Precision | Measures the proportion of predicted positives that are actually positive. Important when false positives are costly. | Higher is better |
| Recall | Measures the proportion of actual positives correctly identified. Important when missing a positive instance is costly. | Higher is better |
| F1 Score | Balances precision and recall into a single metric. Especially useful for imbalanced datasets. | Higher is better |
| Loss | Quantifies the error between predicted outputs and actual values. A core metric used during training and optimization. | Lower is better |
| Log Loss | Measures how well a model predicts probability estimates for each class. Penalizes overconfident incorrect predictions. | Lower is better |
| AUC-ROC | Measures ability to distinguish between classes across all thresholds, independent of any single decision threshold. | Higher is better |
| AUC-PR | Measures precision-recall balance across different thresholds. Especially useful for highly imbalanced datasets. | Higher is better |
| Top-3 Accuracy | Measures how often the true class label appears among the model's top three predictions. | Higher is better |
| Top-5 Accuracy | Measures how often the true class label appears among the model's top five predictions. | Higher is better |
| Cohen's Kappa | Measures agreement between predicted and ground truth labels while accounting for chance agreement. | Higher is better |
| Matthews Correlation Coefficient (MCC) | Classification quality using all parts of the confusion matrix. Balanced even with imbalanced classes. Ranges from -1 to 1. | Higher is better |
| Quadratic Weighted Kappa (QWK) | Measures agreement between predicted and ground truth labels, penalizing larger disagreements more heavily. | Higher is better |
| Brier Score | Mean squared difference between predicted probabilities and actual outcomes. | Lower is better |

#### Text Classification

| Metric | Description | Sort Order |
| - | - | - |
| Accuracy | Proportion of predictions that exactly match the ground truth. Can be misleading on imbalanced datasets. | Higher is better |
| Precision | Measures the proportion of predicted positives that are actually positive. Important when false positives are costly. | Higher is better |
| Recall | Measures the proportion of actual positives correctly identified. Important when missing a positive instance is costly. | Higher is better |
| F1 Score | Balances precision and recall into a single metric. Especially useful for imbalanced datasets. | Higher is better |
| F1 Weighted | Weights each class's F1 Score by its support (number of true instances). Suitable for imbalanced multi-class classification. | Higher is better |
| Micro F1 | Aggregates true positives, false positives, and false negatives across all classes, treating every prediction equally. | Higher is better |
| Loss | Quantifies the error between predicted outputs and actual values. A core metric used during training and optimization. | Lower is better |
| Log Loss | Measures how well a model predicts probability estimates for each class. Penalizes overconfident incorrect predictions. | Lower is better |
| AUC-ROC | Measures ability to distinguish between classes across all thresholds, independent of any single decision threshold. | Higher is better |
| Hamming Loss | Measures the fraction of labels incorrectly predicted. Commonly used in multi-label classification tasks. | Lower is better |
| Jaccard Score | Measures similarity between predicted and ground truth labels by comparing their intersection to their union. | Higher is better |
| Cohen's Kappa | Measures agreement between predicted and ground truth labels while accounting for chance agreement. | Higher is better |
| Matthews Correlation Coefficient (MCC) | Classification quality using all parts of the confusion matrix. Balanced even with imbalanced classes. Ranges from -1 to 1. | Higher is better |
| Quadratic Weighted Kappa (QWK) | Measures agreement between predicted and ground truth labels, penalizing larger disagreements more heavily. | Higher is better |
| Brier Score | Mean squared difference between predicted probabilities and actual outcomes. | Lower is better |

#### Sentence Pair Classification

| Metric | Description | Sort Order |
| - | - | - |
| Accuracy | Proportion of sentence pairs whose predicted label exactly matches the ground truth. | Higher is better |

Training also reports precision, recall, F1 score, MCC and other classification metrics, but Accuracy is the only score you can pick today.

#### Token Classification

Token-level scores are computed over labelled tokens only: padding, special tokens and sub-word continuations are excluded.

| Metric | Description | Sort Order |
| - | - | - |
| Accuracy | Token accuracy: the fraction of labelled tokens whose predicted tag matches the ground truth tag (for example a BIO tag in named entity recognition). | Higher is better |

Training also reports token accuracy (same value as Accuracy) and precision, recall and F1 score, macro-averaged across tag classes. Accuracy is the only score you can pick today.

#### Masked Language Modeling

| Metric | Description | Sort Order |
| - | - | - |
| Masked Token Accuracy | Fraction of masked positions where the model predicts the original token. Only the masked positions (about 15% of tokens) are scored. 1.0 means every masked token is reconstructed. | Higher is better |
| Perplexity | Exponential of the average cross-entropy loss on the validation data: how "surprised" the model is by the correct tokens. 1.0 is a perfect prediction; a value near the vocabulary size means random guessing. Capped at e^100. | Lower is better |

#### Causal Language Modeling

Causal language modeling is next-token prediction, so its scores are token-level. Each validation sequence is scored at every position that has a training target: the model's most likely next token is compared with the actual next token. Padding is excluded, and so is the prompt part of a `prompt<TAB>completion` row, so only completion tokens count.

| Metric | Description | Sort Order |
| - | - | - |
| Accuracy | Next-token accuracy: the fraction of scored positions where the model's most likely next token equals the actual next token. A completion with one wrong token still scores its other tokens as correct. | Higher is better |
| Exact Match | Sequence accuracy: the fraction of validation sequences whose every scored token is predicted correctly, so the model's answer is exactly the expected answer. A sequence with no scored token is left out. Ranges from 0 to 1. | Higher is better |
| Perplexity | Exponential of the average cross-entropy loss on the validation data: how "surprised" the model is by the actual next tokens. 1.0 is a perfect prediction; a value near the vocabulary size means random guessing. Capped at e^100. | Lower is better |

#### Seq2Seq

Seq2Seq scores are token-level over the target (decoder) sequence, with padding excluded.

| Metric | Description | Sort Order |
| - | - | - |
| Accuracy | Token accuracy: the fraction of target positions where the model's most likely token equals the reference token. | Higher is better |

Training also reports token accuracy (same value as Accuracy) and perplexity, but Accuracy is the only score you can pick today.

#### Embeddings

Embeddings are trained with a contrastive objective on anchor/positive pairs, so the scores measure the embedding space rather than class labels.

| Metric | Description | Sort Order |
| - | - | - |
| Retrieval Accuracy | Fraction of anchors whose nearest candidate (among the other pairs in the batch and any hard negatives) is its own positive. Ranges from 0 to 1. | Higher is better |
| Average Cosine Similarity | Mean cosine similarity between each anchor and its positive across the validation set. Roughly -1 to 1; it climbs toward 1 as matching pairs move closer together. | Higher is better |
| Average Contrastive Loss | Mean in-batch contrastive (InfoNCE) loss over the validation set. | Lower is better |

Perplexity is not a score for embeddings: an embeddings run does not report it.

#### Tabular Classification

| Metric | Description | Sort Order |
| - | - | - |
| Accuracy | Proportion of predictions that exactly match the ground truth. Can be misleading on imbalanced datasets. | Higher is better |
| Precision | Measures the proportion of predicted positives that are actually positive. Important when false positives are costly. | Higher is better |
| Recall | Measures the proportion of actual positives correctly identified. Important when missing a positive instance is costly. | Higher is better |
| F1 Score | Balances precision and recall into a single metric. Especially useful for imbalanced datasets. | Higher is better |
| Loss | Quantifies the error between predicted outputs and actual values. A core metric used during training and optimization. | Lower is better |
| AUC-ROC | Measures ability to distinguish between classes across all thresholds, independent of any single decision threshold. | Higher is better |
| AUC-PR | Measures precision-recall balance across different thresholds. Especially useful for highly imbalanced datasets. | Higher is better |
| Balanced Accuracy | Averages recall across all classes, ensuring each class contributes equally regardless of frequency. | Higher is better |
| Specificity (True Negative Rate) | Measures the proportion of actual negatives correctly identified. Important when false positives must be minimized. | Higher is better |
| NPV (Negative Predictive Value) | Measures the proportion of predicted negatives that are actually negative. Important when confirming absence matters. | Higher is better |
| F-beta Score (beta = 0.5) | Balances precision and recall with more emphasis on precision. Suitable when false positives are more costly. | Higher is better |
| F-beta Score (beta = 2) | Balances precision and recall with more emphasis on recall. Suitable when missing positive instances is more costly. | Higher is better |
| Hamming Loss | Measures the fraction of labels incorrectly predicted. Commonly used in multi-label classification tasks. | Lower is better |
| Jaccard Score | Measures similarity between predicted and ground truth labels by comparing their intersection to their union. | Higher is better |
| Cohen's Kappa | Measures agreement between predicted and ground truth labels while accounting for chance agreement. | Higher is better |
| Matthews Correlation Coefficient (MCC) | Classification quality using all parts of the confusion matrix. Balanced even with imbalanced classes. Ranges from -1 to 1. | Higher is better |
| Quadratic Weighted Kappa (QWK) | Measures agreement between predicted and ground truth labels, penalizing larger disagreements more heavily. | Higher is better |
| Brier Score | Mean squared difference between predicted probabilities and actual outcomes. | Lower is better |
| Gini Coefficient | Measures discriminatory power between positive and negative classes. Closely related to AUC-ROC. | Higher is better |
| Normalized Gini | Scales the Gini Coefficient relative to a perfect model, enabling fair comparison across datasets. Ranges from -1 to 1. | Higher is better |

#### Object Detection

| Metric | Description | Sort Order |
| - | - | - |
| Accuracy | Share of images in which every ground-truth box is found by a prediction of the same class with IoU above 0.2. Extra boxes are not penalized and no confidence threshold applies, so read it as image-level recall. Measured on validation only, not during training. | Higher is better |
| Loss | Quantifies the error between predicted outputs and actual values. A core metric used during training and optimization. | Lower is better |
| Mean Average Precision (mAP) | COCO mAP: average precision averaged over IoU thresholds 0.50 to 0.95. | Higher is better |
| Mean Average Precision @ IoU 0.50 | Evaluates object detection requiring at least 50% overlap between predicted and ground truth bounding boxes. | Higher is better |
| Mean Average Precision @ IoU 0.75 | Stricter variant requiring at least 75% overlap between predicted and ground truth bounding boxes. | Higher is better |
| Intersection over Union (IoU) | Overlap between predicted and ground truth boxes. Pairs with IoU below 0.2 are discarded, and missed objects do not lower it. | Higher is better |
| GIoU (Generalized IoU) | Extends standard IoU by penalizing non-overlapping predictions using the smallest enclosing box. Ranges from -1 to 1. | Higher is better |
| Mean Average Recall @ 1 Detection | Measures how well the model retrieves ground truth objects when only the single top-scoring detection is allowed. | Higher is better |
| Mean Average Recall @ 10 Detections | Measures retrieval of ground truth objects when up to 10 detections per image are allowed. | Higher is better |

For ranking detectors, pick **Mean Average Precision @ IoU 0.50**: unlike Accuracy and IoU, it penalizes both missed objects and false boxes.

#### Semantic Segmentation

| Metric | Description | Sort Order |
| - | - | - |
| Accuracy | Proportion of predictions that exactly match the ground truth. | Higher is better |
| Precision | Measures the proportion of predicted positives that are actually positive. | Higher is better |
| Recall | Measures the proportion of actual positives correctly identified. | Higher is better |
| F1 Score | Balances precision and recall into a single metric. | Higher is better |
| Loss | Quantifies the error between predicted outputs and actual values. | Lower is better |
| Intersection over Union (IoU) | Measures overlap between predicted and ground truth regions. | Higher is better |
| Mean Intersection over Union (mIoU) | Averages IoU across all classes, evaluating how well a model predicts each class region. | Higher is better |
| Frequency Weighted IoU | Weights each class's IoU by its relative frequency in the ground truth, giving more importance to dominant classes. | Higher is better |
| Dice Coefficient | Measures similarity between predicted and ground truth segmentation regions. Especially sensitive to small structures. | Higher is better |
| Pixel Accuracy | Proportion of correctly classified pixels across the entire image. Can be dominated by frequent classes. | Higher is better |
| Mean Pixel Accuracy | Averages pixel accuracy per class, giving equal importance to all classes regardless of frequency. | Higher is better |
| Boundary IoU | Measures how well predicted segmentation boundaries align with ground truth boundaries. Focuses on edge accuracy. | Higher is better |
| Boundary F1 Score | Combines boundary precision and recall to evaluate how accurately predicted boundaries match ground truth edges. | Higher is better |
| Hausdorff Distance | Maximum distance between predicted and ground truth boundaries. Captures the worst-case boundary mismatch. | Lower is better |
| Average Surface Distance (ASD) | Average distance between predicted and ground truth boundary points. Stable measure of overall boundary alignment. | Lower is better |

#### Keypoint Detection

| Metric | Description | Sort Order |
| - | - | - |
| Precision | Measures the proportion of predicted positives that are actually positive. | Higher is better |
| Recall | Measures the proportion of actual positives correctly identified. | Higher is better |
| F1 Score | Balances precision and recall into a single metric. | Higher is better |
| Loss | Quantifies the error between predicted outputs and actual values. | Lower is better |
| Mean Absolute Error (MAE) | Average magnitude of errors between predicted and actual values, expressed in the same units as the target. | Lower is better |
| PCK\@0.05 | Keypoints within a normalized distance threshold of 0.05 from ground truth. Strictest variant. | Higher is better |
| PCK\@0.10 | Keypoints within a normalized distance threshold of 0.10 from ground truth. | Higher is better |
| PCK\@0.20 | Keypoints within a normalized distance threshold of 0.20 from ground truth. | Higher is better |
| PCK\@0.30 | Keypoints within a normalized distance threshold of 0.30 from ground truth. | Higher is better |
| PCK\@0.50 | Keypoints within a normalized distance threshold of 0.50 from ground truth. Most lenient variant. | Higher is better |
| Object Keypoint Similarity (OKS) | Measures similarity between predicted and ground truth keypoints, accounting for scale and localization uncertainty. | Higher is better |
| Mean Per Joint Position Error (MPJPE) | Average Euclidean distance between predicted and ground truth joint positions. Standard for pose estimation. | Lower is better |
| Visibility Accuracy | Measures how correctly the model predicts the visibility status of keypoints, independent of spatial localization. | Higher is better |

#### Tabular Regression

| Metric | Description | Sort Order |
| - | - | - |
| Accuracy | For regression this is the R² value, not an exact-match rate. Pick **R²** instead: an upcoming release stops reporting R² as Accuracy. | Higher is better |
| Loss | Quantifies the error between predicted outputs and actual values. | Lower is better |
| Mean Squared Error (MSE) | Average squared difference between predicted and actual values. Penalizes larger errors more heavily. | Lower is better |
| Root Mean Squared Error (RMSE) | Square root of MSE, expressing prediction error in the same units as the target variable. | Lower is better |
| R² (Coefficient of Determination) | Measures how well a regression model explains variance in the target. A value of 1.0 indicates perfect prediction. | Higher is better |
| Root Mean Squared Logarithmic Error (RMSLE) | Measures error on a logarithmic scale. Useful when target values span several orders of magnitude. | Lower is better |
| Median Absolute Error (Median AE) | Uses the median of absolute errors instead of the mean. Highly robust to outliers. | Lower is better |
| Explained Variance | Measures how well the model captures the variance of the target, independent of systematic bias. Ranges up to 1.0. | Higher is better |
| Mean Bias Error (MBE) | Measures the average bias in predictions. Positive = overestimation, negative = underestimation. The best value is 0, but the leaderboard ranks the highest MBE first, so do not pick it as the score. | Higher ranks first |
| Max Absolute Error | The single largest absolute difference between predicted and actual values. | Lower is better |

#### Time Series Forecasting

By default, when the data has feature columns besides `timestamp` and the target, the engine scales the target to the 0–1 range (MinMaxScaler) before training, so error metrics such as MAE and RMSE are in scaled units, not in your data's units.

| Metric | Description | Sort Order |
| - | - | - |
| Accuracy | Not an exact-match rate on the continuous target. Pick an error metric such as MAE or RMSE as the score. | Higher is better |
| Loss | Quantifies the error between predicted outputs and actual values. | Lower is better |
| Mean Absolute Error (MAE) | Average magnitude of errors between predicted and actual values. | Lower is better |
| Mean Squared Error (MSE) | Average squared difference between predicted and actual values. Penalizes larger errors more heavily. | Lower is better |
| Root Mean Squared Error (RMSE) | Square root of MSE. | Lower is better |
| R² (Coefficient of Determination) | Measures how well a regression model explains variance in the target. A value of 1.0 indicates perfect prediction. | Higher is better |
| Mean Absolute Percentage Error (MAPE) | Average percentage difference between predicted and actual values. Easy to interpret across different scales. | Lower is better |
| Symmetric Mean Absolute Percentage Error (SMAPE) | Symmetric variant of MAPE that reduces issues when actual values are close to zero. | Lower is better |
| Median Absolute Percentage Error (MdAPE) | Uses the median of absolute percentage errors. More robust to outliers. | Lower is better |
| Theil's U (U2 Statistic) | Measures forecasting accuracy relative to a naive benchmark. Values below 1.0 mean the model outperforms the baseline. | Lower is better |
| Max Absolute Error | Captures the single largest absolute difference between predicted and actual values. | Lower is better |

Training also reports direction accuracy (how often the model predicts the direction of change correctly), but you cannot pick it as the score today.

#### Time Series Classification

Time series classification offers the same metrics as [Tabular Classification](#tabular-classification), computed per sequence instead of per row.

#### Time-to-Event Prediction

| Metric | Description | Sort Order |
| - | - | - |
| Concordance Index (C-Index) | Measures how well predicted risk scores agree with the observed ordering of event times. Standard for survival analysis. | Higher is better |

***

## Compute budget

The compute budget caps how much compute, in PetaFLOPs (PF), each team spends on the use case. You set it per peer; a team's budget is that value times the number of peers in the team. One value applies to every team of the use case.

To change it, open the use case's **Admin** tab. **Compute usage** shows the compute all teams have used out of the use case's total. Click **Adjust limit**, enter the allocation per peer in TF (1 PF is 1,000 TF), and click **Save**. The value must be at least 100 TF and fit what your account has left. You can't set it below the most any team has already used per peer. The use case's owners, and members of your account with write permission on use cases, can change it.

When a team uses up its budget, the platform pauses the team's running experiments, and the team can't start new ones. Raising the budget doesn't resume them: the team resumes each paused experiment on its **Training** tab. For what peers see, see [When the compute budget runs out](https://docs.tracebloc.io/join-use-case/start-training.md#when-the-compute-budget-runs-out).

***

## Next Steps

Once your use case is published, invite peers to train models on it.

In the use case view, monitor

* total resource consumption
* daily submits and user activity
* overall leaderboard and submissions

Once models have been submitted, you can [compare them in the leaderboard section](https://docs.tracebloc.io/create-use-case/evaluate-models.md) of a use case.

***

## Need Help?

* Email us at [support@tracebloc.io](mailto:support@tracebloc.io)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.