Hyperparameters

TensorFlow is deprecated. New TensorFlow model uploads are no longer accepted — use PyTorch for new models. The TensorFlow-specific optimizers, learning-rate schedules, loss functions, layer freezing, and augmentation flags documented below apply only to existing TensorFlow experiments, which remain readable.

Training Parameters

All parameters are set through the training_plan after linking your model with the dataset.

training_plan = user.link_model_dataset(dataset_id='Dataset ID')

To see all current parameter settings, run training_plan.get_training_plan(). To run consecutive experiments, overwrite parameters and re-start training with training_plan.start().

You can refer to the TensorFlow Documentation for more information on TensorFlow augmentation parameters and the PyTorch Documentation for more information on PyTorch augmentation parameters.

Basic training configuration parameters that control the fundamental aspects of your training process.

Parameter	Description	Default	Example
Epochs	Number of complete passes through the training data in each cycle. Integer from 1 to 1000	10	`training_plan.epochs(100)`
Cycles	Number of federated rounds: each secure environment trains for `epochs`, then the platform averages the models (see Federated aggregation). Integer from 1 to 100	1	`training_plan.cycles(10)`
Batch Size	Number of samples processed at one time. Set automatically from the `batch_size` variable in your model file	Datatype dependent 16 in most cases	Set via `batch_size = 16` in your model `.py` file
Validation Split	Fraction of the training data used for validation. A float from the default up to 0.5	Number of classes divided by the number of samples in the smallest secure environment, rounded to 2 decimals; 0.1 if that rounds to 0; at most 0.5	`training_plan.validation_split(0.2)`
Objective	Free-text description of what the experiment tries to do, stored with the experiment	Empty	`training_plan.objective('Compare a ResNet-18 baseline')`

The default validation split is often very small: 3 classes and 100 images give 0.03. Set it explicitly, for example training_plan.validation_split(0.15), and check it with training_plan.get_training_plan().

To try another batch size, you can copy a model file and change batch_size in the copy. For text models (causal language modeling included), the copy also needs its tokenizer. The SDK looks for it in this order: the tokenizer= argument to user.upload_model(...), then a <model-file-name>_tokenizer.json next to the model file, then a plain tokenizer.json next to it. Without one, the upload fails. So when you copy or rename a model-zoo file, copy and rename its _tokenizer.json too: distilgpt2.py and distilgpt2_tokenizer.json become distilgpt2_bs4.py and distilgpt2_bs4_tokenizer.json. The upload error also mentions tokenizer_id, but don’t declare one: the upload rejects hub references (see Model optimization).

Core Hyperparameters

1. Optimizer

Controls how the model’s parameters are updated during training. Supports different optimizers for TensorFlow and PyTorch. The default optimizer is SGD, except for text tasks: text classification, sentence pair classification, token classification, masked language modeling, causal language modeling, seq2seq and embeddings default to AdamW (SDK 1.2.38 and later). Supported Optimizers:

TensorFlow: adam, rmsprop, sgd, adadelta, adagrad, adamax, nadam, ftrl
PyTorch: adam, adamw, rmsprop, sgd, adadelta, adagrad, adamax

training_plan.optimizer('rmsprop')

2. Learning Rate

Controls the rate at which the model learns. Supports three different types:

Type	Description	Framework Support	Example
Constant	Fixed learning rate throughout training	TensorFlow, PyTorch	`training_plan.learning_rate({'type': 'constant', 'value': 0.002})`
Adaptive	Learning rate that changes based on schedule	TensorFlow	`training_plan.learning_rate({'type': 'adaptive', 'value': {'decay_rate': 0.9, 'decay_steps': 100, 'initial_learning_rate': 0.1, 'scheduler': 'ExponentialDecay'}})`
Custom	User-defined learning rate function	TensorFlow	`training_plan.learning_rate({'type': 'custom', 'value': {'name': custom_function, 'epoch': 5}})`

Default: {'type': 'constant', 'value': 0.001}. Text tasks default to {'type': 'constant', 'value': 5e-05} (SDK 1.2.38 and later). The optimizer and learning rate defaults are a pair. If you set only one of them, the other keeps its default: on a text task, training_plan.learning_rate({'type': 'constant', 'value': 0.001}) alone trains AdamW at 0.001, which can be too high for a pretrained transformer, and training_plan.optimizer('sgd') alone trains SGD at 5e-05, which learns very slowly. Set both when you change either.

Custom for TensorFlow: Define a custom learning rate function, then pass it via learning_rate() with type: 'custom':

def custom_LearningRate_scheduler(epoch):
        if epoch < 5:
            return 0.01
        else:
            return 0.01 * tf.math.exp(0.1 * (10 - epoch))
training_plan.learning_rate({'type': 'custom', 'value': {'name': custom_LearningRate_scheduler, 'epoch': 5}})

3. Loss Function

Defines how the model measures prediction errors. Pass a standard loss by name, or your own custom loss. The allowed standard losses and the default depend on the task:

Task	Allowed standard losses	Default
Image classification, object detection, keypoint detection, tabular regression (PyTorch)	`crossentropy`, `mse`, `l1`	`mse`
Tabular classification (PyTorch)	`mse`, `crossentropy`, `binarycrossentropy`	`mse`
Time series forecasting	`mse`, `l1`	`mse`
Text tasks, time series classification, semantic segmentation	`crossentropy`	`crossentropy`
Time-to-event prediction	`coxph`	`coxph`
scikit-learn models	`mse`, `binarycrossentropy`	`mse`

For object detection, the experiment’s settings show mse, but the platform does not use it: torchvision detectors compute their own losses, and yolo models always use the loss.py shipped with the model.

# Standard loss function
training_plan.loss_function({'type': 'standard', 'value': 'crossentropy'})

On PyTorch you can also pass a custom loss: a top-level function that takes (outputs, targets), or the path to a loss.py that defines Custom_loss. The SDK runs a short local training check with it before you start. scikit-learn models accept standard losses only.

def weighted_mse(outputs, targets):
    return (2.0 * (outputs - targets) ** 2).mean()

training_plan.loss_function({'type': 'custom', 'value': weighted_mse})

An nn.Module class works too, but only when you import it from a .py file: the SDK reads the loss’s source code, and it cannot read the source of a class defined in a notebook cell. Alternatively, put the class in a loss.py as Custom_loss and pass that file’s path. TensorFlow experiments (deprecated) used binary_crossentropy, categorical_crossentropy and mse.

Training Control

Layer Freezing

Specify which layers should remain unchanged during training (TensorFlow only):

training_plan.layers_freeze(['conv1','fc1'])

Layer freezing is currently supported for TensorFlow models only. PyTorch models will receive a “not supported” message.

Callbacks

Control training behavior with various callbacks:

Callback	Purpose	Parameters	Example
Early Stopping	Stop training when metric stops improving	metric, patience	`training_plan.early_stop_callback('loss', 10)`
Reduce LR	Reduce learning rate when metric plateaus	metric, factor, patience, min_delta (float, default `1e-4`)	`training_plan.reduce_lr_callback('loss', 0.1, 10, 0.0001)`
Model Checkpoint	Save model weights at specific intervals	metric, save_best_only	`training_plan.model_checkpoint_callback('val_loss', True)`
Terminate on NaN	Stop training if validation loss becomes NaN	None	`training_plan.terminate_on_nan_callback()`

The metric must be one of accuracy, loss, val_accuracy or val_loss. Object detection does not measure training accuracy, so monitor a loss for object detection models.

Federated aggregation

At the end of each cycle, the platform combines the models trained in each secure environment. The default strategy is fedavg. Choose another one, with its optional hyperparameters, through aggregation_strategy:

training_plan.aggregation_strategy('fedprox', proximal_mu=0.01)

Strategy	Hyperparameters
`fedavg` (default), `fedmedian`	None
`fedtrimmedavg`	`beta`
`krum`	`num_byzantine`, `num_clients_to_keep`
`fedprox`	`proximal_mu`
`fedadam`, `fedyogi`, `fedadagrad`	`eta`, `beta_1`, `beta_2`, `tau`

Hyperparameters you leave out take the strategy’s defaults. An unknown strategy or hyperparameter name is refused.

Preprocessing (Tabular & Time Series)

Configure the platform’s built-in preprocessing per experiment. Available for tabular classification, tabular regression, time-to-event, and time-series classification; time-series forecasting supports the imputation pair (handle_missing_values / imputation_strategy). The calls below are illustrative alternatives, not a recommended combination — for example, handle_missing_values(False) and imputation_strategy('iterative') contradict each other (the first disables the step the second configures).

# Turn built-in imputation off so your model owns missing-data handling
training_plan.handle_missing_values(False)

# Or pick the imputation strategy: median (default), mean, most_frequent,
# constant, iterative (MICE), or 'none'
training_plan.imputation_strategy('iterative')

# Categorical encoding: label (default) or onehot
training_plan.encoding_strategy('onehot')

# Numeric feature scaling on/off + scaler choice (tabular & time-to-event)
training_plan.normalize_features(True)
training_plan.scaler('RobustScaler')

# Time-series classification only: opt-in missingness-indicator channels —
# a 0/1 <column>__missing feature per numeric column carrying the raw NaN
# mask. The model's input width grows, identically in every secure environment.
training_plan.missingness_indicators(True)

When to disable imputation: if your model embeds its own imputer (an sklearn Pipeline) or handles NaNs natively (XGBoost, LightGBM, HistGradientBoosting, CatBoost, EBM), set handle_missing_values(False) — otherwise the built-in step fills every gap first and your model never sees a raw NaN.

knn imputation and the QuantileTransformer scaler are deliberately unavailable: their fitted state memorizes raw training data, which would leave the secure environment inside the preprocessing artifact. iterative (MICE) persists only learned coefficients — the same privacy class as model weights.

Defaults reproduce the historical platform behaviour (median imputation on, label encoding, z-score feature scaling on), so an experiment that never touches these knobs is unchanged.

Data Augmentation for Image Data

Enhance your dataset with real-time image transformations. All parameters support both TensorFlow and PyTorch unless noted otherwise.

Geometric Transformations

Parameter	Description	Default	Framework Support	Example
rotation_range	Degree range for random rotations. For example, if the rotation_range is set to 2, images will be rotated by a random degree between -2 and 2	0	TensorFlow, PyTorch	`training_plan.rotation_range(2)`
width_shift_range	Range for horizontal shifts, as a fraction of the image width from 0 to 1. TensorFlow experiments also accept a pixel count (a value of 1 or more); PyTorch does not	0.0	TensorFlow, PyTorch	`training_plan.width_shift_range(0.1)`
height_shift_range	Range for vertical shifts, as a fraction of the image height. Works like width_shift_range	0.0	TensorFlow, PyTorch	`training_plan.height_shift_range(0.1)`
shear_range	Shear intensity in degrees in counter-clockwise direction (float)	0.0	TensorFlow, PyTorch	`training_plan.shear_range(10.0)`
zoom_range	Range for zooming, a float from 0 to 1. TensorFlow experiments also accept a `[lower, upper]` list; on PyTorch a list fails the experiment	0.0	TensorFlow, PyTorch (float only)	`training_plan.zoom_range(0.1)`
horizontal_flip	Randomly flip images horizontally	False	TensorFlow only; refused on PyTorch	`training_plan.horizontal_flip(True)`
vertical_flip	Randomly flip images vertically	False	TensorFlow only; refused on PyTorch	`training_plan.vertical_flip(True)`

On PyTorch, the platform accepts these ranges: rotation 0 to 180 degrees, width and height shift 0 to 1 (fraction of the image), shear 0 to 45 degrees, zoom 0 to 1, channel shift 0 to 255 and brightness 0 to 1.

Color and Intensity Transformations

Parameter	Description	Default	Framework Support	Example
brightness_range	Range for brightness shifts (float from 0 to 1)	None	TensorFlow, PyTorch	`training_plan.brightness_range(0.2)`
channel_shift_range	Range for random channel shifts	0	TensorFlow, PyTorch*	`training_plan.channel_shift_range(0.4)`
rescale	Rescaling factor for pixel values (float)	None	TensorFlow, PyTorch	`training_plan.rescale(1.0/255.0)`

*PyTorch: Only supported for RGB images

Normalization

Parameter	Description	Default	Framework Support	Example
samplewise_center	Center each image by subtracting mean	False	TensorFlow only; refused on PyTorch	`training_plan.samplewise_center(True)`
samplewise_std_normalization	Standardize each image by subtracting the mean and dividing by the standard deviation of pixel values. Calculated individually	False	TensorFlow only; refused on PyTorch	`training_plan.samplewise_std_normalization(True)`

On PyTorch, horizontal_flip, vertical_flip, samplewise_center and samplewise_std_normalization are refused: the call records an error and training_plan.start() refuses to run while it stands.

Other Parameters

Parameter	Description	Default	Framework Support	Example
fill_mode	Method for filling points outside boundaries. Supported for TensorFlow: “constant”, “nearest”, “reflect”, “wrap”. For PyTorch: “constant”, “edge”, “symmetric”, “reflect”, “wrap”.	‘constant’	TensorFlow, PyTorch	`training_plan.fill_mode("nearest")`
cval	Fill value for points outside the image boundaries when fill_mode=“constant”	0.0	TensorFlow, PyTorch	`training_plan.cval(0.3)`
shuffle	Whether to shuffle the data	True	TensorFlow, PyTorch	`training_plan.shuffle(True)`

LLM Parameters (LoRA for Text Tasks)

LoRA (Low-Rank Adaptation) is available for every text task in PyTorch: text classification, sentence pair classification, token classification, masked language modeling, causal language modeling, seq2seq and embeddings. Other tasks print Operation not allowed and keep standard training. LoRA is off by default. When you enable it, the platform adds low-rank adapters to every nn.Linear (and nn.MultiheadAttention) layer of your model and trains only the adapters instead of every weight. Use it to fine-tune a larger pretrained model, for example a GPT-2 style model in causal language modeling, with less compute.

# Enable LoRA first
training_plan.enable_lora(True)

# Configure LoRA parameters (positional arguments: lora_r, lora_alpha, lora_dropout, q_lora)
training_plan.set_lora_parameters(16, 32, 0.05, False)

set_lora_parameters only works after enable_lora(True). It validates the values, then runs a short local training check of your model with the adapters before you start the experiment. If you only call enable_lora(True), the defaults below apply. From SDK 1.2.38 the defaults are rank 8 and alpha 16; earlier versions used 256 and 512.

Parameter	Description	Type	Default	Example
lora_r	Rank for the LoRA layer	Positive integer, max. 1024	8	`lora_r=16`
lora_alpha	Scaling factor alpha for LoRA	Positive integer	16	`lora_alpha=32`
lora_dropout	Dropout rate for LoRA layers	Float (0-1)	0.05	`lora_dropout=0.05`
q_lora	Enable or disable Q LoRA	Boolean	False	`q_lora=False`

Note: LLM parameters are supported only for PyTorch.

Dataset Parameters (Optional)

Customize your dataset configuration and preprocessing options:

Dataset Customization

Parameter	Description	Example
training_classes	Customize dataset by specifying samples per class	`training_plan.training_classes({'car': 30, 'person': 30})`
data_type	Image format: ‘rgb’ or ‘grayscale’	`training_plan.data_type('rgb')`
seed	Random seed, a non-negative integer. Default 0. Today only the self-supervised text tasks (masked and causal language modeling, seq2seq, embeddings) read it, for their train/validation split and the masking; other tasks split with a fixed seed	`training_plan.seed(42)`

Next Steps

Submit your best model: Evaluate Model

Need Help?

For more info about available functions and methods, call the help function in your notebook:

user.help()

Email us at [email protected]

Overview

Create a Use Case

Join a Use Case

Environment Setup

Tools & Help

Releases

Training Parameters

Core Hyperparameters

1. Optimizer

2. Learning Rate

3. Loss Function

Training Control

Layer Freezing

Callbacks

Federated aggregation

Preprocessing (Tabular & Time Series)

Data Augmentation for Image Data

Geometric Transformations

Color and Intensity Transformations

Normalization

Other Parameters

LLM Parameters (LoRA for Text Tasks)

Dataset Parameters (Optional)

Dataset Customization

Next Steps

Need Help?

​Training Parameters

​Core Hyperparameters

​1. Optimizer

​2. Learning Rate

​3. Loss Function

​Training Control

​Layer Freezing

​Callbacks

​Federated aggregation

​Preprocessing (Tabular & Time Series)

​Data Augmentation for Image Data

​Geometric Transformations

​Color and Intensity Transformations

​Normalization

​Other Parameters

​LLM Parameters (LoRA for Text Tasks)

​Dataset Parameters (Optional)

​Dataset Customization

​Next Steps

​Need Help?

Training Parameters

Core Hyperparameters

1. Optimizer

2. Learning Rate

3. Loss Function

Training Control

Layer Freezing

Callbacks

Federated aggregation

Preprocessing (Tabular & Time Series)

Data Augmentation for Image Data

Geometric Transformations

Color and Intensity Transformations

Normalization

Other Parameters

LLM Parameters (LoRA for Text Tasks)

Dataset Parameters (Optional)

Dataset Customization

Next Steps

Need Help?