Skip to main content

Training

Can I initiate a training using a local environment?​

We support trainings via both Google Colab and local environments using jupyter Notebook. In order to set up your local environment have a look at out Jupyter Notebook training guide.

In Step 3. of the Training Guide, I get the error that Model and Dataset parameters dont match?​

If you get an error message like this:

Assignment failed!
DataSet 'dataset ID' expected parameters :
classes : 5, shape: 224
'model name' parameters :
classes : 6, shape: 224
Please change model parameters to match expected dataset parameters.

when trying to link your model to a dataset (step 3. in the traceblocTrainingGuide.ipynb), it means that the number of classes in the dataset you want to train your model on does not match the number of classes you specified in your model. To fix this, you should update the number of classes in your model to match the number needed for your dataset. If you wonder where you need to change the number of (output-)classes, simply refer to the Model Structure Guide.

How to navigate in the Anaconda Prompt?​

InputDescription
dirDirectory overview
cd + "folder"Move down in the file structure
cd..Move up in the file structure
conda activate + "Environment"Activates your environment created in Anaconda
jupyter-notebookOpens the notebook

What are pre-trained weights and how can I use them in my model?​

Pre-trained weights is short for previously trained weights. Your weights file should contain only the model weights, not the entire model object. Below are examples of how you can create such a file for both PyTorch and TensorFlow models:

For PyTorch models: Using PyTorch, the weights file for your model needs to have the .pth extension. To create your own pre-trained weights, use:

model = MyModel() # load model class here
torch_weight_file = ("<filename>_weights.pth")

torch.save(model.state_dict(), torch_weight_file)

For TensorFlow models: Using TensorFlow, the weights file for your model needs to have the .pkl extension. To create your own pre-trained weights, use:

w = model.get_weights() 

with open('<filename>_weights.pkl', 'wb') as output: 
    pickle.dump(w, output)

By following these examples, you'll be able to create a .pkl/.pth file that is compatible with our federated learning infrastructure.

Can I utilize pre-trained weights from models like ImageNet directly?​

No, our system only accommodates models trained with the same number of classes as our dataset. Therefore, pre-trained weights from models like ImageNet won't function on our platform.

Which libraries can I use to create my models? (Are torchvision or TIMM supported?)​

Our federated learning infrastructure supports model creation specifically with PyTorch and TensorFlow. You have the option to use either a single file format or multiple file formats for these two frameworks, as detailed in our documentation.

Unfortunately, third-party libraries like TIMM are not supported. If you're interested in using a model architecture from TIMM, you'll need to reimplement it using PyTorch or TensorFlow. Be sure to consult our model submission guidelines to ensure that your reimplemented model is compatible with our system.

What kind of training environment does tracebloc use for experiments?​

Tracebloc employs a training environment that utilizes multi-core CPUs for each experiment. In simpler terms, this means that when conducting experiments on Tracebloc, the system uses the processing power of multiple CPU cores to facilitate the training process. This helps in efficiently running and managing experiments, making them faster and more effective. For general users, this means a smoother experience on the platform, while data scientists can appreciate the optimized environment for their experiments.

Can I replicate the Tracebloc environment on my local system?​

Unfortunately, it is currently not feasible to recreate the Tracebloc platform's environment on your local system. The unique configuration and infrastructure of Tracebloc make it impractical for local replication.

What types of model file extensions are supported?​

TensorFlow and PyTorch models with a .py or .zip extension can be uploaded on tracebloc_package.

Can I use multiple subfolders in my model .zip file?​

When uploading the model with a .zip file extension, only the model files should be contained, with no subfolders. So multiple files within one folder with the main file having the required structure is possible.

What operations are not allowed in the model file?​

The following operations are not allowed in the model file:

  • file operations such as open, read, write
  • imports other than tensorflow, keras, torch and torchvision
  • executible operations
  • Using subdirectories
  • subprocess operations
  • terminal operations
On This Page