Hi VexedCat68
You can use argparse and all the parameters will be log automagically to the hyperparameters section like in https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py#L56 example, or just connecting any dict like in https://github.com/allegroai/clearml/blob/master/examples/frameworks/ignite/cifar_ignite.py#L23 example
Quick follow up question. Once I parse args, should they be directly available for i even enque the project for the first time or will i be able to access hyperparameters after running it once?
VexedCat68 there are two possibilities:
This is a new experiment, and you've just run the code locally - an experiment entry will be created in the UI, and the values you used for the args will be in the hyperparams section The experiment already exists in the UI (as a result of running it locally, or even a previous experiment which was already executed remotely) - you can clone it, and than edit the hyper parameters values in the UI. Then, you send this experiment to be executed remotely - when it will run, the values you've just updated in the UI will be the values returned by the argparse parse_args() method
In any case, when sending something to be executed remotely , you can change the values as you see fit. On the other hand, when running locally , it will always use the values explicitly passed in the local run command
I'm using clear-ml agent right now. I just upload the task inside a project. I've used arg parse as well however as of yet, I have not been able find writable hyperparameters in the UI. Is there any tutorial video you can recommend that deals with this or something? I was following https://www.youtube.com/watch?v=Y5tPfUm9Ghg&t=1100s this one on youtube but I can't seem to recreate his steps as he sifts through his code.
Also my execution just completed and as of yet, I can only see the hyperparameters as a report. not in a configurable form. I've just started with ClearML and am having these issues.
from sklearn.datasets import load_iris
import tensorflow as tf
import numpy as np
from clearml import Task, Logger
import argparse
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--epochs', metavar='N', default=64, type=int)
args = parser.parse_args()
parsed_args = vars(args)
task = Task.init(project_name="My Workshop Examples", task_name="scikit-learn joblib example")
iris = load_iris()
data = iris.data
target = iris.target
labels = np.unique(target)
epochs = parsed_args['epochs']
task.connect(args)
model = tf.keras.models.Sequential([
tf.keras.layers.Dense(64, input_dim=4, activation='relu'),
tf.keras.layers.Dense(len(labels), activation='softmax')
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(data, target, epochs=epochs)
print(model.evaluate(data, target))
if name == "main":
main()
Here's the script I'm testing with.
Here's the script I run it with.
clearml-task --project ClearML-Learn --name EpochsConnectReturns --requirements requirements.txt --script demo.py
Try to clone the task (right click on the task and choose “clone”) and you will get a new task in draft mode, that you can configure ( https://clear.ml/docs/latest/docs/getting_started/mlops/mlops_first_steps#clone-an-experiment )
I checked and it seems when i an example from git, it works as it should. but when I try to run my own script, the draft is in read only mode.
My draft is View Only but the cloned toy task one is in normal Draft mode.
You can only edit the cloned copy, not the experiment created from the local run
Basically when I have to re run the experiment with different hyperparameters, I should clone the previous experiment and change the hyperparameters then before putting it in the queue?
It seems that is the case. Thank you for all your help guys.
Basically when I have to re run the experiment with different hyperparameters, I should clone the previous experiment and change the hyperparameters then before putting it in the queue?
You can also reset it instead of cloning