One more related question (I hope there's a similar solution), when I log images, they appear in the UI with http://<my-ip> so they are inaccessible (they should be translated to None . Is there any path_substitution variant for this scenario in the config? I can't seem to find it in the docs. Thanks!

Got it. Is there any way to skip a point at some iteration? If I just don't report it at iteration t I'll get interpolation from t-1 to t+1.

    model_checkpoint = ModelCheckpoint(
        "checkpoint",
        n_saved=2,
        filename_prefix="best",
        score_function=score_function,
        score_name="accuracy",
        global_step_transform=global_step_from_engine(trainer),
    )

    # Save the model after every epoch of val_evaluator is completed
    val_evaluator.add_event_handler(
        Events.COMPLETED, model_checkpoint, {"model": model}
    )

No worries, sorry for pinging, was just making sure you (or anyone else who might help) doesn't miss it 🙂
I use Task.add_requirements("requirements.txt") right before the Task.init.
In main, I parse arguments command-line, add_requirements, initialize Task and call execute_remotely. After that it's all pretty much the usual workflow. Initialize the model, setup dataloaders, optimizer and run the training. I'm using pytorch-ignite and have model checkpoint made on validation evaluator COMPL...

The issue was .ssh wasn't propagated so the git repository couldn't be cloned.

clearml-1.13.1

Task.add_requirements("requirements.txt")
task = Task.init(project_name="My project", task_name="My task")
task.execute_remotely(queue_name="default")
...

Kind ping on this thread, thanks! 🙂

I just added the secrets/keys to docker-compose.yml and restarted everything but no change.

I know about clearml.conf but wanted to avoid ssh-ing through 50 instances to edit it.
task.set_base_docker does the job, but docker_arguments doesn't propagate if I leave docker_image as None (it just uses both image and arguments from clearml.conf of the agent). If I explicitly state docker_image and docker_arguments in task.set_base_docker it works fine.

I have a dataset of ~24GB and I've tried multiple times uploading it with the sync function.

• The cache doesn't work, it attempts to download the dataset every time.
• It "misses" some files somehow. So once the job runs it fails due to missing files.
• I've ran verify afterwards (from the machine I used to upload the data) and it says it's all good. However, once I inspect the zip files on the server (look for the files in the specific zip the state json says they're in) the files are indee...

Makes sense, thanks!

@<1523701087100473344:profile|SuccessfulKoala55> Kind reminder again, thanks and sorry!

Not ClearML employee (just a recent user), but maybe this will help? None

@<1523701087100473344:profile|SuccessfulKoala55> kind reminder not to miss this when you catch time, thanks!

@<1714813627506102272:profile|CheekyDolphin49> You should probably use 'General/coupling' and 'General/rep'

Weird. When I spawn agent with sudo I get this behaviour. Without sudo everything works fine

I'll try to reproduce it and will get back at you. The HPO task (parent of this task) was deleted indeed but that shouldn't matter? One of the models was deleted but the other one wasn't.

@<1523701205467926528:profile|AgitatedDove14> Any ideas on this issue? Thanks!

Neither, metric is a number you report through the Logger: