Sorry both of you, my problem was actually lying somewhere else (both buckets are in the same region) - thanks for you time!

There is no way to filter on long types? I can’t believe it

I am not using hydra, I am reading the conf with:
config_dict = read_yaml(conf_yaml_path) config = OmegaConf.create(task.connect_configuration(config_dict))

So it looks like it tries to register a batch of 500 documents

Ha nice, good one! Thanks!

continue_last_task is almost what I want, the only problem with it is that it will start the task even if the task is completed

AppetizingMouse58 btw I had to delete the old logs index before creating the alias, otherwise ES won’t let me create an alias with the same name as an existing index

Sorry, what I meant is that it is not documented anywhere that the agent should run in docker mode, hence my confusion

SmugDolphin23 Actually adding agent.python_binary didn't work, it was not read by the clearml agent (in the logs dumped by the agent, agent.python_binary = (no value)

Should I open an issue in github clearml-agent repo?

Hi SmugDolphin23 thanks for the input! Will try now but that seems hacky: to have it working I have to specify python3.8 two times:
one in the agent config file (agent.default_python is already python3.8, but seems to be ignored) + make sure it is available (using python:3.8 docker image)Is there a way to prevent this redundancy? Ie. If I want to change the python version, I can control it from a single place?

So either I specify in the clearml-agent agent.python_binary: python3.8 as you suggested, or I enforce the task locally to run with python3.8 using task.data.script.binary

And since I ran the task locally with python3.9, it used that version in the docker container

I think my problem is that I am launching an experiment with python3.9 and I expect it to run in the agent with python3.8. The inconsistency is from my side, I should fix it and create the task with python3.8 with:
task.data.script.binary = "python3.8" task._update_script(convert_task.data.script)Or use python:3.9 when starting the agent

Not really because this is difficult to control: I use the AWS autoscaler with ubuntu AMI and when an instance is created, packages are updated, and I don't know which python version I get, + changing the python version of the OS is not really recommended

I have a mental model of the clearml-agent as a module to spin my code somewhere, and the python version running my code should not depend of the python version running the clearml-agent (especially for experiments running in containers)

DeterminedCrab71 This is the behaviour of holding shift while selecting in Gmail, if ClearML could reproduce this, that would be perfect!

no, simple GET

Hoo I found:
user@trains-agent-1: ps -ax 5199 ? Sl 29:25 python3 -m trains_agent --config-file ~/trains.conf daemon --queue default --log-level DEBUG --detached 6096 ? Sl 30:04 python3 -m trains_agent --config-file ~/trains.conf daemon --queue default --log-level DEBUG --detached

I managed to do it by using logger.report_scalar, thanks!

yes

in the UI the value is correct one (not empty, a string)

as it's also based on pytorch-ignite!

I am not sure to understand, what is the link with pytorch-ignite?

We're in the brainstorming phase of what are the best approaches to integrate, we might pick your brain later on

Awesome, I'd be happy to help!

Ok, so after updating to trains==0.16.2rc0, my problem is different: when I clone a task, update its script and enqueue it, it does not have any Hyper-parameters/argv section in the UI

Now I am trying to restart the cluster with docker-compose and specifying the last volume, how can I do that?

These images are actually stored there and I can access them via the url shared above (the one written in the pop up message saying that these files could not be deleted)

AgitatedDove14 In my case I'd rather have it under the "Artifacts" tab because it is a big json file

I sneaked a PR > https://github.com/allegroai/clearml/pull/319

I tried removing type=str but I got same problem 😕

ubuntu18.04 is actually 64Mo, I can live with that 😛