For some reason the configuration object gets updated at runtime to
resource_configurations = null queues = null extra_trains_conf = "" extra_vm_bash_script = ""

UnevenDolphin73 , task = clearml.Task.get_task(clearml.config.get_remote_task_id()) worked, thanks

Thanks for the hint, I’ll check the paid version, but I’d like first to understand how much efforts it would be to fix the current situation by myself 🙂

I have a custom way of reading the config file

Ok AgitatedDove14 SuccessfulKoala55 I made some progress in my investigation:
I can exactly pinpoint the change that introduced the bug, it is the one changing the endpoint "events.get_task_log", min_version="2.9"
In the firefox console > Network, I can edit an events.get_task_log and change the URL from …/api/v2.9/events.get_task_log to …/api/v2.8/events.get_task_log (to use the endpoint "events.get_task_log", min_version="1.7" ) and then all the logs are ...

So it can be that when restarting the docker-compose, it used another volume, hence the loss of data

The task object

and this works. However, without the trick from UnevenDolphin73 , the following won’t work (return None):
if __name__ == "__main__": task = Task.current_task() task.connect(config) run() from clearml import Task Task.init()

erf, I have the same problem with ProxyDictPreWrite 😄 What is the use case of this one ?

When installed with http://get.docker.com , it works

might be worth documenting 😄

The rest of the configuration is set with env variables

I am happy if I can be of any help to fix that 😄

Ok, I won't have time to venture to check the different database components, the first option (shuting down the server) sounds like the easiest option for me, I would then run manually the script once a month or so

in my clearml.conf, I only have:
sdk.aws.s3.region = eu-central-1 sdk.aws.s3.use_credentials_chain = true agent.package_manager.pip_version = "==20.2.3"

I see 3 agents in the "Workers" tab

Yes, I will try 🙂

Some context: I am trying to log an HTML file and I would like it to be easily accessible for preview

continue_last_task is almost what I want, the only problem with it is that it will start the task even if the task is completed

Yes this is correct. I am trying to create a minimal reproducable example

As to why: This is part of the piping that I described in a previous message: Task B requires an artifact from task A, so I pass the name of the artifact as a parameter of task B, so that B knows what artifact from A it should retrieve

Thanks! Unfortunately still not working, here is the log file:

I would like to try it to see if it solves some dependencies not found eventhough they are installed when using --system-site-packages

sure, will be happy to debug that 🙂

I also don't understand what you mean by unless the domain is different... The same way ssh keys are global, I would have expected the git creds to be used for any git operation

Relevant issue in Elasticsearch forums: https://discuss.elastic.co/t/elasticsearch-5-6-license-renewal/206420

I killed both trains-agent and restarted one to have a clean start. This way it correctly spin up docker containers for services tasks. So probably the bug comes when a bug occurs while setting up a task, it cannot go back to the main task. I would need to do some tests to validate that hypothesis though

I will try to isolate the bug, if I can, I will open an issue in trains-agent 🙂

I tested by installing flask in the default env -> which was installed in the ~/.local/lib/python3.6/site-packages folder. Then I created a venv with flag --system-site-packages . I activated the venv and flask was indeed available

I finally found a workaround using cache, will detail the solution in the issue 👍