RoughTiger69 can you share the python version and the logs?

GreasyPenguin14 I was able to reproduce it, will keep you update when a fix is ready

TL;DR - no 😕

when using task.execute_remotely() , as you wrote, it won’t run on you local environment but with the clearml-agent. When running on the clearml-agent, you task should be stable for running and create a model - for optimize, pipeline and more, but not for debugging.

When running development mode, on your local machine (w/ or w/o remote interpreter), you can debug you code. There are some differences between the two runs - running from a clearml-agent won’t re u...

Was the agent task cloned from the one works as expected (the dev task)?

Hi FlatStarfish45 , can you attach the clearml agent logs?

and same log?

Hi OutrageousSheep60 , can you try with auto_pickle=True when uploading the artifact?

https://clearml.slack.com/archives/CTK20V944/p1642523607145500?thread_ts=1642522392.144800&cid=CTK20V944
You are adding this before your Task.init call?

Hi JitteryCoyote63 , what commit and branch do you see in the UI?

max_spin_up_time_min - the maximum time for an instance to spin up
max_idle_time_min - the maximum time for an instance to stay up with worker on it (the time the ec2 instance is finished running a task with the agent and still up with an worker running on it listening to the queue)

When you run it locally with auto_connect_frameworks={"matplotlib": False} , did it send the matplotlib outputs?

Hi GiganticTurtle0 ,

Not directly with the sdk but you can use the APIClient:
` from clearml.backend_api.session.client import APIClient

api_client = APIClient()
api_client.queues.create("your queue name") `

Hi RotundSquirrel78 ,

Can you check if there are some errors in the UI console? Which server are you using? https://app.clear.ml/ ?

For the agent?

https://allegro.ai/docs/faq/faq/#remote-debugging-trains-pycharm-plugin this is for not getting the git information in the task, but I understand your task logged the changes and the git information.

Are you looking for ssh interpreter in your PyCharm?

Hi TenderCoyote78 ,

You can get it with the APIClient

from clearml.backend_api.session.client import APIClient api_client = APIClient() tasks_in_queue = api_client.queues.get_by_id(<queue_id>).entries

Hi HelpfulHare30

1). Great

🙂

2. As you can see from my screenshot,  Data Processing task is created but I don’t see Datasets tab as I see in 

This dataset tab is part of the enterprise solution, clearml-data is part of the open source ClearML

3. I see. So need to specify with every cli command/SDK method call

How do you use it with the SDK?

👍

So nvidia-container-toolkit and systemctl restart dockerd fixed it?

still i need do this?

dataset.upload() dataset.finalize()if you want to finalize the dataset, yes

if we have uploaded data clearml, how we add data?
this is my way right now.

dataset = Dataset.create( dataset_project=metadata[2], dataset_name=metadata[3], description=description, output_uri=f" ", parent_datasets=[id_dataset_latest] )If you finalized it, you can create a child version - https://clear.ml/docs/latest/docs/clearml_data/data_manage...

Hi EagerStork23 ,

Thanks for catching this bug.
We also caught this issue, so a fix is scheduled to be released in one of the coming versions.
I'll update here once it will be released 🙂

Hi WackyRabbit7

If the trains-agent running docker mode, you can add it to agent.docker_init_bash_script in the ~/trains.conf file.

can you try with the latest? pip install clearml==1.1.4 ?

Hi BattyLion34 ,

Does ResNetFineTune.py is a new file in your git? Do you see its changes in the original task’s uncommitted changes section?

BattyLion34 when you are running the script locally, you have this script ( ResNetFineTune.py ) so you can run it without any issue, but when running the agent, the agent clone the repo, create an env and run the script. Now, the issue is that when trying to run the script, from the cloned repo, it can’t find it, because it’s only on your local machine, in the original git repo.

So running the docker with ‘“device=0,1”’ works? We will check that

To fix it, you can add the file and push the changes to the git. After you should also see the changes in your script (after committed).

 if I build a custom image, do I have to host it on dockerhub for it to run on the agent?

You dont need to host it, but in this case the machine running the agent should have the image (you can verify on the machine with docker images ).

If not how do I make the agent aware of my custom image?

Once the image is the base docker image for this task, and the image was verify on the agent’s machine, the agent should be able to use it

One solution I can think about is having a different image per Task , with the apt-get packages. you can just build a new image based on the one you have with the apt-get packages (or change to one with those packages).

Another one is running more than one agent, each one with different trains.conf file, one for each project.

Currently, task object doesn’t have a parameter for installing packages when running with trains-agent .

how can i check if it is loaded?

When a task is starting, the configuration will be print first

it worked with trains-agent init

Do you have 2 configuration files? ~/trains.conf and ~/clearml.conf ?

Hi GiganticTurtle0 ,

You have all the tasks that are part of the pipeline in an execution table (with links) under plots section, does it helps?