Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
PlainSeaurchin97
Moderator
11 Questions, 32 Answers
  Active since 11 April 2023
  Last activity 2 days ago

Reputation

0

Badges 1

32 × Eureka!
0 Votes
2 Answers
632 Views
0 Votes 2 Answers 632 Views
Hi all! What's the best way to send my (automatically tracked pytorch) models to the ClearML server? My models are curently saved with file:// URI's, but i w...
one year ago
0 Votes
5 Answers
557 Views
0 Votes 5 Answers 557 Views
Hi all! I might have found an issue with the migration guide. None We recently migrated our data to a new server. While testing the migration, we found that ...
11 months ago
0 Votes
2 Answers
605 Views
0 Votes 2 Answers 605 Views
Hi all! I'm trying to migrate a clearml server (that has data on it) to another machine but i´m having problems to make clearml locate the old (backed up) ex...
one year ago
0 Votes
2 Answers
752 Views
0 Votes 2 Answers 752 Views
Hi all! Couldn't find this in the documentation, how do you specify a "setup shell script", so it is used for that specific task?
one year ago
0 Votes
1 Answers
180 Views
0 Votes 1 Answers 180 Views
Hi all, any API-oriented way to get download all the artifacts from a given task?
2 months ago
0 Votes
6 Answers
647 Views
0 Votes 6 Answers 647 Views
Hi all! Quick question: can clearml-agent build a docker image from a Dockerfile before using it for running a task?
one year ago
0 Votes
17 Answers
681 Views
0 Votes 17 Answers 681 Views
Hi all! Is there any simple way to use argparse to pass a clearml task name? I was using an argument called --clearml_task for this, but i ran into an intere...
one year ago
0 Votes
14 Answers
653 Views
0 Votes 14 Answers 653 Views
Hi all! I want to run my task remotely on an agent, but I'm having trouble with the requirements setup. I have a requirements.txt with many packages to insta...
one year ago
0 Votes
6 Answers
644 Views
0 Votes 6 Answers 644 Views
Hi all! Are there any plans to add scatterplots to visualize e.g. Hyperparemeter x Accuracy comparisons between experiments? MLFlow does this in a really nic...
one year ago
0 Votes
2 Answers
22 Views
0 Votes 2 Answers 22 Views
2 days ago
0 Votes
2 Answers
623 Views
0 Votes 2 Answers 623 Views
Is there any way to make a clickable link in the configuration of a task?
one year ago
0 Hi All! I'M Trying To Migrate A Clearml Server (That Has Data On It) To Another Machine But I´M Having Problems To Make Clearml Locate The Old (Backed Up) Experiments Data On The New Machine. I Followed The Section Of The Docs

Hi @<1523701087100473344:profile|SuccessfulKoala55> ,sorry i didn't respond!

We tried it all again and it worked... turns out our backup was probably corrupted

thanks!

11 months ago
11 months ago
0 Hi All! Are There Any Plans To Add Scatterplots To Visualize E.G. Hyperparemeter X Accuracy Comparisons Between Experiments? Mlflow Does This In A Really Nice Way, And I Missed This Feature On Our Transition To Clearml:

Actually, in addition to the parallel coordinates 😄
They're both good ways to visualize, but i think scatterplots are more intuitive for my use case

one year ago
0 Hi All! I Want To Run My Task Remotely On An Agent, But I'M Having Trouble With The Requirements Setup. I Have A

Not sure if i can because of some proprietary stuff on the code.

But i'll try writing a minimum working example on monday!

one year ago
0 Hi All! Is There Any Simple Way To Use

Since I can't use the

torchrun

comand (from my tests, clearml won't use it on the clearm-agent), I went with the

did you check this example?

@<1523701205467926528:profile|AgitatedDove14> actually i did! I based my code adaptation around it, since originally i was running a shell script that called torchrun

But tbh I didn't want to mess too much with my existing code, so i just did a quick and dirty adaptation using the torch.distributed.run command.

one year ago
0 Hi All! Is There Any Simple Way To Use

To be honest, i don't think using this envvar is the best option. I think just getting the task as normal (from the task name using Task.init) is the better option

But for these edge cases like i described, CLEARML_TASK_ID is ok

one year ago
0 Hi All! Is There Any Simple Way To Use

My final solution was to manually detect if i needed to patch the original argparse on the training script ( by using the CLEARML_TASK_ID envvar) and to turn off the automatic argparse connection

one year ago
11 months ago
0 Hi All! I Might Have Found An Issue With The Migration Guide.

is it possible to change an existing model's URL?
would it be smart to try to do this straight on the database? Is it Mongodb?

11 months ago
0 Hi All! Quick Question: Can Clearml-Agent

Either way, thanks for the support 😀

one year ago
0 Hi All! Quick Question: Can Clearml-Agent

From what i understand, what this does is build a container from an existing task. That's not really what i need

I'll describe my use case, maybe it makes it clearer:
I have a Dockerfile which builds an image with a bunch of system dependencies i need for training.

I want clearml-agent to use this image, but run docker build whenever necessary, this way I don't need to keep updating the base image which I want my tasks to be run with

one year ago
0 Hi All! Quick Question: Can Clearml-Agent
  • Not sure about pushing our container to a public registry. But follow-up question: how do I configure secrets (like container registry credentials) for a clearml-agent to use for a task?
  • Is it possible to do this on a task-by-task basis? I thought clearml-agent only installs pip requirements and such, is there a way to configure a setup script for my task environment?
one year ago
0 Hi All! Is There Any Simple Way To Use

Oh wait. Do I need the Task to exist in the subprocesses?
I re-create it on the subprocesses, because I thought my tensorboard stuff wouldn't get logged if the task wasn't initialized

one year ago
0 Hi All! I Want To Run My Task Remotely On An Agent, But I'M Having Trouble With The Requirements Setup. I Have A

I attached three logs:

  • local_console_output : how i setup my local task. Important commands: apt-install that installs the same dependencies that are on the docker_setup_bash_script ; and pip install -r requirements.txt
  • local_task_output: clearml experiment console log. The error "the following arguments are required: config" is the expected behavior
  • remote_task_output: clearml experiment console log obtained when i clone the local task and enqueue it for remote execution. No...
one year ago
0 Hi All! I Want To Run My Task Remotely On An Agent, But I'M Having Trouble With The Requirements Setup. I Have A

Basically: locally, when i run pip install -r requirements.txt , the softgroup.ops package is installed correctly. But not on the remote worker

I install the softgroup.ops package via the last line in requirements.txt , i.e. pip install -e .

one year ago
0 Hi All! I Want To Run My Task Remotely On An Agent, But I'M Having Trouble With The Requirements Setup. I Have A

Is there any way i can do something equivalent to -e . in the agent context?

one year ago
0 Hi All! I Want To Run My Task Remotely On An Agent, But I'M Having Trouble With The Requirements Setup. I Have A

I ned to pip-install the package because i need to build some Cuda extensions

one year ago
0 Hi All! I Want To Run My Task Remotely On An Agent, But I'M Having Trouble With The Requirements Setup. I Have A

Also, if you check the logs my package is actually built at step 4:

2023-05-03 10:07:58
Building wheels for collected packages: softgroup
  Building wheel for softgroup (setup.py) ... ?25l-
2023-05-03 10:08:14
 \ |
2023-05-03 10:08:19
 / - \

Looks like the -e flag is ignored. But it should work either way 🤔

one year ago
0 Hi All! I Want To Run My Task Remotely On An Agent, But I'M Having Trouble With The Requirements Setup. I Have A

in what order does the agent do things?
I assumed it was

  • Start the docker container
  • Run the docker setup bash script
  • Pull the repo , checkout the commit, apply changes
  • Install pip requirementsIn this case, i wouldn't have the correct version of the repo at the time the setup bash script runs
one year ago
0 Hi All! I Want To Run My Task Remotely On An Agent, But I'M Having Trouble With The Requirements Setup. I Have A

Looks like it was a python thing, not a clearml thing!

Clearml correctly installs the . from requirements.txt , but the project from the working directory was conflicting with the installed package, so python couldn't find the compiled extension.

With some small changes to my repo, everything works

one year ago
0 Hi All! Is There Any Simple Way To Use

OK, so i got into this mess with the argparse because i was turning OFF the automatic detection of command line arguments

I was turning it off because i was calling, inside my script, the argparser from torch.distributed.run ( best way i found to run a torchrun command in the clearml-agent)

Because of torch.distributed.run , clearml was automatically tracking inexisting command line arguments, which lead to an error on the remote agent.

In case this happens to anyone else, my ...

one year ago
0 Hi All! Is There Any Simple Way To Use

Hi @<1523701205467926528:profile|AgitatedDove14> , made this mock test real quick, it reproduces the issue:
None

one year ago
0 Hi All! Is There Any Simple Way To Use

just to be clear, this works on my local machine:

distributed_args = torch.distributed.run.parse_args(sys.argv)
distributed_args.nproc_per_node = args.gpus
torch.distributed.run.run(distributed_args)

But not when clearml-agent runs it

So the args are patched on the "main" process, but only on the remote worker

one year ago
0 Hi All! Is There Any Simple Way To Use

Thanks!

Follow-up question: how does clearML "inject" the argparse arguments before the task is initialized?
Does it mess with sys.argv ? Does it inject itself into argparse ?

I had to do another workaround since when torch.distributed.run called it's ArgumentParser , it was getting the arguments from my script (and from my task) instead of the ones I passed it

one year ago
Show more results compactanswers