Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
SarcasticSparrow10
Moderator
13 Questions, 58 Answers
  Active since 10 January 2023
  Last activity one year ago

Reputation

0

Badges 1

58 × Eureka!
0 Votes
2 Answers
635 Views
0 Votes 2 Answers 635 Views
Is it possible to have clearml launch aws instances to run clearml-agent , execute jobs and then shutdown after the jobs have been completed? I have two rele...
aws
2 years ago
0 Votes
2 Answers
575 Views
0 Votes 2 Answers 575 Views
Hi
Hi AgitatedDove14 , I put together some improvements for the parallel coordinate chart on this https://github.com/allegroai/trains/issues/259 . What are your...
3 years ago
0 Votes
7 Answers
567 Views
0 Votes 7 Answers 567 Views
Hi
Hi AgitatedDove14 , I'd appreciate your thoughts on trains-agent on the following topic. To run an experiment by a trains-agent , it must have already been r...
3 years ago
0 Votes
25 Answers
599 Views
0 Votes 25 Answers 599 Views
Hi ClearML, I tried to upgrade the clearml server following this https://clear.ml/docs/latest/docs/deploying_clearml/upgrade_server_aws_ec2_ami but it erased...
2 years ago
0 Votes
14 Answers
592 Views
0 Votes 14 Answers 592 Views
I'm using trains-agent to run an experiment from a private git repo and I run into this error trains_agent: ERROR: Failed cloning repository. 1) Make sure yo...
3 years ago
0 Votes
3 Answers
585 Views
0 Votes 3 Answers 585 Views
Where is the stdout of trains-agent stored in the service mode (running in background)?
3 years ago
0 Votes
4 Answers
622 Views
0 Votes 4 Answers 622 Views
Hello, I'm logging a plotly figure which contains subplots using Logger.report_plotly() method. However, many of the attributes (color, legend label) of the ...
3 years ago
0 Votes
8 Answers
590 Views
0 Votes 8 Answers 590 Views
Quick question on trains-agent and HPO. Say I have 10 experiments enqueued to a trains-agent . I understand the agent runs the experiment one-by-one. But can...
3 years ago
0 Votes
2 Answers
610 Views
0 Votes 2 Answers 610 Views
Is there an upgrade guide from Trains to ClearML? I can't seem to find it.
3 years ago
0 Votes
22 Answers
579 Views
0 Votes 22 Answers 579 Views
Hi
Hi AgitatedDove14 , I upgraded clearml from 0.17.4 to 0.17.5rc2 and the change broke my code as it seems like clearml has started using multiprocessing. I ge...
3 years ago
0 Votes
10 Answers
539 Views
0 Votes 10 Answers 539 Views
2 years ago
0 Votes
5 Answers
575 Views
0 Votes 5 Answers 575 Views
I am using reuse_last_task_id=True to overwrite an existing task (with same project and task name). But the experiment contains the torch model and therefore...
3 years ago
0 Votes
5 Answers
573 Views
0 Votes 5 Answers 573 Views
Hi, can we search tasks using wildcard in the webapp. Say I have task names exp_test1_i1 , exp_test2_i1 and I'd like to search for "exp" and "i1". Is that po...
3 years ago
0 Hi Clearml, I Tried To Upgrade The Clearml Server Following This

Steps 1 and 2 on this https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_mongo44_migration/ say to backup opt/clearml/data/mongo and uncompress into /opt/clearml/data/mongo_4 . Isn't it just copying the old data files?

2 years ago
0 Hi Clearml, I Tried To Upgrade The Clearml Server Following This

There was some complication during the upgrade so I had to resort to the manual process.

I have now been able to upgrade by dumping the mongodb data and restoring it independently.

2 years ago
0 Hi Clearml, I Tried To Upgrade The Clearml Server Following This

I was getting the error in step number 3

2 years ago
0 Hi Clearml, I Tried To Upgrade The Clearml Server Following This

mongo 4.4 image does not launch a container if the data in mongo dir is for previous versions. We should add that comment in the documentation

2 years ago
0 Hi Clearml, I Tried To Upgrade The Clearml Server Following This

Hi AppetizingMouse58

Yes, I tried to perform steps 3-10, however step 3 raised an error because data files for mongo were incompatible between 3.6 and >4.0

2 years ago
0 Hi Clearml, I Tried To Upgrade The Clearml Server Following This

I had to manually create a dump for the mongo data and import it into 4.4. I was just referring to adding a note to the documentation for other users.

2 years ago
0 Hi Clearml, I Tried To Upgrade The Clearml Server Following This

I cannot execute step 4 because I can't get past step 3. Does that make sense?

2 years ago
0 Hi Clearml, I Tried To Upgrade The Clearml Server Following This

Steps 1 and 2 basically copy mongo 3.6 data into a new dir mongo_4 but mongo image of version 4.4 does not accept that data. So I had to perform the following steps:

Launch docker container with mongo=3.6 dump data using mongo dump Launch docker container with mongo=4.4 and empty mongo_4 data dir Restore the dump data using mongo restore
This made sure the data is now compatible with mongo 4.0 or greater

2 years ago
0 Hi Clearml, I Tried To Upgrade The Clearml Server Following This

I'm getting the same error when I followed the instructions to the letter.

Here is one line from the mongo docker output
"This version of MongoDB is too recent to start up on the existing data files. Try MongoDB 4.2 or earlier."

2 years ago
0 Hi, Which Database Services Are Used To Store The Logged Data Such As Scalar, Text, Matrix, Etc? How Can I Query These For A Downstream Process Programmatically Instead Of Just Within The Web Ui? If Scalar Data Is Stored In Mongodb, Can I Use Pymongo To R

Hi AgitatedDove14 Thanks, I'll check these out.

What is the exact use case you have in mind?

I want to store some additional data that is not relevant to training a model. For example, store inference results, explanations, etc and then use them in a different process. I currently use separate database for this.

Btw, I had been busy with another project and hadn't logged in here for some time. I see that you guys have made a lot of progress in the last two months! I'm excited to di...

2 years ago
0 Hi, Which Database Services Are Used To Store The Logged Data Such As Scalar, Text, Matrix, Etc? How Can I Query These For A Downstream Process Programmatically Instead Of Just Within The Web Ui? If Scalar Data Is Stored In Mongodb, Can I Use Pymongo To R

What would be the query ? Are you reporting 100+ diff scalars ?

At the moment I am not reporting any scalars related to inference. I'm only reporting data related to training a model. But I would like to report records that result from an inference process. For example the record would contain key_1, key_2, datetime, pred_1, pred_2 ... pred_n. I would have about 20 scalars if each of these fields is reported as a scalar.

The query can be a simple filtering criteria matching some keys ...

2 years ago
0 Hi, Which Database Services Are Used To Store The Logged Data Such As Scalar, Text, Matrix, Etc? How Can I Query These For A Downstream Process Programmatically Instead Of Just Within The Web Ui? If Scalar Data Is Stored In Mongodb, Can I Use Pymongo To R

Ok, I will look into artifacts. However, I will probably need high performance query functionality. For example, say I have a model and hundreds of thousands of inference records for that model. I want to be able to efficiently query that. My guess is that wouldn't be possible with artifacts. But that should be possible with Task.get_tasks .

2 years ago
0 Is It Possible To Have 

Yes, this is what I'm looking for. Excellent! Thank you!

2 years ago
0 Hi

Thanks for the tip with the config file. I have reverted back to 0.17.4 but will try this.

3 years ago
0 Hi

2. interesting error, maybe we can revert to "thread mode" if running under a daemon. (I have to admit, I'm not sure why python has this limitation, let me check it...)

Yes, I'm not sure either. I have banged my head against the wall in trying to have multiple level of subprocesses, but it gets too complicated with python. Let me know what you find out

3 years ago
0 Hi

Yes, I am using Pool. Here is what I think is happening. clearml launches a subprocess which I assume is a daemonic process. That process in-turn launches a subprocess for training which causes the error I mentioned

3 years ago
0 Hi

The second subprocess is by design. It becomes the primary process when clearml does not use multiprocessing. I hope I'm not confusing you further

3 years ago
0 Hi

This is happening manually. I am not using agent yet

3 years ago
0 Hi

Yes the 'training' is my main code. You can think of it has launching a job (training or inference). My main code launches multiple jobs using multiprocessing. Each job is a seprate task for clearml that gets logged. Does that make sense?

3 years ago
0 Hi

Yes, I am using multiprocessing.Pool to launch each job

3 years ago
0 Hi

Haha.. ok, good to know

3 years ago
0 Hi

Haha.. that would be a problem then!

3 years ago
0 Hi

ok

3 years ago
0 Hi

Hi AgitatedDove14 , I'll wait for you to reply on Github before I add my comments to these points.

3 years ago
Show more results compactanswers