Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
UnevenDolphin73
Moderator
106 Questions, 749 Answers
  Active since 10 January 2023
  Last activity one year ago

Reputation

0

Badges 1

662 × Eureka!
0 Votes
0 Answers
1K Views
0 Votes 0 Answers 1K Views
Also ubuntu 18.04 is losing support so everything in the image needs an update…
one year ago
0 Votes
14 Answers
1K Views
0 Votes 14 Answers 1K Views
Automatic logging for generated matplotlib works nicely, but for some reason it names the graphs as "00 untitled" and uses series "plot image" in ClearML UI ...
3 years ago
0 Votes
11 Answers
1K Views
0 Votes 11 Answers 1K Views
Back to autoscaler; is there any way to ensure the environment variables on the services queue (where the scaler runs) will be automatically exposed to new E...
2 years ago
0 Votes
30 Answers
1K Views
0 Votes 30 Answers 1K Views
Since v1.4.0, our StorageManager.download_folder(..., local_folder='./') is failing - we've had to revert back to 1.3.2. I saw the changelist includes a fix ...
2 years ago
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
Can we use S3 buckets to cache environments?
2 years ago
0 Votes
18 Answers
1K Views
0 Votes 18 Answers 1K Views
What would be the best way to approach this flow? We have a configuration file that defines e.g. the project name to use in ClearML, alongside other experime...
2 years ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
We're running into errors such as this: Action failed <500/0: tasks.add_or_update_artifacts/v2.10 (Update failed (BSONObj size: 18564134 (0x11B4426) is inval...
2 years ago
0 Votes
16 Answers
1K Views
0 Votes 16 Answers 1K Views
We’re running clearml-agent on k8s and I first noticed some warnings from the pod about Python 3.6..? > /usr/lib/python3/dist-packages/secretstorage/dhcrypto...
one year ago
0 Votes
13 Answers
1K Views
0 Votes 13 Answers 1K Views
Two things: The release of clearml-agent was not announced in 🤯 clearml-agent seems outdated wrt requirements, making poetry and clearml-agent incompatible:...
2 years ago
0 Votes
30 Answers
1K Views
0 Votes 30 Answers 1K Views
What happens to file that are downloaded to a remote_execution via StorageManager? Are they removed at the end of the run, or does it continuously increases ...
3 years ago
0 Votes
30 Answers
1K Views
0 Votes 30 Answers 1K Views
Is it possible to avoid the clearml-agent for local installations, and have the file server automatically use an S3 bucket? I've found https://clearml.slack....
3 years ago
0 Votes
42 Answers
84K Views
0 Votes 42 Answers 84K Views
2 years ago
0 Votes
23 Answers
1K Views
0 Votes 23 Answers 1K Views
When using StorageManager.download_folder , I get the following error: Traceback (most recent call last): File "/home/idan/.clearml/venvs-builds/3.7/lib/pyth...
3 years ago
0 Votes
9 Answers
1K Views
0 Votes 9 Answers 1K Views
How do I stop a zombie agent? 🤔 I see it in clearml-agent list , but I do not see it in ps aux and when I try clearml-agent daemon --stop , I get Could not ...
2 years ago
0 Votes
18 Answers
1K Views
0 Votes 18 Answers 1K Views
!! In remote execution, do agents inherit the config from the queue from which they pull the task?
2 years ago
0 Votes
3 Answers
1K Views
0 Votes 3 Answers 1K Views
Does ClearML have any suggestions on GPU and non-GPU AMIs for the AutoScaler? The two default ones in the documentation are either: Non-existent (too old) - ...
2 years ago
Show more results questions
0 Is There Any Testing Suite That Ships With Clearml? If We'D Like To Make Some Unit Tests For Our Code?

Yes exactly that AgitatedDove14
Testing our logic maps correctly, etc for everything related to ClearML

2 years ago
0 If I Clone A Task, I Suppose All Artifacts Are Not Cloned With It, Even If They Are Registered, Right?

There used to be a good example but it's now missing. I'm not sure what does Use only for automation (externally), otherwise use Task.connect_configuration mean when e.g. looking at Task.set_configuration_object , etc.

Could you clarify a bit, CostlyOstrich36 or AgitatedDove14 ?

2 years ago
2 years ago
0 How Would I Go About Adding Multiple Credentials In The Autoscaler? (I.E. Specify Multiple

-ish, still debugging some weird stuff. Sometimes ClearML picks ip and sometimes ip2 , and I can't tell why 🤔

2 years ago
0 Since V1.4.0, Our

StorageManager.download_folder(remote_url=' s3://some_ip:9000/clearml/my_folder_of_interest ', local_folder='./') yields a new folder structure, ./clearml/my_folder_of_interest , rather than just ./my_folder_of_interest

2 years ago
0 Good Week Clearml Team

Would be nice if the second one was a toggle-able feature (either per use or in the server settings) maybe?

2 years ago
0 Back To Autoscaler; Is There Any Way To Ensure The Environment Variables On The Services Queue (Where The Scaler Runs) Will Be Automatically Exposed To New Ec2 Instance? Some Bash Hack Or Similar Would Be Nice, Really

CostlyOstrich36 I'm not sure what you mean by "through the apps", but any script AFAICS would expose the values of these environment variables; or what am I missing?

2 years ago
0 We'Re Trying To Upgrade Our Clearml On K8S But We'Re Getting This Error -

For now this is okay - no data lost, really - but I'd like to make sure we're not missing any steps in the next upgrade

2 years ago
0 Good Week Clearml Team

Because I never know which repo to open it in 😁

2 years ago
0 !! In Remote Execution, Do Agents Inherit The Config From The Queue From Which They Pull The Task?

Okay so the only missing thing of the puzzle I think is that it would be nice if this propagates to the autoscaler as well; that then also allows hiding some of the credentials etc 😮

2 years ago
0 Can I Run An Autoscaler Listening To A Regular Queue (I.E. Combine Autoscaling With On Premise Machines)? Or Does It Run The Danger Of Creating An Ec2 Instance While An On-Premise Machine Takes The Job?

I think now there's the following:
Resource type Queue (name) defines resource + max instancesAnd I'm looking for:
Resource type "pool" of resources (type + max instances) A pool can be shared among queues

2 years ago
0 Can I Run An Autoscaler Listening To A Regular Queue (I.E. Combine Autoscaling With On Premise Machines)? Or Does It Run The Danger Of Creating An Ec2 Instance While An On-Premise Machine Takes The Job?

Thanks CostlyOstrich36 !
And can I make sure the same budget applies to two different queues?
So that for example, an autoscaler would have a resource budget of 6 instances, and it would listen to aws and default as needed?

2 years ago
0 Is There Anywhere In The Web Ui Where One Can See The Clearml Server Version Running? I Keep Getting "Version 1.1.1 Is Now Available" Even Though I'M Pretty Sure I Took All The Steps To Update To The Latest Version

Sure CostlyOstrich36 , sorry it took me so long to reply. I minimized the window a bit here so everything will fill in nicely. Worth mentioning this happens on all pages of course, but I went to the profile page so you can also see the clearml server version.

3 years ago
0 What Happens To File That Are Downloaded To A Remote_Execution Via Storagemanager? Are They Removed At The End Of The Run, Or Does It Continuously Increases Disk Space?

It's okay 🙂 I was originally hoping to delete my "initializer" task, but I'll just archive it if someone is interested in the worker data etc. Setting the queue is quite nice.

I think this should get my team excited enough 😄

3 years ago
0 What Happens To File That Are Downloaded To A Remote_Execution Via Storagemanager? Are They Removed At The End Of The Run, Or Does It Continuously Increases Disk Space?

Honestly, this is all related to issue #340. The only reason we have this to begin with is because we need one separate "initializer" task that downloads the remote cache and prepares the agent environment for execution (downloading the configuration files, etc).
Otherwise it fits perfectly with pipelines, but we're not there yet.

In the local execution we don't have this initializer task, so we use Task.init() before starting to work on a model, and task.close() when we're done....

3 years ago
0 What Happens To File That Are Downloaded To A Remote_Execution Via Storagemanager? Are They Removed At The End Of The Run, Or Does It Continuously Increases Disk Space?

I didn't mention code in #340 nor did I mention data here 😄 The idea was to package non git-specific files for remote execution

3 years ago
0 What Happens To File That Are Downloaded To A Remote_Execution Via Storagemanager? Are They Removed At The End Of The Run, Or Does It Continuously Increases Disk Space?

That could work, given that:
Could we add a preview section? One reason I don't like using the configuration section is that it makes debugging much much harder. Will the clearml-agent download and unzip the files, placing them into the same local folder as needed for execution? What if we want to include non-configuration objects? (i.e. the model case I listed)

3 years ago
0 Is There A Way To Interface With Clearml Agent (Cli?) To Handle Model Repositories And Data Versioning (But So, Not Experimentation, Tight Integration, Pipelining, Etc)?

If everything is managed with a git repo, does this also mean PRs will have a messy metadata file attached to them?

4 years ago
0 Is It Expected That K8S Helm Chart Also Starts A Clearml Worker?

Can I query where the worker is running (IP)?

2 years ago
0 Can I Somehow Change A Project Settings Via The Python Sdk?

Where do I import this APIClient from AgitatedDove14 ? I meanwhile edited it directly in mongo, but editing a db directly on a Friday is a big nono

2 years ago
0 Weird Encounter On Macos (Local Execution, Rerunning After It Failed - Worked Normally):

This happened again 🤔
How many files does ClearML touch? :shocked_face_with_exploding_head:

2 years ago
0 How Would I Go About Adding Multiple Credentials In The Autoscaler? (I.E. Specify Multiple

TimelyPenguin76 CostlyOstrich36 It seems a lot of manual configurations is required to get the EC2 instances up and running.
Would it not make sense to update the autoscaler (and example script) so that the config.yaml that's used for the autoscaler service is implicitly copied to the EC2 services, and then any extra_clearml_conf are used/overwritten?

2 years ago
0 Is There Any Testing Suite That Ships With Clearml? If We'D Like To Make Some Unit Tests For Our Code?

This seems to be fine for now, if any future lookups finds this thread, btw
with mock.patch('clearml.datasets.dataset.Dataset.create'): ...

2 years ago
0 What Happens To File That Are Downloaded To A Remote_Execution Via Storagemanager? Are They Removed At The End Of The Run, Or Does It Continuously Increases Disk Space?

As the meme goes, well yes but actually no, since the input path is provided via argparse? I'm also not sure how this would help debug from the WebUI - you can't really see the contents of a zipped file/the configuration tab is too messy for such a nested configuration as the one we have. It's best suited as an artifact.

EDIT: Or am I missing something? Point being, when the remote execution begins, the entry point tries to run e.g. python train.py --config_file path/to/local/file.yaml ...

3 years ago
Show more results compactanswers