Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
HurtWoodpecker30
Moderator
10 Questions, 43 Answers
  Active since 10 January 2023
  Last activity one year ago

Reputation

0

Badges 1

41 × Eureka!
0 Votes
28 Answers
438 Views
0 Votes 28 Answers 438 Views
I uncommented the line path: ~/.clearml/venvs-cache in the remote agent’s clearml.conf but the remote agent still keeps re-installing all pkgs in the venv. (...
one year ago
0 Votes
0 Answers
444 Views
0 Votes 0 Answers 444 Views
it used to work just fine until a few days ago
one year ago
0 Votes
0 Answers
436 Views
0 Votes 0 Answers 436 Views
Just came here to say how much I appreciate clearML ! It just stays out of the way and tracks so many things, and has been super useful when digging into dif...
one year ago
0 Votes
17 Answers
574 Views
0 Votes 17 Answers 574 Views
Hi folks, Any idea why I am getting this strange error from clearml-data: $ clearml-data list --project myproject clearml-data - Dataset Management & Version...
one year ago
0 Votes
13 Answers
481 Views
0 Votes 13 Answers 481 Views
Are there instructions somewhere on how I can use clearml-agent to run jobs on my google-cloud compute engine?
one year ago
0 Votes
0 Answers
625 Views
0 Votes 0 Answers 625 Views
Hi Could someone please point me to a page that shows how to set up google cloud storage so I can use it to push/pull data from it using clearml-data ? thanks
one year ago
0 Votes
2 Answers
604 Views
0 Votes 2 Answers 604 Views
from the docs it is not clear whether I can use the clearml-data CLI for pushing/pulling data from a remote storage (e.g. google storage) https://clear.ml/do...
one year ago
0 Votes
0 Answers
604 Views
0 Votes 0 Answers 604 Views
I’ve set up a gcp project and service acct, and added creds.json to clearml.conf, but this page is not clear on how to use it with clearml-data https://clear...
one year ago
0 Votes
4 Answers
518 Views
0 Votes 4 Answers 518 Views
gm folks, really liking ClearML so far as my top choice (after looking at dvc, mlflow), and thank you for your help here! I had another Q: Is there a recomme...
one year ago
0 Votes
0 Answers
444 Views
0 Votes 0 Answers 444 Views
is this the right place? https://clear.ml/docs/latest/docs/integrations/storage#configuring-google-storage
one year ago
one year ago
0 Hi Folks, Any Idea Why I Am Getting This Strange Error From Clearml-Data:

CLI doesn’t care about the state of my git repo right?

one year ago
0 Hi Folks, Any Idea Why I Am Getting This Strange Error From Clearml-Data:

Dataset.get works fine from python script, it pulls in the data into cache. Just the cli seems broken

one year ago
one year ago
0 Are There Instructions Somewhere On How I Can Use Clearml-Agent To Run Jobs On My Google-Cloud Compute Engine?

So if I want to train with a remote agent on a remote machine, I have to:
spin up clearml-agent on the remote create a dataset using clearml-data, populate with data… from my local machine use clearml-data to upload data to google gs:// bucket modify my code so it accesses data from the dataset as here https://clear.ml/docs/latest/docs/clearml_data/clearml_data_sdk#accessing-datasetsAm I understanding right?

one year ago
0 I Uncommented The Line

I mean it is in Pip mode and the agent installs deps from git repo that it pulls

one year ago
0 Are There Instructions Somewhere On How I Can Use Clearml-Agent To Run Jobs On My Google-Cloud Compute Engine?

thanks, so I got clearml-task working, sent to a queue and the agent on gcp picked it up. I had a question — for a job that runs on the order of minutes, it’s not worth re-creating the whole python virtual env from scratch on the remote (that itself takes 5mins). So is the --folder ` option meant for running it in an existing folder in an existing virtual env?

one year ago
0 Are There Instructions Somewhere On How I Can Use Clearml-Agent To Run Jobs On My Google-Cloud Compute Engine?

I think I am missing one part — which command do I use on my local machine, to indicate the job needs to be run remotely? I’m imagining something like
clearml-remote run python3 my_train.py

one year ago
0 Hi. I'D Like To Try The Gcp Autoscaler.

I would also be interested in a GCP autoscaler, I did not know it was possible/available yet.

one year ago
0 I Uncommented The Line

I'm not familiar with “installed package s” list in the task

one year ago
0 I Uncommented The Line

Thanks for the quick response . Will look into this later , I think I understand

one year ago
0 I Uncommented The Line

So net-net does this mean it’s behaving as expected, or is there something I need to do enable “full venv cache”? It spends nearly 2 mins starting from
created virtual environment CPython3.8.10.final.0-64 in 97ms creator CPython3Posix(dest=/home/pchalasani/.clearml/venvs-builds/3.8, clear=False, global=False)and then printing several lines lines like this
` Successfully installed pip-20.1.1
Collecting Cython
Using cached Cython-0.29.30-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86...

one year ago
0 I Uncommented The Line

I see, so there’s no way to launch a variant of my last run (with say some config/code tweaks) via CLI, and have it re-use the cached venv?

one year ago
0 I Uncommented The Line

Actually that did not help. Still same behavior

one year ago
0 I Uncommented The Line

I have a strong attachment to a workflow based on CLI, nice zsh auto-suggestions, Hydra and the like. Hence why I moved away from dvc 🙂

one year ago
0 I Uncommented The Line

Actually with base-task-id it uses the cached venv, thanks for this suggestion! Seems like this is equivalent to cloning via UI.
And I will look into the non-cli workflow you’re suggesting.

one year ago
0 I Uncommented The Line

… but I have a feeling they will not give me the “instant venv activation” behavior I’m looking for.

one year ago
0 I Uncommented The Line

I used
task.execute_remotely(queue_name=..., clone=True)and indeed it instantly activates the venv on the remote. I assume clone=True is fine

one year ago
0 I Uncommented The Line

I use a CLI arg remote=True so depending on that it will run locally or remotely.

one year ago
0 I Uncommented The Line

This works great, thanks AgitatedDove14 👍

one year ago
0 I Uncommented The Line

I think I’m starting to “get” this 🙂

one year ago
0 I Uncommented The Line

A quick note for others who may visit this… it looks like you have to do:
Task.force_requirements_env_freeze(force=True, requirements_file="requirements.txt")to ensure any changes in requirements.txt are reflected in the remote venv

one year ago
Show more results compactanswers