Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello, I'M A Bit Lost In The Docs For The Mlops, I Have Script Which Already Integrate Clearml Logging, Should I Use Clearml-Task To Launch It On An Agent ? (I Already Have A Clearml-Server And A Clearml-Agent Running).

Hello, I'm a bit lost in the docs for the MLOps, I have script which already integrate clearml logging, should I use clearml-task to launch it on an agent ? (I already have a clearml-server and a clearml-agent running).
https://github.com/allegroai/clearml/blob/master/docs/clearml-task.md says it's for "codebase that is still not connected to clearml".
But I don't see how I can launch tasks without clearml-task

  
  
Posted 3 years ago
Votes Newest

Answers 18


if the file is untracked by git, it is not saved by clearml

Yep 😞

Does clearml-agent install the repo withΒ 

pip install -e .

It is supported, but the path to the repo cannot be absolute (as it will probably be something else in the agent env)
You can add "git+ https://github.com ...." to the "installed packages" The root path of your repository is always added to the PYTHONPATH when the agents executes it, so in theory there is no need to install it with pipwdyt?

  
  
Posted 2 years ago

Ohh, two options:
From the script itself you can do:
from clearml import Task task = Task.init(...) task.execute_remotely(queue='default')Then run the script locally, it will get until the "execute_remotely call, quit the process and re-launch it on the "default" queue.
Option B:
Use the cleaml-task
$ clearml-task --folder <where the script is> --project ...See https://github.com/allegroai/clearml/blob/master/docs/clearml-task.md#launching-a-job-from-a-local-script

  
  
Posted 3 years ago

The task is registered and is started by the agent, the env seems to be installed well, but then it fails on /home/ubuntu/.clearml/venvs-builds/3.8/bin/python: can't open file 'fastai_classifier.py': [Errno 2] No such file or directory Do you have an idea of what could be wrong ? The agent launch the script in the wrong working dir ? The repo is not copied ? (This script is inside a private git repo, that clearml detects correctly).

I also tried launching the script from the root of the repo but it's the same: /home/ubuntu/.clearml/venvs-builds/3.8/bin/python: can't open file 'train/classification/fastai_classifier.py': [Errno 2] No such file or directory

  
  
Posted 3 years ago

Thanks ! I think .execute_remotely() is exactly what I need

  
  
Posted 3 years ago

not exactly, I want to launch the script (create a new experiment, not clone an existing one in the UI), how can I do it ?

  
  
Posted 3 years ago

Could it be the code is not in a git repository ?
clearml support either a single script or a git repository, but Not a collection of standalone files. wdyt?

  
  
Posted 3 years ago

The script is inside a git repo (and it's the one I launch, I would get an importerror if it was something else missing)

  
  
Posted 3 years ago

I think I found the problem, if the file is untracked by git, it is not saved by clearml

  
  
Posted 2 years ago

which is kind of logical

  
  
Posted 2 years ago

Hmm apparently if I launch the script from the root of the repo (CWD: myrepo python train/classif-custom/train.py ) it works, but from its dir it doesn't work (CWD: myrepo/train/classif-custom python train.py )

  
  
Posted 2 years ago

How does the folder structure look like, and where is the "package" and the entry script ?

  
  
Posted 2 years ago

I think didn't understand, if I'm not at the root of the repo, I have to specify the working dir ?

  
  
Posted 2 years ago

However I have another problem, my git repo is installed with pip install -e . and I import it in my script, but on a task executed by a clearml-agent the module appears not to be installed ?

  
  
Posted 2 years ago

VirtuousFish83
Hmm that is odd, could you send the full log?

  
  
Posted 3 years ago

You can change it the CWD folder, if you put . in working dir it will be the root git repo, but you can do any subfolder, obviously you need to change the script path to match the folder, e.g. ./folder/script.py etc.

  
  
Posted 2 years ago

Does clearml-agent install the repo with pip install -e . if it should be ? (i.e. my local repo is installed with pip install -e . where I launch my script which calls Task.init and .execute_remotely() ).

  
  
Posted 2 years ago

Hi VirtuousFish83
Apologies for the documentation in the docs πŸ™‚ It sounds complicated but actually should be relatively simple. Based on what I understand, you already have the server setup and you code integrated. The question is "can you see an experiment in the UI"? If you do, then you can right click it, clone the experiment , edit parameters and send for execution (enqueue). If the experiment is not in the UI you can either (1) run the code with the Task.init call, it ill automatically populate everything you need, python packages git repo etc. (2) manually provide the necessary definition with clearml-task (i.e. git repo, python packages if you are not using requirements.txt, arguments etc)

Was this what you were looking for?

  
  
Posted 3 years ago

Awesome πŸ™‚

  
  
Posted 3 years ago
571 Views
18 Answers
3 years ago
one year ago
Tags