Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Guys, Until Today I Always Requested Data Scientists To Use Cli To Create Tasks. After That I Usually Reconfigure Them So They Can Be Pointed On Git Repo And So On. Unfortunately This Is Becoming A Big Task Since Now We Have Pipelines With Many Tasks A

Hi guys, until today I always requested data scientists to use CLI to create tasks. After that I usually reconfigure them so they can be pointed on git repo and so on. Unfortunately this is becoming a big task since now we have pipelines with many tasks and I must repeat this task manually for each task (and they are a lot). I'm still thinking on a way to automate this with a gh action and some scripting so I would like to create a project given a folder when every py file is a task. This should automatically create every task needed in ClearML. Anyone already had a look at that?

  
  
Posted 3 years ago
Votes Newest

Answers 21


❯ clearml-task --version ClearML launch - launch any codebase on remote machine running clearml-agent usage: clearml-task [-h] [--version] [--project PROJECT] --name NAME [--repo REPO] [--branch BRANCH] [--commit COMMIT] [--folder FOLDER] [--script SCRIPT] [--cwd CWD] [--args [ARGS [ARGS ...]]] [--queue QUEUE] [--requirements REQUIREMENTS] [--packages [PACKAGES [PACKAGES ...]]] [--docker DOCKER] [--docker_args DOCKER_ARGS] [--docker_bash_setup_script DOCKER_BASH_SETUP_SCRIPT] [--output-uri OUTPUT_URI] [--task-type TASK_TYPE] [--skip-task-init] [--base-task-id BASE_TASK_ID] clearml-task: error: the following arguments are required: --name

  
  
Posted 3 years ago

My bad "ssh://" prefix it not valid, let me try and see why it fails deducing this is a remote repo

  
  
Posted 3 years ago

AgitatedDove14 maybe you can check also --version that returns the help menu

  
  
Posted 3 years ago

I think yes, at least this is whatI saw in docs

  
  
Posted 3 years ago

clearml --help says
--version Display the clearml-task utility version

  
  
Posted 3 years ago

maybe you can check alsoΒ 

--version

Β that returns the helm menu

What do you mean? --version on cleaml-task ?

  
  
Posted 3 years ago

woah.. superfast πŸ˜„

  
  
Posted 3 years ago

how do you point tasks to git repo?

  
  
Posted 3 years ago

Example:

name: ml-project template: nbdev pipelines_runner: gitlab pipelines: pipeline-1: steps: - name: "publish-datasets" task_script: "mlproject/publish_datasets.py" - name: "training" task_script: "mlproject/training.py" parents: ["publish-datasets"] - name: "test" task_script: "mlproject/test.py" parents: ["training"]Have cli which goes through each of the tasks and creates them

  
  
Posted 3 years ago

I can even specify a tag generated on the fly; superinteresting and I can satisfy my obsession for automation πŸ˜„

  
  
Posted 3 years ago

uh, using clearml-task params πŸ˜„

  
  
Posted 3 years ago

ty AgitatedDove14 , your fixes work like a charm. As reward I opened another one https://github.com/allegroai/clearml/issues/423 sorry for that πŸ˜‚

  
  
Posted 3 years ago

interesting, ty

  
  
Posted 3 years ago

I am doing something like this with a yaml based pipelines DSL

  
  
Posted 3 years ago

Yep, found it, the --name is marked as required and the argparser throws an error ...
I'll make sure this is fixed as well πŸ™‚

  
  
Posted 3 years ago

Nice guys! Notice that the clearml-task can auto add the Task.init call on the fly, so you can connect any arbitrary Task and control the argparser arguments (again as parameters to the cleaml-task)
BTW: A fix for the --task-type Issue will be pushed later today πŸ˜‰

  
  
Posted 3 years ago

Hmm good point, it should probably return he clearml python version. Is this what you mean?

  
  
Posted 3 years ago

πŸ˜‚

I'm trying to create a task that is not in repository root folder.

JuicyFox94 If the Task is not in a repo folder, you mean in a remote repository right ?
This means the repo should be in the form of " https://github.com/ " or "ssh://"
It failed in deducing this is a remote repository (maybe we can improve the auto detection?!)

  
  
Posted 3 years ago

not a big issue but you maybe worth a quick fix

  
  
Posted 3 years ago

great πŸ˜„

  
  
Posted 3 years ago

this is what I got

  
  
Posted 3 years ago
905 Views
21 Answers
3 years ago
one year ago
Tags
Similar posts