I am doing something like this with a yaml based pipelines DSL
Example:
name: ml-project template: nbdev pipelines_runner: gitlab pipelines: pipeline-1: steps: - name: "publish-datasets" task_script: "mlproject/publish_datasets.py" - name: "training" task_script: "mlproject/training.py" parents: ["publish-datasets"] - name: "test" task_script: "mlproject/test.py" parents: ["training"]
Have cli which goes through each of the tasks and creates them
I can even specify a tag generated on the fly; superinteresting and I can satisfy my obsession for automation π
Nice guys! Notice that the clearml-task can auto add the Task.init call on the fly, so you can connect any arbitrary Task and control the argparser arguments (again as parameters to the cleaml-task)
BTW: A fix for the --task-type Issue will be pushed later today π
AgitatedDove14 maybe you can check also --version
that returns the help menu
not a big issue but you maybe worth a quick fix
maybe you can check alsoΒ
--version
Β that returns the helm menu
What do you mean? --version on cleaml-task ?
β― clearml-task --version ClearML launch - launch any codebase on remote machine running clearml-agent usage: clearml-task [-h] [--version] [--project PROJECT] --name NAME [--repo REPO] [--branch BRANCH] [--commit COMMIT] [--folder FOLDER] [--script SCRIPT] [--cwd CWD] [--args [ARGS [ARGS ...]]] [--queue QUEUE] [--requirements REQUIREMENTS] [--packages [PACKAGES [PACKAGES ...]]] [--docker DOCKER] [--docker_args DOCKER_ARGS] [--docker_bash_setup_script DOCKER_BASH_SETUP_SCRIPT] [--output-uri OUTPUT_URI] [--task-type TASK_TYPE] [--skip-task-init] [--base-task-id BASE_TASK_ID] clearml-task: error: the following arguments are required: --name
Hmm good point, it should probably return he clearml python version. Is this what you mean?
I think yes, at least this is whatI saw in docs
clearml --help says--version Display the clearml-task utility version
Yep, found it, the --name is marked as required and the argparser throws an error ...
I'll make sure this is fixed as well π
ty AgitatedDove14 , your fixes work like a charm. As reward I opened another one https://github.com/allegroai/clearml/issues/423 sorry for that π
π
I'm trying to create a task that is not in repository root folder.
JuicyFox94 If the Task is not in a repo folder, you mean in a remote repository right ?
This means the repo should be in the form of " https://github.com/ " or "ssh://"
It failed in deducing this is a remote repository (maybe we can improve the auto detection?!)
My bad "ssh://" prefix it not valid, let me try and see why it fails deducing this is a remote repo