Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey, We Were Trying To Run An Experiment On Clearml Using Its Python-Sdk. When I Run An Experiment Using

Hey, we were trying to run an experiment on clearml using its python-sdk. When I run an experiment using Task.create() , it is picking up these random uncommitted changes (which I did not make)
from clearml import Task Task.init() and not running the actual training script that I intend to run. Is there a way to ensure that the whole script runs instead of only uncommitted changes?

Also, How do I make the files other than entry script visible to the job? (The script is not able to import them in the job but the it runs fine locally.)

  
  
Posted 3 years ago
Votes Newest

Answers 14


AgitatedDove14 We are trying to create an interface for our DL team so that they can send local repo path, entrypoint to script, requirements, etc as inputs and the job will get added on the queue.

  
  
Posted 3 years ago

script to create job
from clearml import Task
task = Task.create(project_name=‘example’, task_name=‘test_5_2’.format(i), script=‘~/train-script-old/train1.py’, working_directory=‘~/train-script-old/’, add_task_init_call=False)
status = Task.enqueue(task, ‘test’)
print(status)

  
  
Posted 3 years ago

Ohh StraightCoral86 did you check cleaml-task ? This is exactly what it does
(this is the CLI, from code you basically call Task.create & Task.enqueue)
Will this solve it ?

  
  
Posted 3 years ago

p.s. StraightCoral86 I might be missing something here, please feel free to describe the entire execution scenario and what you are trying to achieve 🙂

  
  
Posted 3 years ago

Right, you need to pass "repo" and direct it to the repository path
(BTW, what's the cleaml version)

  
  
Posted 3 years ago

CLEARML-AGENT version 0.17.2

  
  
Posted 3 years ago

I was able to successfully enqueue the task but only entrypoint script is visible to it and nothing else.

So you passed a repository link is it did not show on the Task ?
What exactly is missing and how the Task was created ?

  
  
Posted 3 years ago

I did that AgitatedDove14 . That is where I faced this issue. I was able to successfully enqueue the task but only entrypoint script is visible to it and nothing else.

  
  
Posted 3 years ago

cleamrl sdk (i.e. python client)
The issue is that the Task.create did not add the repo, link (again as mentioned above, you need to pass the local folder or repo link to the repo argument of the Task.create function). I "think" it could automatically deduce the repo from the script entry point, but I'm not sure. hence my question on the clearml package version

  
  
Posted 3 years ago

clearML console
clearml_agent: ERROR: Can not run task without repository or literal script in script.diff

  
  
Posted 3 years ago

I passed the path to local repo. all files in working directory other than entrypoint are missing.

  
  
Posted 3 years ago

Also, How do I make the files other than entry script visible to the job?

The assumption for clearml (regradless on how you create a Task) is that you code is either a standlone script (or jupyter notebook) or inside a git repository. In case of a git repository cleamrl-agent will clone the git repository of the code, apply the uncommitted changes and run your code.

  
  
Posted 3 years ago

Hi StraightCoral86

When I run an experiment using 

Task.create()

 ,

Use Task.init 🙂
Task.create is meant to create an extranl Task (i.e. Job) ins the system, Not to auto-gernerate a job from the running code. Make sense ?

  
  
Posted 3 years ago

CLI? Code ?

  
  
Posted 3 years ago