Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Can I Run A Random Task From A Queue? Like This

can i run a random task from a queue? like this clearml-agent execute --id <TASK_ID> or do i need to deque it first? for some reason it dosent work for me, just get stuck without any output

  
  
Posted 2 years ago
Votes Newest

Answers 23


can you get the agent to execute the task on the current conda env without setting up new environment?

Wouldn't that break easily ? Is this a way to avoid dockers, or a specific use case ?

is there any other way to get task from the queue running locally in the current conda env?

You mean including cloning the code etc. but not installing any python packages ?

  
  
Posted 2 years ago

"os": "Linux-4.18.0-348.2.1.el8_5.x86_64-x86_64-with-glibc2.28", "python": "3.9.7"

  
  
Posted 2 years ago

, the easiest way possible would be if could just some how run task and let the lsf manage the environment

You mean let the LSF set the conda/venv ? or do you also mean to get the code-base, changes etc ?

  
  
Posted 2 years ago

I'm running hyper parameter optimzation on LSF cluster where every task is an LSF job running without clearml-agent

WOW this is so cool! 🎊

  
  
Posted 2 years ago

now i noticed clearml-agent list gets stuck as well

  
  
Posted 2 years ago

clearml 1.1.6 clearml-agent 1.1.2no output at all, so nothing to paste

  
  
Posted 2 years ago

did i have to configure the environment first maybe? i assumed it just uses the environment where it was called

  
  
Posted 2 years ago

any idea why?

  
  
Posted 2 years ago

can you get the agent to execute the task on the current conda env without setting up new environment? or is there any other way to get task from the queue running locally in the current conda env?

  
  
Posted 2 years ago

Does the machine have connection to the backend?

  
  
Posted 2 years ago

so it would be better just to use the original code files and the same conda env. if possible…

Hmm you can actually run your code in "agent mode" assuming you have everything else setup.
This basically means you set a few environment variables prior to launching the code:
Basically:
export CLEARML_TASK_ID=<The_task_id_to_run> export CLEARML_LOG_TASK_TO_BACKEND=1 export CLEARML_SIMULATE_REMOTE_TASK=1 python my_script_here.py

  
  
Posted 2 years ago

can i run a random task from a queue? like this 

clearml-agent execute --id <TASK_ID>

  or

ChubbyLouse32 This will just work out of the box 🙂
No need to enqueue the Task, just reset it (in the UI)

  
  
Posted 2 years ago

Nice catch! (I’m assuming you also called Task.init somewhere before, otherwise I do not think this was necessary)

i was calling task init and it still somehow tought its the parent task, until i fixed it as i said.
and yes, everything is working now! im running hyper parameter optimzation on LSF cluster where every task is an LSF job running without clearml-agent

  
  
Posted 2 years ago

I think i solved it by deleting the project and running the base_task one time before the hyper parameter optimzation

  
  
Posted 2 years ago

Are you running on your own server or the community server?

  
  
Posted 2 years ago

AgitatedDove14 its the same file system, so it would be better just to use the original code files and the same conda env. if possible…

  
  
Posted 2 years ago

AgitatedDove14 that worked! but i had to add:
os.environ['CLEARML_PROC_MASTER_ID'] = '' os.environ['TRAINS_PROC_MASTER_ID'] = ''or else it tought it was the parent optimizer task i was trying to run.

but now im facing new issue, the details are empty:

  
  
Posted 2 years ago

On what OS are you on?
Regarding your question - I can't recall for sure. I think it still creates a virtualenv

  
  
Posted 2 years ago

my own server

  
  
Posted 2 years ago

i can create tasks and reterive them from the queues

  
  
Posted 2 years ago

os.environ['CLEARML_PROC_MASTER_ID'] = ''

Nice catch! (I'm assuming you also called Task.init somewhere before, otherwise I do not think this was necessary)

I think i solved it by deleting the project and running the base_task one time before the hyper parameter optimzation

So isit working now? everything is there ?

  
  
Posted 2 years ago

Cant you paste the output until the stuck point? Sounds very strange. Does it work when it's not enqueued? Also, what version of clearml-agent & server are you on?

  
  
Posted 2 years ago

Im still trying to figure out what is the best way to execute task on LSF cluster, the easiest way possible would be if could just some how run task and let the lsf manage the environment, on the same filesystem it is very easy to use shared conda env etc

  
  
Posted 2 years ago
1K Views
23 Answers
2 years ago
one year ago
Tags
Similar posts