Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I’M Trying To Use

I’m trying to use task.execute_remotely but the repo I’m trying to use is not being identified correctly. I have it setup to git pull from a master repo and push to my fork of the repo. Is it possible to set that explicitly?

I wrote some code to manually update the remote branch with the latest changes and overrode execute_remotely to do that before delegating to super().execute_remotely but I’m not sure how actually make the task pull from the correct remote. The only place I see where that’s possible is Task.create which seems incompatible with execute_remotely .

  
  
Posted 3 years ago
Votes Newest

Answers 17


$ git remote -v fork git@github.com:salimmj/somerepo.git (fetch) fork git@github.com:salimmj/somerepo.git (push) origin git@github.com:mainuser/somerepo.git (fetch) origin git@github.com:mainuser/somerepo.git (push)I want to keep the above setup, the remote branch that will track my local will be on fork so it needs to pull from there. Currently it recognizes origin so it doesn’t work because the agent then can’t find the commit.

  
  
Posted 3 years ago

LazyTurkey38
The last part makes sense, not sure I get the "if clone", we are calling execute_remotely, so I'm assuming we do not need to clone ourselves, but send the current Task.
Other than that yes, makes sense (BTW, assuming you have upgraded the server >=1.0 you can just do mark_stopped, no need to reset

  
  
Posted 3 years ago

Fixed it by adding this code block. Makes sense.
if clone: task = Task.clone(self) else: task = self # check if the server supports enqueueing aborted/stopped Tasks if Session.check_min_api_server_version('2.13'): self.mark_stopped(force=True) else: self.reset()

  
  
Posted 3 years ago

But these changes haven’t necessarily been merged into main. The correct behavior would be to use the forked repo.

So I would expect the agent to pull from your fork, is that correct? is that what you want to happen ?

  
  
Posted 3 years ago

Hi LazyTurkey38
What do you mean the git repo is not recognized? When execute_remotely leaves you should see on the task a ref to the git repo with the exact commit ID you have locally pulled, do you see it under the Execution tab?

  
  
Posted 3 years ago

LazyTurkey38 , ohh I think you are correct 😞
it should be:
# patch the Task and actually send it for execution if Task.running_locally(): # this will verify all auto repo detection and python is done. task.close() # so that we can edit the task task.reset() # update the repo task.update_task(task_data={'script': {'branch': 'new_branch', 'repository': 'new_repo'}}) # now to actually enqueue the Task Task.enqueue(task, queue_name='default')wdyt?

  
  
Posted 3 years ago

AgitatedDove14 when I try this I get
clearml.backend_interface.session.SendError: Action failed <400/110: tasks.enqueue/v1.0 (Invalid task status (Invalid status change): current_status=in_progress, new_status=queued)> (queue=e78d2fdf2d5140b6b5c6678338c532bb, task=95082c9174a04044b25253d724362ec1)

  
  
Posted 3 years ago

I know this is not the default behavior so I’d be happy with having the option to override the repo when I call execute_remotely

  
  
Posted 3 years ago

It recognizes the main repo, but I want it to push and pull from another one (my own forked repo). AgitatedDove14

  
  
Posted 3 years ago

I want to keep the above setup, the remote branch that will track my local will be on 

fork

 so it needs to pull from there. Currently it recognizes 

origin

 so it doesn’t work because the agent then can’t find the commit.

So you do not want to push the change set ?
You can basically add the entire change set (uncomitted changes) from the last pushed commit).
In your clearml.conf, set store_code_diff_from_remote: true
https://github.com/allegroai/clearml/blob/8708967a5ef4d8529a1a5ea417672e3ebbb258d7/docs/clearml.conf#L157

Will that solve the issue ?
(obviously you can manually change the commit/repo after you call execute_remotely, but I'm trying to find a solution that avoids that)

  
  
Posted 3 years ago

is that what you want to happen ?

  
  
Posted 3 years ago

AgitatedDove14 wouldn’t the above command task.execute_remotely(queue_name=None, clone=False, exit_process=False) fail because
clone==False and exit_process==False is not supported. Task enqueuing itself must exit the process afterwards.I thought it worked earlier 😮

  
  
Posted 3 years ago

yey 🙂 notice that when executed by the agent the call execute_remotely is skipped, and so does the If statement I added (since running_locally will return False when the process is executed by the agent)

  
  
Posted 3 years ago

I already have that set to true and want that behavior. The issue is on the “committed” change set. When I push code to github I push to my fork and pull from the main/master repo (all changes go through PRs from fork to main).

Now when I use execute_remotely , whatever code does the git discovery, considers whatever repo I pull from the repo to use. But these changes haven’t necessarily been merged into main. The correct behavior would be to use the forked repo.

  
  
Posted 3 years ago

Sure LazyTurkey38 here's a nice hack for that:
` # code here
task.execute_remotely(queue_name=None, clone=False, exit_process=False)

patch the Task and actually send it for execution

if Task.running_locally():
task.update_task(task_data={'script': {'branch': 'new_branch', 'repository': 'new_repo'}})
# now to actually enqueue the Task
Task.enqueue(task, queue_name='default') You can also clear the git diff by passing "diff": "" `
wdyt?

  
  
Posted 3 years ago

This is exactly what I was looking for. I thought once you call execute_remotely the task is sent and it’s too late to change anything.

  
  
Posted 3 years ago

Yes

  
  
Posted 3 years ago
1K Views
17 Answers
3 years ago
one year ago
Tags
Similar posts