Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Everyone, I'M Trying To Launch A Clearml Task With Aws Lambda. What I'D Like To Do Is Simply To Use Task.Create() To Launch A Task That Will Be Executed Remotely. The Main Problem Is That Task.Create() Seems To Be Relying On

Hi everyone, I'm trying to launch a ClearML task with AWS Lambda. What I'd like to do is simply to use Task.create() to launch a task that will be executed remotely. The main problem is that Task.create() seems to be relying on multiprocessing and apparently this doesn't work on Lambda as it is not possible to access the /dev/shm directory. I was wondering if someone already tackled this problem and how, or if it is possible to somehow disable parallelism for Task.create(). Thank you!

  
  
Posted 3 years ago
Votes Newest

Answers 4


Hi LovelyHamster1 ,

I had similar issue and used the APIClient ( from clearml.backend_api.session.client import APIClient ).

I cloned a template_task and did some changes, and enqueue it after, here is the code:

` from clearml.backend_api.session.client import APIClient

api_client = APIClient()

BASE_TASK_ID = "MY TASK ID"

Task.get_task

template_task = api_client.tasks.get_by_id(BASE_TASK_ID)

Task.clone

cloned_task = api_client.tasks.clone(task=template_task.id,
new_task_name=f"{template_task.name} from AWS lambda",
new_task_tags=template_task.tags,
new_task_comment=template_task.comment,
new_task_parent=template_task.id if not template_task.parent else template_task.parent,
new_task_project=template_task.project,
)

Do your changes here

.

.

.

Task.enqueue

all_queues = api_client.queues.get_all(name="default", only_fields=["id"])
queue_id = all_queues[0].id
api_client.tasks.enqueue(task=cloned_task.id, queue=queue_id) `I worked with this https://allegro.ai/clearml/docs/rst/references/clearml_api_ref/index.html

I also added a layer with ClearML

  
  
Posted 3 years ago

Hi TimelyPenguin76 , I tried your approach and it works, thank you! However it's a bit different to what I was trying to do: instead of cloning an existing task I'd like to specify the repository and a specific commit tag to use as it is done in Task.create. If this is possible with the API client it would be perfect

  
  
Posted 3 years ago

Editing the cloned task ?

` script = template_task.script

This will override the commit id, you can also override the branch, repository if you like -

script.version_num = COMMIT_ID
t = api_client.tasks.edit(task=t.id,
script=script) `

  
  
Posted 3 years ago

Hi TimelyPenguin76 , I used api_client.tasks.create and It works, thank you!

  
  
Posted 3 years ago
563 Views
4 Answers
3 years ago
one year ago
Tags