Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Have A Reporting Task I Want To Schedule Using Taskscheduler. 2 Main Input Params Are

I have a reporting task I want to schedule using TaskScheduler. 2 main input params are datetime_from and datetime_to (which is datetime.now() by default). Ideally, I want the scheduler to clone the last successful task, take its datetime_to and assign it to datetime_from of the new one. But it looks like schedule_task_id is constant in the context of a TaskScheduler. So I first find a task_id of some previous reporting task when creating a scheduler. And then I need to perform the second search for the last successful task from the reporting task itself (to get its datetime_to ). Is there any way to achieve what I want in a more clear way?

  
  
Posted 2 years ago
Votes Newest

Answers 15


the Task scheduler itself is a Task. What we did is we added a new parameter section on the Task (the task.connect call), so that we can later clone and modify it and use the new value in runtime
(Task.connect will put the data from the Task/UI back into the dict when the agent is running the Scheduler)
Does that make sense?

  
  
Posted 2 years ago

yeah, I think I’ll go with schedule_function right now, but your proposed idea would make it even clearer.

  
  
Posted 2 years ago

slightly related follow-up question: can I add user properties to a scheduler configuration?

  
  
Posted 2 years ago

yes, I’ll try it out

  
  
Posted 2 years ago

can I add user properties to a scheduler configuration?

please expand, what do you mean by user property and how one would use it?

  
  
Posted 2 years ago

not sure I fully get it. Where will the connection between task and scheduler appear?

  
  
Posted 2 years ago

LOL totally 🙂

  
  
Posted 2 years ago

Maybe it makes sense to use schedule_function instead of schedule_task_id and then the schedule function will perform the cloning of the last task and starting the clone?

  
  
Posted 2 years ago

I see that scheduler task UI has the capabilities to edit user properties. But I don’t see how I can read and/or write them through code

  
  
Posted 2 years ago

ideally, I want to hardcode, e.g. use_staging = True, enqueue it; and then via clone-edit_user_properties-enqueue in UI start the second instance

  
  
Posted 2 years ago

I want to have 2 instances of scheduler - 1 starts reporting jobs for staging, another one for prod

  
  
Posted 2 years ago

and ClearML should strive to be clear, amirite? 😄

  
  
Posted 2 years ago

Hi FiercePenguin76

Maybe it makes sense to use

schedule_function

I think you are correct. This means the easiest would be to schedule a function, and have that function do the Task cloning/en-queuing. wdyt?

As a side note , maybe we should have the ability of custom function that Returns a task ID. the main difference is that the Task ID that was created will be better logged / visible (as opposed to the schedule_function, where the fact there was a Task that was created / enqueued is not visible)

  
  
Posted 2 years ago

ideally, I want to hardcode, e.g. use_staging = True, enqueue it; and then via clone-edit_user_properties-enqueue in UI start the second instance (edited)

Oh I see!
Actually the easiest would be to use a Section:
` task = Task.init(...)
my_params = {'use_staging': True}
task.connect(my_params, name="General")
if my_params['use_staging']:
# do something

scheduler = TaskScheduler(...) `wdyt?

  
  
Posted 2 years ago

worked fine, thanks!

  
  
Posted 2 years ago