and ClearML should strive to be clear, amirite? 😄
yeah, I think I’ll go with schedule_function right now, but your proposed idea would make it even clearer.
the Task scheduler itself is a Task. What we did is we added a new parameter section on the Task (the task.connect call), so that we can later clone and modify it and use the new value in runtime
(Task.connect will put the data from the Task/UI back into the dict when the agent is running the Scheduler)
Does that make sense?
Maybe it makes sense to use schedule_function instead of schedule_task_id and then the schedule function will perform the cloning of the last task and starting the clone?
ideally, I want to hardcode, e.g. use_staging = True, enqueue it; and then via clone-edit_user_properties-enqueue in UI start the second instance (edited)
Oh I see!
Actually the easiest would be to use a Section:
` task = Task.init(...)
my_params = {'use_staging': True}
task.connect(my_params, name="General")
if my_params['use_staging']:
# do something
scheduler = TaskScheduler(...) `wdyt?
slightly related follow-up question: can I add user properties to a scheduler configuration?
Hi FiercePenguin76
Maybe it makes sense to use
schedule_function
I think you are correct. This means the easiest would be to schedule a function, and have that function do the Task cloning/en-queuing. wdyt?
As a side note , maybe we should have the ability of custom function that Returns a task ID. the main difference is that the Task ID that was created will be better logged / visible (as opposed to the schedule_function, where the fact there was a Task that was created / enqueued is not visible)
ideally, I want to hardcode, e.g. use_staging = True, enqueue it; and then via clone-edit_user_properties-enqueue in UI start the second instance
can I add user properties to a scheduler configuration?
please expand, what do you mean by user property and how one would use it?
I want to have 2 instances of scheduler - 1 starts reporting jobs for staging, another one for prod
not sure I fully get it. Where will the connection between task and scheduler appear?
I see that scheduler task UI has the capabilities to edit user properties. But I don’t see how I can read and/or write them through code