yeah, I think I’ll go with schedule_function right now, but your proposed idea would make it even clearer.
and ClearML should strive to be clear, amirite? 😄
ideally, I want to hardcode, e.g. use_staging = True, enqueue it; and then via clone-edit_user_properties-enqueue in UI start the second instance (edited)
Oh I see!
Actually the easiest would be to use a Section:
` task = Task.init(...)
my_params = {'use_staging': True}
task.connect(my_params, name="General")
if my_params['use_staging']:
# do something
scheduler = TaskScheduler(...) `wdyt?
not sure I fully get it. Where will the connection between task and scheduler appear?
Maybe it makes sense to use schedule_function instead of schedule_task_id and then the schedule function will perform the cloning of the last task and starting the clone?
I see that scheduler task UI has the capabilities to edit user properties. But I don’t see how I can read and/or write them through code
the Task scheduler itself is a Task. What we did is we added a new parameter section on the Task (the task.connect call), so that we can later clone and modify it and use the new value in runtime
(Task.connect will put the data from the Task/UI back into the dict when the agent is running the Scheduler)
Does that make sense?
I want to have 2 instances of scheduler - 1 starts reporting jobs for staging, another one for prod
can I add user properties to a scheduler configuration?
please expand, what do you mean by user property and how one would use it?
Hi FiercePenguin76
Maybe it makes sense to use
schedule_function
I think you are correct. This means the easiest would be to schedule a function, and have that function do the Task cloning/en-queuing. wdyt?
As a side note , maybe we should have the ability of custom function that Returns a task ID. the main difference is that the Task ID that was created will be better logged / visible (as opposed to the schedule_function, where the fact there was a Task that was created / enqueued is not visible)
slightly related follow-up question: can I add user properties to a scheduler configuration?
ideally, I want to hardcode, e.g. use_staging = True, enqueue it; and then via clone-edit_user_properties-enqueue in UI start the second instance