Is it possible in Clearml to somehow allocate resources so that maybe after running a number of Alice's tasks, Bob's task get processed (Like maybe Round robin fashion)
Hi DeliciousBluewhale87
A few options here:
set the agent with high / low priority queues. Make sure Alice pushes into low priority (aka HPO) then Bob can push into high priority when he needs. This makes a lot of sense when you have automation processes spinning many experiments. expanding (1) you could set different agents with different priorities (for example low compute machines pulling from low priority) etc. The idea is that per agent you can play around with priorities. Per agent you can pass a flag --order-fairness
which will basically pull in a round robin way from all the quques (instead of priority order). That means that if you have Alice Q and Bob Q, the agent will pull once from A then one from B and then again from A and so on. This will ensure that Bob will get a "fair" chance of executing a Task In the Enterprise edition there is actually quotas you can assign, per user/group/queue and limit Alice so it will not hog the queues 🙂