Unanswered
Hey Hey! My Team And I Are Currently Testing Using Clearml Agents For Running Experiments, So Far It Has Been Great And We Really Love The Whole Clearml Ecosystem!! However, There Is Something I Don'T Quite Understand. Basically We Have Two Clusters, A An
Hi @<1673863788857659392:profile|HomelyRabbit25> , in the k8s agent setup, and agent will pick up a task and create a pod for it as soon as it's able to. To limit this behavior, you can set a limit on the number of pods the agent can apply. For example, if each experiment uses a single GPU and cluster A has 8 GPUs, it would make sense to limit the number of pods (using the maxPods setting) for agent A to 8...
115 Views
0
Answers
9 months ago
9 months ago