You can set a dummy step which is executed in parallel of your pre-processing step and which is set to be executed in your GPU queue, provided that your autoscaler doesn't scale back your compute before your pre-processing is complete that should do the trick
Answered
Hey Everyone, I'M Using Clearml Pipeline, I Have A Pre-Processing Step Which I Would Like To Use There Small Instance, And Processing Step Which Executed On A Gpu Queue.
The Initialisation Time Of The Gpu Instance Is Very Long. Is There A Way To Configure
Hey everyone, I'm using ClearML Pipeline, I have a pre-processing step which I would like to use there small instance, and processing step which executed on a GPU queue.
The initialisation time of the GPU instance is very long. Is there a way to configure ClearML to launch him during the pre-processing step?
717 Views
1
Answer
11 months ago
11 months ago
Tags