may be specific to fastai as I cannot reproduce it with another training using yolov5
not sure if related but clearml 1.14 tend to not "show" the gpu_type
Answered
Just Want To Post It Here Before Raising A Github Issue: There Seems To Be A Regression Bug Since Clearml 1.13.0 Where Out Training In Gpu Is 2X Slower
In Our Pipeline Based On
Just want to post it here before raising a github issue: there seems to be a regression bug since clearml 1.13.0 where out training in GPU is 2x slower
in our pipeline based on fastai , an epoch take 8s with clearml==1.12.2 and 17s with clearml==1.13.0 (also tested on 1.14.x), GPU A6000 (and 4070 Ti)
Is there any caveat that I should be aware of after 1.13 ?
109 Views
2
Answers
one month ago
one month ago
Tags