Unanswered
Hi All, I Have A Question Regarding Clearml Task Initialization With Multithreading. I'M Using Python'S Joblib Library And The Parallel Class To Run An Experiment In Multiple Parallel Threads. The Experiment Runs To Completion But I Get Incomplete Mllogge
Hi @<1619867994005966848:profile|HungryTurtle13>
I'm using Python's joblib library and the Parallel class to run an experiment in multiple parallel threads.
I believe joblib creates subprocesses not threads, but yes you are correct,
Basically once Task.init is called, every forked/spawned process will be automatically logged to the main process Task (you can, and probably should call either Task.init or Task.current_task() from the forked processes, but this is just a detial)
The main limitation we cannot "patch" joblib before it spwans the multiple processes so every process knows that it has a "parent" Task to report to. Does that make sense ?
121 Views
0
Answers
12 months ago
12 months ago