Thank you EnviousStarfish54 !
This is very helpful!
I'm looking at Kedro and the project you shared, and a few thoughts came to mind:
I very much like the idea of using functions as "nodes" (and to extend, using notebook cells with tags as nodes). This got me thinking, I'm pretty sure we could have a similar imlmentation with ClearML. My thinking is using inspect or dill to convert the functions/cells into plain text code, automatically analyze the runtime requirements, and create a "single-script" Task from it. I'm also trying to figure out, and I might be totally wrong here, how you are supposed to use Kedro. Or in other words, is it a frame work to basically do Process(target=node_function) for you? is there anything I'm missing (well maybe you might have to add a Queue to pass some data)?
Regrading the actual issue console logging issue.
Try adding the following line to your cleaml.conf , one of the by products is that any os.fork will pass through ClearML and subprocess will be logged as well. This behavior should happen bu default, by on some cases the patching is a must.
sdk.development.report_use_subprocess = false