If there was a "debug mode" for viewing the stack trace before the crash that would've been most helpful...
ZealousSeal58 are you running locally or using a ClearML agent?
SuccessfulKoala55 Locally, I think. Also, I forgot to add that the training script is run on the same machine that's running the ClearML server.
And when exactly are you getting the "user aborted" message)?
How do you start the process (are you manually running it, or is it an agent, or maybe pycharm?)
Can you provide the full log ?
Hi, SmallDeer34 Unfortunately no. The code stack is pretty substantial, and I'm not sure what portion of it would be useful.
Hi ZealousSeal58
What's the clearml version you are using ?
If there was a "debug mode" for viewing the stack trace before the crash that would've been most helpful...
import traceback traceback.print_stack()
Yes, my apologies. I should provide you a more comprehensive log. I am working on pinpoint the line that triggers this message. It behaves almost like a segfault, but I'll do my best to locate the error prone line. Sorry for the incomplete picture -- I'll post an update today.