This is being started as a command line script.
Also tried saving the model with:task.set_model_config(cfg.model) task.update_output_model("best_model.onnx")
But got the same exception,
So downgrading to python 3.8 would be a workaround?
Weird issue, I'll make sure we fix compatibility with python 3.9
Hi SkinnyPanda43
cannot schedule new futures after interpreter shutdown
This seems like a strange exception...
What's the setup here ? jupyter notebook ? how is the interpreter down ?
Is this example working for you?
https://github.com/allegroai/clearml/blob/master/examples/reporting/model_config.py
Its a S3 bucket, it is working since I am able to upload models before this call and also custom artifacts on the same script.
Ubuntu 18.04
Python: 3.9.5
Clearml: 1.0.4
I get the same error:
⋊> /d/c/p/c/e/reporting on master ◦ python model_config.py (longoeixo) 17:48:14
ClearML Task: created new task id=xxx
ClearML results page: xxx
Any model stored from this point onwards, will contain both model_config and label_enumeration
2021-07-05 17:48:18,463 - clearml.storage - INFO - Starting upload: my_best_model.bin => xxx/models/my_best_model.bin
2021-07-05 17:48:18,464 - clearml.Task - INFO - Waiting for repository detection and full package requirement analysis
2021-07-05 17:48:18,466 - clearml.storage - ERROR - Failed uploading: cannot schedule new futures after interpreter shutdown
2021-07-05 17:48:18,466 - clearml.storage - ERROR - Exception encountered while uploading Upload failed
2021-07-05 17:48:18,466 - clearml.Task - INFO - Failed model upload
2021-07-05 17:48:19,373 - clearml.Task - INFO - Finished repository detection and package analysis
What's the output_uri
you are passing ?
And the OS / Python version?
SkinnyPanda43 issue verified, this seems to be related to python 3.9 and subprocesses.
Let me check what we can do