hey there, it happens in the task pod

yes but its limited

kubernetes cluster

Python 3.7.10 and clearml==1.7.0

Hi BoredHedgehog47 ! What version of ClearML and Python3.7 are you using?

It only happens in the clearml environment, works fine local.

Hi BoredHedgehog47
what do you mean by "in the clearml environment" ?

BoredHedgehog47 could it be "python" python points to python 2.7 inside your container, as opposed to python3 on your machine
(this error is python2 trying to run python 3 code)
https://stackoverflow.com/questions/20555517/using-multiple-versions-of-python
"Training classifier with command:\n python -m sfi.imagery.models.bbox_predictorv2.train

Do you have the full exception trace?

hmm that does look really helpful! Let me see if I can fix this using that info! thankyou!

Sure thing

Yes that fixed it, thanks!

It seems like the clearml python sdk might have issues when a subprocess is opened?

Basically, when I have the clearml sdk imported into my python module and I Task.init I'm getting that error only when running the experiment in the task pod.

{"asctime": "2022-09-28 18:45:55,353", "levelname": "INFO", "name": "root", "module": "ldc_train_end_to_end", "threadName": "MainThread", "message": "Training classifier with command:\npython -m sfi.imagery.models.bbox_predictorv2.train ./sfi/imagery/models/training/train_config.json", "filename": "ldc_train_end_to_end.py", "funcName": "train_model"} File "/usr/lib64/python3.7/site.py", line 177 file=sys.stderr) ^ SyntaxError: invalid syntax