ok, it is solved with the force_git_root_python_path: true in clearml.conf

no, i just commented it and it worked fine

ok, thanks jake
what will be the fastest fix for it?

im trying to figure out
i'll play with it a bit and let you know

the successful, which is aborted for some reason (but at least the enviorment is setup correctly)
the end of it is :

• urllib3==1.26.15
• virtualenv==20.23.0
• wcwidth==0.2.6
• Werkzeug==2.3.2
• widgetsnbextension==4.0.7
• xgboost==1.7.5
• yarl==1.9.2
  Environment setup completed successfully
  Starting Task Execution:
  2023-04-29 21:41:02
  Process terminated by user

ignore it, I didn't try and read everything you said so far, I'll try again tomorrow and update this comment
oh, so then we're back to the old problem, when i am using
weights_filename, and it gives me the error
Failed uploading: cannot schedule new futures after interpreter shutdown

from which we run the task

create a queue named services (and subscribe a worker to it)

hey, matrin
this script actuall does work

ok martin, so what i am having troubles with now is understanding how to save the model in our azure blob storage, what i did was to specify:

upload_uri = f'

'
output_model.update_weights(register_uri=model_path, upload_uri=upload_uri, iteration=0)

but it doesn't seem to save the pkl file (which is the model_path) to the storage

yeah

@<1523701070390366208:profile|CostlyOstrich36>
hey john, let us know if you need any more information

thank you! i'll try it

WebApp: 3.16.3-949 • Server: 3.16.1-974 • API: 2.24

that's the one, I'll add a comment (I didn't check the number of connections it opens, so idk the right number)

do you want the entire log files? (it is a pipeline, and i can't seem to find the "Task" itself, to download the logs)

@<1523701205467926528:profile|AgitatedDove14> hey martin, i deleted the task.mark_completed() line
but still i get the same error,
could it possibly be something else?

so i think debian (and python 3.9)

Hey john, i thought this was the end of it, but apperantly the dataset was uploaded in the end

but why does it matter if i ran it on a remote agent?

it is installed as a pip package
but i am not using it in the code

(im running it on docker)

i can send you our pipeline file and task

(still doesn't work)

yes it does work.
looking at the logs, i see that clearml runs the docker with a gpu flag
idk why or if its related

only sometimes, the pipeline runs using local machines

another question: if i save heavy artifcats, should my services worker ram be at least as high? (or is it enough for the default queue workers to have that)

we use the clearml hosted server, so i don't know the version

for some reason the agent tries to install the locally installed pip packages

?