@<1523701070390366208:profile|CostlyOstrich36>
hey john, let us know if you need any more information

do you want the entire log files? (it is a pipeline, and i can't seem to find the "Task" itself, to download the logs)

that's the one, I'll add a comment (I didn't check the number of connections it opens, so idk the right number)

from which we run the task

ok martin, so what i am having troubles with now is understanding how to save the model in our azure blob storage, what i did was to specify:

upload_uri = f'

'
output_model.update_weights(register_uri=model_path, upload_uri=upload_uri, iteration=0)

but it doesn't seem to save the pkl file (which is the model_path) to the storage

why those library need to run on a pipelinecontroller task, this task requires no libraries at all

i'll send you the file in private

just a sec

plus, is there an option to configure the agent configuration? for example we are using:

force_git_root_python_path: true

can we do it there as well?

create a queue named services (and subscribe a worker to it)

ok, it is solved with the force_git_root_python_path: true in clearml.conf

i need to read and write, i do have access from genesis autoscaler when i set off all firewall rules. but this is not recommend by microsoft.

I need to add specific firewall rules for the genesis machines, to allow them to authorize to my azure blob storage

i updated to 1.10
i am uploading the model inside the main() function, using this code:

model_path = model_name + '.pkl'
with open(model_path, "wb") as f:
    pickle.dump(prophet_model, f)

output_model.update_weights(weights_filename=model_path, iteration=0)

basically, only test.py need the packages, but for somereason pipeline_test installs them as well

thanks for the help 🙂

@<1523701205467926528:profile|AgitatedDove14>

it worked! thank you 🙂

ok so, idk why it helped, but setting base_task_id
instead of base_task_name in the pipe.add_step
function, seems to overcome this

im trying to figure out
i'll play with it a bit and let you know

why doesn't it try to use ssh as default? the clearml.conf doesn't contain user name and password

yeah

we use the clearml hosted server, so i don't know the version

martin*

hey martin thanks for the reply.
im doing the calling at the main function

so i think debian (and python 3.9)

ok so i accidentally (probably with luck) noticed the max_connection: 2 in the azure.storage config.
canceled that, and so now everything works

yes it does work.
looking at the logs, i see that clearml runs the docker with a gpu flag
idk why or if its related

the successful, which is aborted for some reason (but at least the enviorment is setup correctly)
the end of it is :

• urllib3==1.26.15
• virtualenv==20.23.0
• wcwidth==0.2.6
• Werkzeug==2.3.2
• widgetsnbextension==4.0.7
• xgboost==1.7.5
• yarl==1.9.2
  Environment setup completed successfully
  Starting Task Execution:
  2023-04-29 21:41:02
  Process terminated by user

not using it in the code

it is installed as a pip package
but i am not using it in the code