gotcha, thanks!

I had a bunch of training tasks each of which outputted a model. I want to apply each one of them to a specific dataset. I have a clearml task ( apply_model ) for that, which takes dataset_id and model-producing task_id as input. First time I initiate apply model by hardcoding ids and starting the run from my machine (it is then goes into cloud, when it reaches execute_remotely )

I got it working!

nope, I need to contact devops team for that, that can happen not earlier than Monday

I guess that’s because ngrok is not like a Dynamic DNS

I tried both, didn’t work

well, I first run clearml-session to start everything on the remote machine, then I close the local process (while Interactive is still running on the remote machine)

it is missing in CLI, but I was able to set external_ssh_port and external_address in GUI. It was certainly a step forward, but still failed

` Remote machine is ready
Setting up connection to remote session
Starting SSH tunnel
Warning: Permanently added '<CENSORED>' (ECDSA) to the list of known hosts.
Enter passphrase for key '/Users/jevgenimartjushev/.ssh/id_rsa': <CENSORED>

SSH tunneling failed, retrying in 3 seconds `

https://github.com/allegroai/clearml-session/issues/1

I use ngrok to connect to it

yes I can.

Then I ssh into the remote machine using ngrok hostname and tunnel the port for Jupyter

Here’s my workaround - ignore the fail messages, and manually create an SSH connection to the server with Jupyter port forwarded.

For others, who haven’t heard about ngrok:
Ngrok exposes local servers behind NATs and firewalls to the public internet over secure tunnels.

and I have no way to save those as clearml artifacts

ideally, I want to hardcode, e.g. use_staging = True, enqueue it; and then via clone-edit_user_properties-enqueue in UI start the second instance

DisgustedDove53 ?

I’m rather sure that after restart everything will be back to normal. Do you want me to invoke smth via SDK or REST while the server is still in this state?

I found this in the conf:
# Default auto generated requirements optimize for smaller requirements # If True, analyze the entire repository regardless of the entry point. # If False, first analyze the entry point script, if it does not contain other to local files, # do not analyze the entire repository. force_analyze_entire_repo: false

as I understand this: even though force=false, my script is importing another module from same project and thus triggering analyze_entire_repo

AgitatedDove14 I did exactly that.

yes, same

I think we can live without mass deleting for a while

One workaround that I see is to export commonly used code not to a local module, but rather to a separate in-house library.

“supply the local requirements.txt” this means I have to create a separate requirements.txt for each of my 10+ modules with different clearml tasks

example here: https://github.com/martjushev/clearml_requirements_demo

task_trash_trash is probably irrelevant, as the latest entry there is from Dec 2021

we certainly modified some deployment conf, but lets wait for answers tomorrow

we’ll see, thanks for your help!