this is great… so it looks like best to do it in a new dir

… but I have a feeling they will not give me the “instant venv activation” behavior I’m looking for.

I use a CLI arg remote=True so depending on that it will run locally or remotely.

Thanks, I guess I need to have a bucket under Cloud Storage?

But “cloning” via UI runs an exact copy of the code/config, not a variant, unless I edit those via UI (which is not ideal). So it looks like the following workflow that is trivial to do locally is not possible via remote agents:
run exp tweak code/configs in IDE, or tweak configs via CLI have it re-rerun in exact same venv (with no install overhead etc)
So maybe the remote agents are more meant for enqueuing a whole collection of settings (via code) and checking back in a few hours (in which ...

I think I’m starting to “get” this 🙂

Dataset.get works fine from python script, it pulls in the data into cache. Just the cli seems broken

I have been using 1.6.2

I would also be interested in a GCP autoscaler, I did not know it was possible/available yet.

I guess I follow these steps on a GCP instance?
https://clear.ml/docs/latest/docs/clearml_agent

thanks, so I got clearml-task working, sent to a queue and the agent on gcp picked it up. I had a question — for a job that runs on the order of minutes, it’s not worth re-creating the whole python virtual env from scratch on the remote (that itself takes 5mins). So is the --folder ` option meant for running it in an existing folder in an existing virtual env?

I used
task.execute_remotely(queue_name=..., clone=True)and indeed it instantly activates the venv on the remote. I assume clone=True is fine

should I nuke the .clearml/cache

The cached venv path .clearml/venvs-cache is nowhere mentioned in the logs

will try what you suggested above

CLI doesn’t care about the state of my git repo right?

A quick note for others who may visit this… it looks like you have to do:
Task.force_requirements_env_freeze(force=True, requirements_file="requirements.txt")to ensure any changes in requirements.txt are reflected in the remote venv

got it, nice, thanks

created a new dataset 5GB, no update since 20 mins, is that normal?

(and a way to specify which remote server)

Great, and this would show up in the description column in the dashboard ?

👍

I think I am missing one part — which command do I use on my local machine, to indicate the job needs to be run remotely? I’m imagining something like
clearml-remote run python3 my_train.py

didn’t help

AgitatedDove14 thanks yes I assume I would follow these instructions:
https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_gcp

So if I want to train with a remote agent on a remote machine, I have to:
spin up clearml-agent on the remote create a dataset using clearml-data, populate with data… from my local machine use clearml-data to upload data to google gs:// bucket modify my code so it accesses data from the dataset as here https://clear.ml/docs/latest/docs/clearml_data/clearml_data_sdk#accessing-datasetsAm I understanding right?

no containers for me 😁
So if I do this in my local repo, will it mess up my git state, or should I do it in a fresh directory?

it finally finished no worries

thanks

saas