
Reputation
Badges 1
41 × Eureka!I have a strong attachment to a workflow based on CLI, nice zsh auto-suggestions, Hydra and the like. Hence why I moved away from dvc š
CLI doesnāt care about the state of my git repo right?
it finally finished no worries
created a new dataset 5GB, no update since 20 mins, is that normal?
ā¦ but I have a feeling they will not give me the āinstant venv activationā behavior Iām looking for.
should I nuke the .clearml/cache
I mean it is in Pip mode and the agent installs deps from git repo that it pulls
Thanks for the quick response . Will look into this later , I think I understand
Yes after installing , it listed the installed packages in the console , with version of each
I guess I follow these steps on a GCP instance?
https://clear.ml/docs/latest/docs/clearml_agent
I would also be interested in a GCP autoscaler, I did not know it was possible/available yet.
no containers for me š
So if I do this in my local repo, will it mess up my git state, or should I do it in a fresh directory?
So if I want to train with a remote agent on a remote machine, I have to:
spin up clearml-agent on the remote create a dataset using clearml-data, populate with dataā¦ from my local machine use clearml-data to upload data to google gs:// bucket modify my code so it accesses data from the dataset as here https://clear.ml/docs/latest/docs/clearml_data/clearml_data_sdk#accessing-datasetsAm I understanding right?
Dataset.get
works fine from python script, it pulls in the data into cache. Just the cli seems broken
I see, so thereās no way to launch a variant of my last run (with say some config/code tweaks) via CLI, and have it re-use the cached venv?
I think I am missing one part ā which command do I use on my local machine, to indicate the job needs to be run remotely? Iām imagining something likeclearml-remote run python3 my_train.py
AgitatedDove14 thanks yes I assume I would follow these instructions:
https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_gcp
got it, nice, thanks
thanks, so I got clearml-task working, sent to a queue and the agent on gcp picked it up. I had a question ā for a job that runs on the order of minutes, itās not worth re-creating the whole python virtual env from scratch on the remote (that itself takes 5mins). So is the --folder
` option meant for running it in an existing folder in an existing virtual env?
(and a way to specify which remote server)