Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Are There Instructions Somewhere On How I Can Use Clearml-Agent To Run Jobs On My Google-Cloud Compute Engine?

Are there instructions somewhere on how I can use clearml-agent to run jobs on my google-cloud compute engine?

  
  
Posted one year ago
Votes Newest

Answers 13


HurtWoodpecker30 currently in the open source only AWS is supported, I know the SaaS pro version supports it (I'm assuming enterprise as well).
You can however manually spin an instance on GCP and launch an agent on the instance (like you would on any machine)

  
  
Posted one year ago

I guess I follow these steps on a GCP instance?
https://clear.ml/docs/latest/docs/clearml_agent

  
  
Posted one year ago

Exactly !

  
  
Posted one year ago

👍

  
  
Posted one year ago

So if I want to train with a remote agent on a remote machine, I have to:
spin up clearml-agent on the remote create a dataset using clearml-data, populate with data… from my local machine use clearml-data to upload data to google gs:// bucket modify my code so it accesses data from the dataset as here https://clear.ml/docs/latest/docs/clearml_data/clearml_data_sdk#accessing-datasetsAm I understanding right?

  
  
Posted one year ago

Yes, which looks like a lot, but you only need to d that once.
Auto scheduler would make (1) redundant (as it would spin the instance up/down based on the jobs in the queue)

  
  
Posted one year ago

I think I am missing one part — which command do I use on my local machine, to indicate the job needs to be run remotely? I’m imagining something like
clearml-remote run python3 my_train.py

  
  
Posted one year ago

AgitatedDove14 thanks yes I assume I would follow these instructions:
https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_gcp

  
  
Posted one year ago

got it, nice, thanks

  
  
Posted one year ago

thanks, so I got clearml-task working, sent to a queue and the agent on gcp picked it up. I had a question — for a job that runs on the order of minutes, it’s not worth re-creating the whole python virtual env from scratch on the remote (that itself takes 5mins). So is the --folder ` option meant for running it in an existing folder in an existing virtual env?

  
  
Posted one year ago

... indicate the job needs to be run remotely? I’m imagining something like

clearml-task and you need to specify the queue to push your Task into.
See here: https://clear.ml/docs/latest/docs/apps/clearml_task

  
  
Posted one year ago

(and a way to specify which remote server)

  
  
Posted one year ago

Actually, no. This is ti spin the clearml-server on GCP, not the agent

  
  
Posted one year ago
589 Views
13 Answers
one year ago
one year ago
Tags