While creating a GCP credentials using None
What values should I insert in the following step so that the autoscaler has access, as of now I left this field blank

Can you tell me how clearml get access to my repo even if I didn't pass any information about it?

Do I need not make changes into clearml.conf so that it doesn't ask for my credentials or is there another way around

Hey, so I am able to spin up the GCP instance using the autoscaler, I wanted to confirm one thing does the autoscaler spins up the agent automatically in the VM or do I need to add the script for that to the bash script

is there a way that there is only one environment for the whole pipeline?

how's that?

so inside /Users/adityachaudhry/.clearml/venvs-builds.1/3.10/task_repository/ I have my git repo, I have one component that make a dataset directory inside this git repo, but when the other component starts executing this dataset directory is not there

So, one of my tasks requires GCP credentials json file, is there a way that I can pass in the json file and set the environment variable for that?

Well the VM is running in the default docker nvidia/cuda:10.2-cudnn7-runtime-ubuntu18.04, but it's not spinning up the agent when the VM is intialized

I am not able to see cu117 there

I was able to set up a GCP VM manually earlier, like without the autoscaler

so you mean when i ssh into my VM i need to do a git clone and then spin up the agent, right?

I think I got it resolved

Also @<1523701087100473344:profile|SuccessfulKoala55> when autoscaler spins up my GCP instance, when I look inside it I am not able to find the clearml.conf file, does it not install clearml automatically when it spins up the VM?

When the package installation is done in the task

Hey, @<1523701070390366208:profile|CostlyOstrich36> @<1523701087100473344:profile|SuccessfulKoala55> I would very much appreciate it if you could help me with this

Heyy guys, I was able to run the pipeline using autoscaler, thanks to @<1523701070390366208:profile|CostlyOstrich36> @<1523701087100473344:profile|SuccessfulKoala55> for all your help and suggestions!!

So funny thing I was making a typo while writing the GPU type, I was writing NVIDIA T4 instead of nvidia-tesla-t4

So the issue I am facing is, I am running the pipeline controller task on my local system agent and the steps of the pipeline on an agent running on GCP VM, the first step of the pipeline is failing showing clearml_agent: ERROR: Failed cloning repository.

Ok so here's what I want to do, I want to export Google Application credentials to my docker container. Here's what I have tried so far

agent.extra_docker_shell_script: [
"echo -E '{
  'type': 'xxx',
  'project_id': 'xxx',
  'private_key_id': 'xxx',
  ....
}' > google-api-key.json", "export GOOGLE_APPLICATION_CREDENTIALS=google-api-key.json"]

This didn't work, is there a way that I can set up this environment variable in my docker container?

Self hosted

File "/opt/conda/envs/bumlo/lib/python3.10/site-packages/clearml/binding/artifacts.py", line 745, in upload_artifact
pickle.dump(artifact_object, f)
_pickle.PicklingError: Can't pickle <class 'mongoengine.base.metaclasses.samples.6627e5ecc60879fe5e49cee6'>: attribute lookup samples.6627e5ecc60879fe5e49cee6 on mongoengine.base.metaclasses failed

Thanks!

dataset = fo.Dataset.from_dir(
labels_path=labels_path,
dataset_type=fo.types.COCODetectionDataset,
label_field="ground_truth",
use_polylines=True
)
task.upload_artifact(
name="Dataset",
artifact_object=dataset,
)

Note: switching to 'commit_id'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting ...

@<1523701205467926528:profile|AgitatedDove14> I was able to resolve that, but now I am having issues with fiftyone, it's showing me the following error

import fiftyone as fo
File "/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/fiftyone/init.py", line 25, in <module>
from fiftyone.public import *
File "/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/fiftyone/public.py", line 15, in <module>
_foo.establish_db_conn(config)
File "/root/.clearml...

Oh ok, thanks though!!

yeah