I tried it on another laptop and I also tried it on the server machine where the clearml server is deployed. I get same error. @<1523701070390366208:profile|CostlyOstrich36>

import yaml
from clearml.automation.auto_scaler import AutoScaler, ScalerConfig
from gcp_driver import GCPDriver

with open('gcp_autoscaler.yaml') as f:
    conf = yaml.load(f, Loader=yaml.SafeLoader)

    driver = GCPDriver.from_config(conf)

    conf = ScalerConfig.from_config(conf)
    autoscaler = AutoScaler(conf, driver)
    autoscaler.start()

That is the python code.

That is the configuration yaml.

@<1523701070390366208:profile|CostlyOstrich36> I don't get that.

@<1523701070390366208:profile|CostlyOstrich36> I have been exploring. The problem seems to be when the docker container is using the cached dir.

Using cached repository in "/root/.clearml/vcs-cache/****.git.0081a6bc4d7afe6adde369e6aeab9406/****.git"

When inside that directory and tries to fetch, it asks for credentials. when it clones, it doesn't.

cloning: git@github.com:****/****.git
Using user/pass credentials - replacing ssh url 'git@github.com:****/****.git' with https ...

clearml==1.14.1
That is the version.

created virtual environment CPython3.10.13.final.0-64 in 511ms
  creator CPython3Posix(dest=/root/.clearml/venvs-builds/3.10, clear=False, no_vcs_ignore=False, global=True)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
    added seed packages: pip==23.3.1, setuptools==69.0.2, wheel==0.42.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
...

or can I put something like the following in clearml.conf?

cat << EOF >> ~/clearml.conf
agent.enable_git_ask_pass=true
agent.git_user="{GIT_USER}"
agent.git_pass="{GIT_PASSWORD}"

The

agent.git_user="{GIT_USER}"
agent.git_pass="{GIT_PASSWORD}"

already existed. I only added

`agent.enable_git_ask_pass=true

configurations:
  extra_clearml_conf: 'sdk.aws.s3.region="us-west-2"
                        agent.extra_docker_arguments=["--shm-size=90g"]
                        agent.extra_docker_shell_script=["git config --global credential.helper cache --timeout=604800",]'
  extra_trains_conf: ''
  extra_vm_bash_script: ''
  queues:
    gcp-v100:
    - - gcp-v100
      - 4
    gcp-l4:
      - - gcp-l4
        - 4
    gcp-cpu:
    - - gcp-cpu
      - 4
  resource_configurations:
    gcp-v100:
    ...

The worker machines are on gcp

@<1523701070390366208:profile|CostlyOstrich36>

You are right. My colleague wrote it I think starting from the aws autoscaler.

I don't think I am on the pro version, is that a paid one?

Exactly, @<1523701070390366208:profile|CostlyOstrich36>

It works on other machines. Can I clearml-init on a virtual environment? I installed clearml in a virtual environment.

It doesn't work when I insert the credentilas individually either. I am using EC2 as clearml server.

Sorry, It just worked now. I think It was slow internet connection issue. It just went away today.

Thanks, it did

they are different tasks. I start a new task but it can be same commit sometimes.

I have started the autoscaler with python? @<1523701070390366208:profile|CostlyOstrich36>

CostlyOstrich36

I see. So, is it same thing when network is slow and there is a mistake in url?

Not this key pair. I removed all my previous key pairs and created a new one

It is an autoscaler for gcp. I think, there are unnecessary configs that were used in aws.

I will have to go for matplotlib or seaborn options.

I didn't write this conf, but it works.

Thank you Jake. is that the same for bitbucket and other repos as well? AND is there a specific part of the doc that talks about it?

Thank you so much.

Can I see visualization of for example categorical columns as bar graphs?

Pefect, thanks. I will take a look at that.