Do you have any insights on the missing fileserver @<1523701205467926528:profile|AgitatedDove14> ?

@<1523701205467926528:profile|AgitatedDove14> Got the overrides working with Hydra/params.batch_size thank you 🙏

None

Thanks @<1523701205467926528:profile|AgitatedDove14> happy to PR on the docs 😉

Hey @<1593051292383580160:profile|SoreSparrow36> I am trying to test that if I delete a project the S3 storage gets also deleted. But I am not sure this is even a good assumption as I haven’t found anywhere what the expected/default behaviour is. Do you happen to know anything about this? Thanks.

Update

I ran:

clearml-agent build --id <task-id> --docker <custom-docker> --log-level DEBUG --entry-point reuse_task

and got a similar problem:

Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
error: Could not fetch origin

hmmm… probably not if I don’t have a reference that clearml can update right?….

What about:

hpo_params = OmegaConf.to_object(cfg)
...
task.connect(hpo_params)

And then I use hpo_params in the code. This way I give clearml a chance to update the object.

Would this work? Thanks

Hey @<1523701205467926528:profile|AgitatedDove14> in the WebUI the hydra configuration object is under CONFIGURATION OBJECTS > OmegaConf

So should this be OmegaConf/trainer.batch_size ?

Also @<1619867994005966848:profile|HungryTurtle13> 👆

This is what I see:

Responding to my own question, in case someone else has the same issue. You have to edit the security group and enable TCP 8080.

I haven’t figure out the missing fileserver? :man-shrugging:

What I am referring to is this information about the Storage Configuration:
None

Hey @<1523701087100473344:profile|SuccessfulKoala55> just updating you here. I started from scratch, new EC2 instance, follow the installation step by step and the only change that I made was selecting rsa instead of ed255190 for the generation of the SSH key (as per github docs ), and now I my agent can connect consistently to GitHub. Just thought of p...

Thanks @<1523701205467926528:profile|AgitatedDove14> reading …

I just ran a dummy experiment logging images, plots, etc and I can see them in my server’s Web UI.

there under fileserver it should read /opt/clearml/data/fileserver

hmm….. probably simpler/cleaner if I do

hpo_params = {
'param1':cfg.param_1, ...
}

task.connect(hpo_params)

Thoughts?

if that were the case it explains why I see /opt/clearml/data/fileserver but no /mnt/fileserver ….

Will this work?

task.connect(OmegaConf.to_object(cfg))

assuming cfg is my Hydra dict

will give it a try with debug….

@<1523701087100473344:profile|SuccessfulKoala55> I changed my agent to poetry mode it and it worked like magic. Thanks Jake!

Hi @<1523701087100473344:profile|SuccessfulKoala55> it’s failing again.. I haven’t rebooted the agent or changed anything and I am able to connect with ssh with ssh -vT git@github.com on a different tmux sess.

This is the error I am seeing running the agent with the -debug flag:

Using cached repository in "/home/ubuntu/.clearml/vcs-cache/clearml-tutorial.git.e1c2351b09f3d661b6f0dbf85e92be2e/clearml-tutorial.git"
git@github.com: Permission denied (pub...

Hi @<1523701087100473344:profile|SuccessfulKoala55> thanks for your response. What I mean is that in the Web UI when you are creating a project you have storage (S3) field at the bottom of the create project pop-up, where you enter the S3 bucket that you want to associate with the project. Now, the thing is, you can’t visualize that information after the project is created, anywhere in the UI, as far as I can tell. So, it would be great to be able to see the configured bucket somewhere in...

A related question… how does the server know how to delete artifacts when the project is deleted if it doesn’t have a clearml.conf with the S3 credentials to do so?

Also @<1619867994005966848:profile|HungryTurtle13> 👆

Hi @<1523701435869433856:profile|SmugDolphin23> thanks for your answer. I am not sure that I understand. I ran a test by cloning and experiment and editing the OmegaConf object under Configuration > Hyperparameters > OmegaConf.

Unless I also change the allow_omegaconf_edit flag to True , I won’t see my changes reflected. That is my question. As a new user, it seems counterintuitive that I have to also change the flag. Does this make sense to you? Thanks.

@<1547028031053238272:profile|MassiveGoldfish6> check this:

• does your local clearml.conf should use use_credentials_chain:true ?
• Do you have the needed AWS credentials in your local environment?
• Do you have an S3 bucket as the storage for your project (did you set this up when you created the project)?
• Do your local AWS credentials give you write access to that S3 bucket?

but from a terminal I can do:

ubuntu@***:~/sw/clearml-tutorial$ git fetch --all --recurse-submodules
Fetching origin

and it works

Hei @<1523701087100473344:profile|SuccessfulKoala55> it just worked. Maybe there was some github refresh delay … not sure but thanks anyways for the debug suggestion. 👍