My only suggestion is to maybe route it into a "debug_pipeline" project and then just delete it altogether when development is done? 🙂

can you please run:
nslookup app.clear.ml

Hmm, can you give a small code snippet of the save code? Are you using a wandb specific code? If so it makes sense we don't save it as we only intercept torch.save() and not wandb function calls

I think that's a hydra issue 🙂 I was able to reproduce this locally. I'll see what can be done

PricklyRaven28
You mean that even though you called Task.init(), nothing will show in the UI?

Hi ZanyPig66 , do you want to have an agent per GPU? If so just add --gpus and specify the GPU number (0 or 1) that would be associated with this worker

ZanyPig66 , the 2 agents can run from the same ubuntu account and use the same clearml.conf. if you want each to have its own configuration file just add --config-file PATH_TO_CONF_FILE and it would take another config file. Makes sense?

Well...I'll make sure we do something about it 🙂

GiganticTurtle0 So 🙂 had a short chat with one of our R&D guys. ATM, what you're looking for isn't there. What you can do is use OutputModel().update_weights_package(folder_here) and a folder will be saved with EVERYTHING in it. Now I don't think it would work for you (I assume you want to donwload the model all the time, but artifacts just some times, and don't want to download everything all the time) but it's a hack.
Another option is to use model design field to save links to a...

So all training machines will be exposed to the server?

Yeah, makes sense. We actually thought that the "best practice" would be to launch the "actual code" (as opposed to the pipeline controller) from agents. But obviously we were wrong, or at least it doesn't cover the fact that a lot of the time, code is being written for debugging. So yeah, that's where we're at, ATM

That would simplify things 🙂

@<1590514584836378624:profile|AmiableSeaturtle81> yeah I can see what you mean. So you reuploaded everything from the ClearML file server into S3 and just changed the links?

Thanks MotionlessMonkey27 , we're looking into that! Thanks for the info

Yup indeed! Let us know how it goes!

Cool!

Hi OutrageousSheep60 , The plan is to release this week \ early next week a version that solves this.

VexedCat68 you mean the artifact in the previous step is called "merged_dataset_id"? Is it an artifact or is it a parameter? And what issues are you having with accessing the parameter?

EnviousStarfish54 VivaciousPenguin66 Another question if we're in a sharing mood 😉 Do you think a video \ audio session with one of our experts, where you present a problem you're having (let's say large size of artifacts) and he tries to help you, or even can give some example code \ code skeleton. Would something like that be of interest? Would you spend some time in such monthly session?

Am I doing something differently from you?

you can do this