Does it wok if you remove the Task.init call?

This doesn't seem to be running inside a container...
What's the clearml-agent launch command you are using ? (i.e. do you have --docker flag)

MelancholyElk85

After I set base docker for pipeline controller task, I cannot clone the repo...

What do you mean by that?
Also, how do you set the PipelineController base_docker_image (I'm assuming the is needed to run the pipeline logic?!, is that correct?)

docstring ?
Usually the preferred way is StorageManager
https://clear.ml/docs/latest/docs/references/sdk/storage
https://clear.ml/docs/latest/docs/integrations/storage

MelancholyElk85 that looks great, let me see how quickly we can push it (I think 1.1.5 needs to be pushed very soon, I'll check if we can have it before 🙂 )

Hi @<1625303806923247616:profile|ItchyCow80>
Could you add some prints ? Is it working without the Task.init call? the code looks okay and the - No repository found, message basically says it logs it as a standalone script (which makes sense)

MelancholyElk85 if you are manually adding models OutputModel, then when you call update_weights(...) upload will start in the background (if the process ends it will wait until the upload is competed). You can also specify auto_delete_file which will delete the local copy once the upload completes

Check on which queue the HPO puts the Tasks, and if the agent is listening to these queues

basically @<1554638166823014400:profile|ExuberantBat24> you can think of hyper-datasets as a "feature-store for unstructured data"

HealthyStarfish45
No, it should work 🙂

Can I assume that if we have two agents spinning the same experiment, your code will take it from there?

Is this true ?

Can you try to set this in your clearml.conf:

agent.pip_download_cache.enabled: false

this should disable the local caching, of your wheel, I suspect there is some issue with the local cache file in windows...

This is cleaml python client, no need to change the server

GorgeousSeagull44 I think this should have worked (basically replacing all the links on the mongo DB with the new IP)

GiddyPeacock64 and you see the kale (KF) jobs in the kubeflow UI ?

Ok, but when 

nvcc

 is not available, the agent uses the output from 

nvidia-smi

 right? On one of my machine, 

nvcc

 is not installed and in the experiment logs of the agent runnin there, 

agent.cuda =

 is the version shown with 

nvidia-smi

Already added to the next agent's version 😉

Yes exactly !

tell me please, does the agent always create a virtual environment?

Yes, but it inherits from the container preinstalled system environment

is it possible to make the agent run the script in an already prepared docker container without creating a virtual environment in the container?

You can set the CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=1 environment variable

So are you saying the large file size download is the issue ? (i.e. network issues)

AFAIK that's the only way right now (see my comment here - https://clearml.slack.com/archives/CTK20V944/p1657720159903739?thread_ts=1657699287.630779&cid=CTK20V944 )
Or then if you have the ClearML paid service, I believe there is a "vaults" service, right AgitatedDove14 ?

Yep UnevenDolphin73 :)

Hi PlainSquid19
Any model stored by TF/Keras/PyTorch/Joblib will automatically appear in the artifact/models tab.
Are you asking on how to add one manually ?

When you login with user/pass in the UI the same "process" happens and you get a Token to work with, this is the same as secret/key
Since in both cases you provide credentials and get back access token, it should work
(This is of course only if you are setting user/pass manually and disabling pass_hashed as you have)

Hi JuicyOtter4
Seems like you cannot access your sever ?

Hi @<1552101447716311040:profile|SteadySeahorse58>

ValueError: Could not find queue named "services"

Did you set an agent / auto-scaler ? where is the pipeline and its components will be running ?

If you are using the "default" queue for the agent, notice you might need to run the agent with --services-mode to allow for multiple pipeline components on the same machine

Hi @<1544853695869489152:profile|NonchalantOx99>
I would assume the clearml-server configuration / access key is misconfigured in your copy of example.env

DistressedGoat23
you can now access the weights model object
pip install 1.8.1rc0
then:
` def callback(_, model_info):
model_info.weights_object # this is your xgboost object
model_info.name = "my new name"
return model_info

WeightsFileHandler.add_pre_callback(callback) `

1. yes they will! This is exactly the idea :)
2. yes it will store it as text file (as is raw text) notice the return value is the file you should open. This is because when running via agent the return file will contain the conf file from the UI. Make sense?

Hi MiniatureCrocodile39
I would personally recommend the ClearML show 😉
https://www.youtube.com/watch?v=XpXLMKhnV5k
https://www.youtube.com/watch?v=qz9x7fTQZZ8

Yes very