Hi LazyFish41
Could it be some permission issue on /home/quetalasj/.clearml/cache/ ?

Hi SarcasticSparrow10
I think the default search is any partial match, let me check if there is a way to do some regexp / wildcard

Hi RattySeagull0

I'm trying to execute trains-agent in docker mode with conda as package manager, is it supported?

It should, that said we really do not recommend using conda as package manager (it is a lot slower than pip, and can create an environment that will be very hard to reproduce due to internal "compatibility matrix" of conda, that might be changing from one conda version to another)

"trains_agent: ERROR: ERROR: package manager "conda" selected, but 'conda' executable...

ItchyJellyfish73
Unfortunately this needs backend support, and only available in the enterprise version, what is your use case for it? (It was designed to allow out of the box bare-metal multi gpu dynamic allocation, think DGX with 8 GPUs that instead of spinning down agents when you want to change the queue->num-gpu mapping you can do it on the fly)

Should be under Profile -> Workspace (Configuration Vault)

ConvolutedSealion94 Let me try to explain how it works, I hope this will help in debugging.
There are two different entities here
Clearml-server: In this context clearml server acts as a control-plane, it stores configuration on the different endpoints, models, preprocessign code etc. It does Not perform any compute or serving clearml-serving-inference https://github.com/allegroai/clearml-serving/blob/e09e6362147da84e042b3c615f167882a58b8ac7/docker/docker-compose-triton-gpu.yml#L77 . This ...

Hi MagnificentPig49 unfortunately it's only in the trains-server docker, we are working on making it "presentable" and uploading it to it's repo.
It's written in Angular (v8 I think). Do you want to help out, it will definitely incentive the guys to tidy up the code and upload it :)

MagnificentPig49 quick update, front-end guys updated me that with the next trains-server update they will have the web client code available on the repository , ETA probably mid May or so :)

I started running it again and it seems to have passed the phase where it failed last time

Yey!

Yes it is a common case....

I have the feeling ShinyLobster84 WackyRabbit7 you are not alone in this one 🙂 let me make sure we change the default value of Yes it is a common case to False, so the code looks cleaner

Ohh, sorry 🙂
:param run_pipeline_steps_locally: (default False) If True, run the pipeline steps themselves locally as a subprocess (use for debugging the pipeline locally, notice the pipeline code is expected to be available on the local machine)

I can definitely feel you!
(I think the implementation is not trivial, metrics data size is collected and stored as commutative value on the account, going over per Task is actually quite taxing for the backend, maybe it should be an async request ? like get me a list of the X largest Tasks? How would the UI present it? As fyi, keeping some sort of book keeping per task is not trivial either, hence the main issue)

Hmm that is odd.
Can you verify with the latest from GitHub?
Is this reproducible with the pipeline example code?

Like get the tasks that uses the most metrics API?

Yes, I find myself trying to select "points" on the overview tab. And I find myself wanting to see more interesting info in the tooltip.

Yep that's a very good point.

The Overview panel would be extremely well suited for the task of selecting a number of projects for comparing them.

So what you are saying, this could be a way to multi select experiments for detailed comparison (i.e. selecting the "dots" on the overview graph), is this what you had in mind?

Hi @<1571308003204796416:profile|HollowPeacock58>

parameters = task.connect(config, name='config_params')

It seems that your DotDict does not support the python copy operator?
i.e.

from copy import copy
copy(DotDict())

fails ?

What about output_uri?

If you are using StorageManager directly, output_uri is not relevant

Can you share the storagemanager usage, and error you are getting ?

So you are uploading a local file (stored in a Dataset) into GS bucket? may I ask why ?
Regrading usage (I might have a typo but this is the gist):
torageManager.upload_file( local_file=separated_file_posix_path, remote_url=remote_file_path + separated_file_posix_path.relative_to(files_rgb) )Notice that you need to provide the full upload URL (including path and file name to be used on your GS storage)

Hi @<1575656665519230976:profile|SkinnyBat30>
Streamlit apps are backend run (i.e. the python code drives the actual web app)
This means running your Tasks code and exposing the web app (i.e. http) streamlit.
This is fully supported with ClearML, but unfortunately only in the paid tiers 😞
You can however run your Task with an agent, make sure the agent's machine is accessible and report the full IP+URL as a hyper-parameter or property, and then use that to access your streaml...

Hi RoundMosquito25
This is a bit old but probably a good start:
https://clear.ml/blog/stacking-up-against-the-competition/
tl;dr
ClearML advantages (at least a few I can think of)
Scales way better Enables out of the box experiment orchestration (i.e. remote execution etc) Data management Nicer UI Full RestAPI Full MLops platform Model serving Query-able model repositoryProbably more 🙂

What's your clearml version (python and server) ?

It seems that once the job as completed once, it doesn't accept any new report...

completed can be forced, published cannot ...
What's the error you are getting ?

@<1523711619815706624:profile|StrangePelican34> are you saying that after the " with " block the task is marked completed? how is that possible? is this done manually ?

Hi UnevenHorse85

As far as I understand, users use logins and passwords specified in config/apiserver.conf to access webserver UI and key/secret key from their local ~/clearml.conf to access apiserver.

Correct 🙂

access apiserver. What is the use of all other security keys

To be able to configure the SDK client (i.e. clearml package) from OS environment and not clearml.conf file

Correct 🙂

When are those keys used?

They are the default keys for internal access, basically just make up something, otherwise someoune could access the server with the default keys

Oh I see, these are to secure your server (basically we recommend you replace the default key/secret 🙂 )
Make sense ?

Sen the full Task log, you can DM it if it is easier

are you referring to 

extra_docker_shell_

scrip

t

Correct

the thing is that this runs before you create the virtual environment, so then in the new environment those settings are no longer there

Actually that is better, because this is what we need to setup the pip before it is used. So instead of passing --trusted-host just do:
` extra_docker_shell_script: ["echo "[global] \n trusted-host = pypi.python.org pypi.org files.pythonhosted.org YOUR_S...

🤞

but I'd prefer to have a new instance deployed for each new experiment and that it also terminates when no new experiments are queued

I'm not objecting, just wondered on the rational behind the decision 🙂
Back to the AWS autoscaler:
Basically if you have the services-agent running on your cluster, it will just run the aws-autoscaler for you 🙂
The idea of the service-agent is to run logic/monitoring Tasks suck as the aws autoscaler. Notice that service-mode means multiple job per...