Welp, it's been a day with the new settings, and stats went up 140K for API calls

... going to check again tomorrow to see if any of that was spill over from yesterday

140K calls a day, how often are you sending scalars ? how long is it running? how many experiments are running ?

Hi MassiveBat21
CLEARML_AGENT_GIT_USER is actually git personal token
The easiest is to have a read only user/token for all the projects.
Another option is to use the ClearML vault (unfortunately not part of the open source) to automatically take these configuration on a per user basis.
wdyt?

I think you are correct the env variable is not resolved in "time". It might be it's resolved at import not at Task.init

So this is a UI visual issue?

Just to make sure I understand, running locally creates the Args/command correctly, then when actually executed on the remote machine (i.e. execute_remotely creates the correct Args/command But when the agent actually executes it) it updates back the Args/command as a list. Is that a correct description ?

Hi CheekyAnt38

However now I would like to evaluate directly my machine learning model via api requests, directly over clearml. It’s possible?

This basically means serving the model, is this what you mean?

BTW: you will be loosing the comments 😞

Why do you ask? is your server sluggish ?

Hi IrritableJellyfish76
If you are running a code that uses clearml from kubeflow, you have out of the box integration between the two, what am I missing?

Hi @<1683648242530652160:profile|ApprehensiveSeaturtle9>

I send a request to the endpoint but never unload (the gpu memory keep increasing when I infer with a new model).

They are not unloaded after the request is done. see discussion here: None
You can however remove the model from the serving session (but I do not think this is what you meant)
I'm assuming you want to run multiple models on a single GPU with not en...

LovelyHamster1 Now I see... Interesting credentials ability. Specifically all the S3 access on trains is derived from the ~/clearml.conf credentials section :
https://github.com/allegroai/clearml/blob/ebc0733357ac9ead044d0ed32d41447763f5797e/docs/clearml.conf#L73
( or the AWS S3 environment variables )

I'm not sure how this AWS feature works, I suspect it is changing the AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY variables on the ec2 instance. If this is the case, it should work out of...

JitteryCoyote63

are the calls from the agents made asynchronously/in a non blocking separate thread?

You mean like request processing on the apiserver are multi-threaded / multi-processed ?

So the thing is clearml automatically detects the last iteration of the previous run, my assumption you also add it hence the double shift.
SourOx12 could that be it?

Hmm can you try with additional configuration, next to "secure: true" in your clearml.conf, can you add "verify: false"

Okay let me check if we can reproduce, definitely not the way it is supposed to work 😞

Hi UnsightlySeagull42
Just making sure, the two scripts are on your git repo ?

I would like to put table with url links and image thumnails.

StraightParrot3 links will work inside table (your code sample looks like the correct way to add them), but I think plotly (which is the UI package that displays the table) does not support embedding images into tables 😞
When they add it, the support will be transparent and it would work as you expect

Does that help ?

And command is a list instead of a single str

"command list", you mean the command argument ?

LazyLeopard18 you can point the artifact directly to your azure object storage and have StorageManager download and cache it for you:

Hmm that is odd, could it be you are changing the sys.path ?
(What I'm assuming is happening is that it detects the packages in the PYTHONPATH and for some reason the order is different so it finds the "system" package before the "venv" package, hence the incorrect version)

or "General/artifact_name" (if you are connecting it to a dict)

Hi SkinnyPanda43
Are you trying to access the same Task or an external one ?

Thanks @<1527459125401751552:profile|CloudyArcticwolf80> ! let me see if we can reproduce it

Hi @<1569858449813016576:profile|JumpyRaven4>

task.add_requirements()

This is the problem, if you look closely this is a class method, meant for helping the Task.init better capture python packages, it does Not change the task requirements.
To do that, use " task.set_packages "

Apologies on the typo ;)
There is also a global "running_remotely" but it's not on the task

I think it is only in get_task (and by default it is true)
I think query task does not filter the

Hi FierceHamster54
I would take a look at the decorator example here
https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_decorator.py
Think of every function as a stand-alone task running on a different machine. The controller itself is the logic that creates the jobs and passes data, and the clearml agent / autoscaler does the actual orchestration

Hi ClumsyElephant70
So do you need both requirements.txt combined ?
How will the agent be able to reproduce both repo on the remote machine ?

ImmensePenguin78 this is probably for a different python version ...