The pipeline itself is also a task, so this line works in a pipeline. Task.current_task is a class method that returns the running task (pipeline in our case), then then the usual interface. BTW what are you having in the conf file ?

👍

Okay this seems correct:

pytorch=1.8.0=py3.7_cuda11.1_cudnn8.0.5_0

I can't seem to find what's the diff between the two.
Give me a second let me check if I can reproduce it somehow.

LOL @<1545216070686609408:profile|EnthusiasticCow4>
I assume this is a hidden folder?
for example datasets are hidden folders that can be viewed if you go to the settings page and turn on "show hidden folders"

Can you see that the environment is actually being passed ?

ElegantCoyote26 I don't think Keras logs it anywhere unless you have TB, so nowhere to take the data from...
In short, yes you have to have TB :)

Hi ElegantCoyote26 , yes I did 🙂
It seems cometml puts their default callback logger for you, that's it.

Nice!!!!!

hmm... try to run the trains-agent from the ml environment with "system_site_packages: true", it might do the trick. Anyhow please let me know if it worked 🙂

For example:
examples/k8s_glue_example.py --queue k8s_gpu - --namespace pod-clearml-conf ~/trains.conf --template-yaml example/base.yml

Hi @<1536518770577641472:profile|HighElk97>

Is there a way to change the smoothing algorithm?

Just like with TB, this is front-end, not really something you can control ...
That said you can report a smoothed value (i.e. via python) as additional series, wdyt ?

DeliciousBluewhale87 could you restart the pod and ssh to the Host and make sure the folder /opt/clearml/agent exists and there is not *.conf file in it ?

It should preserve the order as the order of the update back (i.e. when executed by the agent) is the same as the order of the keys (obviously py3.7+ becuase it creates dict not Ordered Dicts)

Ohh, I see now, yes that should be fixed as well 🙂

hurray 🎊

VexedCat68
delete the uploaded file, or the artifact from the Task ?

I know that there is possibility to set up some budget - for example seconds of running after which optimization stops. But is there a possibility to specify a boolean condition when work should stop?

RoundMosquito25 you mean when you reach a limit of loss<Threshold or something similar ?

ReassuredTiger98 I think it is using moviepy for the encoding... No?

👍

Hi GrievingTurkey78

How can I check the server dashboard to make sure everything is working? I have tried to access the external ip through https but the browser is not able to connect.

What do you mean by the server dashboard ?

regrading (2) see here: https://allegro.ai/docs/faq/faq/#web-auth

What do you mean by a custom queue ?
In the queues page you have a plus button, this will just create a new queue

Hi @<1641611252780240896:profile|SilkyFlamingo57>

. It is not taking a new pull from Git repository.

When you are saying it's not trying to get the latest, are you referring to a new run of the pipeline, and then the component being pulled is Not pulling the latest from the branch, is that the issue?
When you click on the component Task details (i.e. right hand side panel "Full details"), what's the commit ID you have?
Lastly, is the component running on the same machine as the prev...

clearml-agent deployment file

What do you mean by that? is that the helm of the agent ?

Thanks @<1523702652678967296:profile|DeliciousKoala34> I think I know what the issue is!
The container has 1.3.0a and you need 1.3.0 this is why it is re-downloading (I'll make sure the agent can sort it out, becuase this is Nvidia's version in reality it should be a perfect match)

ContemplativeCockroach39 unfortunately No directly as part of clearml 😞
I can recommend the Nvidia triton serving (I'm hoping we will have the out-of-the-box integration soon)
mean while you can manually run it , see docs:
https://developer.nvidia.com/nvidia-triton-inference-server
docker here
https://ngc.nvidia.com/catalog/containers/nvidia:tritonserver

each of it gets pushed as a separate Model entity right?

Correct

But there’s only one unique model with multiple different version of it

Do you see multiple lines in the Model repository ? (every line is an entity) basically if you store it under the same local file, it will override the model entry (i.e. reuse it and upgrade the file itself), otherwise you are creating a new model, "version" will be progress in time ?

"warm" as you do not need to sync it with the dataset, every time you access the dataset, clearml will make sure it is there in the cache, when you switch to a new dataset the new dataset will be cached. make sense?

Sounds like something very similar, I'll try to use it,

You can set it per container with -e CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=1

Or add it here:
https://github.com/allegroai/clearml-agent/blob/51eb0a713cc78bd35ca15ed9440ddc92ffe7f37c/docs/clearml.conf#L149
extra_docker_arguments: ["-e", "CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=1"]

Hi UnevenOstrich23
--cpu-mode means no GPU's are passed to the Tasks it executed.
--services-mode means that instead of the agent running a single job at a time, it will spin as many jobs as you need on the same machine

See here:
https://pip.pypa.io/en/stable/user_guide/#environment-variables

Pass these environment variables as part of the YAML template you are using with the k8s.
Should work for both 🙂