Thanks. The examples uses upload_artifact which stores the files in output_uri. What if I do not want to save it but simply pass to next step, is there a way to do so?

Hi TimelyPenguin76 , nope. I don't see any errors. That's why not sure what went wrong

@<1526734383564722176:profile|BoredBat47> Just to check if u need to do update-ca-certificates or equivalent?

Is there a way to troubleshoot further?

By the way, will downloading still happen if the datasets is available in the cache folder? Any specific settings to add to Dataset.get_local_copy()?

OK let me try by adding to vol mount.

@<1523701205467926528:profile|AgitatedDove14> when my codes get the clearml datasets, it stores in the cache e.g. /$HOME/.clearml/cache....

I wanted it to be in a mounted PV instead, so other pods (in same node) who needed same datasets can use without pulling again.

Hi SuccessfulKoala55 Thanks for pointing me to this repo. Was using this repo.

I didn't manage to find in this repo that if we still require to label the node app=clearml, like what was mentioned in the deprecated repo. Although from the values.yaml, the node selector is empty. Would u be able to advise?

How is the clearml data handled now then? Thanks

Example i build my docker image using a image in docker hub. In this image, i installed torch and cupy packages. But when i run my experiment in this image, the packages are not found.

Yes, I ran the experiment inside.

I was browsing clearml agent gihub and saw this. Isn't this for spinning up clearml-agent in a docker and perform like a daemon?

Nice. It is actually dataset.id .

Yes. The training working well with cuda

This is what I got. and when I see http400 error in the console.

Thanks AgitatedDove14 . Specifically, I wanted to use my own clearml server and Triton. Thus, I attempted to use --engine-container-args during launch but error saying no such flag. Looked into --help but I guessed it is not updated yet.

I see. Was wondering any advantage to do it any of the ways.

I not very sure tbh. Just want to see if this is useful....

I got SSL error few days back and I solved it by adding cert to /etc/ssl/certs and perform update-ca-certificates .

export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

Add this. Note that verify might not work with sdk.aws.s3.verify but sdk.aws.s3.credentials . Pls see the attached image.

Example:
aws {
s3 {
credentials: [
{
` ...

Hello CostlyOstrich36 I am facing an issue now. basically i installed all necessary python packages in my docker image. But somehow, the clearml-agent does not seems to be able to detect these global packages. I don't see them in the "installed packages". Any advice?

SdK meaning I run the agent using clearml-agent daemon ....
Alternatively I understand I can also run the agent using docker run allegroai/clearml-agent:latest. But I cannot figure out how to add --restart, --queue, -- gpus flag to the container

Alright. Thanks CostlyOstrich36

Thanks I just realised I didn't add --docker

By the way, how can I start up the clearml agent using the clearml-agent image instead of SDK? Do u have an example of the docker run command that includes the queue, gpus etc?

May I know where to set the cert to in env variable?

seems like it was broken for numpy version 1.24.1.
Tried with numpy 1.23.5 and it works.

Yup, was thinking of bash script.

The intent is to generate some outputs from the clearml task and thinking probably to package it into a docker image for ease of sharing to others that are not plug into our network and able to run the image directly.

https://clear.ml/docs/latest/docs/integrations/storage/
Try add the <path to your cert> for s3.credentials.verify.

I have yet to figure out how to do so, would appreciate if u could give some guidance

SuccessfulKoala55 Nope. I didn't even get to enter my name. I suspect there is some mistake in mapping the data folder.
Was using the template in https://github.com/allegroai/clearml-helm-charts to deploy.

Cool thanks guys. I am clearer now. Was confused by the obsolete info. Thanks for the clarification.

Not exactly sure yet but I would think user tag for deployed make sense as it should be a deliberated user action. And additional system state is required too since a deployed state should have some pre-requitise system state.

I would also like to ask if clearml has different states for a task, model, or even different task types? Right now I dun see differences, is this a deliberated design?

@<1523701070390366208:profile|CostlyOstrich36> This is output_uri or where do I put this url?