When I run as regular remote task it works. But when I run as a step in pipeline, it cannot access the same folder in my local machine.

Just to add, when I run the pipeline locally it works as well.

Thanks AgitatedDove14 . Specifically, I wanted to use my own clearml server and Triton. Thus, I attempted to use --engine-container-args during launch but error saying no such flag. Looked into --help but I guessed it is not updated yet.

Hi Bart, yes. Running with inference container.

Yes. The training working well with cuda

Clearml 1.1.1. Yes, i have boto3 installed too.

OK I will send here shortly.

I figured out that it maybe possible to do these
experiment_task = Task.current_task()
OutputModel(experiment_task ).update_weights(' http://model.pt ') to attach it to the ClearML experiment task.

https://clear.ml/docs/latest/docs/integrations/storage/
Try add the <path to your cert> for s3.credentials.verify.

Yup. But I happened to reinstall my server and the data is lost. And the agent continue running.

U want to share your clearml.conf here?

Yes. But I not sure what's the agent running. I only know how to stop it if I have the agent id

Nice. That should work. Thanks

Hello CostlyOstrich36 I am facing an issue now. basically i installed all necessary python packages in my docker image. But somehow, the clearml-agent does not seems to be able to detect these global packages. I don't see them in the "installed packages". Any advice?

I see. Was wondering any advantage to do it any of the ways.

I have yet to figure out how to do so, would appreciate if u could give some guidance

SuccessfulKoala55 Nope. I didn't even get to enter my name. I suspect there is some mistake in mapping the data folder.
Was using the template in https://github.com/allegroai/clearml-helm-charts to deploy.

It gets rerouted to http://app.clearml.home.ai/dashboard . with the same network error.

Hi TimelyPenguin76 , nope. I don't see any errors. That's why not sure what went wrong

This is what I got. and when I see http400 error in the console.

@<1526734383564722176:profile|BoredBat47> Just to check if u need to do update-ca-certificates or equivalent?

Do u have an example of how I can define the packages to be installed for every steps of the pipeline?

I was browsing clearml agent gihub and saw this. Isn't this for spinning up clearml-agent in a docker and perform like a daemon?

seems like it was broken for numpy version 1.24.1.
Tried with numpy 1.23.5 and it works.

CostlyOstrich36 I mean the dataset object in clearml as well as the data that is tied to this object.

The intent is to bring over to another clearlml setup and keep some form of traceability.

Thanks @<1523701205467926528:profile|AgitatedDove14> . what I could think of is to write a task that may run python subproecss to do "helm install". In those python script, we could point to /download the helm chart from somewhere (e.g. nfs, s3).

Does this sound right to u?
Anything that I was wondering is if we could pass the helm charts /files when we uses clearml sdk, so we could minimise the step to push them to the nfs/s3.

@<1523701205467926528:profile|AgitatedDove14> I still trying to figure out how to do so. Coz when I add a task in queue, clearml agent basically creates a pod with the container. How can I make a task that does a helm install or kubectl create deployment.yaml?

Can clearml-serving does helm install or upgrade? We have cases where the ml models do not come from the ml experiments in clearml. But would like to tap on clearml q to enable resource queuing.

@<1523701205467926528:profile|AgitatedDove14> I looking at a queue system which clearml q offers that allow user to queue job to deploy an app / inference service. This cam be as simple as a pod or a more complete helm chart.

A more advanced case will be to decide how long this job should run amd terminate after that. This is to improve the usage of gpu