@<1523701205467926528:profile|AgitatedDove14> when my codes get the clearml datasets, it stores in the cache e.g. /$HOME/.clearml/cache....

I wanted it to be in a mounted PV instead, so other pods (in same node) who needed same datasets can use without pulling again.

U want to share your clearml.conf here?

Is there a way to troubleshoot further?

Do u have an example of how I can define the packages to be installed for every steps of the pipeline?

And just a suggestion which maybe I can post in GitHub issue too.
It is not very clear what are the purpose of the project name and name, even after I read the --help. Perhaps this is something that can be made clearer when updating the docu?

Nice. It is actually dataset.id .

It gets rerouted to http://app.clearml.home.ai/dashboard . with the same network error.

Hi CostlyOstrich36 I have run this task locally at first. This attempt was successful.

When I use this task to run in a pipeline (task was run remotely), it cannot find the external package. This seems logical but I not sure how to resolve this.

Ok. Can I check that only the main script was stored in the task but not the dependent packages?

I guess the more correct way is to upload to some repo where the remote task can still pull from it?

SuccessfulKoala55 i tried comment off fileserver, clearml dockers started but it doesn't seems to be able to start well. When I access clearml via webbrowser, site cannot be reached.

Just to confirm, I commented off these in docker-compose.yaml.

apiserver:
command:
- apiserver
container_name: clearml-apiserver
image: allegroai/clearml:latest
restart: unless-stopped
volumes:
- /opt/clearml/logs:/var/log/clearml
`...

May I know where to set the cert to in env variable?

@<1526734383564722176:profile|BoredBat47> Just to check if u need to do update-ca-certificates or equivalent?

It return false. Just to share abit more, I have the requirements.txt in gitlab with my codes and are in folders. Do I need to provide a gitlab path?

This is what I got. and when I see http400 error in the console.

@<1523701205467926528:profile|AgitatedDove14> do u mean not using helm but fill up the values and install with the yaml files directly? E.g. kubectl apply ...

OK I will send here shortly.

By the way, how can I start up the clearml agent using the clearml-agent image instead of SDK? Do u have an example of the docker run command that includes the queue, gpus etc?

Ah I think I was not very clear on my requirement. I was looking at porting project level, not entire clearml data over. Is it possible instead?

Thanks

By the way, will downloading still happen if the datasets is available in the cache folder? Any specific settings to add to Dataset.get_local_copy()?

Thanks AgitatedDove14 . Specifically, I wanted to use my own clearml server and Triton. Thus, I attempted to use --engine-container-args during launch but error saying no such flag. Looked into --help but I guessed it is not updated yet.

I see. Was wondering any advantage to do it any of the ways.

I was browsing clearml agent gihub and saw this. Isn't this for spinning up clearml-agent in a docker and perform like a daemon?

https://clear.ml/docs/latest/docs/integrations/storage/
Try add the <path to your cert> for s3.credentials.verify.

Hello CostlyOstrich36 I am facing an issue now. basically i installed all necessary python packages in my docker image. But somehow, the clearml-agent does not seems to be able to detect these global packages. I don't see them in the "installed packages". Any advice?

Yes. But I not sure what's the agent running. I only know how to stop it if I have the agent id

SuccessfulKoala55 Nope. I didn't even get to enter my name. I suspect there is some mistake in mapping the data folder.
Was using the template in https://github.com/allegroai/clearml-helm-charts to deploy.

Nope. In my local drive

Hi TimelyPenguin76 , nope. I don't see any errors. That's why not sure what went wrong

Just to add, when I run the pipeline locally it works as well.