Cool thanks guys. I am clearer now. Was confused by the obsolete info. Thanks for the clarification.

Yes. But I not sure what's the agent running. I only know how to stop it if I have the agent id

I not very sure tbh. Just want to see if this is useful....

I got SSL error few days back and I solved it by adding cert to /etc/ssl/certs and perform update-ca-certificates .

export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

Add this. Note that verify might not work with sdk.aws.s3.verify but sdk.aws.s3.credentials . Pls see the attached image.

Example:
aws {
s3 {
credentials: [
{
` ...

U want to share your clearml.conf here?

Example i build my docker image using a image in docker hub. In this image, i installed torch and cupy packages. But when i run my experiment in this image, the packages are not found.

Yes, I ran the experiment inside.

By the way, how can I start up the clearml agent using the clearml-agent image instead of SDK? Do u have an example of the docker run command that includes the queue, gpus etc?

Hi Bart, yes. Running with inference container.

Yup, was thinking of bash script.

The intent is to generate some outputs from the clearml task and thinking probably to package it into a docker image for ease of sharing to others that are not plug into our network and able to run the image directly.

I guess we need to understand the purpose of the various states. So far only "archive, draft, publish". Did I miss any?

Do u have an example of how I can define the packages to be installed for every steps of the pipeline?

https://clear.ml/docs/latest/docs/integrations/storage/
Try add the <path to your cert> for s3.credentials.verify.

Nice. It is actually dataset.id .

Hi @<1523701070390366208:profile|CostlyOstrich36> , basically

• I uploaded dataset using clearml Datasets. The output_uri is pointed to my s3, thus the dataset is stored in s3. My s3 is setup with http only.
• When I retrieve the dataset for training, using Dataset.get() , I encountered ssl cert error as the url to retrieve data was https://<s3url>/... instead of s3://<s3url>/... which is http. This is weird as the dataset url is without https.
• I am not too sure why and I susp...

Thanks AgitatedDove14 . Specifically, I wanted to use my own clearml server and Triton. Thus, I attempted to use --engine-container-args during launch but error saying no such flag. Looked into --help but I guessed it is not updated yet.

And just a suggestion which maybe I can post in GitHub issue too.
It is not very clear what are the purpose of the project name and name, even after I read the --help. Perhaps this is something that can be made clearer when updating the docu?

Thanks AgitatedDove14 and TimelyMouse69 . The intention was to have some traceability between the two setups. I think the best way is to enforce some naming convention (for project and name) so we can know how they are related? Any better suggestions?

Ah I think I was not very clear on my requirement. I was looking at porting project level, not entire clearml data over. Is it possible instead?

Hello CostlyOstrich36 I am facing an issue now. basically i installed all necessary python packages in my docker image. But somehow, the clearml-agent does not seems to be able to detect these global packages. I don't see them in the "installed packages". Any advice?

CostlyOstrich36 I mean the dataset object in clearml as well as the data that is tied to this object.

The intent is to bring over to another clearlml setup and keep some form of traceability.

This is what I got. and when I see http400 error in the console.

@<1526734383564722176:profile|BoredBat47> Just to check if u need to do update-ca-certificates or equivalent?

I have yet to figure out how to do so, would appreciate if u could give some guidance

OK I will send here shortly.

I figured out that it maybe possible to do these
experiment_task = Task.current_task()
OutputModel(experiment_task ).update_weights(' http://model.pt ') to attach it to the ClearML experiment task.

Is there a way to troubleshoot further?

Yup. But I happened to reinstall my server and the data is lost. And the agent continue running.

By the way, will downloading still happen if the datasets is available in the cache folder? Any specific settings to add to Dataset.get_local_copy()?

It return false. Just to share abit more, I have the requirements.txt in gitlab with my codes and are in folders. Do I need to provide a gitlab path?

JuicyFox94 and SuccessfulKoala55 Thanks alot. Indeed it is caused by dirty cookies.

OK let me try by adding to vol mount.