It return false. Just to share abit more, I have the requirements.txt in gitlab with my codes and are in folders. Do I need to provide a gitlab path?

@<1523701070390366208:profile|CostlyOstrich36> This is output_uri or where do I put this url?

Not exactly sure yet but I would think user tag for deployed make sense as it should be a deliberated user action. And additional system state is required too since a deployed state should have some pre-requitise system state.

I would also like to ask if clearml has different states for a task, model, or even different task types? Right now I dun see differences, is this a deliberated design?

SdK meaning I run the agent using clearml-agent daemon ....
Alternatively I understand I can also run the agent using docker run allegroai/clearml-agent:latest. But I cannot figure out how to add --restart, --queue, -- gpus flag to the container

I guess we need to understand the purpose of the various states. So far only "archive, draft, publish". Did I miss any?

@<1523701205467926528:profile|AgitatedDove14> when my codes get the clearml datasets, it stores in the cache e.g. /$HOME/.clearml/cache....

I wanted it to be in a mounted PV instead, so other pods (in same node) who needed same datasets can use without pulling again.

@<1523701205467926528:profile|AgitatedDove14> do u mean not using helm but fill up the values and install with the yaml files directly? E.g. kubectl apply ...

Cool thanks guys. I am clearer now. Was confused by the obsolete info. Thanks for the clarification.

May I know where to set the cert to in env variable?

Example i build my docker image using a image in docker hub. In this image, i installed torch and cupy packages. But when i run my experiment in this image, the packages are not found.

Yes, I ran the experiment inside.

Thanks AgitatedDove14 and TimelyMouse69 . The intention was to have some traceability between the two setups. I think the best way is to enforce some naming convention (for project and name) so we can know how they are related? Any better suggestions?

@<1523701070390366208:profile|CostlyOstrich36> Yes. I'm running on k8s

Yup, was thinking of bash script.

The intent is to generate some outputs from the clearml task and thinking probably to package it into a docker image for ease of sharing to others that are not plug into our network and able to run the image directly.

Hi ExasperatedCrab78 I managed to get it. It was due to ip address set in examples.env.

Is there a way to troubleshoot further?

To clarify, there might be cases where we get helm chart /k8s manifests to deploy a inference services. A black box to us.

Users may need to deploy this service where needed to test out against other software components. This needs gpu resources which a queue system will allow them to queue up and eventually get this deployed instead of hard resource allocation to this purpose

Hi @<1523701070390366208:profile|CostlyOstrich36> , basically

• I uploaded dataset using clearml Datasets. The output_uri is pointed to my s3, thus the dataset is stored in s3. My s3 is setup with http only.
• When I retrieve the dataset for training, using Dataset.get() , I encountered ssl cert error as the url to retrieve data was https://<s3url>/... instead of s3://<s3url>/... which is http. This is weird as the dataset url is without https.
• I am not too sure why and I susp...

Thanks. The examples uses upload_artifact which stores the files in output_uri. What if I do not want to save it but simply pass to next step, is there a way to do so?

Ok. Can I check that only the main script was stored in the task but not the dependent packages?

I guess the more correct way is to upload to some repo where the remote task can still pull from it?

Thanks I just realised I didn't add --docker

Thanks

By the way, how can I start up the clearml agent using the clearml-agent image instead of SDK? Do u have an example of the docker run command that includes the queue, gpus etc?

Hi SuccessfulKoala55 Thanks for pointing me to this repo. Was using this repo.

I didn't manage to find in this repo that if we still require to label the node app=clearml, like what was mentioned in the deprecated repo. Although from the values.yaml, the node selector is empty. Would u be able to advise?

How is the clearml data handled now then? Thanks

Ah I think I was not very clear on my requirement. I was looking at porting project level, not entire clearml data over. Is it possible instead?

SuccessfulKoala55 i tried comment off fileserver, clearml dockers started but it doesn't seems to be able to start well. When I access clearml via webbrowser, site cannot be reached.

Just to confirm, I commented off these in docker-compose.yaml.

apiserver:
command:
- apiserver
container_name: clearml-apiserver
image: allegroai/clearml:latest
restart: unless-stopped
volumes:
- /opt/clearml/logs:/var/log/clearml
`...

Nice. It is actually dataset.id .

By the way, will downloading still happen if the datasets is available in the cache folder? Any specific settings to add to Dataset.get_local_copy()?

OK let me try by adding to vol mount.

Nope. In my local drive

Hi CostlyOstrich36 I have run this task locally at first. This attempt was successful.

When I use this task to run in a pipeline (task was run remotely), it cannot find the external package. This seems logical but I not sure how to resolve this.