Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Everyone, I’M Facing The Issue That When Archive And Remove My Project (Including Tasks, Datasets), I Still Seeing That Data In My Minio Server. Is This A Bug Or Clearml Does Not Have That Feature Yet?

Hi everyone, I’m facing the issue that when archive and remove my project (including tasks, datasets), I still seeing that data in my MinIO server. Is this a bug or ClearML does not have that feature yet?

  
  
Posted 8 months ago
Votes Newest

Answers 16


The latest server version should be able to support it, but you need to configure the required credentials sin the server configuration for the server to be able to delete these files

  
  
Posted 8 months ago

Yes, I already config it and I seeing that my files are being uploaded on MinIO. But when I delete project on clearML, it does not remove files in MinIO

  
  
Posted 8 months ago

See here: None

  
  
Posted 8 months ago

Hi @<1603198153677344768:profile|ExasperatedSeaurchin40> , I assume you're using a self-hosted server?

  
  
Posted 8 months ago

No no, the upload is done on the client side. This is for the server

  
  
Posted 8 months ago

Yes, I self-host a clearml server and minio as well

  
  
Posted 8 months ago

I’m deploying my MinIO server on local network, and the URL to it contains port 9000, when I setup clearml to sync data to the storage, I have to set default output uri to sdk.development.default_output_uri=" None "

And the rest of my config will look like the image below. Is there anything I have to config in my clearml server on default output uri (for example the bucket name or port…)?
image

  
  
Posted 8 months ago

In short - you have to map config folder to async_delete container like in apiserver container.
image
image

  
  
Posted 8 months ago

You can put it under /opt/clearml/config/services.conf and inside the keys specified in the file need to be inside the storage_credentials { } section

  
  
Posted 8 months ago

Hi! I've faced the same problem and solved it like in this thread

None

  
  
Posted 8 months ago

@<1523701087100473344:profile|SuccessfulKoala55> I already try to setup bucket which point to clearmlstorage but it didn’t work

  
  
Posted 8 months ago

For the server to be able to delete these files, you need to set up the same credentials on the server side, not just on the client side

  
  
Posted 8 months ago

yes, I upload on the client, but the information about dataset is manage on the clearml server right?

  
  
Posted 8 months ago

Hmm, let me check that again. Is that storage config file goes into /opt/clearml/config?

  
  
Posted 8 months ago

@<1523701087100473344:profile|SuccessfulKoala55> @<1585078752969232384:profile|FantasticDuck7> Thank you very much. It works now 😊

  
  
Posted 8 months ago

Thanks @<1585078752969232384:profile|FantasticDuck7> , I'll make sure to fix that in the docker-compose deployment

  
  
Posted 8 months ago
488 Views
16 Answers
8 months ago
8 months ago
Tags