Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Ist It Possible To Move Artifacts From Local Storage To S3? Or Do I Have To Delete The Old One And Create A New One With A Location In S3?

Ist it possible to move artifacts from local storage to s3? Or do I have to delete the old one and create a new one with a location in s3?

  
  
Posted 2 years ago
Votes Newest

Answers 12


What is your scenario? Can you elaborate?

  
  
Posted 2 years ago

Hi EnormousWorm79 , are you using a self-hosted server?

  
  
Posted 2 years ago

Well, that can be done by running a custom script on the server

  
  
Posted 2 years ago

Yes, I am using a self-hosted server.

So we ran some experiments(tasks) and uploaded the artifacts to local storage. We want to move them to a s3 storage

Furthermore we want to move existing datasets on one s3 server to another s3 server

  
  
Posted 2 years ago

Yeah, but I still need to update the links in the clearml server

  
  
Posted 2 years ago

around 100-200

  
  
Posted 2 years ago

For anyone else struggling with this:
Shell into the mongo container and run
db.task.find({ "project": "<PROJECT_ID>" }, { "execution.artifacts.$": 1 }).forEach(function (doc) { Object.keys(doc.execution.artifacts).forEach(function (k) { doc.execution.artifacts[k].uri = doc.execution.artifacts[k].uri.replace("old_host", "new_host"); db.task.updateOne({ _id: doc._id }, { $set: { execution: doc.execution } }) }) });

  
  
Posted 2 years ago

You might be able to write a script to override the links ... wdyt?

  
  
Posted 2 years ago

You would also need to edit the links somehow that are connected to the task

  
  
Posted 2 years ago

Yeah, but I still need to update the links in the clearml server

yes... how many are we talking about here?

  
  
Posted 2 years ago

Nice ๐Ÿ™‚ ๐Ÿ‘

  
  
Posted 2 years ago

Thatโ€™s not as straightfoward as it might seem.. the clearml client lib handles artifacts as read only dict. I am not taking a look at changing the values directly in mongodb

  
  
Posted 2 years ago
530 Views
12 Answers
2 years ago
one year ago
Tags