Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey Clearml, Pipeline From Tasks: Following This Example

Hey ClearML,
Pipeline from tasks: following this example None
Problem Description:

  • I have two steps / tasks in the pipeline, first task create some data id, and upload the string as an artifact
  • second task use that string data id to run this experiment, (this is an separate task doesn't process the uploaded artifact as an pickle file)
  • i am using the parameter override in the the creation of second step of pipeline, like this
parameter_override={
        "General/dataid": "${step1.artifacts.dataid}",
    }
  • but i am getting a strange string (which contains the storage link, mode and ...) in the data id of second task.
  • Is there a way to upload a string as an artifact, and access that exact uploaded string (without any meta and additional details) in the second task? if so how can I do that?
  
  
Posted 11 months ago
Votes Newest

Answers 7


Hi @<1523701283830108160:profile|UnsightlyBeetle11> , I think you can store txt artifacts so you can store the string there. If it's not too long, you can even fetch it from the preview

  
  
Posted 11 months ago

yes, i tried to store the string in txt format, still getting all the metadatas, and the needed string is in preview
how can i fetch it from the preview?

  
  
Posted 11 months ago

the artifacts is stored as a dictionary with the name of the parametr as the key, and values

{'data_id': {'name': 'data_id', 'size': 39, 'type': 'string', 'mode': <ArtifactModeEnum.output: 'output'>, 'url': 'a big string', 'hash': 'some numbers', 'timestamp': datetime.datetime(2023, 5, 10, 8, 56, 10), 'metadata': {}, 'preview': '(data.UID:>147)AND(data.UID:<148)'}}

i need to access the string in the preview using parameter override logic, how can i get the value of the preview key of the artifact

  
  
Posted 11 months ago

I think it is one of the parameters of the task. Fetch a Task and see what properties the artifact has 🙂

  
  
Posted 11 months ago

task.upload_artifact("data_id", data_id), name of the artifact, and created string

  
  
Posted 11 months ago

Can you give an example of how you're saving this artifact?

  
  
Posted 11 months ago

Can you provide a self contained contained snippet that reproduces this behavior?

  
  
Posted 11 months ago
522 Views
7 Answers
11 months ago
11 months ago
Tags