Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello All, I'M Trying To Download A File Hosted In The Clearml Server And I Receive The Following Error :

hello all,

I'm trying to download a file hosted in the clearml server
and I receive the following error :
2022-12-16 11:52:40,423 - clearml.storage - ERROR - Could not download http://xxx.xxx.xx.xxx:8081/V3M/.pipelines/V3M%20MF/V3M_step.8ba3d92dc7b0454dad2c52dc66fc554d/artifacts/Detections/output.tsv , err: Failed getting object /V3M/.pipelines/V3M%20MF/V3M_step.8ba3d92dc7b0454dad2c52dc66fc554d/artifacts/Detections/output.tsv (404): NOT FOUND

code :
detection_file_path = StorageManager.get_local_copy(input['detection_file_path'])
the file exists this problem occurs, only when executing as an agent

  
  
Posted 2 years ago
Votes Newest

Answers 5


Hi ShallowCormorant89 ,

When does 1. happen? Can you add the full log?

Regarding 2, can you please elaborate? What is your pipeline doing and what sort of configurations would you want to add? On the pipeline controller level or steps?

  
  
Posted 2 years ago

Hey CostlyOstrich36

Happens when I try to execute a pipeline remotely.2022-12-19 15:13:47,884 - clearml - WARNING - Could not retrieve remote configuration named 'RUN_CONFIG' Using default configuration: {...}It happens in my pipeline and here is the code :

` pipe = PipelineController(
name="mypipe", project="myproject", version="0.0.1", add_pipeline_tags=False
)
pipe.set_default_execution_queue("default")

my_json = "jsons/my_json.json"
clearml_input_path = "jsons/clearml_input.json"
my_json = load_json_if_path(my_json)
clearml_input_path = load_json_if_path(clearml_input_path)

pipe.add_parameter(name="my_json", default=my_json, param_type='dictionary')
pipe.add_parameter(name="RUN_CONFIG", default=clearml_input_path, param_type='dictionary')

pipe.add_step(
name="my_Step_1",
base_task_project="my_1",
base_task_id="e82582054be642e38541ab06770",
execution_queue="my_queue",
configuration_overrides={"my_json": "${pipeline.my_json}",
'RUN_CONFIG': "${pipeline.RUN_CONFIG}"}
)

pipe.add_step(
name="my_Step_2",
parents=["my_Step_1"],
base_task_project="my_2",
base_task_id="703cdbd4f46f4386b4e338f56f34",
execution_queue="my_queue",
parameter_override={"INPUTS/detection_file_path" : "${my_Step_1.artifacts.Detections.url}"},
configuration_overrides={
"my_json": "${pipeline.my_json}",
'RUN_CONFIG': "${pipeline.RUN_CONFIG}"
}
)

pipe.start()
print("done") I want my_json and RUN_CONFIG ` to be configuration object in a pipeline. (so that we can paste our dictionarys)
thank you for your help!

  
  
Posted 2 years ago

Hi ShallowCormorant89
Can you verify the http link is valid? Can you download it from code on your machine (i.e. not via an agent), maybe 8081 port is blocked from the agent machine to the server?

  
  
Posted 2 years ago

hello again,

It will be helpful to know why we experience this when running a pipeline
2022-12-19 15:13:47,884 - clearml - WARNING - Could not retrieve remote configuration named 'RUN_CONFIG'

  1. how do I add a configuration object to a pipeline.
    the dictionary is split into multiple values.. when using it as a param in the pipeline
  
  
Posted 2 years ago

hello AgitatedDove14
thanks for your reply.

yes, the HTTP link is valid I was able to download it using wget I'm doubtful if it was an inconsistency
right now this seems to be solved for me.
previously I was using ${V3M_step.artifacts.Detections} - which will return a dictionary
on changing it to ${V3M_step.artifacts.Detections.url} (the url of the artifact) is returned and this seemed to have helped.

  
  
Posted 2 years ago