Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey, I’Ve Deployed Clearml On K8S With The Helm Charts In Repo “Allegroai/Clearml-Helm-Charts”. Is There A Way To Bind All Logs From Tasks Running On The Agents To Stdout So That They Will Be Visible On The Pod And Not Only Through The Ui? With “Kubectl L

Hey, I’ve deployed ClearML on k8s with the helm charts in repo “allegroai/clearml-helm-charts”.
Is there a way to bind all logs from tasks running on the agents to stdout so that they will be visible on the pod and not only through the UI?
with “kubectl logs <POD NAME>”

Thanks!

  
  
Posted 2 years ago
Votes Newest

Answers 10


JuicyFox94 WDYT - can we get these files out into the pod log somehow?

  
  
Posted 2 years ago

SuccessfulKoala55 after looking at the issue I’m a bit confused 😄 ; as far as I can see there is no way to pass any parameter to clearml-agent in daemon mode to push log to stdout. Can you confirm it? (If yes I need to find some workaround)

  
  
Posted 2 years ago

ok, will try to find a solution then, ty

  
  
Posted 2 years ago

Awesome guys! thanks a lot 🙌

  
  
Posted 2 years ago

I need to investigate, ScrawnyLion96 can you pls open an issue on https://github.com/allegroai/clearml-helm-charts ?

  
  
Posted 2 years ago

Hi SuccessfulKoala55 ,

On the agent pod the tasks logs are written locally to /tmp/.clearml_agent_out.<random num>.txt

I wan’t those logs to also bind to the hosts stdout, with a console handler for example

  
  
Posted 2 years ago

Hi ScrawnyLion96 - as far as I remember these logs are intercepted by the agent from the task's process stdout/stderr, and sent to the server - unless we modify the agent to output them to another location as well, I'm not sure getting them in the way you are looking for will be possible...

  
  
Posted 2 years ago

I’m going to investigate (and fix it if possible) in some day

  
  
Posted 2 years ago

I think by default it does that, it's only that the filename itself is temporary, so right now the only way to get it is parse stdout...

  
  
Posted 2 years ago

Oh, might be my bad - there is a mode in which the logs are actually aggregated there

  
  
Posted 2 years ago
588 Views
10 Answers
2 years ago
one year ago
Tags