Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi

Hi 🙂 I use Clearml agents to execute tasks, I would like to remove irrelevant info logs (logs of the agent setup - such as: "Summary - installed python packages:","Running task id", "Executing: ['docker', 'run', '-t', '...")
Is there any way doing it? I had tried find the relevant configuration at the clearml.conf but I couldn't see any log configuration for agents.

  
  
Posted one year ago
Votes Newest

Answers 12


@Is there a way to choose other logs format ? to get a json or something like that?
or setting logging level?

  
  
Posted one year ago

Actually what my service do is to collect

stdout/stderr

from the Docker socket

That's exactly how the agent works, it cannot really filter it, it logs everything by default for full visibility ...

  
  
Posted one year ago

Hi @<1523701205467926528:profile|AgitatedDove14> , I send the agent's logs to log management and monitoring service, and we get big amount of info logs (especially if we use schedulers and launch agents again and again as part of the production pipeline). I think that it is important to set level also for other levels then debugging, WDYT?

  
  
Posted one year ago

send the agent's logs to log management and monitoring service,

These are stored into ELK, it was built to store large amounts of logs, I cannot see any reason why one would want to remove it?

Maybe if there would be a way to change their format, it could also help filtering them from my side.

You mean in the UI?

  
  
Posted one year ago

Hi CostlyOstrich36 ,Thanks for your answer, I would like it to be configurable , just like I set logging level for my logs.

  
  
Posted one year ago

In the agent, no, it pipes stdout/stderr of the container and logs everything 😞

to get a json or something like that?

There is an api to get all the console logs, is this what you are after?

  
  
Posted one year ago

@<1523701205467926528:profile|AgitatedDove14> Even if there is a way to set the format of those logs, that will be fine for now

  
  
Posted one year ago

@<1523701205467926528:profile|AgitatedDove14> I am not sure if it would help but lets try 🙂

  
  
Posted one year ago

@<1523701205467926528:profile|AgitatedDove14> Actually what my service do is to collect stdout/stderr from the Docker socket. I meant to the stdout/stderr .

  
  
Posted one year ago

Maybe if there would be a way to change their format, it could also help filtering them from my side.

  
  
Posted one year ago

Hi SmugTurtle78
Unfortunately there is no actual filtering for these logs, because they are so important for debugging and visibility. I have to ask, what's the use case to remove some of them ?

  
  
Posted one year ago

Hi SmugTurtle78 , I'm not sure it's possible. Maybe SuccessfulKoala55 knows some workaround. Do you want to get rid of them in all scenarios or in just some specific use cases?

  
  
Posted one year ago
996 Views
12 Answers
one year ago
one year ago
Tags