Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi! I Was Wondering Why Clearml Recognize Scikit-Learn Scalers As Input Models... Am I Missing Something Here? For Me It Would Make Sense To Include The Scalers As A Configuration Object Of The Trained Model, Not Outside

Hi! I was wondering why ClearML recognize Scikit-learn scalers as Input Models... Am I missing something here?
For me it would make sense to include the scalers as a configuration object of the trained model, not outside

  
  
Posted 2 years ago
Votes Newest

Answers 11


Thank you!

  
  
Posted 2 years ago

I'm using the last commit. I'm just fitting a scikit-learn MinMaxScaler object to a dataset of type tf.data.Dataset inside a function (which represents the model training step) decorated with PipelineDecorator.component . The function does not even return the scaler object as an artifact. However, the scaler object is logged as an artifact of the task, as shown in the image below.

  
  
Posted 2 years ago

Hi! I was wondering why ClearML recognize Scikit-learn scalers as Input Models...

Hi GiganticTurtle0
any joblib.load/save is logged by clearml (it cannot actually differentiate what it is used for ...)
You can of course disable it with Task.init(..., auto_connect_frameworks={'joblib': False})

  
  
Posted 2 years ago

GiganticTurtle0 Hi!

Which versions are you using? Also do you have an snippet example by chance?

  
  
Posted 2 years ago

GiganticTurtle0 is it just --stop that throws this error ?
btw: if you add --queue default to the command line I assume it will work, the thing is , without --queue it will look for any queue with the "default" tag on it, since there are none, we get the error.
regardless that should not happen with --stop I will make sure we fix it

Just so we do not forget, can you please open an issue on clearml-agent github ?

  
  
Posted 2 years ago

Sadly, I think we need to add another option like task_init_kwargs to the component decorator.
what do you think would make sense ?

  
  
Posted 2 years ago

Oh, I see. This explains the surprising behavior. But what if Task.init code is created automatically by PipelineDecorator.component ? How can I pass arguments to the init method in that case?

  
  
Posted 2 years ago

Well, just as you can pass the 'task_type' argument in PipelineDecorator.component , it might be a good option to pass the rest of the 'Task.init' arguments as they are passed in the original method (without using a dictionary)

  
  
Posted 2 years ago

Yes, before removing the 'default' queue I was able to shut down agents without specifying further options after the --stop command. I just had to run clearml-agent daemon --stop as many times as there were agents. Of course, I will open the issue as soon as possible :D

  
  
Posted 2 years ago

That is a good question ... let me check 🙂

  
  
Posted 2 years ago

BTW, let's say I accidentally removed the 'default' queue from the queue list. As a result, when I try to stop an agent using clearml-agent daemon --stop , I get the following error:
clearml_agent: ERROR: APIError: code 400/707: No queue is tagged as the default queue for this company
I have already created another queue also called 'default' but it had no effect :/

  
  
Posted 2 years ago
654 Views
11 Answers
2 years ago
one year ago
Tags