That is a good question ... let me check 🙂
I'm using the last commit. I'm just fitting a scikit-learn MinMaxScaler object to a dataset of type tf.data.Dataset inside a function (which represents the model training step) decorated with PipelineDecorator.component . The function does not even return the scaler object as an artifact. However, the scaler object is logged as an artifact of the task, as shown in the image below.
Sadly, I think we need to add another option like task_init_kwargs to the component decorator.
what do you think would make sense ?
Yes, before removing the 'default' queue I was able to shut down agents without specifying further options after the --stop command. I just had to run clearml-agent daemon --stop as many times as there were agents. Of course, I will open the issue as soon as possible :D
Hi! I was wondering why ClearML recognize Scikit-learn scalers as Input Models...
Hi GiganticTurtle0
any joblib.load/save is logged by clearml (it cannot actually differentiate what it is used for ...)
You can of course disable it with Task.init(..., auto_connect_frameworks={'joblib': False})
Oh, I see. This explains the surprising behavior. But what if Task.init code is created automatically by PipelineDecorator.component ? How can I pass arguments to the init method in that case?
GiganticTurtle0 Hi!
Which versions are you using? Also do you have an snippet example by chance?
GiganticTurtle0 is it just --stop that throws this error ?
btw: if you add --queue default to the command line I assume it will work, the thing is , without --queue it will look for any queue with the "default" tag on it, since there are none, we get the error.
regardless that should not happen with --stop I will make sure we fix it
Just so we do not forget, can you please open an issue on clearml-agent github ?
BTW, let's say I accidentally removed the 'default' queue from the queue list. As a result, when I try to stop an agent using clearml-agent daemon --stop , I get the following error:clearml_agent: ERROR: APIError: code 400/707: No queue is tagged as the default queue for this company
I have already created another queue also called 'default' but it had no effect :/
Well, just as you can pass the 'task_type' argument in PipelineDecorator.component , it might be a good option to pass the rest of the 'Task.init' arguments as they are passed in the original method (without using a dictionary)