Ok, this makes more sense. Thank you very much. I'll take a closer look at your code when I have a better picture of ClearML.

Thank you very much, Martin. Step by step I am understanding better the platform (and the more I do, the more I like it!). If you don't mind, I will write down a summary of a use case for the reusing of Tasks, taken from a recent project I made using Luigi.

Ok, thanks a lot. This is not exactly what I expected, so I don't fully understand. For example, let's say you have a basic project in which the workflow is:
You read a csv stored in your filesystem. You transform this csv adding some new features, scaling and things like that. You train a model (usually doing several experiments with different hyperparameters). You deploy the model and is ready for making predictions. How would you structure this workflow in Tasks in ClearML?

Thank you very much, Martin. Yes, it makes sense and I can see this approach has also some benefits when compared to Kedro (for example, if you want to run only one node, you'd have to create one pipeline having only this node, and define inputs and outputs...). This kind of information is very valuable! And regarding the other stuff of Luigi I mentioned, how does ClearML address that? I mean: if the same Task is run with different parameters, do their artifacts collide or not? And on rerun o...

Thanks a lot. Yes, in this week I have met at least two people combining ClearML with other tools (one with Kedro and the other with luigi). In the beginning, I would rather prefer sticking with ClearML alone, so that I won't need to learn more than one tool. But I don't discard trying this integration in the future if I find some benefits.

I am sorry but I did not fully understand your answer. Well, from what you say it seems that everything is very flexible and programmable, which is some...

Thank you very much!

I have replied on the thread too, to haver all the discussion unified.

Thank you so much, Martin. You are really nice. 🙂

One of them told me they translated complete pipelines of Luigi to tasks in ClearML. It can be a way of working, but you lack the flexibility of running tasks by themselves.

In my case, I need to understand ClearML better in order to make a decision (I mean whether using ClearML with another tool for designing pipelines or not).

And regarding the possible approach, I will say the same: I will try to understand ClearML better and then ma...

Regarding git/svn, I think the same as you, but I'm afraid it won't be an easy battle... Is it hard to configure a git server?

Thanks a lot for your really informative answers. I like ClearML, to be honest, but I like the way Kedro (with its somehow tight and forced project structure) can guide us in the setting of standards. So, do you think that we could benefit from both tools, and use ClearML for experiment tracking while having our code structured within Kedro pipelines?