From this movie, its clear that we are not using all trains (clearml) functionality. Because kubeflow is manage our pipeline. So for example, trains cannot record the git changes

The first pipeline
step is calling init

🙂

AgitatedDove14 , ideally yeah, but we manually adds a line to the generate script from kale before running it.

So main question, is there a best partice how to connect clearml and kubeflow ?

well. we are using it from our code and the trains dashboard works just fine. But this is without the agent. Moreover, Kubeflow uses docker images for the training pipelines, there for trains cannot capture the diff

and there is not trains agent on kubeflow. At least not that I'm aware of

Kale just translate the notebooks into kubeflow pipeline dsl. So In kubeflow when the pipeline is running we can see the pipeline elements.

am.. it depends, it can be a small function but can also be a task. We are using kale so it easier to group/create task and move data from one task to another. Each step is independent but the next is being creating from the data of the pervious one.

not only the diff, but also the git itself. because this make no sense to push the git into the docker image

Hey AgitatedDove14 ! Thanks!

For now we are using Kale to build kubeflow pipeline. Then we run this pipeline, so I'm not sure where the agent fits inside the Kubeflow ecosystem, can you elaborate more ?