Great discussion, I agree with you both. For me, we are not using clearml-data, so I am a bit curious how does a "published experiment" locked everything (including input? I assume someone can still just go inside the S3 bucket and delete the file without Clearml noticing).

From my experience, absolute reproducibility is code + data + parameter + execution sequence. For example, random seed or some parallelism can cause different result and could be tricky to deal with sometimes. We did bu...

VivaciousPenguin66 What's your thought on Prefect? There are so many pipeline library and I wasn't so sure how different are they. I have experience with Airflow. With Kedro, we were in hope that data scientist will write the pipeline themselves with minimal effort to handover to another engineer to work on. For serious production (need to scale), we consider convert Kedro pipeline to Airflow, there are plugin to do that, tho I am not sure how mature they are.

AnxiousSeal95 At first sight, the pipeline logic of ClearML seems binding with ClearML quite a bit. Back then I was considering I need something that can convert to Production pipeline (e.g. Airflow DAGs) easily, as we need pipelines not just for Experiments, Airflow seems to be the default one.

Also, clearml-data was not available when we started the development of internal framework. As for clear-agent, from my previous experience, it seems not working great with Window sometimes, and als...

Yup, I am only more familiar with the experiment tracking part, so I don't know if I have a good understanding before I have reasonable knowledge of the entire ClearML system.

VivaciousPenguin66 How are you using the dataset tool? Love to hear more about that.

Cool, versioning the difference is useful. It also depends on what kind of data. For example, for tabular data, database might be a natural choice, however, how to integrate it and keep track of the metadata could be tricky. While for images, it probably more suitable for blob storage or per file basis.

It's good that you have version your dataset with name, I have seen many trained model that people just replace the dataset directly.

lol...... mine is best_model_20210611_v1.pkl and better_model_20210611_v2.pkl or best_baseline_model_with_more_features.pkl

I have been using this line to prevent experiments won't accidentally sent to the public server (I have my custom self-hosted server)
Task.set_credentials("PLACEHOLDER", "PLACEHOLDER","PLACEHOLDER")

However, when I upgraded from 0.17.5 -> > 1.0.0. Weird stuff happen.
Since upgrade from v0.17.5 -> > 1.0.0, it has issue replacing the credentials.

Expected Behavior:

Conf should replace the "PLACEHOLDER" is the conf file exist. Else it should fails the experiment.

What happened:

The ...

Supplement the error.

Oh I did not realize I asked this in a old thread, sorry about that.

AgitatedDove14 Git is fine, I just create a local repository for this. The code is two line
from trains import Task
task = Task.init(project_name="my project", task_name="my task3")

one does record the package, the other does not

TimelyPenguin76 No, I didn't see it.

repository detection is fine

conda create -n trains python==3.7.5
pip install trains==0.16.2.rc0

AgitatedDove14
are the data versioning completely different from the Trains Artifact/storage solution? or it's some enhanced feature.

I am interested in machine learning experiment mangament tools.

I understand Trains already handle a lot of things on the model side, i.e. hyperparameters, logging, metrics, compare two experiments.

I also want it to help reproducible. To achieve that, I need code/data/configuration all tracked.

For code and configuration I am happy with current Trains solution, but I am not sure about the data versioning.

So if you have more details about the dataset versioning with the enterprise offer...

potentially both, but let just say structure data first, like CSV, pickle (may not be a table, could be any python object), feather, parquet, some common data format

GrumpyPenguin23 yes, those features seems to related to other infrastructure, not Trains (ML experiment management)

for the most common workflow, I may have some csv, which may be updated from time to time

Do you know what is the "dataset management" for the open-source version?

for the open source version, if I use artifact, if I already have a local file, does it knows to skip downloading it or it will always replace the file? As my dataset is large (~100GBs), I cannot afford it to be re-downloaded everytime

Ok, then maybe it can be still used as a data versioning solution. Except that I have to manually track the task id (those generate artifact) for versioning myself.

I wonder what's the extra features is offered in the enterprise solution tho

TimelyPenguin76 It works fine. I may need to check on my side, I just notice it was caused by @funcy.log_durations decorator. It may changes the function signature and causing some issue with it. I don't have time to look into it yet, but the example works fine.

Hi, I think I can confirm this is a bug of Trains. Is that ok if I submit a PR to fix this?

https://github.com/allegroai/trains/issues/250
https://github.com/allegroai/trains/pull/251

TimelyPenguin76

Ok, will prepare a PR and script to reproduce the error

It would be nice if there is an "export" function to just export all/selected experiment table view

So I found that if I change this plot, it seems changes across all the expeiment too? How did it cache this setting? could this be shared among users or only per user or it is actually cached by the browser only?