I think the example here should help you.
Together with this
Tell me if it helped 🙂
Using the https://allegro.ai/clearml/docs/rst/references/clearml_python_ref/task_module/task_task.html?highlight=upload_artifact#clearml.task.Task.upload_artifact method. It works well, but only saves it as a
csv (which is very problematic since when loading the artifact none of the data types of the columns are preserved...)
Hi OutrageousSheep60 , can you try with
auto_pickle=True when uploading the artifact?
Thx CostlyOstrich36 for your reply
Can't see the reverence to
parquet . we are currently using the above functionality , but the
pd.DataFrame is only saved as
csv compressed by
How do you currently save artifacts now?
CostlyOstrich36 - but we will use any method that will allow us to save the files as
We are not yet using
clearml Dataset - i'm not sure if this is a solution
Well - that will convert it to a binary pickle format but not as parquet -
since the artifact will be accessed from other platforms we want to use parquet
TimelyPenguin76 - I'm I mistaken?