CostlyOstrich36

0 Questions, 3697 Answers

Active since 10 January 2023

Last activity one year ago

Reputation

Answers 3697

0 Hi, In The New Datasets Ui. It Doesn'T Seem To Display The Entire Lineage Of The Datasets. For Example. If A Dataset Is Create As Such Id1 (Parent)->Id2, Then Another Dataset Created As Id2(Parent)-> Id3. When You Look At Id3, It Only Shows Id2 As Parent.

Hi SubstantialElk6 ,

Just making sure I understand DS1 is parent of DS2 which is parent of DS3. DS3 doesn't show DS1 as one of it's parents. correct?

2 years ago

0 Hi, It Seems Like When I Upload The Dataset To Clearml With The Methods From The Dataset Class. It Packed The Whole Dataset Into Several .Zip Files In /Tmp Folder Before Uploading. I'D Like To Know Whether I Can Point The .Zip Files To Be Saved Somewhere

Hi DeterminedOwl36 , what about simply adding files that are already sitting on some storage?
https://clear.ml/docs/latest/docs/references/sdk/dataset#add_external_files

2 years ago

0 Hello All

Also, can you add a screenshot please?

one year ago

0 Has Anybody Encountered:

Hi @<1584716373181861888:profile|ResponsiveSquid49> , what optimization method are you using?

one year ago

0 Hello All

No need, that was intended for another chat 🙂

one year ago

0 Hello All

Hi @<1527096757119356928:profile|FrothyShrimp23> , what server are you using? Is it self hosted or the community, if self hosted, what version?

one year ago

0 Heya, I Hope You'Re All Well In This Beautiful Day, My Gcp Autoscaler Just Died With That Strange But Short Backtrace, Wondered If It Rang A Bell To Any Of You ?

Can you please add a larger chunk of the autoscaler log?

2 years ago

0 Hi Folks, What Is The Process To Get An Ml Algo/Engine Added To The List Of Integrations?

Hi FrustratingShrimp3 , which framework would you like added?

2 years ago

0 Hey Currently Trying To Run A Pipeline Locally To Test A Pipeline Component With

Didn't have a chance to try and reproduce it, will try soon 🙂

2 years ago

0 Hello Everyone! I Setup A Gcp Autoscaler On My Pro Saas Deployment But I Keep Getting This Error:

Hi FierceHamster54 , I'm taking a look 🙂

2 years ago

0 Hey I Hope Everyone Is Having A Good Day, Two Quick Questions About Datasets:

This description in the

add_tags()

doc intrigues me tho, I would like to remove a tag from a dataset and add it to another version (eg: a

used_in_last_training

tag) and this method seems to only add new tags.

I see. Then I think you would need to do this via the API:
https://clear.ml/docs/latest/docs/references/api/tasks#post-tasksupdate
or
https://clear.ml/docs/latest/docs/references/api/tasks#post-tasksupdate_batch

2 years ago

0 Hi There, Is It Possible To Configure The Clearml-Agent To Run Some Commands Before Running Each Experiment It Launches? Eg.

JitteryCoyote63 , If you mean storage secrets (aws,azure, etc..) then you can configure them in your ~/clearml.conf 🙂

3 years ago

0 Hi Folks, What Is The Process To Get An Ml Algo/Engine Added To The List Of Integrations?

Is this something that the ClearML team could add? Would it be helpful to have a call?

FrustratingShrimp3 , I think you can open a github issue to request the new feature

2 years ago

0 I'M Trying To Read The Runtimes Of Tasks After The Pipeline Finished, For Analytics. Currently, I'M Doing Something Like This:

I'd suggest using the API directly, fetch a task and compare it's start to end time.

5 months ago

0 Hello There! I Have A Question Regarding The Web Ui, On The Project Page: I Have The Following Use Case: I Need To Add Two Custom Columns, Each Reporting One Metric. Currently, This Shows Me The Best (Min/Max) Values Reached By The Model, But Not Necessar

How about the comparison?

2 years ago

0 Hi! Quick Question. Can I Use The Ui To Search For An Artifact By Id?

Hi 🙂
When viewing the experiments tables you have a tab called 'models'. If the artifact was saved as a model, then you can find it there 🙂

2 years ago

0 Hi There, Is It Possible To Configure The Clearml-Agent To Run Some Commands Before Running Each Experiment It Launches? Eg.

JitteryCoyote63 , I'm afraid currently not and only available in docker mode.

What do you need it for if I may ask?

3 years ago

0 Hello, My Dl Workflow Includes Post-Training Quantization. Is There A Way To Implement These Procedures In Clearml?

Can you please give an example?

2 years ago

0 Hi There, Is It Possible To Configure The Clearml-Agent To Run Some Commands Before Running Each Experiment It Launches? Eg.

JitteryCoyote63 Hi! Are you running in docker mode?

3 years ago

0 Hi Folks, What Is The Process To Get An Ml Algo/Engine Added To The List Of Integrations?

FrustratingShrimp3 , it could be quite simple to do yourself as well. You can take a look at the example bindings here:
https://github.com/allegroai/clearml/tree/master/clearml/binding
It shouldn't be difficult to implement a new framework

2 years ago

0 Hey I Hope Everyone Is Having A Good Day, Two Quick Questions About Datasets:

And additionally does the

When executing a Task (experiment) remotely, this method has no effect).

part means that if it is executed in a remote worker inside a pipeline without the dataset downloaded the method will have no effect ?

I think this means the add tags specifically will have no effect

2 years ago

0 Heya, I Hope You'Re All Well In This Beautiful Day, My Gcp Autoscaler Just Died With That Strange But Short Backtrace, Wondered If It Rang A Bell To Any Of You ?

Thanks for the info! This happened when you had 2 spot instances running something, correct?

2 years ago

0 Hi Everyone, I Have Been Using Clearml Self Hosted For Experiment Visualisation - Logging Loss And Uploading Debug Samples Etc. Our Clearml Server Is Running On A Aws Instance, With The Fileserver And Databases (Mongo, Redis Etc) On That Instance. We Run

Hi TimelyCrab1 , directing all your outputs to s3 is actually pretty easy. You simply need to configure api.files_server: <S3_BUCKET/SOME_DIR> in clearml.conf of all machines working on it.

Migrating existing data is more difficult since everywhere in the system everything is saved as links. I guess you could change the links in mongodb but I would advise against it.

one year ago

0 Hey Currently Trying To Run A Pipeline Locally To Test A Pipeline Component With

Can you please elaborate on what you're trying to do and what is failing?

2 years ago

0 Hi Everyone. I’M Struggling To Setup Minio Storage. Below Is What I’M Adding In My Credentials And When I Try To Create A New Dataset Using Below Command; I Get Errors: Configs:

try None :port/bucket

5 months ago

0 Hi, I Am At The Process Of Migrating From Trains To Clearml. While Running The 'Pull' Command I Got Some Warnings. Please Let Me Know If Anything Needs To Be Done With These Warnings:

Doesn't look like anything critical 🙂

2 years ago

0 Hello, Is There A Way To Selectively Disable Logging Of Certain Scalars? My Pytorch Callback Reports Certain Scalars For Informative Purposes, But I Don'T Want Those To Clutter Up The "Scalars" Plot, And I Only Want To See Certain Scalars Plotted. I Am Aw

Hi JuicyOtter4 , not sure you can disable certain scalars but you can disable auto logging of specific frameworks using the method showed here:
https://www.youtube.com/watch?v=etGjxOKG9lo

Hope this helps 🙂

one year ago

0 Hi Everyone. I’M Struggling To Setup Minio Storage. Below Is What I’M Adding In My Credentials And When I Try To Create A New Dataset Using Below Command; I Get Errors: Configs:

Can you provide a full print of the terminal?

5 months ago

0 Hi Everyone. I’M Struggling To Setup Minio Storage. Below Is What I’M Adding In My Credentials And When I Try To Create A New Dataset Using Below Command; I Get Errors: Configs:

@<1709740168430227456:profile|HomelyBluewhale47> , how did you set the output_uri ?

5 months ago

0 Hi Everyone. I’M Struggling To Setup Minio Storage. Below Is What I’M Adding In My Credentials And When I Try To Create A New Dataset Using Below Command; I Get Errors: Configs:

Also a paste of your config

5 months ago

Show more results