TrickySheep9

71 Questions, 428 Answers

Active since 10 January 2023

Last activity 8 months ago

Reputation

Badges 1

383 × Eureka!

Questions 71
Answers 428

0 Votes

3 Answers

958 Views

0 Votes 3 Answers 958 Views

Is There A Way I Can Create A Dataset As Part Of A Pipeline And Be Able To See That This Dataset Came From This Pipeline / Task ?

Is there a way I can create a dataset as part of a pipeline and be able to see that this dataset came from this pipeline / task ?

clearml

3 years ago

0 Votes

1 Answers

1K Views

0 Votes 1 Answers 1K Views

Currently Clearml-Agent In Services-Mode Supports Cpu Only Configuration.

Currently clearml-agent in services-mode supports cpu only configuration. Any particular reason for this limitation? Any inputs on what’s needed to do this i...

clearml

3 years ago

0 Votes

12 Answers

1K Views

0 Votes 12 Answers 1K Views

Any Info On The Lifecycle Of Datasets Downloaded To $Home/.Clearml/Cache/Storage_Manager/Datasets Via Get_Local_Copy I Have A Task Running And I Was Watching The Above Path And Datasets Were Being Downloaded And Then They Are All Removed And For A Partic

Any info on the lifecycle of datasets downloaded to $HOME/.clearml/cache/storage_manager/datasets via get_local_copy I have a task running and I was watching...

clearml

3 years ago

0 Votes

12 Answers

1K Views

0 Votes 12 Answers 1K Views

Apart From Having Packages In Requirements.Txt, Does Clearml Expect Them To Be Actuall Installed To Add Them As Installed Packages For A Task?

Apart from having packages in requirements.txt, does clearml expect them to be actuall installed to add them as installed packages for a task?

clearml

3 years ago

0 Votes

0 Answers

1K Views

0 Votes 0 Answers 1K Views

@AgitatedDove14 - similarly seeing some difference in way packages are installed. How does clearml agent cache packages? having issues with Click 7 vs 8.

AgitatedDove14 - similarly seeing some difference in way packages are installed. How does clearml agent cache packages? having issues with Click 7 vs 8.

mlops

3 years ago

0 Votes

19 Answers

1K Views

0 Votes 19 Answers 1K Views

I Saw Some Talk Of Clearml + Kedro On Reddit. Is That A Good Approach?

I saw some talk of ClearML + Kedro on reddit. Is that a good approach?

clearml

3 years ago

0 Votes

14 Answers

1K Views

0 Votes 14 Answers 1K Views

Hi Folks Any Info On When The Helm Chart Will Be Updated For 1.0.1 ?

Hi folks any info on when the helm chart will be updated for 1.0.1 ?

clearml

3 years ago

0 Votes

4 Answers

1K Views

0 Votes 4 Answers 1K Views

Is It Not Possible To Fire And Forget A Pipeline? In This Example, What’S The Step To Just Trigger A Pipeline Without Waiting? Without The Pipe.Wait() I Don’T See The Stages Running

Is it not possible to fire and forget a pipeline? In this example, what’s the step to just trigger a pipeline without waiting? Without the pipe.wait() I don’...

clearml

3 years ago

0 Votes

8 Answers

955 Views

0 Votes 8 Answers 955 Views

Is It Possible To Do Additional Setup (The

Is it possible to do additional setup (the extra_bash_init_script , container_bash_script etc per task? Say a task requires a system package, install it befo...

clearml

3 years ago

0 Votes

4 Answers

975 Views

0 Votes 4 Answers 975 Views

Similar question but when running a pipeline, can I control the tags that the tasks a pipeline creates? add_pipeline_tags adds tags from pipeline to the task...

clearml

3 years ago

0 Votes

30 Answers

1K Views

0 Votes 30 Answers 1K Views

Can Someone Help Me With Deploying This Example Model (From Triton Inference Server) Deployed In Clearml-Serving? Too Many Random Errors For Me To Figure It Out

Can someone help me with deploying this example model (from triton inference server) deployed in clearml-serving? Too many random errors for me to figure it ...

clearml

3 years ago

Show more results

0 Has Anyone Done This Exact Use Case - Updates To Datasets Triggering Pipelines?

AgitatedDove14 - any thoughts?

3 years ago

0 Apart From Having Packages In Requirements.Txt, Does Clearml Expect Them To Be Actuall Installed To Add Them As Installed Packages For A Task?

Is it not possible to say just look at my requirements.txt file and the imports in the script?

3 years ago

0 Why Would Every Submitted Task Be Aborted Directly?

I was having this confusion as well. Did behavior for execute_remote change that it used to be Draft is Aborted now?

3 years ago

0 Apart From Having Packages In Requirements.Txt, Does Clearml Expect Them To Be Actuall Installed To Add Them As Installed Packages For A Task?

Ok i did a pip install -r requirements.txt and NOW it picks them up correctly

3 years ago

0 Why Would Every Submitted Task Be Aborted Directly?

Any specific use case for the required “draft” mode?

Nothing except that Draft makes sense feels like the task is being prepped and Aborted feels like something went wrong

3 years ago

0 Why Would Every Submitted Task Be Aborted Directly?

Yeah sounds good to me

3 years ago

0 I Saw Some Talk Of Clearml + Kedro On Reddit. Is That A Good Approach?

This actually ties well with the next version of pipelines we are working on

Is there a way to see a roadmap on such things AgitatedDove14 ?

3 years ago

0 Last Reminder For Our First Community Talk! We'Ll Be Discussing Our Latest Feature - Clearml Apps! We'Ll Also Review The New Clearml Roadmap And Have A Q&A Session With Our Cto !

Chance of recording being available?

3 years ago

0 I Am Not Familiar With Pytorch, But Is It Expected That So Many “Models” Are Created? These Are Being Repeated As Well For A Single Task (This Is Training A T5_Model With Transformers):

Yeah concerns make sense.

The underlying root issue is unnecessary models being added or at least what I think are unnecessary and even happening when you load a model to test.

Do people use ClearML with huggingface transformers? The code is std transformers code.

Will create an issue.

3 years ago

0 I Am Not Familiar With Pytorch, But Is It Expected That So Many “Models” Are Created? These Are Being Repeated As Well For A Single Task (This Is Training A T5_Model With Transformers):

AgitatedDove14 - yeah wanted to see what’s happening before disabling as I wasn’t sure if this is what’s expected.

3 years ago

0 Sorry Folks Too Many Questions - If I Have A Project (And I Set The Output Uri In It While Creating, To A S3 Folder) How Can I Ensure That A Experiment (Task) That I Run On My Local Outputs The Model To The Uri?

Ref of dvc doing about the same - https://github.com/iterative/dvc/blob/master/dvc/fs/s3.py#L127-L134

3 years ago

0 Has Anyone Done This Exact Use Case - Updates To Datasets Triggering Pipelines?

What could those triggers be?

3 years ago

0 Playing Around With Hpo For First Time. I Am Giving This As Hyperparameter:

how do you see things being used as the most normal way?

3 years ago

0 Would Be Great If Clearml Is Represented In The Matrix. (It'S Mentioned As A Mlops Platform And The Origin Year)

Seems like usual? https://github.com/thoughtworks/mlops-platforms#contributions

3 years ago

0 Does Clearml Have A Way To Do Human In The Loop Activities As Part Of A Pipeline That I Might Have Missed?

Something like this - https://aws.amazon.com/blogs/machine-learning/processing-pdf-documents-with-a-human-loop-using-amazon-textract-and-amazon-augmented-ai/

2 years ago

0 Playing Around With Hpo For First Time. I Am Giving This As Hyperparameter:

It completed after the max_job limit (10)

3 years ago

0 If I Have A Task And A Dataset Is Being Created In A Task, How Can I Get A “Link” That This Dataset Is Created In This Task, Similar To How Model Has The Task Where It Came From

Doesn’t set the parent though 🤔

3 years ago

0 If I Have A Task And A Dataset Is Being Created In A Task, How Can I Get A “Link” That This Dataset Is Created In This Task, Similar To How Model Has The Task Where It Came From

id works, thanks 👍

3 years ago

0 How Do People Generally Handle Moving From Experimental Mode With Notebooks And Then Running Pipelines For Production Training And Beyond?

Would like to get to the Maturity Level 2 here

https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning#mlops_level_2_cicd_pipeline_automation

3 years ago

0 Anyone Doing Sagemaker With Clearml - Something Like The K8S Glue But The Tasks Are Pulled Into Sagemaker Training Jobs

Sagemaker will make that easy, especially if I have sagemaker as the long tail choice. Granted at a higher cost

3 years ago

0 Anyone Doing Sagemaker With Clearml - Something Like The K8S Glue But The Tasks Are Pulled Into Sagemaker Training Jobs

Got it. Never ran GPU workload in EKS before. Do you have any experience and things to watch out for?

3 years ago

0 Apart From Having Packages In Requirements.Txt, Does Clearml Expect Them To Be Actuall Installed To Add Them As Installed Packages For A Task?

So packages have to be installed and not just be mentioned in requirements / imported?

3 years ago

0 Playing Around With Hpo For First Time. I Am Giving This As Hyperparameter:

I am running from noebook and cell has returned

3 years ago

0 How Do People Generally Handle Moving From Experimental Mode With Notebooks And Then Running Pipelines For Production Training And Beyond?

GrumpyPenguin23 both in general and clearml 🙂

3 years ago

0 Apart From Having Packages In Requirements.Txt, Does Clearml Expect Them To Be Actuall Installed To Add Them As Installed Packages For A Task?

I am doing Task.init but it’s not adding expected libraries imported in the script or from requirements.txt

3 years ago

0 Has Anyone Done This Exact Use Case - Updates To Datasets Triggering Pipelines?

More interested in some way for doing this ssytem wide

3 years ago

0 I Am Not Familiar With Pytorch, But Is It Expected That So Many “Models” Are Created? These Are Being Repeated As Well For A Single Task (This Is Training A T5_Model With Transformers):

Does a pipeline step behave differently?

3 years ago

0 Apart From Having Packages In Requirements.Txt, Does Clearml Expect Them To Be Actuall Installed To Add Them As Installed Packages For A Task?

Thanks for the confirmation.

3 years ago

0 Is There A Way To Do S3 -> S3 Copy While Doing A Dataset? I Don’T Want To Get It To Local From S3 And Then Upload As A Dataset To S3

Yes using clearml-data.

Can I pass a s3 path to ds.add_files() essentially so that I can directly store a dataset without having to get the files to local and then upload again. Makes sense?

3 years ago

0 Is There A Way I Can Create A Dataset As Part Of A Pipeline And Be Able To See That This Dataset Came From This Pipeline / Task ?

Will try it out. Pretty impressed 🙂

3 years ago

Show more results