TrickySheep9

71 Questions, 428 Answers

Active since 10 January 2023

Last activity one year ago

Reputation

Badges 1

383 × Eureka!

Answers 428

0 I Am Not Familiar With Pytorch, But Is It Expected That So Many “Models” Are Created? These Are Being Repeated As Well For A Single Task (This Is Training A T5_Model With Transformers):

Yeah concerns make sense.

The underlying root issue is unnecessary models being added or at least what I think are unnecessary and even happening when you load a model to test.

Do people use ClearML with huggingface transformers? The code is std transformers code.

Will create an issue.

4 years ago

0 I Am Seeing That Some Steps In A Pipeline Are Being Skipped. Like For Example, In A Pipeline With 4 Steps, It’S Directly Starting At Step 3. Is There Some Reason For This, Some Optimization Kicking In?

From the code - it’s supposed to not cache if task override is different? I also have task_override that adds a version which changes each run

4 years ago

0 Sorry Folks Too Many Questions - If I Have A Project (And I Set The Output Uri In It While Creating, To A S3 Folder) How Can I Ensure That A Experiment (Task) That I Run On My Local Outputs The Model To The Uri?

Yeah sure

4 years ago

0 Clearml Server Deployment Uses Node Storage. If More Than One Node Is Labeled As App=Clearml, And You Redeploy Or Update Later, Then Clearml Server May Not Locate All Your Data.

Beyond this have the UI running, have to start playing with it. Any suggestions for agents with k8s?

4 years ago

0 After I Have Create A Task And Closed It In A Notebook, Any Activity Seems To Trigger Another Task. For Example:

another place I was using was to see if i am in a pipeline task

4 years ago

0 Getting This Error At

4 years ago

0 After I Have Create A Task And Closed It In A Notebook, Any Activity Seems To Trigger Another Task. For Example:

How can I ensure that additional tasks aren’t created for a notebook unless I really want to?

4 years ago

0 How Can I Clone A Task And Execute_Remotely The Cloned Task With Exit_Process=False. It Currently Kills The Notebook Kernel. If I Say Exit_Process=False, It Says Clone Cannot Be False. Why The Restriction? What To Do In A Notebook To Run A Task Remotely

workflow

4 years ago

0 Can Someone Help Me With Deploying This Example Model (From Triton Inference Server) Deployed In Clearml-Serving? Too Many Random Errors For Me To Figure It Out

don’t know what’s happening there

4 years ago

0 When Using Something Like Pdf2Image Which Requires Poppler (Which Can Be Installed With Conda), How Can I Ensure That The Task Can Run On An Agent Correctly? As Of Now It Doesn’T Know About Poppler

Basic question - i am running clearml agent in a ubuntu ec2 machine. Does it use docker by default? I thought it uses docker only if I add the --docker flag?

4 years ago

0 When Running An Experiment From A Notebook, It Knows It’S A Notebook And Automatically Adds The Notebook As An Artifact Right? And The Uncommited Changes Becomes The Nottebook Converted To A Script? In One Case I Am Seeing Actual Git Diff Coming In Instea

Is there some doc or relevant code on exactly what's happening? The behaviour has been random

4 years ago

Ref of dvc doing about the same - https://github.com/iterative/dvc/blob/master/dvc/fs/s3.py#L127-L134

4 years ago

0 If I Have A Dataset And I Process It And I Want The Processed Data As Another Dataset, Is Parent The Right Approach?

Yeah got it

4 years ago

0 What’S The Easiest Way To Update The Repo Url Alone For A Task? Need - In My Ci, The Url Used Is Https But I Need The Ssh Url To Be Used. I See That We Can Pass Repo To Task.Create But Not Task.Init

Only allowed to have ssh key, not username-password

4 years ago

0 Clearml Server Deployment Uses Node Storage. If More Than One Node Is Labeled As App=Clearml, And You Redeploy Or Update Later, Then Clearml Server May Not Locate All Your Data.

AlertBlackbird30 - got it running. Few comments:

Nodeport is set by default despite being parameter in values.yml. For example:` webserver:
extraEnvs: []

service:
type: NodePort
port: 80 `2. Ingress was using 8080 for webserver but service was 80
3. Had to change path in ingress to “/*” instead of “/” to get it working for me

4 years ago

Yeah getting a better sense

4 years ago

0 Sorry If This Is Addressed In Docs Somewhere - Does An Agent Map To One Worker (1:1) All The Time? Can I Have Multiple Workers In An Agent? Does Docker Mode Enable Multiple Workers Per Agent?

As of now my understanding is agent and worker are just interchangeable words

4 years ago

0 Why Would Every Submitted Task Be Aborted Directly?

Yeah sounds good to me

4 years ago

0 Does Clearml-Session Work In A Kubernetes Environment?

But ok the summary is I guess it doesn’t work in a k8s env

4 years ago

0 Quick Qn, When Using The Clearml-Task, How To Specify The Output_Uri.

BTW when I started using s3, I was thinking I needed to specify ouput_uri for each task. Soon realized that you just need the prefix where you want to put it into, and clearml will take care of project etc being appended to the path. So for most usecases, a single output uri set in conf should work.

4 years ago

0 Is It Not Possible To Add Artifacts To A Completed Task?

Thanks that works. Had to use Task.completed() for my version

4 years ago

0 After I Have Create A Task And Closed It In A Notebook, Any Activity Seems To Trigger Another Task. For Example:

I do it to get project name

4 years ago

0 Is There A Way To Do S3 -> S3 Copy While Doing A Dataset? I Don’T Want To Get It To Local From S3 And Then Upload As A Dataset To S3

Yes using clearml-data.

Can I pass a s3 path to ds.add_files() essentially so that I can directly store a dataset without having to get the files to local and then upload again. Makes sense?

4 years ago

0 If I Have A Dataset And I Process It And I Want The Processed Data As Another Dataset, Is Parent The Right Approach?

In this case I have data and then set of pickles created from the data

4 years ago

0 I Might Be Missing Something, But Is There No Way To Clear All Tags Or Remove A Tag In A Task? I Only See A Add_Tags()

Ok couldn’t see it in the docs - https://clear.ml/docs/latest/docs/references/sdk/task

4 years ago

0 I Am Not Familiar With Pytorch, But Is It Expected That So Many “Models” Are Created? These Are Being Repeated As Well For A Single Task (This Is Training A T5_Model With Transformers):

Maybe two thing here:

If Task.init() is called in an already running task, don’t reset auto_connect_frameworks? (if i am understanding the behaviour right) Option to disable these in the clearml.conf

4 years ago

0 Playing Around With Hpo For First Time. I Am Giving This As Hyperparameter:

how do you see things being used as the most normal way?

4 years ago

0 Playing Around With Hpo For First Time. I Am Giving This As Hyperparameter:

Ah ok there’s only optimizer.stop in the example

4 years ago

0 Playing Around With Hpo For First Time. I Am Giving This As Hyperparameter:

The job itself doesn’t have any other param

4 years ago

0 Playing Around With Hpo For First Time. I Am Giving This As Hyperparameter:

Yeah, Curious - is a lot of clearml usecases not geared for notebooks?

4 years ago

Show more results