TrickySheep9

71 Questions, 428 Answers

Active since 10 January 2023

Last activity one year ago

Reputation

Badges 1

383 × Eureka!

Questions 71
Answers 428

0 Votes

6 Answers

2K Views

0 Votes 6 Answers 2K Views

Off Topic - Is There A Slack / Discord Community For General Ml Discussions?

Off topic - Is there a slack / discord community for general ML discussions?

clearml

4 years ago

0 Votes

7 Answers

2K Views

0 Votes 7 Answers 2K Views

How Do People Generally Handle Moving From Experimental Mode With Notebooks And Then Running Pipelines For Production Training And Beyond?

How do people generally handle moving from experimental mode with notebooks and then running pipelines for production training and beyond?

clearml

4 years ago

0 Votes

18 Answers

2K Views

0 Votes 18 Answers 2K Views

Looking At Clearml-Serving - Two Questions - 1, What’S The Status Of The Project 2. How Does One Say How A Model Is Loaded And Served Etc? For Example, If I Have A Spacy Ner Model, I Need To Specify Some Custom Code Right?

Looking at clearml-serving - two questions - 1, what’s the status of the project 2. how does one say how a model is loaded and served etc? For example, if i ...

clearml

4 years ago

0 Votes

0 Answers

2K Views

0 Votes 0 Answers 2K Views

Can I Do These From A Notebook?

Can I do these from a notebook? https://github.com/allegroai/clearml/blob/master/examples/pipeline/step2_data_processing.py

clearml

4 years ago

0 Votes

9 Answers

2K Views

0 Votes 9 Answers 2K Views

Is There An Efficient Way To Query All Unique Models (Ie Excluding Versions) In A Project?

Is there an efficient way to query ALL unique models (ie excluding versions) in a project?

clearml

4 years ago

0 Votes

12 Answers

2K Views

0 Votes 12 Answers 2K Views

When Using Something Like Pdf2Image Which Requires Poppler (Which Can Be Installed With Conda), How Can I Ensure That The Task Can Run On An Agent Correctly? As Of Now It Doesn’T Know About Poppler

When using something like pdf2image which requires poppler (which can be installed with conda), how can I ensure that the task can run on an agent correctly?...

mlops

4 years ago

0 Votes

19 Answers

2K Views

0 Votes 19 Answers 2K Views

I Saw Some Talk Of Clearml + Kedro On Reddit. Is That A Good Approach?

I saw some talk of ClearML + Kedro on reddit. Is that a good approach?

clearml

4 years ago

0 Votes

9 Answers

2K Views

0 Votes 9 Answers 2K Views

When Using Docker Mode (And Specifically K8S Glue), What Are The Options For Caching? One Option Is Definitely Having A Base Image That Has The Things Needed. Anything Else? Thanks!

When using docker mode (and specifically k8s glue), what are the options for caching? One option is definitely having a base image that has the things needed...

clearml

4 years ago

0 Votes

27 Answers

2K Views

0 Votes 27 Answers 2K Views

Playing Around With Hpo For First Time. I Am Giving This As Hyperparameter:

Playing around with HPO for first time. I am giving this as hyperparameter: DiscreteParameterRange("General/num_iterations", values=[10, 20, 30]), Is it expe...

clearml

4 years ago

0 Votes

17 Answers

2K Views

0 Votes 17 Answers 2K Views

Is It Possible To Increase The Polling Interval For K8S Glue? Currently It Is 5 Seconds I Believe. Would Adding An Argument For It Help? Can Do A Pr If So

Is it possible to increase the polling interval for k8s glue? Currently it is 5 seconds I believe. Would adding an argument for it help? Can do a PR if so

clearml

4 years ago

0 Votes

12 Answers

2K Views

0 Votes 12 Answers 2K Views

Apart From Having Packages In Requirements.Txt, Does Clearml Expect Them To Be Actuall Installed To Add Them As Installed Packages For A Task?

Apart from having packages in requirements.txt, does clearml expect them to be actuall installed to add them as installed packages for a task?

clearml

4 years ago

Show more results

0 How Can I Clone A Task And Execute_Remotely The Cloned Task With Exit_Process=False. It Currently Kills The Notebook Kernel. If I Say Exit_Process=False, It Says Clone Cannot Be False. Why The Restriction? What To Do In A Notebook To Run A Task Remotely

I am providing a helper to run a task in queue after running it locally in the notebook

4 years ago

0 Wondering How The Services To Access Can Be Setup

Hey SuccessfulKoala55 - this was related to the previous message. Had it clarified with AgitatedDove14

4 years ago

0 What’S The Best Way To Get The List Of All Projects (While In A Notebook)? Hitting The Api?

Yeah got it, thanks!

4 years ago

0 Hi, I'M Trying To Understand If Clearml Supports My Usecase: I Generate My Own Data (Problem-Specific) And Generation Process Is Uploading A Parquet Dataset (In Fact, Parquet Contains Multiple Files Obv.) To S3. Is There A Way To "Register" The Dataset In

+1 to this question

4 years ago

0 Playing Around With Hpo For First Time. I Am Giving This As Hyperparameter:

Only one param, just playing around

4 years ago

0 Is It Not Possible To Add Artifacts To A Completed Task?

tasks.add_or_update_artifacts/v2.10 (Invalid task status: expected=created, status=completed

4 years ago

0 I Want To Use The Pipelinecontroller Which Assumes A Task Already Been Executed And As Ready As Template. I Never Executed The Task Before And It Is A Very Long Running Task. Is There A Way To Create The Template Without Executing All The Way Through?

Here’s an example - https://github.com/allegroai/clearml-serving/blob/b5f5d72046f878bd09505606ca1147d93a5df069/clearml_serving/serving_service.py#L58

4 years ago

0 Clearml Server Deployment Uses Node Storage. If More Than One Node Is Labeled As App=Clearml, And You Redeploy Or Update Later, Then Clearml Server May Not Locate All Your Data.

AgitatedDove14 - these instructions are out of date? https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_server_kubernetes_helm.html

4 years ago

0 Any Pointers On Running Gpu Tasks With K8S Glue?

Can you let me know if i can override the docker image using template.yaml?

4 years ago

0 Hi, How Do I Switch Clearml Server To Run On Https With A Self Signed Cert?

I deploy to kubernetes and have an ingress and ALB on top that terminates ssl

4 years ago

0 Clearml Server Deployment Uses Node Storage. If More Than One Node Is Labeled As App=Clearml, And You Redeploy Or Update Later, Then Clearml Server May Not Locate All Your Data.

The repo seems to be

4 years ago

0 Is It Possible To Add A Callback For A Pipeline From A Step?

AgitatedDove14 is it possible to get the pipeline task running a step in a step? Is task.parent something that could help?

4 years ago

0 This Verify Param To Decode Of Pyjwt Seems To Be Removed And In A Fresh Intallation We Are Seeing This Error:

As the verify param was deprecated and now removed

4 years ago

0 Is It Possible To Increase The Polling Interval For K8S Glue? Currently It Is 5 Seconds I Believe. Would Adding An Argument For It Help? Can Do A Pr If So

https://github.com/allegroai/clearml-agent/blob/aede6f4bac71c8fc56e7cf982318a48527953a3c/clearml_agent/glue/k8s.py#L217

4 years ago

0 Is It Possible To Add A Callback For A Pipeline From A Step?

What happens if I do blah/dataset_url ?

4 years ago

0 Is There A Way To Do S3 -> S3 Copy While Doing A Dataset? I Don’T Want To Get It To Local From S3 And Then Upload As A Dataset To S3

Yes using clearml-data.

Can I pass a s3 path to ds.add_files() essentially so that I can directly store a dataset without having to get the files to local and then upload again. Makes sense?

4 years ago

0 How Do People Solve This? If I Am Pip Installing A Custom Package From .Tar.Gz, How Can I Ensure That If I Run The Experiment (Initially Run From A Notebook) Via The Queueing It Can Be Properly Installed Steps - Notebook -> Get A Tar.Gz From S3 -> Pip I

AgitatedDove14 - apologies for late reply. So to give context this in a Sagemaker notebook which has conda envs.

I use a lifecycle like this to pip install a package (a .tar.gz downloaded from s3) in a conda env- https://github.com/aws-samples/amazon-sagemaker-notebook-instance-lifecycle-config-samples/blob/master/scripts/install-pip-package-single-environment/on-start.sh

In the notebook I can do things like create experiments and so on. Now the problem is in running the cloned experimen...

4 years ago

0 I Am Seeing That Some Steps In A Pipeline Are Being Skipped. Like For Example, In A Pipeline With 4 Steps, It’S Directly Starting At Step 3. Is There Some Reason For This, Some Optimization Kicking In?

pipeline code itself is pretty standard

4 years ago

0 Any Pointers On Running Gpu Tasks With K8S Glue?

So didn’t want to update the defaults

4 years ago

0 Any Info On The Lifecycle Of Datasets Downloaded To $Home/.Clearml/Cache/Storage_Manager/Datasets Via Get_Local_Copy I Have A Task Running And I Was Watching The Above Path And Datasets Were Being Downloaded And Then They Are All Removed And For A Partic

Will try it out. A weird one this.

4 years ago

0 Sorry Folks Too Many Questions - If I Have A Project (And I Set The Output Uri In It While Creating, To A S3 Folder) How Can I Ensure That A Experiment (Task) That I Run On My Local Outputs The Model To The Uri?

Anything I am missing

4 years ago

If i were to push the private package to, say artifactory, is it possible to use that do the install?

4 years ago

0 When Running An Experiment From A Notebook, It Knows It’S A Notebook And Automatically Adds The Notebook As An Artifact Right? And The Uncommited Changes Becomes The Nottebook Converted To A Script? In One Case I Am Seeing Actual Git Diff Coming In Instea

Can I switch off git diff (change detection?)

4 years ago

0 Anyone Doing Sagemaker With Clearml - Something Like The K8S Glue But The Tasks Are Pulled Into Sagemaker Training Jobs

AgitatedDove14 - i had not used the autoscaler since it asks for access key. Mainly looking for GPU use cases - with sagemaker one can choose any instance they want and use it, autoscaler would need set instance configured right? need to revisit. Also I want to use the k8s glue if not for this. Suggestions?

4 years ago

0 Anyone Doing Sagemaker With Clearml - Something Like The K8S Glue But The Tasks Are Pulled Into Sagemaker Training Jobs

For different workloads, I need to habe different cluster scaler rules and account for different gpu needs

4 years ago

0 How Do People Generally Handle Moving From Experimental Mode With Notebooks And Then Running Pipelines For Production Training And Beyond?

Would like to get to the Maturity Level 2 here

https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning#mlops_level_2_cicd_pipeline_automation

4 years ago

0 Anyone Doing Sagemaker With Clearml - Something Like The K8S Glue But The Tasks Are Pulled Into Sagemaker Training Jobs

Running multiple k8s_daemon rightt? k8s_daemon("1xGPU") and k8s_daemon('cpu') right?

4 years ago

0 Can Someone Help Me With Deploying This Example Model (From Triton Inference Server) Deployed In Clearml-Serving? Too Many Random Errors For Me To Figure It Out

forking and using the latest code fixes the boto issue at least

4 years ago

0 Is There An Efficient Way To Query All Unique Models (Ie Excluding Versions) In A Project?

If i publish a keras_mnist model and experiment on, each of it gets pushed as a separate Model entity right? But there’s only one unique model with multiple different version of it

4 years ago

0 More An Internal Utilities Usage Question - If I Have A Script (.Py) Can I Use An Utility That Clearml Uses To Get The Packages Used By The Script? Was Looking At

Get the imports, yes.

4 years ago

Show more results