CostlyOstrich36

0 Questions, 4173 Answers

Active since 10 January 2023

Last activity 2 years ago

Reputation

Answers 4173

0 Hello Folks! I Have A Pipeline With Three Tasks: A, B, And C I Want To Set It Up So That: A Gets Assigned A Machine (E.G. Based On The Queue) B Always Gets Assigned To The Same Machine As A (But May Run In A Different Docker Etc.) C Will Be Submitted To

Is there a specific reason you would want them executed on the same machine? Cache?

3 years ago

0 In My Current Project I Generate The Data From An Sql Query. Is The Only Way To Register The Dataset With Clearml To Write The Files To Disk First Or Is There Another Method? This Leads Into The Second Issue I Have, Which Is What Happens When I Store The

@<1545216070686609408:profile|EnthusiasticCow4> , I think add_files always generates a new version. I mean, you add files to your dataset, so the version has changed. Does that make sense?

2 years ago

Hi RoughTiger69 , you can specify a queue per step with execution_queue parameter in add_function_step
https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller

Same goes for the docker image - docker parameter add_function_step

3 years ago

0 Hi!, Is There An Installation Guide For

Yes 🙂

3 years ago

0 Is There Any Simple Way To Orchestrate A Batch To Train A Model With Different Features (In Order To Do Feature Selection, For Example) Through A Single .Py File? I Saw The Following Example

Hi ShallowGoldfish8 , can you elaborate please? You mean train with different data?

3 years ago

0 Another Hpo App Question: Using Ui In "Optimization Configuration" I Can Choose Only One Metric. Is This By Design Or I'M Missing Something?

Hi @<1523701062857396224:profile|AttractiveShrimp45> , I think this is currently by design. How would you suggest doing multiple metric optimization - priority between metrics after certain threshold is met?

2 years ago

0 I'M Trying To Import Offline Tasks, But Before I Do So I Want To Make Sure I'M Not Creating Duplicates By Accessing The Task_Name And Project_Name I'M Going To Sync. I'M Trying To Run

Can you please elaborate what AWS Lambda is and what your use case is with it? When running in a regular state does this error occur?

3 years ago

0 Hey All, I'M Running A Task Inside Docker, And One Of The Functions Call: Latest_Dataset = Dataset.Get(Dataset_Project=Dataset_Project]) A I Get: Raise Valueerror('Could Not Load Dataset Id={} State'.Format(Task.Id)) Valueerror: Could Not Load Dataset

I think you need to do latest_dataset = Dataset.get(dataset_id=<DATASET_ID>)

3 years ago

0 Is There Anywhere In The Web Ui Where One Can See The Clearml Server Version Running? I Keep Getting "Version 1.1.1 Is Now Available" Even Though I'M Pretty Sure I Took All The Steps To Update To The Latest Version

UnevenDolphin73 , can you please provide a screenshot of the window, message and the URL in sight?

4 years ago

0 Hey Everyone! First Post Here. My Issue Is The Python_Binary I Set In The .Conf File Doesn'T Seem To Get Used By The Task I'M Running. The Task'S Output Confirms

Is there any specific reason you're not running in docker mode? Running in docker would simplify things

one year ago

0 Hello, I Want To Use Custom Packages In My Code.It Works Fine When I Run It Locally.But When I Clone The Experiments And Run On Agent It Shows Package Not Found Error In Agent. Can Anyone Help Me Out How To Use Custom Packages In Clearml? Which Parameter

Sorry, not available for calls

2 years ago

0 Hi, I Noted That If I Run My Codes On My Laptop With Remote_Execute Off A Python3.8 Venv, And When The Remote Task Starts Executing But The Image Is Installed With A Different Version Of Python, Say Python3.8, We Would Encounter Errors With Venv. At This

This can be a bit of a problem as well since not all packages for 3.8 have the same versions available for 3.6 for example. It's recommended to run on the same python versions OR have the required python version installed on the remote machine

3 years ago

0 Hello People, Is There An Easy Way For Clearml To Work With

TartSeal39 , Hi 🙂
Do I understand correctly that you want to push parameters for Task.create() from a .yml file?

4 years ago

0 What Determines The Value Of User? (See Screenshot) Right Now All My Teammates' Experiments Show Up Under My Own User Name. Thanks

WittyOwl57 , It determines the user that created the object. What is the sign in method that you and your team are using?

3 years ago

0 Hi All, I Tried To Create A Aws Autoscaler With A Cpu Only Instance

Hi @<1539780258050347008:profile|CheerfulKoala77> , it seems that you're trying to use the same 'Workers Prefix' setup for two different autoscalers, workers prefix must be unique between autoscalers

2 years ago

0 Hi, I Would Like To Understand How I Can Set The Pip Cache Location For My Agent, I Thought That I Already Had The Right Setting With

Hi ClumsyElephant70 ,
What about
# pip cache folder mapped into docker, used for python package caching docker_pip_cache = ~/.clearml/pip-cache # apt cache folder mapped into docker, used for ubuntu package caching docker_apt_cache = ~/.clearml/apt-cache

3 years ago

0 Hi, I'Ve Multiple Tasks Setup In A Complex Pipeline. How Can I;

SubstantialElk6 ,

We were trying with 'from task' at the moment. But the question apply to all methods.

You can specify this using add_function_step(..., execution_queue="<QUEUE>")

Make certain tasks in the pipeline run in the same container session, instead of spawning new container sessions? (To improve efficiency)

I'm not sure this is possible currently. This could a be nice feature request. Maybe open a github request?

3 years ago

0 Hi, I Am Trying To Use The Parameterset For Hyper-Parameter Tuning With Dependencies, An Example Of How I Use It: Parameterset([{“Prm1”:1, “Prm2": 1},{“Prm1”:2, “Prm2":2}]) But I Get A Warning :

Can you please add a screenshot of how the hyper params show in the UI for you?

3 years ago

0 Hi, I Notice A New Behavuour With Clearml-Agent=1.1.0. When It Is Installing The Packages I Nrequirements.Txt, It Failed With.

SubstantialElk6 , can you please verify that you have all the required packages installed locally ? Also in your ~/clearml.conf what is the setting of agent.package_manager.system_site_packages

3 years ago

0 Hi, I’M Running An Agent In K8S Using The Glue. I’M Trying To Set The Setup Shell Script To Install Some Packages Through Apt But I Don’T Think It Even Runs. I Dont See Anything In The Console Logs That Says It Does, At Least. Is There Anyway I Can Debug

Setup shell script works in docker mode

one year ago

0 Is There A Way To Programatically Access The Tasks A

WackyRabbit7 I don't believe there is currently a 'children' section for a task. You could try managing the children to access them later.

One option is add_pipeline_tags(True) this should mark all the child tasks with a tag of the parent task

3 years ago

0 Hey All. I'M Getting This Error:

Hi @<1671689469218000896:profile|PleasantWalrus77> , is this AWS S3 or something like minio?

one year ago

0 Q: Hello, I Am A Clearml Begineer, Can I Use Clearml Open-Source To Build Our Customized Platform? What Are The Limits Of Clearml While Building Mlops Platform?

I am not very familiar with KubeFlow but as far as I know it is mainly for orchestration whereas ClearML offers a full E2E solution 🙂

3 years ago

0 Q: Hello, I Am A Clearml Begineer, Can I Use Clearml Open-Source To Build Our Customized Platform? What Are The Limits Of Clearml While Building Mlops Platform?

I would advise using ClearML 😄

3 years ago

0 Hey All, Is There A Way To Transfer A Workspace To Someone Else? One Of My Reports Did The Original Spike On Which Experiment Tracking Platform To Use And The Company Workspace Is Under His Name. With A New Hire, We'Re Over The Seat Count Now And Blowing

Hi @<1828241063677005824:profile|ReassuredAlligator91> , I think as long as you have access to the email account that signed up to the original workspace, you should be OK. Just pass down the credentials to the account to the relevant people (yourself for example) and manage it from there.

WDYT?

4 months ago

What do you mean by organization? In Enterprise, you have users, roles & access controls based on those roles.

4 months ago

Looks like it's not running in docker mode 🙂
Otherwise you'd have the 'docker run' command at the sttart

one year ago

0 Hi Team! Is There A Way To Make Clearml’S Aws Autoscaler And Queues Resource-Aware Please? I.E. If We Can Say, As We Enqueue Our Job, How Much Ram Or Gpu-Ram Or Even Gpus It Needs, Have The Scheduler/Autoscaler Dispatch The Job To Instances That Are Of Th

Yep, although I'm quite sure you could build some logic on top of that to manage proper queueing

2 years ago

0 For Model Monitoring, Is There A Good Feature In Clearml? If Not Is There A Go-To Tool That Works Well With This Framework?

Hi @<1545216070686609408:profile|EnthusiasticCow4> , I suggest you try ClearML-Serving
None

2 years ago

0 Hello, I’M Trying To Log (Via

Hi RattyLouse61 ,

Do you have an example of the parameters you're trying to connect?

3 years ago

Show more results