CostlyOstrich36

0 Questions, 4213 Answers

Active since 10 January 2023

Last activity 2 years ago

Reputation

Answers 4213

0 I Wanted To Ask, How To Run Pipeline Steps Conditionally? E.G If Step Returns A Specific Value, Exit The Pipeline Or Run Another Step Instead Of The Sequential Step

Hmmmm this looks like what you're looking for:
https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller#stop-1
Tell me if this helps 🙂

3 years ago

0 Hi All, I Have A Setup Where Data Scientists Can Run Notebooks In Headless Mode Remotely Using Sagemaker Processing Jobs, However I Have An Issue With Clearml Credentials. I Don'T Want Them To Send Their Own Api Key And Secret Key To A Remote Server Every

Hi @<1523703397830627328:profile|CrookedMonkey33> , you can also set the credentials with an env variable. Would that work?

8 months ago

0 I Was Trying To Use The Callback Option In

Hi @<1719162252994547712:profile|FloppyLeopard12> , not sure what you're trying to do, can you please elaborate?

one year ago

0 Hello, I Have A Question Regarding The Use Of Hydra For Config In Clearml Pipelines. I Have Setup A Pipeline Using The Pipeline Decorator (Based On The Example You Provided In Github), With The Config Values Hard-Coded In Code, And It Works Well. I Then T

YummyLion54 , let me take a look 🙂

4 years ago

0 Hi! My Agent Has Very Limited Disk Space. And When A New Task Is Launched And The Loading Of New Data Begins, There May Not Be Enough Space. Is There A System For Automatically Clean Up Old Currently Unused Data In

DepressedFish57 , I'm not sure there is something for this. There is a cleanup service to kill old failed/aborted tasks but to clear caches you'd have to handle it manually 🙂

3 years ago

0 Hi There, I Have A Bit Of A Problem With Aws Secrets: I Pass Keys As Env Var To Clearml-Agents To Retrieve Data From A Bucket In Us-East-1 But I Use A Bucket To Store Task Artifacts In A Bucket In Eu-Central-1. So When I Pass Aws Keys As Env Vars, The Tas

Do you explicitly specify the region?
Also, please try specifying the buckets directly in your ~/clearml.conf

The relevant section is sdk.aws.s3.credentials

It would look something like this:

{ host: "<BUCKET_A_HOST>" key: "<KEY>" secret: "<SECRET>" region: "<REGION>" }

4 years ago

0 Hi All! Quick Question: Can Clearml-Agent

Build a Docker container that when launched executes a specific experiment, or a clone (copy) of that experiment.

From the docs

2 years ago

0 If You Rely On Task To Automatically Upload Artifacts, It Seems You Cannot Retrieve Them Using

TrickyRaccoon92 , Hi!

Yes I believe this is the intended behavior. Since if you upload automatically you can upload many artifacts during a single run, whereas when you upload manually you create the object yourself.

4 years ago

0 Hi, Question Regarding

If you killed all processes directly, there can't be any workers on that machine. It means that these two workers are running somewhere else...

3 years ago

0 Is It Possible To Change The

SuperiorPanda77 , yes! 🙂

3 years ago

0 Hey, I Was Planning To Save The Best Trained Model And Get A Task Id For It, So I Could Use It Later As

DeliciousSeal67 , something along these lines:
task.upload_artifact('<ARTIFACT_NAME>', artifact_object=os.path.join('<FOLDER>', '<FILE_NAME>'))
So in your case it would be along the lines of
task.upload_artifact('trained_model', 'model_folder/best_mode.pt')

3 years ago

0 Hello Folks! I Have A Pipeline With Three Tasks: A, B, And C I Want To Set It Up So That: A Gets Assigned A Machine (E.G. Based On The Queue) B Always Gets Assigned To The Same Machine As A (But May Run In A Different Docker Etc.) C Will Be Submitted To

Oh, you want the same machine to execute the two tasks/steps?

3 years ago

0 Hello, Clearml! Let'S Say That I Have A Git Repository In Format

What about if you specify the repo user/pass in clearml.conf?
I think it removes the user/pass so it wouldn't be shown in the logs

2 years ago

0 Hi, I Wanted To Report A Matplotlib Figure. In The Experiment Tab Everything Works As Expected And Is Displayed Correctly. However, When Embedding This Figure In A Clearml Report, Everything Changes Colour From Black To White, But Not The Axis Labels (See

Hi @<1717350310768283648:profile|SplendidFlamingo62> , are you using a self hosted server or the community?

11 months ago

0 Can I Connect Wtih Some One Who Was Able To Successfully Deploy Clearml On Gcp. Are There Tutorials For This ?

Sounds like an issue with your deployment. Did your Devops deploy this? How was it deployed?

one year ago

0 Hello, I Have A Question Regarding Clearml Task And Pytorch Lightning I Am Training A Model And I Want Clearml To Plot The Accuracy And Loss. According To

Hi @<1547028031053238272:profile|MassiveGoldfish6> , what version of clearml & pytorch-lightning ? Does this happen to you with the example as well? Are you on a self deployed or the community server?

2 years ago

0 Hello Everyone, I'M Currently Working On

Hi @<1610445887681597440:profile|WittyBadger59> , how are you reporting the plots?

I would suggest taking a look here and running all the different examples to see the reporting capabilities:
None

2 years ago

0 Hi! We'Re Using Clearml In Our Pipeline And Have A Question Regarding Datasets. Currently, We Export Training Data Directly From Clickhouse To S3 Using A Command Like:

Hi, I think this is what you're looking for - None

6 months ago

0 Hello Everyone ! Problem Description: I Have My Virtual Environment (Conda) In Which I Do Have Detectron2 Installed. When I Run The Task Locally It Is Working (Some Training Script). I Also Have Clearml Agent Installed. In My Config I Do Have Python Binar

Can you please add the ~/clearml.conf for the agent? Also, are you trying to run everything on the same machine or different ones?

3 years ago

0 Clearml Hyperparameters Optimization App Produces "Summary" Table With 'Task Id', Hyperparameters Values And 'Obective'. Is There A Way (== Function) For Adding Some Custom Data For This Table?

Hi @<1523701062857396224:profile|AttractiveShrimp45> , I'm afraid not. But you can always export these tables and plots into a report and add your custom data into the ClearML report as well

2 years ago

0 Hey, Guys, Are Any Of Your Have An Experience Of Configuring Clearml Agent To Use A Custom Non-Aws S3 Cloud Which Uses Http Endpoint. If So, How Did You Do It? I Would Really Like To Use Clearml But I Can'T Figure Out How To Set Up It Properly. I Tried Do

Hi @<1526734383564722176:profile|BoredBat47> , it should be very easy and I've done it multiple times. For the quickest fix you can use api.files_server in clearml.conf

2 years ago

0 Hi, I Have An Error With Clearml-Agent 1.5.1 When Importing Tensorflow 2.10

Can you please open a GitHub issue to follow up on this issue?

2 years ago

0 Hi, All I Am Trying To Setup Clear-Ml Environment In Local. I Have One Question That Clear-Ml Has Some Plan To Integrated Some Data Label Tool(For Example, The Picture Label Tool) Into The Clear-Ml Webapp?

Hi @<1572395190897872896:profile|ShortWhale75> , this capability exists as part of the HyperDatasets feature which is present in the Scale/Enterprise licenses.

2 years ago

0 Hi There All! Is There A Way To Set Empty Requirements When Creating A Remote Task? I Have Tried To Set Packages To None Or Empty Array, But It Doesn’T Work, Which Is Corroborated By The Documentation Of That Function

Yes, however I think you might be able to expose this via an env variable on the Task object itself

one year ago

0 Dear Team - I Have Some Gui Related Requests / Questions:

BoredPigeon26 , do you run them manually or with the agent? If you run manually then I'm afraid it doesn't show the config currently. However if you run with the agent, then it will also print out the entire config (excluding the secrets, of course) at the start of the run and it will be shown in the console output in the UI 🙂

Or do you mean the different parameters you've changed about in the task itself?

4 years ago

0 Hello, I Am Trying To Compare Plots Of Two Experiments With The Same Name ( For Instance The Distribution Of Features When I Change A Hyperparameter). How Can I Know Which One Belongs To Which Experiment? I Know I Could Use The Name Of The Clearml Task A

Is it a self deployed server or the Community?

one year ago

0 Hey Everyone! I’M Currently Trying To Set Up Hyperparameter Optimization With Clearml On A Base Experiment Using Hydra. I Got Everything Working From The Examples. However, The Child Experiments Started By The Optimization Don’T Seem To Actually Get The N

In the task hyper parameters section you have a section called Hydra. In that section there should be a configuration called _allow_omegaconf_edit_ , what is it set to?

3 years ago

0 Question About The Task And Queue In The Clearml Sdk. There Are Three Machine, Machine A(Running The Clearml Server), Machine B (Linux, Target Machine To Run Task), Machine C(Windows, My Woring Machine). B And C Already Installed Clearml-Agent. I Want T

Hi @<1572395190897872896:profile|ShortWhale75> , that is not the correct way to use workers & queues.

First of all, Task.init will mark your task as running so this error makes sense.

The idea is first you run the code locally on your machine, once everything is logged (packages, repo, uncomitted changes & configurations) you can clone the task and then enqueue it into the agent.

Programmatically, you would watch to fetch an existing task in the system, clone it and then enqueue the n...

2 years ago

0 Hi, I Want To Run A Task On A Remote Docker. I Have Several Dockers, Some Manage The Packages With Pip And Some With Conda. How Can I Deal With This By Running One Clearml Agent? Thanks

Hi @<1564060263047499776:profile|ThoughtfulCentipede62> , you manage it with the docker arguments you can provide (you can inject env vars this way) or with a setup shell script.
None

2 years ago

0 Hi, I'Ve Added External Links To The A

Hi OutrageousSheep60 , how did you add external links to a dataset? Can you provide a snippet?

3 years ago

Show more results