CostlyOstrich36

0 Questions, 3690 Answers

Active since 10 January 2023

Last activity one year ago

Reputation

Answers 3690

0 Hi, We Are Using Clearml For Our Experiment Tracking But Now Investigating Using The Pipeline Functionality As Well For Scheduling. We Also Want To Be Able To Trigger A Pipeline Run When There Is New Data In An External Database. Is This Possible? From Wh

Hi @<1654294828365647872:profile|GorgeousShrimp11> , long story short - you can.

Now to delve into it a bit - You can trigger entire pipeline runs via the API.

I can think of two options from the top of my head. First being some sort of "service" task running constantly and listening to something and then triggering pipeline runs.

The second, some external source sending an POST request via API to trigger a pipeline.

What do you think?

10 months ago

0 Hello! I Have An Issue On A Last-Week Self-Hosted Clearml Installation (Server: 1.16.0-494): After Starting Containers With Docker Compose Everything Goes Well Except For

Hi @<1722061354531033088:profile|TroubledCamel37> , what do you see in the apiserver logs?

4 months ago

0 Hello, Friends! I'Ve Got A Problem. I Am Trying To Restore Clearml Server From Backup. I Did What Docs Tell: Archived "Config" And "Data" Folders, Moved Them To New System. I Copied Docker Compose File And Set Correct Paths. When I Try To Run This Apiserv

@<1526734383564722176:profile|BoredBat47> , that could indeed be an issue. If the server is still running things could be written in the databases, creating conflicts

3 months ago

Hi @<1526734383564722176:profile|BoredBat47> , do you see any errors in the elastic container?

3 months ago

0 Hi, I Am Fairly New To Clearml And Would Appreciate Some Help With This Problem. My Team Currently Uses The Virtual Environment Mode To Run The Experiments. But We Also Have A Docker File That We Use For Local Debug So I Want To Consolidate Everything And

This is exactly what the build command is for. I suggest reviewing the documentation

2 months ago

0 In My Current Project I Generate The Data From An Sql Query. Is The Only Way To Register The Dataset With Clearml To Write The Files To Disk First Or Is There Another Method? This Leads Into The Second Issue I Have, Which Is What Happens When I Store The

@<1545216070686609408:profile|EnthusiasticCow4> , I think add_files always generates a new version. I mean, you add files to your dataset, so the version has changed. Does that make sense?

one year ago

0 Hello! How Will Clearml React, If I Have Combine Two Datasets Into A New Dataset, And Both Contain A File “Ids.Txt”. Will One Be Overwritten, Will Both Files Be Merged, Will There Be Two Renamed Files? Thank You!

I think as long as they have different hashes you will have two different files

4 months ago

0 Hi Community, I Want To Add 2 Args For Clearml Agent Dockers " ["--Network=Host", "--Ipc=Host"]". I'Ve Tried With Changing Agent.Default_Docker.Arguments => But It Didn'T Work. Later When I Tried With Agent.Extra_Docker_Arguments => It Worked. So I Was Wo

Hi @<1749965229388730368:profile|UnevenDeer21> , can you add the log of the job that failed?

Also, note that you can set these arguments from the webUI on the task level itself as well, Execution tab and then container section

one month ago

0 Hi Community, I'M Installing Multiple Clearml-Agent On Some Gpu Workstations, So I Have A Question: How Can I Use The Cache Of Dataset As When I Run Only 1 Clearml-Agent? I Don'T Want To Download Full Dataset Everytime When A Task Run Because Now Since I

Hi @<1749965229388730368:profile|UnevenDeer21> , an NFS is one good option. You can also point all agents on the same machine to the same cache folder as well. Or just like you suggested, point all workers to the same cache on a mounted NFS

one month ago

0 Hi Again

Hi @<1750327622178443264:profile|CleanOwl48> , you need to set the output_uri in Task.init() for example to True to upload to the files server or to a string if you want to use s3 for example

one month ago

0 Hello Admin, How Do I Download The Clearml Documents In A Pdf Format?

You have the open source repository of the documents - None

I think you could generate a pdf from that with some code.

one month ago

0 Hello Admin, Please Add More Mlops Hands On Learning Resources On Youtube.

Hi @<1753589101044436992:profile|ThankfulSeaturtle1> , what sort of materials do you think you're missing?

one month ago

0 Hi Everyone! Quick Question Does Clearml Offer Also Direct Access To Compute Resources, Or You Always Need To Bring Your Own Cloud Provider?

Hi @<1752501940488507392:profile|SquareMoth4> , you have to bring your own compute. ClearML only acts as a control plane allowing you to manage your compute. Why not use AWS for example as a simple solution?

one month ago

0 Hi, The Clearml-Serving Cli --Preprocess Argument Accepts Either A Local File Or Folder. I Cannot Find Guidelines On How To Setup This Folder, And Simply Adding

Hi @<1755038652741718016:profile|LuckyRobin32> , how are you pointing to the folder?

one month ago

0 Hi, Can I Run A Single Hyperparameter Optimization Task With At Least 100 Experiments On Multiple Machines By Using Clearml-Agent And Queue?

Did you run the code locally first? I don't see the agent installing the packages themselves, did you remove it from the log or how are the packages being installed?

4 months ago

0 Hi, Can I Run A Single Hyperparameter Optimization Task With At Least 100 Experiments On Multiple Machines By Using Clearml-Agent And Queue?

You can add torch to the installed packages section manually to get it running but I'm curious why it wasn't logged. How did you create the original experiment?

4 months ago

0 I Executed A Hyperparameter Optimization Task, But Just One Single Experiment Was Performed And Nothing. The Task Wasn'T Terminated Automatically, But Still Not Starting The Next Experiments. What'S Wrong Here? How Can I Fix This?

@<1664079296102141952:profile|DangerousStarfish38> , can you provide logs please?

4 months ago

0 In The Pipeline Examples, Components Have The Following Note:

Maybe a single step pipeline?

one year ago

0 How Does Clearml Store Some Of The More Intrinsic Data (Scalars, Plots, Etc)? We Recently Recovered Mongo From A Backup And While The Projects And Tasks Are There, The Console, Info, Etc Of All Of Them Is Missing

Hmmm I would guess something would be possible but I'm not sure how to go about it. Maybe @<1523701087100473344:profile|SuccessfulKoala55> or @<1523701994743664640:profile|AppetizingMouse58> can give some more input.

one year ago

0 Hi. I Am Using Clearml Server On Premises To Manage Ml Data And Tasks/Experiments. Is It Possible To Modify File Path From Artifacts? The File Server Ip Changed After Server Migration And Previously Generated Artifacts Are No Longer Reachable Due To File

Hi @<1751777178984386560:profile|ConfusedGoat3> , I think you might need to run some migration script on the database, basically changing the paths of the artifacts registered to the new IP

one month ago

I think you'd have to write it yourself. Basically, the artifact paths in experiments are saved in mongo. You would need to write a script that would modify those values in Mongo directly

one month ago

I think this is what you're looking for - None

one month ago

0 Hi There, Would It Be Possible For The Autoscaler To Support Stopping Instances Instead Of Terminating Them? My Use Case Is The Following: I Am Continuing My Journey With The Clearml-Session Tool, And In Case The Clearml-Session Is Running In A Ec2 Inst

Hi JitteryCoyote63 ,

Regarding Edit 2: This seems like a nice idea.

Regarding adding option to only stop them - Please open a feature request on GitHub 🙂

2 years ago

0 Hi, I Am Giving Another Try To Clearml-Session And I Am Blocked At The Current Error Shown When The Cli Try To Establish The Tunneling:

Hi JitteryCoyote63 , can I assume you can ssh into the machine directly?

2 years ago

0 Hello

@<1523701977094033408:profile|FriendlyElk26> , try upgrading to the latest version, I think it should be fixed on the latest version

2 months ago

0 Hi. I Am Using Clearml Server On Premises To Manage Ml Data And Tasks/Experiments. Is It Possible To Re-Create/Upload From Disk A Final Dataset? One Dataset Part Is Corrupted On Disk At The File Server And Cannot Longer Be Downloaded. Thanks In Advance!

ClearML only keeps links to the data, so if you simply put the copy you have in the same paths, everything will work as before

one month ago

0 Hi, I'M Trying To Use Clearml On Pytorch-Lightning With Multiple Gpus, But It Seems As If The Server Does Not Monitor The Experiment. I Can See No Progress In The Console (Steps Counter Stays On 0) Nor Any Tensorboard Loggings. On A Single Gpu Everything

Also, what if you try using only one GPU with pytorch-lightning? Still nothing is reported - i.e. console/scalars?

2 years ago

0 Hi There I'M Trying Out Clearml. I Saw Mention That Clearml Can Capture Tensorboard Output So I Tried It With This Little Script (Image Below). The Events File Is Filled, The Clearml Task Is Created, And Marked Complete However There Is Nothing In The Sc

Hi PanickyMoth78 ,

What version of ClearML are you using?

2 years ago

Strange, I'm not familiar with tensorboard_logger package. I see it's latest package on pypi is also 0.1.0 with latest supported python 3.5.

Scalers are usually reported and auto captured through SummaryWriter if I'm not mistaken. I found an example here:
https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_tensorboard.py

Anyhow I'll take a look into it 🙂

2 years ago

0 Hi People, I Cant Force Deleting Datasets It Tells Me Storageerror: Can'T Find Azure Configuration For

Is it possible that you don't have permissions for deletion on that Azure account with your credentials?

2 months ago

Show more results