CostlyOstrich36

0 Questions, 4213 Answers

Active since 10 January 2023

Last activity 2 years ago

Reputation

Answers 4213

0 Hi Guys, I Encountered A Very Frustrating Problem About Virtual Enviroment Management When I Use Clearml. I Use Virtualenv To Create A New Python Env And Installed All The Needed Python Pakcages In The Env. The Env Path Is

I think the 3rd one, let me know what worked for you

2 years ago

0 Hi, I'M Eric. I'M An Mlops Engineer At A Company With 9 De'S, 6 Ds'S, And 2 Mlops Engineers. I Just Learned About Clearml A Few Hours Ago And I'M Getting Excited About It!! I'M Wondering If We Could Replace Our Current Mlops Platform With Clearml. Right N

@<1557175205510516736:profile|ShallowSwan53> , ClearML automatically logs all your models in it's model registry during your experiment's run. You can also use the OutputModel class for manual reporting as well.

2 years ago

0 Hi, I Am Using Pytorch Lightning In Combination With Clearml. I Would Like To Report Figure In Clearml Debug Samples. I Use `Self.Logger.Experiment.Add_Figure("Test/Name", Fig, Global_Step=Self.Global_Step)`, Where Self Is Module. Indeed, The Figure Ends

I'm afraid I don't have anything like that 😞

3 years ago

0 Hi All, Can I Synchronize All My Artefacts On Clearml Server With S3? I'M Trying To Deploy Clearml-Server In A Limited Disk Space Environment

I think you can periodically upload them to s3, I think the StorageManager would help with that. Do consider that artifacts are logged in the system with links (each artifact is a link in the end) So even if you upload it to and s3 bucket in the backend there will be a link leading to the file-server so you would have to amend this somehow.

Why not upload specific checkpoints directly to s3 if they're extra heavy?

2 years ago

0 Hi Folks I Have A Problem I Can'T Understand. Plots Are Not Shown When Experiments Are Executed From The Ui. For Example, If I Run The Code On My Laptop, And I Go To The Experiment Page I Can See Correctly The Plots: But If I Then Clone The Task, And Ex

Also, please go into the UI - go to the experiment that was executed remotely. Open developer tools (F12) and see what is returned when you navigate to the plots page in the UI

3 years ago

0 Hello, Is There A Way To Running Multiple Environments (Like Staging, Production) On The Same Clearml Instance? Or Do We Need Separate Instance For Each Environment? Or Is There A Way To Restrict What Each User Can Do? (Like Not Modify Other User Experime

Hi :)
I'm guessing you're running a self hosted version? I think that access rules are a feature in the enterprise version only.

3 years ago

0 Hi Everyone. If I Edit A File In Configuration Objects In Clearml Ui, Will The New Parameters Be Injected In My Code When I Run This?

Hi PetiteRabbit11 , can you please elaborate on what you mean?

2 years ago

0 Hi Everyone, I Got Several Error Messages Like This When Trying To Upload Data On The Clearml Which We Self-Hosted. Do You Have Any Ideas?

Looks like you're having some connectivity to the files server


2024-11-14 07:05:30,888 - clearml.storage - INFO - Uploading: 5.00MB / 12.82MB @ 35.88MBs from /tmp/state.vykhyxpt.json
2024-11-14 07:05:31,111 - clearml.storage - INFO - Uploading: 10.00MB / 12.82MB @ 22.36MBs from /tmp/state.vykhyxpt.json
1731567938707 labserver error WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=None)) after connection broken by 'NewConnectionError('<urllib...

one year ago

0 Hello, I Am Having A Problem That Debug Images Are Not Shown After Clearml Server Migration. I Found A Solution On This Page:

Hi @<1526734383564722176:profile|BoredBat47> , do you mean new debug samples or old ones? Please note that older debug samples were registered to the previous URL

one year ago

0 Hi There, Is There A Maximum Number Of Plots (I.E. Losses/Scalars Being Logged During Training)? I Have An Application With Many Different Losses (About 200) And I Can See That Some Are Missing From The Dashboard. When This Number Is Smaller Everything Ru

I recall a big fix to plots in server version 1.6.0, can you try upgrading to see if it fixes the issue?

3 years ago

0 I Am Curious If Clear Has Any Sort Of Multi Tenant Capabilities. I Have Just Found You Guys And Plan On Spinning Up An Instance To Play Around, But Wanted To Get Some Direction There.

That is fully supported on premise, even in air gapped environments 🙂

5 months ago

0 What Is The Right Way To Increase Number Of Retries When Using

DilapidatedDucks58 , I think this is what you want. You just configure it and don't need to changing anything in code.
https://github.com/allegroai/clearml/blob/92210e2c827ff20e06700807d25783691629808a/docs/clearml.conf#L106
You can pass any boto3 parameter here!

3 years ago

0 I'M Getting This Error Across Different Machines

@<1547028079333871616:profile|IdealElephant83> , what are you trying to do during the code execution?

Is it a self hosted server?

2 years ago

0 Hi Everyone, I Have One Quick Question Regarding The Artifact Uploading. I See The Output Models From Training Are Stored Under The "Output Models" Section In Artifacts Tab, And If I Do Upload Other Artifacts Using .Upload_Artifact() Api, They Are Stored

I have tried

task.upload_artifact('/text/temp', 'temp.txt')

but it's not working (I can access the task, but as soon as I click artifacts tab, it shows 404 error).

Can you please elaborate on this? Can you please share a screenshot?

3 years ago

0 Hi, I'Ve Noticed That In A Few Experiments That Got Stuck With No Indication That Something Went Wrong And It Kept On "Running" Until I Manually Aborted The Experiments, Is There A Way To Create A Timeout For If The Server Is Stuck? For Clarification I'M

Can you provide a task id for such a task?

one year ago

0 Hello

@<1523701977094033408:profile|FriendlyElk26> , try upgrading to the latest version, I think it should be fixed on the latest version

one year ago

0 Hi, I'M Not Sure When Exactly But At Some Point I'Ve Got No Access To My Debug Samples. We'Re Using Self-Hosted Clearml, Which Runs Within A Docker Container. Any Ideas? Thanks!

Hi @<1717350332247314432:profile|WittySeal70> , where are the debug samples stored? Have you recently moved the server?

one year ago

0 Hi There! I'M Getting An Error Whenever Trying To Queue Experiments Using Conda Package Manager That Require Python>=3.10. Locally (Non-Queued) The Training Runs Just Fine. In The Ui I See The The Following Console Output:

I'll try to see if it reproduces on my side 🙂

3 years ago

0 Hi. Is There A Way To Transfer Clearml Workspace Ownership To Another Team Member?

Hi @<1523706826019835904:profile|ThoughtfulGorilla90> , it's not possible since workspaces are connected to the email itself. I would suggest writing some automation to extract the relevant projects/experiments from one workspace and register them into the new workspace. The API would be the best way to go. You would need to extract all information about the experiment itself and then also extract all the logs/scalars/plots and then simply register everything in the new workspace.

[None](h...

9 months ago

0 Hi All! Could Do With Some Help On Running Registered Task On A Clearml-Agent. My Workflow So Far Is As Follows:

Is this commit local or was it pushed to some branch?

3 years ago

0 Hi, I'M Using Clearml Over Aws And I Can See That My Ec2 Instance Disk Space Is Filling Up Really Fast. I First Started With 100Gb Disk Space And Once It Filled Up I Increased The Size To 150Gb Which Is Soon To Be Filled Up As Well. I Wanted To Check If T

I see. I'm guessing you have pretty extensive use in the form of artifacts/debug samples. You can lower the storage usage by deleting some experiments/models though the UI. That should free up some space 🙂

3 years ago

0 Documentation Is Not Good Because Beginner Or Intermiate People Cant Undeerstand, Only More 20 Years Of Exp Can I Understand.. I Checked With Exp People So I Am Saying. I Am Intermiate Guy

It's totally possible, I think you need to do research on it. There are probably a few ways to do it too. I see CLEARML_API_ACCESS_KEY & CLEARML_API_SECRET_KEY in the docker compose - None

You should do some more digging around. One option is to see how you can generate a key/secret pair and inject them via your script into mongoDB where the credentials are stored. Another way is to see how the UI ...

2 years ago

0 Just A Small Question

@<1772433273633378304:profile|VexedWoodpecker50> , I think there is some misconfiguration or misunderstanding. Can you elaborate on how you set up the server and worker?

11 months ago

0 Hi All, I Am Testing The New

GiganticTurtle0 , are you using ClearML 1.1.1 or 1.1.0?

4 years ago

0 Where Can I Find A More Extensive Documentation Of Serving? I'M Looking For

Hi WackyRabbit7 ,

Is this what you're looking for?
https://github.com/allegroai/clearml-serving

3 years ago

0 `I'M Attempting My First Project In Clearml And Have A Few Questions. I'M Migrating My Video Encoding Pipeline In Python3.12 To Clearml Using The Decorators.: @Pipelinedecorator.Pipeline( Name="Run Video Encoder Pipeline", Project="Video Encoder

Regarding pipelines, did you happen to play with this example? - None

The idea is that each step in the pipeline including the pipeline controller are tasks in the system. So you have to choose separate queues for steps and also the controller. The controller by default maps the 'services' queue, but you can control also that.

8 months ago

0 Q: Hello, I Am A Clearml Begineer, Can I Use Clearml Open-Source To Build Our Customized Platform? What Are The Limits Of Clearml While Building Mlops Platform?

I am not very familiar with KubeFlow but as far as I know it is mainly for orchestration whereas ClearML offers a full E2E solution 🙂

3 years ago

0 Hi, I'Ve Run Into A Problem And Would Appreciate Some Help. I Installed Clearml Locally. When I Run A New Task On A Remote Server And In The Python Training Code I Set It To Only Train On One Gpu. Everything Works Fine And I See All The Scalars Automatica

Just to make sure we're on the same page, you're referring the machine statistics or ALL scalars don't show up?

11 months ago

0 Hi All, While Training Some Models I Noticed That My Api Calls Are Going Through The Roof, Is There A Way To Control The Amount Of Api Calls? Is There A Way To Update All The Plots And Console Print On The App.Clear.Ml For Every Some Epoch? Or Aggregating

Hi @<1523701295830011904:profile|CluelessFlamingo93> , are you self hosting or using the community server?

2 years ago

0 Hi. I Have Been Using Clearml For A While, And Wanted To Know How Can I Run My Tasks In A Pipeline, Without It Detecting It Is From A Git Repository, Since The Remote Machine Won'T Have Access To Our Repository. I Have Wrote The Code In A Way That Each Ta

Hi @<1625303791509180416:profile|ExasperatedGoldfish33> , I would suggest trying pipelines from decorators. This way you can have very easy access to the code.
None

2 years ago

Show more results