CostlyOstrich36

0 Questions, 4213 Answers

Active since 10 January 2023

Last activity 2 years ago

Reputation

Answers 4213

0 Hi Everyone! We Are Trying To Run Pipelines From Gitlab Ci Runners, But Are Faced With The Following Error When Performing

Hi PreciousParrot26 ,

Why are you running from gitlab runner - Are you interested in specific action triggers?

3 years ago

0 How Do You Save Models And Store The Classes As Well? When We Save The Model Using Pytorch We Just Save The State Dict And Not The Custom Classes For Different Models. I Was Thinking Of Just Pickling The Model But If Anyone Has A Better Suggestion That Wo

Can you please clarify? What does torch.nn.Module hold within? Isn't it just the weights configuration?

3 years ago

0 Hi, Was There Any Change In Clearml-Agent Recently That Caused It To Ignore The Environment Variable $Clearml_Agent_Skip_Pip_Venv_Install? I Had To Spin New Agents And Now They Try To Create Venv For No Reason ( The Environment Already Exists)

HugeArcticwolf77 , what if you specify the exact python binary from conda using CLEARML_AGENT_SKIP_PIP_VENV_INSTALL?

2 years ago

0 Hi There, I Would Like To Know How We Can Pass An Environment Variable For The Running Script When The Script Is Run Remotely By An Agent?

Hi @<1529995795791613952:profile|NervousRabbit2> , if you're running in docker mode you can easily pass it in the docker_args parameter for example so you can set env variables with -e docker arg

2 years ago

0 Do I Understand Correctly That Python Versions Must Match Between Client (My Mac, Sends Task For Remote Execution) And Clearml-Agent? I Don’T Really Get How The Environments Are Managed. All I Want To Do Is Take My Code And Execute It On The Agent Machin

So how do you attach the pytorch requirement?

3 years ago

0 What

Hi HugeLion86 , I think that's the way

2 years ago

0 I Am Using Clearml Pro And Pretty Regularly I Will Restart An Experiment And Nothing Will Get Logged To Clearml. It Shows The Experiment Running (For Days) And It'S Running Fine On The Pc But No Scalers Or Debug Samples Are Shown. How Do We Troubleshoot T

My bad, if you set auto_connect_streams to false, you basically disable the console logging... Please see the documentation:

auto_connect_streams (Union[bool, Mapping[str, bool]]) – Control the automatic logging of stdout and stderr.

one year ago

0 Hello. Please Tell Me How To Make Sure That When You Start The Pipeline, Nothing Superfluous Is Installed In The Service Queue?

Hi, Can you please elaborate on what you mean and what is happening?

2 years ago

0 Hello, I Am Using A Nas Storage To Save My Models And Dataset Artifacts. My Pipeline Consists Of Two Steps:

How is the model saved in the code?

3 years ago

0 [Clearml-Session Question] Why Does Jupyter Lab Have Only Token In Url While Code-Server Doesn’T?

Hi @<1524922424720625664:profile|TartLeopard58> , can you elaborate on what do you mean by code-server?

2 years ago

0 Hi Clearml, I Tried To Upgrade The Clearml Server Following This

SarcasticSparrow10 , please note that during the upgrade you do NOT copy /opt/clearml/data/mongo into /opt/clearml/data/mongo_4 , you create the folder like in the instructions: sudo mkdir /opt/clearml/data/mongo_4

This is the reason that it is giving out errors - You've got old mongo data in your mongo 4 folder...

Please follow the instructions to the letter - this should work 🙂

3 years ago

0 Need Help ! I Am Able To Train Models From Our Local Machines And Log Everything On The Clearml Server Without Any Issues, The Same Training Gets Stuck When I Use Remote Training. The Logs Do Not Provide Any Useful Information, And The Last Line In The L

Hi @<1562973095227035648:profile|ThoughtfulOctopus83> , what version of ClearML server are you running? Also what versions of clearml & clearml-agent

2 years ago

0 Hi When I Am Using Clearml For Checking The Performance Of My Llm Will I Be Able To See The Time An Agent Will Be Using For A Particular Task To Execute In A Visualized Way Where I Will Be Getting From Other Competitors How Can I Edit My Report By Adding

Hi @<1719162252994547712:profile|FloppyLeopard12> , not sure I understand what you're trying to do, can you elaborate step by step?

one year ago

0 Hey Everyone! Is There A Possibility To Move Artifacts From Existing Tasks To Another Storage Host? We Have A Number Of Datasets On

Hi @<1523702439230836736:profile|HomelyShells16> , I'm afraid that's not really possible since the links themselves are saved on the backend

2 years ago

0 Two Questions Today. First, Is There Some Way To Calculate The Number Of Gpu-Hours Used For A Project? Could I Select All Experiments And Count Up The Number Of Gpu-Hours/Gpu-Weeks? I Realize I Could Do This Manually By Looking At The Gpu Utilization Grap

Regarding 1 & 2 - I suggest always keeping the API docs handy - https://clear.ml/docs/latest/docs/references/api/definitions

I love using the API since it's so convenient. So to get to business -
To select all experiments from a certain project you can use tasks.get_all with filtering according to the API docs (I suggest you also use the web UI as reference - if you hit F12 you can see all the API calls and their responses. This can really help to get an understanding of it's capabilities ...

3 years ago

0 I Just Found Out That I Have A Lot Of Data In /Opt/Clearml/Fileserver For Tasks That I Already Have Deleted. No Artifacts Are Displayed In The Webui. Anyway I Can Automatically Delete All Unused Data?

Hmmmm you can automate the cleanup. Iterate through folders, if such an experiment exists, skip, if no experiment exists, delete folder

3 years ago

0 Hi Clearml, Does Clearml Orchestration Have The Ability To Break Gpu Devices Into Virtual Ones?

Hi, do you mean out of the box virtualization of your gpu or using virtual gpus on the machine?

3 years ago

0 I Noticed When I Restore Entire Clearml App (Tar And Untar

Hi @<1523701842515595264:profile|PleasantOwl46> , I'm not sure. Do you see any errors in the API server on such a startup?

11 months ago

0 Hello Everyone ! I Would Like To Form My Problem As Best As I Can. I Have My Project (For This Project I Used Pip To Install Every Dependency), But I Also Had Virtualenv Created Via Conda. For My Config Binary Path I Set It To The Virtual Environment. For

Hi ExasperatedCrocodile76 , I think this is the quickest and easiest solution you're looking for:
https://clear.ml/docs/latest/docs/references/sdk/task/#taskforce_requirements_env_freeze
Just make sure to call it before Task.init() as documentation states

3 years ago

0 Hey All. Question Regarding Scheduling And Orchestration. Does Clearml Provide Any Tooling To Schedule Entire Training Pipelines And To Trigger Training Pipelines In Response To Events, E.G. Degraded Model Performance Alerting?

Hi TenseOstrich47 ,

Yes 🙂
Please take a look here:
https://github.com/allegroai/clearml/blob/master/examples/scheduler/trigger_example.py
https://github.com/allegroai/clearml/blob/master/examples/scheduler/cron_example.py

3 years ago

0 Hi, Can I Ask How I Can Make Clearml-Datasets In Comparison With Pytorch Datasets/Dataloader? In Particular, Pytorch Dataloaders Would Be Able To Batch Pull And Then Preprocess Data Using Multi-Cpus, Feed It Into The Training Loop And Achieve As High Util

https://clear.ml/docs/latest/docs/references/sdk/dataset/#get_num_chunks
I think this might also be helpful. Gloss over the functions available in the documentation, I think you might find what you're looking for 🙂

3 years ago

0 Hi! If I Want To Programmatically Extract Information On All (Clearml) Tasks Of A Given (Clearml) Project, What Is The Recommended Way? For Example - Let'S Say I Want To Extract Some Specific Metric Curves Points From All Of The Tasks

Hi GorgeousMole24 , I think for this your best option would be using the API to extract this information.
` from clearml.backend_api.session.client import APIClient

client = APIClient() `is the pythonic usage

3 years ago

0 Hello Everyone ! Problem Description: I Have My Virtual Environment (Conda) In Which I Do Have Detectron2 Installed. When I Run The Task Locally It Is Working (Some Training Script). I Also Have Clearml Agent Installed. In My Config I Do Have Python Binar

you can edit the requirements section directly

3 years ago

0 I Am Working On Versioning My Data With Clearml-Data. I Have Clearml Server Deployed On One Machine And On Another Machine I Have My Clearml.Conf Setup To Use The Deployed Clearml Server. Also, On Both Machines I Have A Nas Storage Mounted On The Same Pat

RipeAnt6 , you have to manage your storage on the NAS yourself. We delete data only on the fileserver.

However, you could try mounting the NAS to the fileserver docker as a volume and then deletion should also handle files on the NAS 🙂

4 years ago

0 Hey, When We Change In Our Clearml Conf Local Cache To Our Mounted Nas Drive (Default_Base_Dir: "/Nfs_Client/.Clearml/Cache") Download Speed Is ~50 Times Slower. Did Anybody Encountered This Issue And Or Know How To Fix It ?

We use the GCP SDK under the hood. Can you try downloading to the same folder in the NFS using GCP SDK?

one year ago

0 Hi, I’M Running An Agent In K8S Using The Glue. I’M Trying To Set The Setup Shell Script To Install Some Packages Through Apt But I Don’T Think It Even Runs. I Dont See Anything In The Console Logs That Says It Does, At Least. Is There Anyway I Can Debug

Looks like it's not running in docker mode 🙂
Otherwise you'd have the 'docker run' command at the sttart

one year ago

0 Hi Everybody. When I Want To Force The Agent To Not Reproduce My Local Pip Environment, I Add

I think if you provide an absolute path it should work 🙂

3 years ago

0 Hello Community, I'M Planning To Deploy A Self-Hosted Clearml Server On Aws. Our Usage Pattern Is To Run It Intensively Once A Week, With A Few Additional Sessions During The Week To Review Results And Navigate The Ui. I Noticed Clearml Offers A Custom Am

Hi @<1717350332247314432:profile|WittySeal70> , just to clarify, are you talking about the ClearML server itself or about agents?

From my understanding the AMI is simply an image with the ClearML server preloaded to it.

9 months ago

0 Hi, We Have A Self Hosted Clearml Server Which I Mainly Use For Experiment Tracking. There Is One Issue I Have Noticed Recently: Whenever I Archive And Delete An Experiment (With The Box " Remove All Related Artifacts And Debug Samples From Clearml File

If you go to settings, the versions should appear at the bottom right

2 years ago

0 After Resetting And Enqueueing A Task, The Pip Installation Procedure In A Remote Agent Tries To Install Packages Different From Packages That Were Installed In The Execution Session Before The Resetting And Enquiring Of The Task. How Can I Control That

Can you copy paste the error you got?

3 years ago

Show more results