AgitatedDove14

49 Questions, 8126 Answers

Active since 10 January 2023

Last activity one year ago

Reputation

Badges 1

25 × Eureka!

Answers 8126

0 Correct Way To Configure Ssh Authentication For Git In Agent With Docker Mode

Here are my extra_docker_arguments that make the thing working:

GentleSwallow91 Nice!
BTW: in theory there should not need to be any need to add the specific: "-v","/home/nino/.ssh:/home/testuser/.ssh", the agent should do that automatically

3 years ago

0 Hi! I Was Wondering Regarding This Issue:

Thanks WittyOwl57 ! let me check

4 years ago

0 <image>

And only the main one ?

4 years ago

0 Hi All—First Off, Thanks For Being Such A Helpful And Thorough Group Of People. I Learn A Ton Just Searching Through The Channel For Problems. I’M Seeing A Weird Issue. I Have A Conda Env On My Linux Machine, And I Can Successfully Run A Training Script

I think the main issue is running with python -m module.name --args
Which is a bit different, when trying to "understand" what is the actual repository.
Can you try to run it from the repository folder (same command, just to see if it will have any effect on the detected packages)

4 years ago

0 Also, Not Sure Where To Ask This Question. I Am Following The Instructions From Here:

Try to upload something to the file server ?
None

2 years ago

0 Dear Clearml Community, I Am Trying To Optimize Storage On My Clearml File Server When Doing A Lot Of Experiments. To Achieve This, I Already Upload Only The Newest And Best Checkpoints To Clearml File Server Instead Of All Checkpoints. Another Component

None
notice there is a scroll_id there, you might need to call the API multiple times until you scroll over All the events
could that be it?

one year ago

0 Hello, Is It Possible To Disable Lazy Loading ? It’S Quite Horrible To Use In Console Logs For Instance, Where Search Is Useless As It Doesn’T Request Anything But Only Filter Currently Loaded Logs, And From My Browser Info The Ui Loads Previous Logs By 7

Hi @<1523706645840924672:profile|VirtuousFish83>

Hello, is it possible to disable lazy loading ?

You mean in the UI for loading the console ?
The logs can be huge 10s and 100s of MB...

We have the same issue for hyperparameters even with only ~100 keys,

100+ parameters that is quite a lot.
So are you saying the search in the UI only filter the lazily loaded elements and not the entire param list?

2 years ago

0 Hello, I Have Another, Hopefully Minor Question. What Is The Recommended Way Of Checking If A Code Segment Is Running As A Part Of A Task Or Not? I Tried Checking The Return Value Of

Hi ArrogantBlackbird16

but it returns a task handle even after the Task has been closed.

It should not ... That is a good point!
Let's fix that 🙂

4 years ago

0 Question About The Storage Manager. Assuming I Have An Object That Updates Frequently And Always Saved At The Same Path (E.G.

Well I guess you can say this is definitely not self explanatory line 😉
but, it is actually asking whether we should extract the code, think of it as:
if extract_archive and cached_file: return cls._extract_to_cache(cached_file, name)

5 years ago

0 Hi! Does Anyone Have Experience Using Clearml Inside A Vertex Ai Job?

Hi MuddySquid7
Hmmm what would be the use case ? (I mean how are we using Vertex ?)

4 years ago

0 So, Here'S A Question. Does Clearml Automatically Save Everything Necessary To Continue Training A Pytorch Language Model? Specifically, I'Ve Been Looking At The Checkpoint Folders Created When I'M Training A Huggingface Robertaformaskedlm. I Checked What

yep 🙂

4 years ago

0 Is There A Way To Set Precedence On Package Managers? If We Set An Agent To Use

so you have a repo with poetry that some users update and some do not?
All working on the same branch ?

3 years ago

0 Hi All! Please Tell Me Why The Almost The Last Version Of Docker-Compose Is Used (In The Example From The Site 1.24.1, Link

My question is, which version do you need docker compose?

Ohh sorry, there is no real restriction, we just wanted easy copy-paste for the installation process.

3 years ago

0 Is It Possible To Avoid The Clearml-Agent For Local Installations, And Have The File Server Automatically Use An S3 Bucket? I'Ve Found

Are you aware of any other way then (other than the

secure: false

flag?

Actually self -signing and providing certificate file is already supported with boto (and thus clearml)
AWS_CA_BUNDLE
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html

3 years ago

0 Hello, This Is The Following Python Code I Had Saved As Main.Py.

Seems like credentials error
Do you have everything setup correctly in your ~/clearml.conf ?

4 years ago

0 I Try To Use A Ssh Key For Git. When I Push A File To Git I Still Have To Write My Name And Password. I Solved It By The Command: Git Remote Set-Url Origin Git@Rz-S-Git:C.Huber/Allegro.Git How Can I Use This With Allegro? At The Moment I Get Get Error: Fa

UnsightlySeagull42 the assumption is that the agent has a read-only all access user.
As the moment there is no way to configure it to have diff user/pass per repository in the clearml.conf
You can however:
embed the user/pass on the repository link (not very secure) Use ssh-key and have it on .ssh on the host machine Use .git-credentials and configure them (with per project user/pass)

4 years ago

0 Can Anyone Recommend A Good Workflow For

I guess only if autoscaling is used (one worker one machine)?

yes, basically depending on how you set autoscaling / k8s integration 🙂

2 years ago

0 Any Pointers On Running Gpu Tasks With K8S Glue?

--docker or in clearml.conf https://github.com/allegroai/clearml-agent/blob/21c4857795e6392a848b296ceb5480aca5f98e4b/docs/clearml.conf#L153

4 years ago

0 Having Issues Running Trains-Server On Win10. Trains-Elastic Exited With Code 137 Trains-Mongo Exited With Code 100 Trains-Apiserver Exited With Code 1 Some Errors=> Requests.Exceptions.Connectionerror: Httpconnectionpool(Host='Elasticsearch', Port=9200

Many thanks LazyLeopard18 ! 🙂

5 years ago

0 Can Anyone Point Me To Web Ui Source Code Out, I’M Wondering How Can I Customizing Web App Ui A Little Bit

Thanks MagnificentPig49 !

5 years ago

0 I Have A Questions About Queue Priorities With Clearml-Agent. I Have Two Queues,

but it is not optimal if one of the agents is only able to handle tasks of a single queue (e.g. if the second agent can only work on tasks of type B).

How so?

4 years ago

0 The Histograms I Plot With Matplotlib Look Different On Clearml Compared To The Original Plots, How Can I Make Them Look The Same?

SoreDragonfly16
btw: The difference between the two graphs is the ratio pf the graph display , that it 🙂

4 years ago

0 Hi, I Run The Trains Server In An Docker Container And Started Making Use Of Tasks ... My Tests Are Showed On The Projects Dashboard Which Is Realy Cool. What I Haven'T Found So Far Is A Way To Clean Up The System From The Tests I Did. I'M Able To Archive

Hi WickedGoat98

but is there also a way to delete them, or wipe complete projects?

https://github.com/allegroai/trains/issues/16

Auto cleanup service here:
https://github.com/allegroai/trains/blob/master/examples/services/cleanup/cleanup_service.py

5 years ago

0 Does Dataset.Add_Files Support Uploading From S3 Uri? I Have No Problem Uploading To S3 But Cant Use Data That Is Already In S3? Or Am I Dong Something Wrong? I Read In Documentation That Add_External_Files Supports This Feature, But I Want To Be Able To

Yes, but does add_external_files makes chunked zips as add_files do?

No it references them, (i.e. meta-data not actually doing something with the files themselves)

I need the zipping, chunking to manage millions of files

That makes sens, if that's the case you will have to download those files anyway, and then add them with add_files
you can use the StoargeManager to download them, and then add them from the local copy (this will zip/chunk them)
[None](https://clear.ml/docs/la...

one year ago

0 Is There A Way To Run Additional Shell Commands After Setup In Agent Virtual Environment Mode? I Have Found This Feature For Docker Mode But Not For Venv Mode.

You are correct, it is currently not supported in venv mode. We could not find a good use case for it. What is yours?

4 years ago

0 What Sort Of Integration Is Possible With Clearml And Sagemaker? On The Page

. I'm thinking it's generically a kernel gateway issue, but I'm not sure if other platforms are using that yet

The odd thing is that you can access the notebook, but it returns zero kernels ..

2 years ago

0 Is There A Way To Interface With Clearml Agent (Cli?) To Handle Model Repositories And Data Versioning (But So, Not Experimentation, Tight Integration, Pipelining, Etc)?

UnevenDolphin73 FYI: clearml-data is documented , unfortunately only in GitHub:
https://github.com/allegroai/clearml/blob/master/docs/datasets.md

4 years ago

Hi LazyLeopard18 ,
See details below, are you using the win10 docker-compose yaml?
https://github.com/allegroai/trains-server/blob/master/docs/install_win.md

5 years ago

0 Hey Everyone, I'M Having An Issue Due To Conflicting Git Credentials On The Clearml-Agent (Running Inside The Docker). I'M Using Ssh Settings (

We're lucky that they let the developers see their code...

LOL 😄

and it is also set in the

/clearml-agent/.ssh/config

and it still can't clone it. So it must be some security issue internally.

Wait, are you using docker mode or venv mode ? in both cases your SSH credentials should be at the default ~/.ssh

3 years ago

0 Hey Everyone, I'M Having An Issue Due To Conflicting Git Credentials On The Clearml-Agent (Running Inside The Docker). I'M Using Ssh Settings (

PleasantGiraffe85 can you send examples of the different git repo links (one internal one public) ?

3 years ago

Show more results