SubstantialElk6

115 Questions, 310 Answers

Active since 10 January 2023

Last activity one year ago

Reputation

Badges 1

282 × Eureka!

Answers 310

0 Hi, We Are Having An Interesting Issue Here. We Serve Many Users And Each User Has Their Own Credentials In Accessing The Private Git Repo. We Can'T Seem To Find A Way For The End User To Pass In Their Git Credentials When They Run Their Codes In Both Age

unfortunately, our security posture is so strict that we cannot have an agent git user that have unfettered read access to all repos.

3 years ago

0 Hi, Is There A Command I Can Use To Generate A Report That Can

Hi, any idea if i can acheive this? I just need a list of usernames.

3 years ago

0 There Is An Annoying Bug In Docker Mode Which I Have Noticed Several Months Ago But Put Up With It For Some Reason, Despite I Kept Running Into It All The Time. Sometimes Agents Don'T Pull Layers For Updated Docker Images If The Image With The Same Name

Try set docker_force_pull: true under agent section of your agent's clearml.conf.

2 years ago

0 Hi, I Shifted My Clearml Setup To An On-Premise Disconnected Env, Which Has A Pip Repo Setup. I Noted This Warning,

AlertBlackbird30 , Actually the log says 10.2.
docker_cmd = nvidia/cuda:10.2-devel-ubuntu18.04 -e GIT_SSL_NO_VERIFY=true

3 years ago

0 Hi, My Devsecops Team Has Raised Some Issues Of Us Deploying Clearml For Use. In Particular, They Are Not Happy With Docker.Sock Configuration As It Would Potentially Expose The Entire Cluster To Unauthorised View. Can We Do Without It?

We are deploying ClearML Server via the docker-compose.

For ClearML-Agent. We have the choice of Docker or K8S preferred (Using the Glue).
For K8S, we can't get the glue to work ( https://clearml.slack.com/archives/CTK20V944/p1614525898114200?thread_ts=1613923591.002100&cid=CTK20V944 ) so we can't make an assessment of whether it actually works for us.

3 years ago

0 Hi, I Was Using The K8S Glue And It Worked Fine On One Project But Didn'T Work On Another. At The Point Just Before A Git Clone Was Executed, I Get The Error

I'm using the k8s Glue.

3 years ago

0 Hi, I Shifted My Clearml Setup To An On-Premise Disconnected Env, Which Has A Pip Repo Setup. I Noted This Warning,

Hi AgitatedDove14 , i changed everything to cuda 10.1 and tried again with the same rrror. the section as follows. I made sure torch==1.6.0+cu101 and torchvision==0.8.2+cu101 are in the pypi repo. But the same error still came up.
` # Python 3.6.9 (default, Oct 8 2020, 12:12:24) [GCC 8.4.0]
boto3 == 1.14.56
clearml == 0.17.4
numpy == 1.19.1
torch == 1.6.0
torchvision == 0.7.0

Detailed import analysis

**************************

IMPORT PACKAGE boto3

clearml.storage: 0

IMPORT PACKAG...

3 years ago

0 Hi, I Shifted My Clearml Setup To An On-Premise Disconnected Env, Which Has A Pip Repo Setup. I Noted This Warning,

Hi AgitatedDove14 , what version i should change it to? I'm currently on v0.17.2rc3.

3 years ago

0 Hi, I Had A Task Successfully Completed. Then I Cloned It And Enqueued It Again Without Any Changes. But The Task Ends Up With An Error. Here'S The Logs, Not Sure What Went Wrong.

I'm also beginning to think this is related to https://clearml.slack.com/archives/CTK20V944/p1620664770492400 . Previously when i set force_repo_requirements_txt=true and system_site_packages: true , it seems to work. upgrading to v1.02 seems to change things.

3 years ago

0 Hi, I Have A Future Roadmap Question On Clearml-Datasets. The Current Implementation Works Well For Small Datasets But Its Rather In Effective For Very Large Datasets. For Example, Let'S Say I Have 10 Million Images Just For The Training Dataset, And My T

This one can be solved with shared cache + pipeline step, refreshing the cache in the shared cache machine.

Would you have an example of this in your code blogs to demonstrate this utilisation?

3 years ago

0 Hi, I Have Been Getting The Following For A While. Is There A More Detailed Log I Can Look Into? This Happens On Both Https And Http.

Hi, we are still not getting the model repo to work, mainly due to clearml.storage failing to save the models.
We tried a vanilla boto3 code and it works, but we can't figure out why we get connectionreseterror 104 when clearml does it.

How do we configure clearml in correspondence to following boto code?

S3= boto3.resource('s3', endpoint_url=' https://ecs.ai ', aws_access_key_id='mykey', aws_secret_access_key='mysevret', config=Config(signature_version='s3v4'), region_name='us-east-1', ve...

3 years ago

0 Hi, I Would Like To Check What Would Be The Recommended Hardware Specs For The Server Host Clearml Server. I Had One Configured With 32 Cpu Cores, 64Gb Ram And I Noticed That If We Have A Surge In Remote Task Creation, The Following Delays Occurs.

We are running on a 1gbps backend.

3 years ago

0 Hi, We Noted That Using K8S Glue, There Are Some Situations Where The Task Cannot Be Registered As Error And Will Be Stuck At Pending. An Example Of One Situation Is When The Task Is Pulling A Docker Image That Doesn'T Exist. Is There A Way To Catch Such

Oh, this meant i have been using the latest agent which is v1.0.0. The problems were still there.

3 years ago

0 Hi I Saw This On The Clearml-Agent Docs But Other Than The Docker Image, I'M Not Sure How To Integrate This With Clearml Py And Clearml-Server. Please Advise.

the default for base_pod_num is 1.

3 years ago

0 Hi, There'S Something I Don'T Find Too Logical When Using Clearml And Its Agents. I Will Need To Run My Code Once On My Client Computer This Is Without Gpus. And Then I Will Need To Run It Via The Ui On Clearml Server That Has Gpus. Why Can'T I Configure

Hi, it make sense if i only had to change hyperparameters, but it's not so when i am still changing the model architecture (training code) and train and repeat.

3 years ago

0 Hi, In The New Datasets Ui. It Doesn'T Seem To Display The Entire Lineage Of The Datasets. For Example. If A Dataset Is Create As Such Id1 (Parent)->Id2, Then Another Dataset Created As Id2(Parent)-> Id3. When You Look At Id3, It Only Shows Id2 As Parent.

Hi CostlyOstrich36 , That's correct.

2 years ago

0 Hi I'M Using Clearml Datasets. How Do I Tell From The Clearml Ui Which Datasets Version Am I Using?

AgitatedDove14 , i'm Jax, not Manoj! lol. 😅 😅

3 years ago

0 I'M Getting This When Running With Keras Framework. Clearml.Storage - Error - Failed Uploading: [Errno 21] Is A Directory: 'Model.Savedmodel'.

thanks. That seems to work. I got a question, does it save the best model or the model in the last epoch?

3 years ago

0 Hi

Thanks. Which brings me to the question. How does ClearML deal with all the CVEs? What is your process for response?

2 years ago

0 Hi, How Might I Use The Sdk To Pull Parameters Of The Agent'S Clearml.Conf Into My Code During Runtime? For Example, If I Wish To Pull The Configuration For Aws.S3.Credentials.Key And Aws.S3.Credentials.Secret?

Yup that works. Thanks.

2 years ago

Yes! I definitely think this is important, and hopefully we will see something there

(or at least in the docs)

Hi AgitatedDove14 , any updates in the docs to demonstrate this yet?

2 years ago

0 Hi

Hi i missed this. Is there a recording?

2 years ago

0 Hi, I'M Getting This Long Error When Running

Alright thanks, i will work on that.

3 years ago

0 Hi, I'M Using The K8S Glue And Have A Few Questions.

Any comments on using the global python libraries without the need to 'pip install' anything?

3 years ago

0 Hi, I'M Getting This Long Error When Running

[root@2c7498711bef elasticsearch]# curl `
{
"cluster_name" : "clearml",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 4,
"active_shards" : 4,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 8,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" ...

3 years ago

0 Hi, I Am Running Several Python Scripts But All For The Same Project/Task. Is It Possible To Task.Init To Existing Running/Completed Task And Adding On The Results?

Hi, for both of them, args.lastiter is the exact same value. But when plotted out, they are 2 actually iterations apart.

3 years ago

0 Hi, Clearml Console Leaks Credentials Passed In As Env Vars. The Issue Remains With Clearml Version==1.1.1.135 - 1.1.1 - 2.1.4 (As Listed On The Profile Page) I Am Using K8S Glue And The Clearml.Conf Has The Following In The Agent Section.

Hi SuccessfulKoala55 ,i managed to install clearml-agent==1.0.1rc5. However, the same issues occur.

3 years ago

alright. 🙂

3 years ago

0 Hi, V1 Of Agent Seems To Have Removed Agent.Package_Manager.Force_Repo_Requirements_Txt. Is This Still Available In Other Forms?

so the clearml-agent daemon needs higher privilege?

3 years ago

0 Hi, I Can'T Seem To Set A Password To Clearml, Anyone Seems To Be Able To Just Enter The Username And They Can Enter That Username'S Workspace.

It's a local deployment. I was only presented with username without a need to enter passwords. When I'm in, I don't see an option in my profile to set a password as well. Neither is there integration with ldap for example.

3 years ago

Show more results