AgitatedDove14

49 Questions, 8126 Answers

Active since 10 January 2023

Last activity one year ago

Reputation

Badges 1

25 × Eureka!

Answers 8126

0 Hello, I'M Trying To Save A Keras Model As A Task Artifact, And Then Upload It From Another Task. Does Anyone Know The Syntax For That? What I'Ve Seen Is Not Quite Working.

🤞

4 years ago

0 Is It Possible To Add Just A String Or Some Other Object As An Artifact? If Yes, Then How?

Hi VexedCat68

txt file or pkl file?

If this is a string , it just stored it (not as a file, this is considered a "link")
https://github.com/allegroai/clearml/blob/12fa7c92aaf8770d770c8ed05094e924b9099c16/clearml/binding/artifacts.py#L521

3 years ago

0 Is There Any Customization Options With Respect To The Ui Of The Debug Samples Tab In Results? Specifically I Am Looking For Something More Similar To Tensorboard, Namely The Slider That Lets You Scroll Conveniently Through The Debug Samples Across The E

Hi RipeGoose2
So the http://app.community.clear.ml already contains it.
Next release of the standalone server (a.k.a clearml-server) will include it as well.
I think the ETA is end of the year (i.e. 2 weeks), but I'm not sure on the exact timeframe.
Sounds good ?

4 years ago

0 Regarding The “Classic” Datasets (Not Hyper Datasets): Is There An Option To Do Something Equivalent To Dvc’S “

you can run md5 on the file as stored in the remote storage (nfs or s3)

s3 is implementation specific (i.e. minio weka wassaby etc, might not support it) and I'm actually not sure regrading nfs (I mean you can run it, but it actually means you are reading the data, that said, nfs by definition I'm assuming is relatively fast access)
wdyt?

3 years ago

0 Hi, I Am Trying To Pull Api Data From /Tasks.Get_All Endpoint

Yeah the doctring is always the most updated 🙂

3 years ago

0 So I Bumped Onto This Comparison Shared By Dagshub. It Kinda Placed Clearml Is A Rather Bad Position Compared To Everything Else In The Industry.

TrickySheep9
you are absolutely correct 🙂

4 years ago

0 Hello, I Have A Trained Model (Saved As

Hi OddShrimp85
If you pass 'output_uri=True' to task init, it will upload the model automatically, or as you said manually with outputmodel class

2 years ago

0 Getting This Error At

This is a Sagemaker notebook instances

Yes I think this is the issue

4 years ago

0 Hi Everyone! I Built A Task That Processes Data And In The End Generates A Large Folder Containing Images (Aprox 10Gb). I Would Like To Be Able To Upload This Folder As An Artifact But When I Try To Do This With Task.Upload_Artifact I Get This Error. Is T

Hi SmoggyGoat53
There is a storage limit on the file server (basically 2GB per file limit), this is the cause of the error.
You can upload the 10GB to any S3 alike solution (or a shared folder). Just set the "output_uri" on the Task (either at Task.init or with Task.output_uri = " s3://bucket ")

3 years ago

0 Hi There! Is There Any Way To Boost Creating Sha2 Hashes During

It uses only one CPU core, could I use multiprocessing somehow?

Hi EcstaticMouse10
Hmm, yes it should be multi core:
https://github.com/allegroai/clearml/blob/a9774c3842ea526d222044092172980ae505e24f/clearml/datasets/dataset.py#L1175
wdyt?

3 years ago

0 I Am Back With Another Question: Is There A File Similar To The

ReassuredTiger98 no, but I might be missing something.
How do you mean project-specific?

4 years ago

0 It Seems Like Clearml Agent Does Not Support Arparse Subparsers, Right?

I can verify the behavior, I think it has to do with the way the subparser was setup.
This was the only way for me to get it to run:
script.py test blah1 blah2 blah3 42When I passed specific arguments (for example --steps) it ignored them...

4 years ago

0 Hi Guys, I Am Having Some Trouble Running Some Training Scripts With The Agent Functionality:

Done 🙂

3 years ago

0 I'M Trying To Configure The Glue Agent To Use Aws Ecr Via Helm Charts. Below Is My Configuration. It Is Not Pulling The Image Though, It Is Failing With

Test it on your local setup (I would hate to push a broken fix)
Is that possible?

3 years ago

0 I Buried This Issue In Another Thread To Do With Deployment, But I Was Wondering If Anyone Else Has Had Problems Using

So can you verify it can download the model ?

4 years ago

0 Is It Possible To Avoid The Clearml-Agent For Local Installations, And Have The File Server Automatically Use An S3 Bucket? I'Ve Found

👍

3 years ago

0 Hi There, I Have A Problem With Pyjwt: I Am Using

Sure. JitteryCoyote63 so what was the problem? can we fix something?

4 years ago

0 And One More Question. How Can I Get Loaded Model In Preporcess Class In Clearml Serving?

I try to add it to ClearML Serving, but it call

forward

method by default

If this is the case, then the statement above is odd to me, if this is a custom engine, who exactly is calling " forward " ?
(in you code example you specifically call generate, as you should)

3 years ago

0 Hi, I Noted That Clearml-Serving Does Not Support Spacy Models Out Of The Box And That Clearml-Serving Only Supports Following;

Correct. Basically the order is restapi body dictionary-> preprocess -> process -> post-process -> restapi dictionary return

3 years ago

0 Hello, Is There A Way To Update A Task Diff Programatically? Eg, I'M Creating A Task Using

Thanks ShakyJellyfish91 this really helps to narrow it down!
Let me see what I can find

4 years ago

0 Hello, Is There A Way To Update A Task Diff Programatically? Eg, I'M Creating A Task Using

pip install clearml==1.0.6rc2Did not work?!

4 years ago

0 Hi Folks, I Did A Deployment Of Clearml Using The K8S Helm Chart, And I Set The Agent Using K8S Glue. I Run A Task Locally, And I Went To The Ui Cloned The Experiment And Scheduled It In The Default Queue. After Doing This, I See That The Experiment Is Q

okay this points to an issue with the k8s glue, I think it somehow failed to launch the pod. Can you send me the log of the clearml-k8s-glue ?

3 years ago

0 Hi Everyone, Additional Arguments To The Script Execution, Is It Possible? How Can It Be Done? So At The Moment When My Script Is Being Executed The

PompousBeetle71 you can check this example:
https://github.com/allegroai/trains/blob/master/examples/distributed/example_torch_distributed.py

I think it should help, if you want a more manual approach, you can check the POpen subprocesses here:
https://github.com/allegroai/trains/blob/master/examples/distributed/example_subprocess.py

5 years ago

0 Hi There, Executing Remotely, The Script I'M Running The Clearml Task From, Has An Import Command For A Module Located In A Different Repository, But When Running The Script It Only Clones The Repo The Clearml Task Is On, How Can It Get The Other Repo Als

👍

4 years ago

0 Hi, Just Want To Report A Small Bug In The Clearml Dashboard: After Queuing An Experiment, If I Change The Experiment Queue, Then Go Back To The Experiment Info Tab, The Queue Property Still Shows The Previous Queue

JitteryCoyote63 , just making sure, does refresh fixes the issue ?

4 years ago

0 Hi, I’M Trying To Figure Out What Do The Clearml Agents Use The Webserver Endpoint For And What Would Break If One Didn’T Have Access? For Context: I’M Trying To Have A Self-Hosted Server With Endpoints Accessible Publicly, But Securely. The Webserver En

Hi HollowFish37
I think I have good news for you, the clearml-agent is only communicating with the api endpoint, so as long as this is secure, you should be fine. Do notice that the default files server endpoint should be secure as well, as by default it will allow any upload/download

4 years ago

0 Hello! I’M Wondering If There Is An Option To Run A Termination Hook Script

A single query will return if the agent is running anything, and for how long, but I do not think you can get the idle time ...

3 years ago

0 What Is

PipelineController works with default image, but it incurs overhead 4-5 min

You can try to spin the "services" queue without docker support, if there is no need for containers it will accelerate the process.

Repository cloning failed: Command '['git', 'fetch', '--all', '--recurse-submodules']' returned non-zero exit status 1.

This error is about failing to clone the pipeline code repo, how is that connected to changing the container ?!
Can you provide the full log?

4 years ago

0 Is Clearml Able To Intercept (Automatically) Metrics Gathered Via

Where are they stored? I could not find a backend they work with, what am I missing?

2 years ago

0 Hi, Currently It Seems That Trains-Agent Writes Files With The User "Nobody", Group "Nogroup" And Permissions 777 To Created Files. How Can I Change That? To The Very Least, Change The User Group It Uses? Running On Linux Ubuntu

The agent cannot use another user (it literally has no way of getting credentials). I suspect this is all a by product of the actual mount point)

4 years ago

Show more results