AgitatedDove14

49 Questions, 8124 Answers

Active since 10 January 2023

Last activity one year ago

Reputation

Badges 1

25 × Eureka!

Answers 8124

0 When Using Azure Blob Storage For Storing Artifacts, Is There A Possibility To Use Sas Token Instead Of Storage Account Key For Authentication? When I Specify Sas Token (With All Permissions) Instead Of Account Key In Clearml.Conf I'M Getting Authenticati

Hi ScaryLeopard77
Could that be solved with this PR?
https://github.com/allegroai/clearml/pull/548

3 years ago

0 Hello, We Are Currently Working On A Hyperparameter Tuning Job For Object Detection Following This Tutorial

DeterminedToad86 were you running a jupyter notebook or a jupyter console ?

4 years ago

0 Hi

The problem is of course filling in all the configuration details, so that they are viewable.
Other than that, check out:
https://allegro.ai/docs/task.html#trains.task.Task.export_task
https://allegro.ai/docs/task.html#trains.task.Task.import_task
Sounds good ?

4 years ago

0 [Pipeline] Hey, Is It Possible To Specify The Output Uri For Pipelines And Their Components Using Pipeline Decorators? I Would Like To Store Pipeline Artifacts And Component Artifacts On S3.

Hi ReassuredOwl55
The easiest is to configure it as default output_uri in the clearml.conf of file the agent, wdyt?
https://github.com/allegroai/clearml-agent/blob/ebb955187dea384f574a52d059c02e16a49aeead/docs/clearml.conf#L430

2 years ago

0 Hi, I Wanted To Try Model Versioning, Suppose That I'Ve A Model And Want To Have Multiple Versions Of The Same Model And To Be Able To Have Inference On These Models(For Example

@<1671689437261598720:profile|FranticWhale40> could you test the fix? just pull & run

allegroai/clearml-serving-triton:1.3.1
allegroai/clearml-serving-inference:1.3.1

one year ago

0 I Wanted To Ask, I'M Versioning My Data Using Clearml Data. And I'Ll Have A Training Task With Clearml Task. My Question Is, Does Clearml Keep Track Of The Data Versions Fetched From Clearml Data? Basically I Want To See How Much Of Tracking And Informati

VexedCat68 actually a few users already suggested we auto log the dataset ID used as an additional configuration section, wdyt?

3 years ago

0 Does K8S Glue Support Running Service Agent? Slightly Confused Here

I guess it won’t due to the nature of services?

Correct, k8s glue works differently, that said I would actually use the helm to spin a pod woth the agent in services mode and venv mode.

4 years ago

0 My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I

Actually, dumb question: how do I set the setup script for a task?

When you clone/edit the Task in the UI, under Execution / Container you should have it
After you edit it, just push it into the execution with the autoscaler and wait 🙂

2 years ago

0 For The Frameworks Which Are Supported In Built, Trains Stores The Trained Model As Output Model E.G. For Xgboost Here

so what should the value of "upload_uri" to set to,

fileserver_url

e.g.

?

yes, that would work.

5 years ago

0 It Seems Like Clearml Agent Does Not Support Arparse Subparsers, Right?

and does the code above reproduce the issue/bug? because obviously should not happen

2 years ago

0 Hi, I Have A Future Roadmap Question On Clearml-Datasets. The Current Implementation Works Well For Small Datasets But Its Rather In Effective For Very Large Datasets. For Example, Let'S Say I Have 10 Million Images Just For The Training Dataset, And My T

SubstantialElk6 I just realized 3 weeks passed, wow!
So the good news we have some new examples:
https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_decorator.py
https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_functions.py
The bad news the documentation was postponed a bit, as we are still messaging the interface (the community is constantly pushing for great ideas and uses cases , and they are just too good to miss out 🙂 )...

3 years ago

0 Hey I’M Running This Script And Initialise The Clearml Task Also In This File

Or can it also be right after

Task.init()

?

That would work as well 🙂

2 years ago

0 Hey All, We Are Trying To Clone A Task That Uses Custom Pip Installed Packages And Run It Via An Agent. When Running Locally, We Simply “

@<1523701079223570432:profile|ReassuredOwl55> did you try adding manually ?

./path/to/package

You can also do that from code:

Task.add_requirements("./path/to/package")
# notice you need to call Task.add_requirements before Task.init
task = Task.init(...)

2 years ago

0 Hey - I'M Trying To Compare Voxel Versus Clear Ml In Image Data Exploration.

Hi @<1523701111020589056:profile|DefiantSpider5>
So there are two answers here, I'll start with the open-source version of both

Is there a way in clear ml to interactively view subsets of images based on a lasso of embedding plots

The ClearML Datasets have no "query" capabilities of the data inside the dataset. That means you can see preview images, statistics and download the datasets, but no query capabilities. On the other hand, there is no limitation on the type and format of me...

2 years ago

0 Hello, I Have A Question Regarding Creating A Clearml Pipeline Using Pytorch Lightning. I Am Not Really Sure Where To Begin. Should I Create A Task For Each Pytorch Lightning Class In My Pipeline? Is There A Demo Or Clearml Project That Specifically Uses

How do I best utilize clearml in this scenario such that any coworker of mine is able to reproduce my work with the same pipeline?

Basically this sounds to me like proper software developemnt design (i.e. the class vs stages).
In order to make sure Anyone can reproduce it, you mean anyone can rerun the "pipeline" ? If this is the case just add Task.init (maybe use a specific Task type) and the agents will make sure this is Fully reproducible.
If you mean the data itself is stored, the...

2 years ago

0 Hi

send the agent's logs to log management and monitoring service,

These are stored into ELK, it was built to store large amounts of logs, I cannot see any reason why one would want to remove it?

Maybe if there would be a way to change their format, it could also help filtering them from my side.

You mean in the UI?

2 years ago

0 Hi All, I'Ve Successfully Run A Task Locally, And Now I'M Trying To Clone It And Send It To A Queue. It Looks Like The Environment Is Built Successfully, But It Hangs Here:

Retrying (Retry(total=239, connect=240, read=240, redirect=240, status=240)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)'))': /auth.login

OH that makes sense I'm assuming on your local machine the certificate is installed but not on remote machines / containers
Add the following to your clearml.conf:

api.verify_certificate: false

[None](https...

one year ago

0 Hi Everyone, I'M Trying To Deploy My First Clearml Pipeline With A Configuration For Logging And Caching To Avoid Repeating Already Computed Steps. However, The Caching Doesn'T Seem To Be Working Correctly. Despite Not Changing The Configuration, The Firs

Hi @<1730396272990359552:profile|CluelessMouse37>

However, the caching doesn't seem to be working correctly. Despite not changing the configuration, the first step runs every time.

How are you creating the cached component?
is this a standalone script or a git repo link?

These parameters are dictionaries of specific configurations (dict of dict) that are the same but might not be taken into account properly by the caching mechanism.

hmm for the component to be cached (or reuse...

one year ago

0 Hi Folks, I Did A Deployment Of Clearml Using The K8S Helm Chart, And I Set The Agent Using K8S Glue. I Run A Task Locally, And I Went To The Ui Cloned The Experiment And Scheduled It In The Default Queue. After Doing This, I See That The Experiment Is Q

Click on the "k8s_schedule" queue, then on the right hand side, you should see your Task, click on it, it will open the Task page. There click on the "Info" Tab, there look for "STATUS MESSAGE" and "STATUS REASON". What do you have there?

3 years ago

0 Is There A Way To Get Tasks By Hyperparameters Values? When I Use The Search In The Ui I Get The Relevant Task, But When I Try The Following I Get An Empty List:

HandsomeCrow5 Ideas on improvement are always welcome 🙂

5 years ago

0 Hi Again, Is There A Way To Pass Secrets As Parameters Of A Task? I Have An Experiment That Requires Connecting To A Database, And I Need To Be Able To Pass The Creds As Task Params (Or In Another Way, I Don'T Know Yet). But I Don'T Want To Expose My Cred

TenseOstrich47 Nice!

4 years ago

0 I Just Deployed Clearml Into K8 Cluster Using Clearml Helm Package. When I Ran A Job, It Gave This Error In The Clearml Web Server (Attached Below). I Sshed Into The Pod Running The Clearml-Agent. Upon Typing Clearml-Agent Init, I Realised The Clearml.Con

Hi DeliciousBluewhale87
My theory is that the clearml-agent is configured correctly (which means you see it in the clearml-server). The issue (I think) is that the Task itself (running inside the docker) is missing the configuration. The way the agent passes the configuration into the docker is by mapping a temporary configuration file into the docker itself. If the agent is running bare-metal, this is quite straight forward. If the agent is running on k8s (or basically inside a docker) th...

4 years ago

0 Hi, I Am Running A File Like This

The warning just let's you know the current processes stopped and itis being launched on a remote machine.
What am I missing? Is the agent failing to run the job that you create manually ?
(notice that when creating a job manually, there is no "execute_remotely", you just enqueue it, as it is not actually "running locally")
Make sense ?

4 years ago

0 Hi, I Am Trying To Do Some Hyperparameter Search With Clearml. I Would Like To Define A Logarithmic Uniform Range For Some Hyperparams But The Options I Find Are Uniformparameterrange And Uniformintegerparameterrange Which I Assume Are Linear Ranges. Is T

I'm having another problem now because I am using the OptunaOptimizer.

Hmm let me check a sec

3 years ago

0 Pytorch-Lightning-Bols.Loggers.Trainslogger

YummyWhale40 from the code snippet, it seems like the argument is passed.
"reuse_last_task_id=True" is the default, and it means that if the previous run of the task did not create any artifacts/models and was executed 72 hours ago (configurable), The Task will be reset (i.e. all logs cleared) and will be reused in the current run.

5 years ago

0 Hi, Where Can I Find The Logs Of Trains-Agent By Default?

Hi JitteryCoyote63 when you run the trains-agent it tells you where it puts the logs, it's a temp auto generated filename usually under /tmp/
Running TRAINS-AGENT daemon in background mode, writing stdout/stderr to /tmp/.trains_agent_daemon_out4uahki3i.txt

5 years ago

0 When Running An Experiment From A Notebook, It Knows It’S A Notebook And Automatically Adds The Notebook As An Artifact Right? And The Uncommited Changes Becomes The Nottebook Converted To A Script? In One Case I Am Seeing Actual Git Diff Coming In Instea

(I'll make sure it is added to the docstring because apparently it was not there

4 years ago

0 Hi, I Have An Agent That Is Running Two Experiments At The Same Time: One That Was Running For A Long Time (11H) And One That The Agent Picked Up Afterwards, While The First One Was Still Running. Context: I Have 3 Agents Up (Not In Docker Mode) And All O

Hi JitteryCoyote63
I think that what happens is that the agent are registered on the same name (id). How many agent do you see in the "Workers" tab?

5 years ago

0 Hi

LOL

4 years ago

0 The “Manage Queue” Option In The Right Tab On A Queued Experiment Is Broken In V1.0 (It Does Nothing)

It should move you directly into the queue pages.
Let me double check (working on the community server)

4 years ago

Show more results