Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
TenseOstrich47
Moderator
22 Questions, 100 Answers
  Active since 10 January 2023
  Last activity 11 months ago

Reputation

0

Badges 1

90 × Eureka!
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
Has anybody integrated Airflow with ClearML here? If so, what has been your experience?
3 years ago
0 Votes
3 Answers
1K Views
0 Votes 3 Answers 1K Views
Also, is there a way to remove the Examples from my server deployment? I can't delete the tasks. I tried to archive the task prior to deletion and I get the ...
4 years ago
0 Votes
12 Answers
1K Views
0 Votes 12 Answers 1K Views
3 years ago
0 Votes
30 Answers
1K Views
0 Votes 30 Answers 1K Views
4 years ago
0 Votes
4 Answers
818 Views
0 Votes 4 Answers 818 Views
Can't access the docs at the moment. SSL error. Can you guys verify whether your SSL certificate has expired or not please? Unsure whether the error is on my...
3 years ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hey all. Question about the bucket config for clearml. Is it possible to have two buckets in the config? We have partners in EU and US, and due to data gover...
3 years ago
0 Votes
13 Answers
1K Views
0 Votes 13 Answers 1K Views
Why would every submitted task be aborted directly?
3 years ago
0 Votes
3 Answers
1K Views
0 Votes 3 Answers 1K Views
3 years ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
3 years ago
0 Votes
15 Answers
1K Views
0 Votes 15 Answers 1K Views
Hey all. Question regarding scheduling and orchestration. Does ClearML provide any tooling to schedule entire training pipelines and to trigger training pipe...
2 years ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hey all. Thinking about organisation of artefacts and how we store them in S3. We have been using the StorageManager to upload files to target S3 destination...
3 years ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
Can the StorageManager upload directly from memory or does it have to be from a local file?
3 years ago
0 Votes
14 Answers
1K Views
0 Votes 14 Answers 1K Views
Any ideas why this is happening? It was fine yesterday
4 years ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hey. This may be a silly question ... but I am unsure about how the clearml-agent works. Does it need to be installed on the local server (from which the exp...
4 years ago
0 Votes
18 Answers
1K Views
0 Votes 18 Answers 1K Views
4 years ago
0 Votes
9 Answers
1K Views
0 Votes 9 Answers 1K Views
Hey. May be a question asked previously , but does clearml support ECR? I have a series of pre-created containers that I would like to use during training. I...
2 years ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
Hi. I'm struggling with executing a python script which is a replica of a notebook, which executes fine. The issue I am encountering is clearml_agent: ERROR:...
4 years ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
Hey all. Is there a best practice approach to deploying models trained in clearml? Does anyone have a standard workflow that they employ?
3 years ago
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
2 years ago
0 Votes
25 Answers
1K Views
0 Votes 25 Answers 1K Views
Hey all. Quick question about the ~/.clearml directory on the clearml-agent. It seems to be filling up disk storage quite quickly 440M /home/ubuntu/.clearml/...
4 years ago
0 Votes
11 Answers
1K Views
0 Votes 11 Answers 1K Views
Hey all. I'm seeing a strange error when trying to run hyperparameter optimisation by cloning a base training task Action failed <500/0: tasks.clone/v1.0 (ke...
4 years ago
0 Votes
11 Answers
1K Views
0 Votes 11 Answers 1K Views
4 years ago
3 years ago
0 Hey. Seeing A Strange Error When Trying To Install A Private Python Package When The Agent Executes The Task. We Use Aws Codeartifact As The Private Pypi Repository. The Agent Attempts To Install The Package From Git, Rather Than A Regular Pip Install And

Nope, from a remote server. It was that I had installed the package from git locally, so when pushing the task, clearml assumed it should also install from git. I since installed the package from the private pypi and it all works as expected now 🙂

3 years ago
0 Hey All. Question Regarding Scheduling And Orchestration. Does Clearml Provide Any Tooling To Schedule Entire Training Pipelines And To Trigger Training Pipelines In Response To Events, E.G. Degraded Model Performance Alerting?

I can't figure out from the examples how the external trigger works. All of our model performance stats are in the DWH, and we want to build triggers based on that, Is that possible to integrate with Clearml triggers and schedulers?

2 years ago
0 Hey All. Another Question - How Are Private Packages Handled/Installed So That Clearml-Agent Can Execute A Task? I Have A Bunch Of Private Repos For Communicating With The Data Warehouse. I Could Do A System-Wide Installation For It On The Clearml-Agent I

Hey Martin. We have managed to resolve this. FYI the issue was with the resolving of the host. It had to be changed from @github.com to what the host is in the ssh config file!

4 years ago
0 Hi All. I Am Struggling With Integrating Plots Into My Task. Without The Plotting Code, The Task Never Completes The Execution And Seems To Hang. Also, The Plots Are Not Visible In The Plots Tab. I Am Running A For Loop For Different Models And Attemptin

I dont think its that. its a 20kb file upload. This was the last message just printed
ClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-star

4 years ago
0 Hi All. I Am Struggling With Integrating Plots Into My Task. Without The Plotting Code, The Task Never Completes The Execution And Seems To Hang. Also, The Plots Are Not Visible In The Plots Tab. I Am Running A For Loop For Different Models And Attemptin

` # Plot the confusion matrix for predictions
sns.heatmap(
preds_confusion_percentage, annot=True, fmt=".3f", linewidths=.5,
square=True, cmap='Blues_r'
)
plt.ylabel('Actual label')
plt.xlabel('Predicted label')
title_str = f'Accuracy Score: {round(score, 2)}\n{TRANSFORM_TYPE}'
plt.title(title_str, size=15)

task.logger.report_matplotlib_figure(
title=f"Performance Heatmap - {model_export_name}",
series="Device Brand Predictions",
iteration=0,
figure=pl...

4 years ago
0 Hey All. Another Question - How Are Private Packages Handled/Installed So That Clearml-Agent Can Execute A Task? I Have A Bunch Of Private Repos For Communicating With The Data Warehouse. I Could Do A System-Wide Installation For It On The Clearml-Agent I

While we are here - excuse my ignorance for now if this has already been stated in the docs ..

Is it possible to launch multiple clearml-agents on a dedicated clearml-agent server? I noticed that with one agent, only one task gets executed at one time

4 years ago
0 Hi Again, Is There A Way To Pass Secrets As Parameters Of A Task? I Have An Experiment That Requires Connecting To A Database, And I Need To Be Able To Pass The Creds As Task Params (Or In Another Way, I Don'T Know Yet). But I Don'T Want To Expose My Cred

How we resolved this issue is by developing a package to deal with the connecting and querying to databases. This package is then used inside the task for pulling the data from the data warehouse. There is a devops component here for authorising access to the relevant secret (we used SecretsManager on AWS). The clearml-agent instances are launched with role permissions which allow access to the relevant secrets. Hope that is helpful to you

4 years ago
0 Hey All. I'M Seeing A Strange Error When Trying To Run Hyperparameter Optimisation By Cloning A Base Training Task

Hey Martin. By labels map, I'm referring to the labels map assigned to the model. The one you can view in the models tab // labels

4 years ago
0 Hey All. Another Question - How Are Private Packages Handled/Installed So That Clearml-Agent Can Execute A Task? I Have A Bunch Of Private Repos For Communicating With The Data Warehouse. I Could Do A System-Wide Installation For It On The Clearml-Agent I

Here is the error message from the console
Collecting git+ssh://****@github.com/15gifts/py-db.git Cloning ssh://****@github.com/15gifts/py-db.git to /tmp/pip-req-build-xai2xts_ Running command git clone -q 'ssh://****@github.com/15gifts/py-db.git' /tmp/pip-req-build-xai2xts_ ERROR: Repository not found. fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.

4 years ago
4 years ago
0 Any Ideas Why This Is Happening? It Was Fine Yesterday

I literally cannot reset a single task

4 years ago
0 Hey All. I'M Seeing A Strange Error When Trying To Run Hyperparameter Optimisation By Cloning A Base Training Task

Yeah, it's not urgent. I will change the labels around to avoid this error 🙂 thanks for checking!

4 years ago
0 Good Morning, I'M Wondering If Someone Has Any Advice/Experience Configuring Clearml-Agent To Include Private Packages From Aws Codeartifact? So Far I Know I Have To Edit The

Did the shell script route work? I have a similar question.

It's a little more complicated because the index URL is not fixed; it contains the token which is only valid for a max of 12 hours. That means the ~/.config/pip/pip.conf file will also need to be updated every 12 hours. Fortunately, this editing is done automatically by authenticating AWS codeartefact in the command line by logging in.

My current thinking is as follows:

Install the awscli - pip install awscli (c...

3 years ago
0 Good Morning, I'M Wondering If Someone Has Any Advice/Experience Configuring Clearml-Agent To Include Private Packages From Aws Codeartifact? So Far I Know I Have To Edit The

One question - you can also set the agent.package_manager.extra_index_url , but since this is dynamic, will pip install still add the extra index URL from the pip config file? Or does it have to be set in this agent config variable?

3 years ago
0 Hi All, We'Ve Been Using Clearml For Quite Some Time Now. Our Deployment Is A Simple Docker Container On A Dedicated Ec2 Instance. More Recently, We Have Been Running Out Of Storage, And It Looks To Be That Elasticsearch Is The Main Culprit. What Data I

SuccessfulKoala55 thanks for your help as always. I will try to create a DAG on airflow using the SDK to implement some form of retention policy which removes things that are not necessary. We independently store metadata on artefacts we produce, and mostly use clearml as the experiment manager, so a lot of the events data can be cleared.

2 years ago
0 Hey.

Oh great, thanks! Was trying to figure out how the method knows that the docker image ID belongs to ECR. Do you have any insight into that?

2 years ago
0 Hey. Seeing A Strange Error When Trying To Install A Private Python Package When The Agent Executes The Task. We Use Aws Codeartifact As The Private Pypi Repository. The Agent Attempts To Install The Package From Git, Rather Than A Regular Pip Install And

I can authorise CodeArtifact if I ssh into the server, and install the private package with no issues. Seems like something is forcing clearml-agent to use github cloning to install, rather than directly pip. Not sure if this is a configuration I have set up myself, or whether the server is configured to do this

3 years ago
0 Hey All. Is There A Best Practice Approach To Deploying Models Trained In Clearml? Does Anyone Have A Standard Workflow That They Employ?

In particular, I am trying to find a neat way to query all models available, and use tags to know the context. As it stands, I log the model accuracies/RMEs as part of the metadata, alongside the training data filepath. Issue is that this is not the neatest way of querying models across tasks without a lot of laborious manual lifting. Suggestions welcome

3 years ago
0 Hey.

And how will it know that the container is on ECR instead of some other container repository?

2 years ago
0 Hey.

yes it does, but that requires me to manually create a new agent every time I want to run a different env no?

2 years ago
0 Hey All. Question Regarding Scheduling And Orchestration. Does Clearml Provide Any Tooling To Schedule Entire Training Pipelines And To Trigger Training Pipelines In Response To Events, E.G. Degraded Model Performance Alerting?

Our model store consists of metadata stored in the DWH, and model artifacts stored in S3. We technically use ClearML for managing the hardware resource for running experiments, but have our own custom logging of metrics etc. Just wondering how tricky integrating a trigger would be for that

2 years ago
Show more results compactanswers