Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
AgitatedDove14
Moderator
49 Questions, 8094 Answers
  Active since 10 January 2023
  Last activity 10 months ago

Reputation

0

Badges 1

25 × Eureka!
0 Hi, We Are Using Clearml For Our Experiment Tracking But Now Investigating Using The Pipeline Functionality As Well For Scheduling. We Also Want To Be Able To Trigger A Pipeline Run When There Is New Data In An External Database. Is This Possible? From Wh

why is pushing into the services queue required ...

The services queue is usually connected with an agent running in "services mode" which means this agent is executing multiple tasks in parallel (as opposed to regular agent that only launches one Task at a time, the assumption is that "service" Tasks are usually not heavy on cpu/ram so multiple instances make sense)

one year ago
2 years ago
0 Hi I Saw This On The Clearml-Agent Docs But Other Than The Docker Image, I'M Not Sure How To Integrate This With Clearml Py And Clearml-Server. Please Advise.

TypeError:Β 

init

() got an unexpected keyword argument 'base_pod_num'

Could you post the entire log?

3 years ago
0 How Can I Tell Clearml To Ignore Certain Submodules Existing In The Project? My Projects Consists Of Multiple Git Submodules And It Is Rather Annoying That The Task Always Tries To Fetch All Submodules, When They Are Not Even Necessary. I Don'T Know How I

Hi @<1694157594333024256:profile|DisturbedParrot38>
You mean how to tell the agent to pull only some submodules of your git?
If this is the case you can actually remove them on your git branch, submodule is a file with a soft link. Wdyt?

9 months ago
0 Hi Everyone! Is There A Way To Specify The Working Directory In A Pipeline Component? I’M Using Pipelines From Decorators, I Can Set The Repo Url Just Fine, But I’M Running Everything From A Subfolder, And The Working Dir Is Set To

Hmm yes, @<1570220858075516928:profile|SlipperySheep79> I think you are right in your case it make sense to do add this option.
Could you add GH issue with the feature request? it should be fairly easy to add and we use GH to make sure we track those requests
wdyt?

one year ago
0 Crazy Idea:

This is good stuff! Keep us posted here, I'll do my best to support live 😍

one year ago
0 Hi, Would Like To Check. So An Agent Pulled A Docker Image And Install The Pip Dependencies On It. What If I Have Os Library Dependencies As Well? (Apt Install, Rpm Install...Etc).

Hi SubstantialElk6

What if I have OS library dependencies as well? (Apt install, rpm install...etc).

If these are OS libraries that you always need you can put them here:
https://github.com/allegroai/clearml-agent/blob/d9b9b4984bb8a83914d0ec6d53c86c68bb847ef8/docs/clearml.conf#L136
agent.extra_docker_shell_script: ["apt-get install -y bindfs", ]In the next version, this could be controlled on a per Task basis.
FYI: the default apt package that are installed:
` apt-get update
a...

3 years ago
5 months ago
0 More Of Pushing Clearml To It'S Data Engineering Limits

Hi @<1541954607595393024:profile|BattyCrocodile47>

Can you trigger a pre-existing Pipeline via the ClearML REST API?

Yes

'd want to have a Lambda function trigger the Pipeline for a batch without needing to have all the Pipeline code in the lambda function.

Easiest is to use clearml SDK, which basically is clone / enqueue (notice that pipeline is also a kind of a Task). See here: [None](https://github.com/allegroai/clearml/blob/3ca6900c583af7bec18792a4a92592b94ae80cac/example...

one year ago
0 Hi Again Everybody, Is There A Way To Cache The Docker Containers Used By The Agents? As Far As I Understood Every Time It Spawns A New Container With A Shared Apt Cache (And Venv Cache If Configured)

Hi TrickyFox41

is there a way to cache the docker containers used by the agents

You mean for the apt get install part? or the venv?
(the apt packages themselves are cached on the host machine)
for the venv I would recommend turning on cache here:
https://github.com/allegroai/clearml-agent/blob/76c533a2e8e8e3403bfd25c94ba8000ae98857c1/docs/clearml.conf#L131

2 years ago
0 Apart From Having Packages In Requirements.Txt, Does Clearml Expect Them To Be Actuall Installed To Add Them As Installed Packages For A Task?

Is it not possible to say just look at my requirements.txt file and the imports in the script?

I think there is a GitHub Issue for this feature
(basically the issue is, requirements.txt are very often not updated, and have no real version lock, so replicating a working env is always safer)

3 years ago
3 years ago
0 Hi, I'Ve A Few Questions On Clearml-Session.

Hi SubstantialElk6

We will be running some GUI applications so is it possible to forward the GUI to the clearml-session?

If you can directly access the machine running the agent, yes you could. If not reverse proxy is in the working πŸ˜‰

We have a rather locked down environment so I would need a clear view of the network view and the ports associated.

Basically all connections are outgoing only, with the exception of the clearml-server (listening on ports 8008 8080 8081)

3 years ago
0 Hi Everyone! I'M Trying To Upload Roc Figure From Matplotlib To Clearml. Unfortunately Clearml Adds Invalid Legend Item To The Plot As You Can See On The Attached Image. Is There Any Way To Hide This Junk?

Hi SpicyOtter88
plt.plot([0, 1], [0, 1], 'r--', label='')ti cannot have a legend without a label, so it gives it "anonymous" label, I think it should just get "unlabeled 0" wdyt?

2 years ago
0 Hi, In This

Hi OddAlligator72

itΒ 

Β that they do not support PBT.

The optimization algorithm themselves are usually external (although the trivial stuff are in within Trains)
Do you have a specific PBT implementation you are considering ?

4 years ago
0 Okay Another Question !! Okay So I Would Like To Edit Parameters Through The Ui And Run It. So This Is My Script

Hi CluelessElephant89
When you edit the args (General section) in the UI, you are editing the args for "remote execution"
(i.e. when executed by the agent, the args dict will get the values from the UI , as oppsed to "manual execution" where there UI gets the values from code)
In order to simulate the "remote execution" inside your development environment
Try:
` from clearml import Task

simulate remote execution of a specific Task instance

Task.debug_simulate_remote_task(task_id='R...

3 years ago
0 Hi, I'M Trying To Use

SoggyBeetle95 is this secret a per Task secret, or is it for the agent itself (I.e. for all Tasks the agent will spin)?

2 years ago
0 Hi, Currently It Seems That Trains-Agent Writes Files With The User "Nobody", Group "Nogroup" And Permissions 777 To Created Files. How Can I Change That? To The Very Least, Change The User Group It Uses? Running On Linux Ubuntu

nfs version 3

That's the thing, NFS will automatically set file access and flags based on the mount options you cannot change them post mount.
How about creating a new user just for the agent, it makes sense from security / credentials perspective

4 years ago
0 Hi, I Have Been Getting The Following For A While. Is There A More Detailed Log I Can Look Into? This Happens On Both Https And Http.

Hi SubstantialElk6
If you are using boto to acess anything that is Not AWS S3 you have to add both address and port, and make sure you configure the "security" flag.
See example in clearml.conf :
https://github.com/allegroai/clearml-agent/blob/176b4a4cdec9c4303a946a82e22a579ae22c3355/docs/clearml.conf#L247
` aws {
s3 {

            {
                host: "my-minio-host:9000"
                key: "12345678"
                secret: "12345678"
              ...
3 years ago
0 Hey, Great Product! I'Ve Installed Trains Agent On A Python3 Venv, But When I Run A Script On The Worker, It Calls Python2 Instead Of Python 3. How To Change It?

VivaciousWalrus99
Yes this is odd:
1608392232071 spectralab:gpu0 DEBUG New python executable in /cs/usr/gal.hyams/.trains/venvs-builds/3.7/bin/python2So it thinks it has python v3.7 but it is using python2 in the venv...
In your trains.conf file, set agent.python_binary to the python3.7 binary. It should be something like:
agent.python_binary=/path/to/python/python3.7

4 years ago
0 Hi

In the agent, no, it pipes stdout/stderr of the container and logs everything 😞

to get a json or something like that?

There is an api to get all the console logs, is this what you are after?

2 years ago
0 Hi!

Or is it already expected behavior that ClearML agent rewrites ...

Yep, that should work

3 years ago
0 Hi , I Am New To Clearml , I Am Trying To Create A Pipeline That Have Multiple Steps That Run Inside A Gitlab Repo , So I Pass It Add_Function_Step With A Token Like : Repo=F"

Hi @<1798887590519115776:profile|BlushingElephant78>

have multiple steps that run inside a gitlab repo

one thing to make sure that is not missed, "inside a gitlab repo" , notice the actual pipeline is running on agents/workers/k8s gitlab will be the trigger and monitor it, but the actual execution should probably happen somewhere with proper compute

this works fine when i run the pipeline from the script directly but when i run it from the web interface

try to configure your...

10 days ago
0 Hi, I Am Wondering After A Task Submitted To Remote Server Finishing Running. Will The Docker Container / Disk Space (Really I Am More Interested About The Dataset That Download By The Task) Get Automatically Clean Up?

Hi @<1797800418953138176:profile|ScrawnyCrocodile51>

Will the docker container / disk space (really I am more interested about the dataset that download by the task) get automatically clean up?

Yes, the agent is running the container with --rm πŸ™‚

3 days ago
Show more results compactanswers