Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
CooperativeFox72
Moderator
11 Questions, 92 Answers
  Active since 10 January 2023
  Last activity 11 months ago

Reputation

0

Badges 1

92 × Eureka!
0 Votes
17 Answers
1K Views
0 Votes 17 Answers 1K Views
Hi, I am new here, can I ask question on trains-server also?
4 years ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hi all, I think their is a UI bug. When trying to add experiment to compare:
4 years ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
Hi, I upgraded the ClearML client to 1.0.5 and now I am getting an upload log message: ... 2021-08-12 17:47:59,188 - clearml.storage - INFO - Uploading: 150....
3 years ago
0 Votes
30 Answers
1K Views
0 Votes 30 Answers 1K Views
Hi all, I am starting to use clearml-agent. run it with clearml-agent daemon --foreground --gpus 3 --queue default --docker MyDockerImage:v0then I enqueued n...
4 years ago
0 Votes
12 Answers
1K Views
0 Votes 12 Answers 1K Views
Hi, Another question There is a way to know if a job is running locally or remotely? Like execute_remotely knows ... > Note If the > task > is running remote...
4 years ago
0 Votes
20 Answers
1K Views
0 Votes 20 Answers 1K Views
Hey, I am trying to move the fileserver to S3. As here: https://github.com/allegroai/trains-server/issues/35 I update the trains.conf with my s3 bucket and t...
4 years ago
0 Votes
17 Answers
1K Views
0 Votes 17 Answers 1K Views
Hi again, I tried to upgrade Trains package to 15.1 from 13.1 that I was using for a while.. After the upgrade my code stuck when trying to use "Pool" (from ...
4 years ago
0 Votes
22 Answers
1K Views
0 Votes 22 Answers 1K Views
Hi all, I have a trains-server (self-host) on a EC2 machine. The version of it is older then 0.16. I like to move to ClearML-server on different machine but ...
4 years ago
0 Votes
31 Answers
61K Views
0 Votes 31 Answers 61K Views
Hi all, I like to upgrade trains-server:0.16.1 to clearml-server:0.17 In the https://github.com/allegroai/clearml-server#upgrading- the process looks the sam...
4 years ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hi all 🙂 , There is a way to freeze the iteration monitoring? I like to download my data/model at the start of my code, but it takes a while and I am gettin...
3 years ago
0 Votes
10 Answers
1K Views
0 Votes 10 Answers 1K Views
Hi all 🙂 I have a question regarding task.connect_configuration() . Does it possible to update the file on the server when running remotely or locally? for ...
4 years ago
0 Hi All, I Like To Upgrade

` [2021-01-24 17:02:25,660] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_all in 2ms
[2021-01-24 17:02:25,674] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_next_task in 8ms
[2021-01-24 17:02:26,696] [8] [INFO] [trains.service_repo] Returned 200 for events.add_batch in 36ms
[2021-01-24 17:02:26,742] [8] [INFO] [trains.service_repo] Returned 200 for events.add_batch in 78ms
[2021-01-24 17:02:27,169] [8] [INFO] [trains.service_repo] Returned 200 for projects.get_al...

4 years ago
0 Hi All

From the UI it will since it getting the temp file from there.
I mean from the code (let say remotely)

4 years ago
0 Hi All, I Like To Upgrade

Hi SuccessfulKoala55 ,
I down the server:
` [ec2-user@ip-172-31-26-41 ~]$ sudo docker-compose -f /opt/clearml/docker-compose.yml down
WARNING: The CLEARML_HOST_IP variable is not set. Defaulting to a blank string.
WARNING: The CLEARML_AGENT_GIT_USER variable is not set. Defaulting to a blank string.
WARNING: The CLEARML_AGENT_GIT_PASS variable is not set. Defaulting to a blank string.
Stopping clearml-webserver ... done
Stopping clearml-agent-services ... done
Stopping clearml-apiserver...

4 years ago
0 Hi All, I Have A

The script is running for more then 45 min, does it regular?

4 years ago
0 Hi All, I Am Starting To Use Clearml-Agent. Run It With

Ok looks It is starting the training...
Thanks 💯

4 years ago
0 Hi, Another Question There Is A Way To Know If A Job Is Running Locally Or Remotely? Like

my docker has my project on it all ready so I know where to mount. Maybe the agent moves/create copy of my project somewhere else?

4 years ago
0 Hi All, I Am Starting To Use Clearml-Agent. Run It With

So for now I am leaving this issue...
Thanks a lot 🙏 🙌

4 years ago
0 Hi All, I Have A

Thanks I will take a look 👀

4 years ago
0 Hi Again, I Tried To Upgrade Trains Package To 15.1 From 13.1 That I Was Using For A While.. After The Upgrade My Code Stuck When Trying To Use "Pool" (From Multiprocessing Import Pool) The Code Snip:

Thanks AgitatedDove14 ,
I need to check with my boss that it is OK to share more code, will let you know..

But I will give 0.16 a try when it will release.
🙏

4 years ago
0 Hi All, I Like To Upgrade

the index creation:
[ec2-user@ip-172-31-26-41 ~]$ sudo docker exec -it clearml-mongo /bin/bash root@3fc365193ed0:/# mongo MongoDB shell version v3.6.5 connecting to: mongodb://127.0.0.1:27017 MongoDB server version: 3.6.5 Welcome to the MongoDB shell. For interactive help, type "help". For more comprehensive documentation, see Questions? Try the support group `
Server has startup warnings:
2021-01-25T05:58:37.309+0000 I CONTROL [initandlisten]
2021-01-25T05:58:37.309+0000 I C...

4 years ago
0 Hey, I Am Trying To Move The Fileserver To S3. As Here:

If I will mount the S3 bucket to the trains-server and link the mount to /opt/trains/data/fileserver does it will work?

4 years ago
0 Hey, I Am Trying To Move The Fileserver To S3. As Here:

Thanks for the reply,
I saw that it prefer to change the fileserver in trains.conf to s3://XXX
So, I changed this as I wrote before.

4 years ago
0 Hey, I Am Trying To Move The Fileserver To S3. As Here:

Ohh I understood, so can you give me a short explanation on how to change the meta data?

4 years ago
0 Hi, I Upgraded The Clearml Client To

Hi, AgitatedDove14 Thanks for the answer.

I think the upload reporting (files over 5mb) was added post 0.17 version,

That what I thought...

I think it can be helpful to add it to the conf since 5MB is really small and my files are ~300MB, meaning 60 messages for each upload.
Another option is maybe to configure it as Task.init() parameter

I think both are OK 🙂

3 years ago
0 Hi All, I Am Starting To Use Clearml-Agent. Run It With

ARG USER_ID=1000 RUN useradd -m --no-log-init --system --uid ${USER_ID} appuser -g sudo RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers USER appuser WORKDIR /home/appuser

4 years ago
0 Hi All, I Am Starting To Use Clearml-Agent. Run It With

Thanks, I will make sure that all the python packages install as root..
And will let you know if it works

4 years ago
0 Hi, Another Question There Is A Way To Know If A Job Is Running Locally Or Remotely? Like

SuccessfulKoala55 Thanks 🙏 ..

Another related question:
My remote job fails because it cannot find the data.
FileNotFoundError: [Errno 2] No such file or directory: './data/XXXXXXXX I mounted the data to the same place relative to my project inside the docker with: extra_docker_arguments

I am using execute_remotely for enqueue the job.
I know it works locally since the job reads from ./data/XXXX before execute_remotely() and working.
but when the agent create ...

4 years ago
0 Hi, Another Question There Is A Way To Know If A Job Is Running Locally Or Remotely? Like

Hi SuccessfulKoala55 ,
Dose running_remotely() will return True even if the task was enqueued from UI and not by execute_remotely ?

4 years ago
4 years ago
0 Hi All, I Like To Upgrade

SuccessfulKoala55 and AppetizingMouse58 Thanks you very much!!

I have a future question:
Does this fix should harm in future cleraml-server upgrade?
Or what the best practice to upgrade after doing it?

4 years ago
0 Hi All, I Am Starting To Use Clearml-Agent. Run It With

Hi AgitatedDove14 ,
Sorry for the late response It was late at my country 🙂 .

This what I am getting
appuser@219886f802f0:~$ sudo su root root@219886f802f0:/home/appuser# whoami root

4 years ago
0 Hi Again, I Tried To Upgrade Trains Package To 15.1 From 13.1 That I Was Using For A While.. After The Upgrade My Code Stuck When Trying To Use "Pool" (From Multiprocessing Import Pool) The Code Snip:

AgitatedDove14 Hi, sorry for the long delay.
I tried to use 0.16 instead of 0.13.1.
I didn't have time to debug it (I am overwhelming with work right now).
But it doesn't work the same as 0.13.1. I am still getting some hanging in my eval process.
I am don't know if it just slower or really stuck since I killed it and move back to 0.13.1 until my busy time will pass.
Thanks

4 years ago
0 Hi, I Am New Here, Can I Ask Question On Trains-Server Also?

I an running trains-server on AWS with your AMI (instance type t3.large)

The server runs very good, and works amazing!
Until we start to run more training in parallel (around 20).
Then, the UI start to be very slow and getting timeouts often.
Does upgrading the instance type can help here? or there is some limit to parallel running?

4 years ago
0 Hey, I Am Trying To Move The Fileserver To S3. As Here:

Thanks I just want to avoid giving the credentials to every user.
If it won't possible, I will do it..

4 years ago
0 Hi, I Am New Here, Can I Ask Question On Trains-Server Also?

Thanks!! you are the best..
I will give it a try when the runs will finish

4 years ago
0 Hi All

OK thanks for the answer.. I will use
task.set_resource_monitor_iteration_timeout(seconds_from_start=1800)as you suggested for now..

If you will add something like I suggest can you notify me?

3 years ago
4 years ago
0 Hi All

I am sure you add this timeout for a reason.

Probably since increasing the timeout can affect other functionality. .

Am I wrong?

3 years ago
0 Hi All, I Like To Upgrade

I update to the new version 0.16.1 few weeks away and it works using the elastic_upgrade.py

4 years ago
Show more results compactanswers