Hi, I've run into a problem and would appreciate some help. I installed clearml locally. When I run a new task on a remote server and in the Python training ...
one day ago
SmoggyGoose12
1
Question,
5
Answers
Active since 04 December 2024
Last activity
one day ago
Reputation
0
Badges 1
5 × Eureka!
0
Hi, I'Ve Run Into A Problem And Would Appreciate Some Help. I Installed Clearml Locally. When I Run A New Task On A Remote Server And In The Python Training Code I Set It To Only Train On One Gpu. Everything Works Fine And I See All The Scalars Automatica
No, i have one queue, with one server in that queue. This server have 2 gpus. Using my training code i can choose if i use 2 gpus or just one in the training....
one day ago
0
Hi, I'Ve Run Into A Problem And Would Appreciate Some Help. I Installed Clearml Locally. When I Run A New Task On A Remote Server And In The Python Training Code I Set It To Only Train On One Gpu. Everything Works Fine And I See All The Scalars Automatica
Clearml-agent on worker: 1.9.2
Clearml on my computer: 1.16.4
2 gpus - NVIDIA GeForce RTX 3080
I reffer to only the training statistic, in the scalars tab. I can see the monitoring of the gpu's and cpu, memory...
Also i will say again then with only one gpu in the training everything is working great.
one day ago
0
Hi, I'Ve Run Into A Problem And Would Appreciate Some Help. I Installed Clearml Locally. When I Run A New Task On A Remote Server And In The Python Training Code I Set It To Only Train On One Gpu. Everything Works Fine And I See All The Scalars Automatica
Yes no issue with nvidia-smi, it recognize the 2 gpu's and successfuly use them for the training. The only problem is the metrics, scalars in the ui when i use 2 gpus
one day ago
0
Hi, I'Ve Run Into A Problem And Would Appreciate Some Help. I Installed Clearml Locally. When I Run A New Task On A Remote Server And In The Python Training Code I Set It To Only Train On One Gpu. Everything Works Fine And I See All The Scalars Automatica
Even when i dont add the --gpus flag, it doesnt work.
one day ago
0
Hi, I'Ve Run Into A Problem And Would Appreciate Some Help. I Installed Clearml Locally. When I Run A New Task On A Remote Server And In The Python Training Code I Set It To Only Train On One Gpu. Everything Works Fine And I See All The Scalars Automatica
I use in clearml-agent the --gpus 0,1 flag.
And also i dont use docker mode. I use virtual env mode.
one day ago