Good morning folks, I am setting up ClearML on a (self-hosted) K8s cluster using the https://github.com/allegroai/clearml-helm-charts/blob/main/charts/clearm...
3 years ago
Good day to all. I am moving from one MLOps framework to clearml and have a question on git integration. In my prev framework each ML project had a specified...
3 years ago
Hi everyone, I’ve been using ClearML for a while now and I wanted to add the option to execute my code remotely as a command line argument. I have the ClearM...
3 years ago
Hello, I am using the autoscaler to start jobs. Previously, everything was working. However, now I get this error: Using cached repository in "/home/ubuntu/....
3 years ago
Hi! We're trying to orchestrate an experiment through the ClearML ui with a remote agent on k8s. Our training job requires a package only available on our in...
3 years ago
Hi, when I use the autoscaler to start jobs, I noticed some of them randomly abort in the middle of the jobs and give the following error: Process failed, ex...
3 years ago
I am trying to run a task on an agent for the first time but I am running into some things I do not understand, I hope someone can help me out with this. I g...
3 years ago
Hi everyone, is there any difference between the service agent that comes with the server and the normal clearml-agent in --services-mode ?
3 years ago
Hi, I'm using the autoscaler and getting the error Process terminated by user even though I did not terminate anything. This error occurs randomly during tra...
3 years ago
Hi everyone! I've noticed that if I run an experiment and it fails, the ClearML agent will delete all datasets that have been downloaded during the run. Is i...
3 years ago
How can i modify the line executed by the agent at the beginning Executing: ['docker', 'run', '-t', '--gpus', 'all', '-l', 'clearml-worker-id=islin-hdplnod05...
3 years ago
Hi guys, newbe question: my clearml.conf do not have the agent text included, but a had installed it. Now I'm trying to use a pipeline in one of my projects,...
3 years ago
Hi, I'm looking for documentation on GCP autoscalers. When I search on the docs site, it shows me the AWS autoscaler but not the GCP one. Can someone point m...
3 years ago
Hi, I would like to follow-up in this https://clearml.slack.com/archives/CTK20V944/p1646123127790389 happening on clearml server 1.2.0 (self hosted on a sing...
Hi everybody. When I want to force the agent to not reproduce my local pip environment, I add Task.force_requirements_env_freeze(force=False, requirements_fi...
3 years ago
Hi! My agent has very limited disk space. And when a new task is launched and the loading of new data begins, there may not be enough space. Is there a syste...
3 years ago
Hi, trying to spin up a clearml agent and gettting this error: ERROR: Could not find a version that satisfies the requirement pywin32==303 (from -r /tmp/cach...
3 years ago
Hi all, I have a problem with setting up an agent. I set up local server on ubuntu behind proxy, it seems to work correctly. I can access web UI and set up a...
3 years ago
I assume I can ask a question here. The ClearML Orchestrator looks interesting. But the website suggests that K8S is required. We have a Linux training box (...
3 years ago
Hi everybody, I'm running experiments inside a docker which includes multiple python instances, some of them are inside conda environments. How can I specify...
3 years ago
Hi, when trying to use a remote agent to train a model, the initial environment setup on the remote machine fails because the list of requirements located in...
Hey, is there a way to limit the number of tasks run at the same time by an agent in service mode?
3 years ago
Hi, I'm conceptually struggling to understand how to get my models to be saved permanently by my self-hosted ClearML server. As it stands now, when I train a...
3 years ago
After resetting and enqueueing a task, the pip installation procedure in a remote agent tries to install packages different from packages that were installed...
3 years ago
Hi, I'd very much like a feature to support clearml agent stopping specified by the exact bringup command: when trying for example to bring down a gpu agent ...
3 years ago
Hi, when running a clearml agent with services mode, is there a way to limit the number of concurrent services crunning?
3 years ago
Hey, is there a way to run tasks not on ClearML agent but rather as a k8s job directly? Thanks!
3 years ago
Hi, I have a question regarding the aws_autoscaler: It usually takes ~hours to get a GPU instance nowadays. I was thinking, it would be much more interesting...
3 years ago
Hi guys! Is there a way to tell an agent to run a task in an existing venv (without creating a new one)?
3 years ago