Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
LazyTurkey38
Moderator
17 Questions, 56 Answers
  Active since 10 January 2023
  Last activity 8 months ago

Reputation

0

Badges 1

54 × Eureka!
0 Votes
2 Answers
211 Views
0 Votes 2 Answers 211 Views
There’s a typo in this repo name: https://github.com/allegroai/nvidia-cleamrl-integration
2 years ago
0 Votes
1 Answers
190 Views
0 Votes 1 Answers 190 Views
Also, since this Slack workspace only allows searching up to 10,000 messages, useful information will be lost overtime. It’s been very useful for me personal...
2 years ago
0 Votes
3 Answers
197 Views
0 Votes 3 Answers 197 Views
Is it possible to interact with the job queues directly from the python SDK?
2 years ago
0 Votes
13 Answers
209 Views
0 Votes 13 Answers 209 Views
Another issue is the agent uses Python 2 for some reason even though locally I’m using python 3 and the agent is supposed to use a python 3 venv. $ /home/smj...
2 years ago
0 Votes
2 Answers
193 Views
0 Votes 2 Answers 193 Views
Do you have some documentation on what https://app.community.clear.ml/applications are useful for and how to use them? Are they going to be available for sel...
2 years ago
0 Votes
17 Answers
205 Views
0 Votes 17 Answers 205 Views
I’m trying to use task.execute_remotely but the repo I’m trying to use is not being identified correctly. I have it setup to git pull from a master repo and ...
2 years ago
0 Votes
7 Answers
199 Views
0 Votes 7 Answers 199 Views
Hey all! Great job on ClearML by the way 🙂 I’m currently exploring whether it could be a fit for me and my team and I have a few questions: If I want to sen...
2 years ago
0 Votes
1 Answers
227 Views
0 Votes 1 Answers 227 Views
Is there an example of how to use standalone mode? As in, if the packages, venv and git repo will be provided by the user, will they still get tracked in the...
2 years ago
0 Votes
5 Answers
257 Views
0 Votes 5 Answers 257 Views
Related to my above thread, I think there is some more customization that needs to be supported to use docker mode properly for different use-cases. It seems...
2 years ago
0 Votes
1 Answers
265 Views
0 Votes 1 Answers 265 Views
I’m running the clearml agent in standalone mode but it still installs the virtual env, is that the expected behavior? clearml-agent daemon --queue venv_queu...
2 years ago
0 Votes
4 Answers
198 Views
0 Votes 4 Answers 198 Views
I see that in the default setup, this command is part of the docker bash setup script: chown -R root /root/.cache/pip Does the agent assume it will be runnin...
2 years ago
0 Votes
1 Answers
238 Views
0 Votes 1 Answers 238 Views
If we’re using the same git repo over and over for almost all jobs, is it possible to have the agents keep a local version and only download the diff of the ...
2 years ago
0 Votes
9 Answers
196 Views
0 Votes 9 Answers 196 Views
There is some specificity with the way we setup our environment at my company that prevents me from using the full features of clearml-agent and forces me to...
2 years ago
0 Votes
17 Answers
239 Views
0 Votes 17 Answers 239 Views
I’m getting these errors when using agent in docker mode -bash: /etc/apt/apt.conf.d/docker-clean: Permission denied chown: cannot access '/root/.cache/pip': ...
2 years ago
0 Votes
3 Answers
208 Views
0 Votes 3 Answers 208 Views
When scheduling a job for remote execution, is it possible to decide whether to use the repo’s formal requirements sheet vs the currently installed venv libr...
2 years ago
0 Votes
1 Answers
249 Views
0 Votes 1 Answers 249 Views
Can someone please explain the difference between clearml-agent-services (that comes with the official docker compose) and clearml-agent deamon?
2 years ago
0 Votes
30 Answers
216 Views
0 Votes 30 Answers 216 Views
Is it possible to give the agent access to install private pip packages (needs to be installed from the repo)? ERROR: Could not find a version that satisfies...
2 years ago
0 I’M Getting These Errors When Using Agent In Docker Mode

btw, AgitatedDove14 I launch the agent daemon outside docker (with --docker ) , that’s the way it is supposed to work right?

$ clearml-agent daemon --detached --queue manual_jobs automated_jobs --docker --gpus 0
And then the worker itself will run the docker run command for me and start another non-daemon agent inside.
I guess the failure happens when it tries to switch to docker because the same experiment works with agents not started with --docker flag

2 years ago
0 I’M Getting These Errors When Using Agent In Docker Mode

AgitatedDove14 no I mean I can do:
` docker run -t --gpus "device=1" -dit -e APP_ENV=kprod -e CLEARML_WORKER_ID=ada:gpu1 -e CLEARML_DOCKER_IMAGE=922531023312.dkr.ecr.us-west-2.amazonaws.com/jym-coach:202108080511.7e8d6d1 -v /home/smjahad/.gitconfig:/root/.gitconfig -v /tmp/.clearml_agent.kjx6r9oo.cfg:/root/clearml.conf -v /tmp/clearml_agent.ssh.l8cguj81:/root/.ssh -v /home/smjahad/.clearml/apt-cache.1:/var/cache/apt/archives -v /home/smjahad/.clearml/pip-cache:/root/.cache/pip -v /home/smjah...

2 years ago
0 I’M Getting These Errors When Using Agent In Docker Mode

I tried with and without. I’m having the issue where if I run the task from the queue it will complete as soon as it goes into docker but if I run the same docker run it works.

2 years ago
0 Hi

I’m wondering, would an older version of the agent work well with a newer server version and vice-versa?

2 years ago
0 I’M Getting These Errors When Using Agent In Docker Mode

AgitatedDove14 should I try running the above command with privileged user?

2 years ago
0 Do You Have Some Documentation On What

I think it’s great to let users build their own UI-connected apps, I’d use that for sure!

2 years ago
0 I’M Trying To Use

$ git remote -v fork git@github.com:salimmj/somerepo.git (fetch) fork git@github.com:salimmj/somerepo.git (push) origin git@github.com:mainuser/somerepo.git (fetch) origin git@github.com:mainuser/somerepo.git (push)I want to keep the above setup, the remote branch that will track my local will be on fork so it needs to pull from there. Currently it recognizes origin so it doesn’t work because the agent then can’t find the commit.

2 years ago
0 I’M Trying To Use

I know this is not the default behavior so I’d be happy with having the option to override the repo when I call execute_remotely

2 years ago
0 I’M Trying To Use

AgitatedDove14 wouldn’t the above command task.execute_remotely(queue_name=None, clone=False, exit_process=False) fail because
clone==False and exit_process==False is not supported. Task enqueuing itself must exit the process afterwards.I thought it worked earlier 😮

2 years ago
0 I’M Trying To Use

It recognizes the main repo, but I want it to push and pull from another one (my own forked repo). AgitatedDove14

2 years ago
0 I’M Trying To Use

I already have that set to true and want that behavior. The issue is on the “committed” change set. When I push code to github I push to my fork and pull from the main/master repo (all changes go through PRs from fork to main).

Now when I use execute_remotely , whatever code does the git discovery, considers whatever repo I pull from the repo to use. But these changes haven’t necessarily been merged into main. The correct behavior would be to use the forked repo.

2 years ago
0 I’M Trying To Use

AgitatedDove14 when I try this I get
clearml.backend_interface.session.SendError: Action failed <400/110: tasks.enqueue/v1.0 (Invalid task status (Invalid status change): current_status=in_progress, new_status=queued)> (queue=e78d2fdf2d5140b6b5c6678338c532bb, task=95082c9174a04044b25253d724362ec1)

2 years ago
0 I’M Trying To Use

This is exactly what I was looking for. I thought once you call execute_remotely the task is sent and it’s too late to change anything.

2 years ago
0 I’M Trying To Use

Fixed it by adding this code block. Makes sense.
if clone: task = Task.clone(self) else: task = self # check if the server supports enqueueing aborted/stopped Tasks if Session.check_min_api_server_version('2.13'): self.mark_stopped(force=True) else: self.reset()

2 years ago
0 There Is Some Specificity With The Way We Setup Our Environment At My Company That Prevents Me From Using The Full Features Of

For your second question, those are generated using custom tooling, it relies on the build system to be setup which is guaranteed by the docker image used. So I don’t think this is a case of supporting a specific env setup or build tool but just allowing custom script for env setup step / building code.
WDYT?

2 years ago
0 There Is Some Specificity With The Way We Setup Our Environment At My Company That Prevents Me From Using The Full Features Of

That won’t work 😕
The docker shell script runs too early in the process.
I want to inject a bash command after the repo has been clone (and maybe even after the venv has been installed).

2 years ago
0 There Is Some Specificity With The Way We Setup Our Environment At My Company That Prevents Me From Using The Full Features Of

TimelyPenguin76 After creating the venv (so I don’t have to do it myself). Once an env is there, I need to run a script while the env is activated from the root of the repo.

2 years ago
0 There Is Some Specificity With The Way We Setup Our Environment At My Company That Prevents Me From Using The Full Features Of

So when the repo is cloned and venv is created and activated I want to executed this from the repo: tools/setup_dependencies.sh

2 years ago
0 I’M Getting These Errors When Using Agent In Docker Mode

SuccessfulKoala55 I tried to make a docker image by combining one of our dockerfiles with this https://github.com/allegroai/clearml-agent/blob/master/docker/agent/Dockerfile . I modified the entrypoint to also be a combination of both.

Right now I’m not seeing that error, but the the process seems to exit (as completed) after the docker run . I’m wondering if my Dockerfile is not properly setup and it’s exiting before the deamon is started.

2 years ago
2 years ago
0 I’M Getting These Errors When Using Agent In Docker Mode

ugh, sudo actually makes it fail explicitly because
` error: Could not fetch origin
Repository cloning failed: Command '['git', 'fetch', '--all', '--recurse-submodules']' returned non-zero exit status 1.

  1. Make sure you pushed the requested commit:
    (repository='git@github.com:salimmj/clearml-demo.git', branch='main', commit_id='f76f3affd28d5558928d7ffd9a6797890ffdd708', tag='', docker_cmd='nvidia/cuda:11.4.0-runtime-ubuntu20.04', entry_point='mnist.py', working_dir='.')
  2. Check if remote-wo...
2 years ago
0 I’M Getting These Errors When Using Agent In Docker Mode

It’s not that I think because it works if I run the same command manually.

2 years ago
0 Another Issue Is The Agent Uses Python 2 For Some Reason Even Though Locally I’M Using Python 3 And The Agent Is Supposed To Use A Python 3 Venv.

$ python --version Python 3.6.8 $ python repo/toy_workflow.py --logtostderr --logtoclearml --clearml_queue=ada_manual_jobs 2021-08-07 04:04:16,844 - clearml - WARNING - Switching to remote execution, output log page https://...
On the webpage logs I see this:
2021-08-07 04:04:12 ClearML Task: created new task id=f1092bcbe30249639122a49a9b3f9145 ClearML results page: `
2021-08-07 04:04:14
ClearML Monitor: GPU monitoring failed getting GPU reading, switching off GPU monitoring
2021-08...

2 years ago
0 Hey All! Great Job On Clearml By The Way

In fact, if there is a good python API to list/duplicate/edit/run experiments by ID, it seems straightforward to do that from Airflow (or any other job scheduler). I’m just wondering if there is some built-in scheduler.

2 years ago
0 Hey All! Great Job On Clearml By The Way

I don’t mean a serving endpoint, just the equivalent of “cloning an experiment” and running it on a different (larger) dataset.

2 years ago
0 Another Issue Is The Agent Uses Python 2 For Some Reason Even Though Locally I’M Using Python 3 And The Agent Is Supposed To Use A Python 3 Venv.

Issue seems fixed now, thanks! Is the fact that clearml-agent needs to be installed from system python mentioned anywhere in the docs, if not I suggest it gets added.

Thank you so much for helping.

2 years ago
Show more results compactanswers