Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All, I'M Having An Issue Running Clearml-Agent In Docker Mode. When Enqueuing A Task To The Docker Agent I Get The Following Error:

Hi all,
I'm having an issue running clearml-agent in docker mode. When enqueuing a task to the docker agent I get the following error:
error: could not write config file /root/.gitconfig: Device or resource busy

which if followed a git credentials error:
Using cached repository in "/root/.clearml/vcs-cache/not_sharing.git/my_secrets_here.git"
The fingerprint for the RSA key sent by the remote host is
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
error: Could not fetch origin
Repository cloning failed: Command '['git', 'fetch', '--all', '--recurse-submodules']' returned non-zero exit status 1.
clearml_agent: ERROR: Failed cloning repository.
1) Make sure you pushed the requested commit:
(repository='git@bitbucket.org:foo/bar.git', branch='master', commit_id='abcdefg', tag='', docker_cmd='foo/bar:latest -v /foo:/bar', entry_point='validation/drive_matcher_ros.py', working_dir='.')
2) Check if remote-worker has valid credentials [see worker configuration file]

I have SSH credentials defined and non-docker workers have no issues with git. Any idea how to solve this?

I did notice that clearml mounts the current user's .gitconfig to the docker's /root/.gitconfig which is what causes the write error. My question is, why does the docker even try to write to .gitconfig if it already exists?

  
  
Posted 5 months ago
Votes Newest

Answers 9


yes.

I checked the base docker image I use and noticed that a $HOME/.gitconfig file already exists. would that be an issue given that the .gitconfig of the current user is mounted to that path once the clearml task is run?

  
  
Posted 5 months ago

Hi @<1523701235335565312:profile|HugeArcticwolf77> , is it possible you do not have root privileges inside the docker container?

  
  
Posted 5 months ago

@<1523701087100473344:profile|SuccessfulKoala55> the docker user is root, but it does not have the "sudo" group if that's what you mean. Is that required?

EDIT: I just ran "sudo ls" and it returned with no issues so I guess I do have sudo permission :man-shrugging:

  
  
Posted 5 months ago

And is the $HOME variable set correctly?

  
  
Posted 5 months ago

I could not reproduce it... I think the issue might be that this folder is mounted to some host folder which causes a problem - can you check the docker command and see where it's mounted to?

  
  
Posted 4 months ago

IS it /home ? I think it's only a subdir of home

  
  
Posted 4 months ago

these are the mounts I add:
-v /home/some_username/workspace/:/root/workspace -v /software:/software -v /images:/images -v /data:/data -v /processedData:/processedData -v /disk1:/disk1 -v /disk2:/disk2 -v /disk3:/disk3 -v /disk4:/disk4 -v /disk5:/disk5 -v /disk6:/disk6 -v /disk8:/disk8

None of these seem problematic to me. The only issue I can think of is that /home is an external mount on the host machine (outside of docker). Should I mount it somewhere?

  
  
Posted 4 months ago

@<1523701087100473344:profile|SuccessfulKoala55> any update on how to solve this?

  
  
Posted 4 months ago

it is /home This is how we share user dirs across multiple servers. I think this might be a permissions issue though. will try changing the permissions on .gitconfig and let you know if that helps

  
  
Posted 4 months ago
335 Views
9 Answers
5 months ago
4 months ago
Tags
Similar posts