Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All, I'M Having An Issue Running Clearml-Agent In Docker Mode. When Enqueuing A Task To The Docker Agent I Get The Following Error:

Hi all,
I'm having an issue running clearml-agent in docker mode. When enqueuing a task to the docker agent I get the following error:
error: could not write config file /root/.gitconfig: Device or resource busy

which if followed a git credentials error:
Using cached repository in "/root/.clearml/vcs-cache/not_sharing.git/my_secrets_here.git"
The fingerprint for the RSA key sent by the remote host is
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
error: Could not fetch origin
Repository cloning failed: Command '['git', 'fetch', '--all', '--recurse-submodules']' returned non-zero exit status 1.
clearml_agent: ERROR: Failed cloning repository.
1) Make sure you pushed the requested commit:
(repository='git@bitbucket.org:foo/bar.git', branch='master', commit_id='abcdefg', tag='', docker_cmd='foo/bar:latest -v /foo:/bar', entry_point='validation/drive_matcher_ros.py', working_dir='.')
2) Check if remote-worker has valid credentials [see worker configuration file]

I have SSH credentials defined and non-docker workers have no issues with git. Any idea how to solve this?

I did notice that clearml mounts the current user's .gitconfig to the docker's /root/.gitconfig which is what causes the write error. My question is, why does the docker even try to write to .gitconfig if it already exists?

  
  
Posted one year ago
Votes Newest

Answers 9


@<1523701087100473344:profile|SuccessfulKoala55> the docker user is root, but it does not have the "sudo" group if that's what you mean. Is that required?

EDIT: I just ran "sudo ls" and it returned with no issues so I guess I do have sudo permission :man-shrugging:

  
  
Posted one year ago

Hi @<1523701235335565312:profile|HugeArcticwolf77> , is it possible you do not have root privileges inside the docker container?

  
  
Posted one year ago

And is the $HOME variable set correctly?

  
  
Posted one year ago

these are the mounts I add:
-v /home/some_username/workspace/:/root/workspace -v /software:/software -v /images:/images -v /data:/data -v /processedData:/processedData -v /disk1:/disk1 -v /disk2:/disk2 -v /disk3:/disk3 -v /disk4:/disk4 -v /disk5:/disk5 -v /disk6:/disk6 -v /disk8:/disk8

None of these seem problematic to me. The only issue I can think of is that /home is an external mount on the host machine (outside of docker). Should I mount it somewhere?

  
  
Posted one year ago

it is /home This is how we share user dirs across multiple servers. I think this might be a permissions issue though. will try changing the permissions on .gitconfig and let you know if that helps

  
  
Posted one year ago

@<1523701087100473344:profile|SuccessfulKoala55> any update on how to solve this?

  
  
Posted one year ago

I could not reproduce it... I think the issue might be that this folder is mounted to some host folder which causes a problem - can you check the docker command and see where it's mounted to?

  
  
Posted one year ago

IS it /home ? I think it's only a subdir of home

  
  
Posted one year ago

yes.

I checked the base docker image I use and noticed that a $HOME/.gitconfig file already exists. would that be an issue given that the .gitconfig of the current user is mounted to that path once the clearml task is run?

  
  
Posted one year ago
1K Views
9 Answers
one year ago
one year ago
Tags
Similar posts