Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, We Are Having An Interesting Issue Here. We Serve Many Users And Each User Has Their Own Credentials In Accessing The Private Git Repo. We Can'T Seem To Find A Way For The End User To Pass In Their Git Credentials When They Run Their Codes In Both Age

Hi, we are having an interesting issue here. We serve many users and each user has their own credentials in accessing the private git repo. We can't seem to find a way for the end user to pass in their git credentials when they run their codes in both agent and non-agent scenarios. Any advice here?

  
  
Posted 2 years ago
Votes Newest

Answers 20


alright. 🙂

  
  
Posted 2 years ago

Hi AgitatedDove14 . I'm trying out passing env via the code instead.
task.set_base_docker("nvcr.io/nvidia/tensorflow:19.11-tf2-py3 --env TRAINS_AGENT_GIT_USER=git_username_here --env TRAINS_AGENT_GIT_PASS=git_password_here")So the strange thing is when my k8sglue pulls a task, this happens.
Pulling task xxxxxxxxxx launching on kubernetes cluster Pushing task xxxxxxxxxx into temporary pending queue Kubernetes scheduling task id=xxxxxxxxxxxx skipping docker argument TRAINS_AGENT_GIT_USER=git_username_here (only -e --env supported) TRAINS_AGENT_GIT_PASS=git_username_here (only -e --env supported) FATAL ERROR: blah blah.. clear_agent/glue/k8s.py env=docker_args.pop().strip() IndexError: pop from empty listAfter looking through, i noted that the pop method pops from the last item (instead of first item), so by the time it reaches the --env, there's nothing else to pop hence the error.

I fixed that and got another error pertaining to a key in this line [ https://github.com/allegroai/clearml-agent/blob/cb6bdece39751eaef975287609b8bab603f116e5/clearml_agent/glue/k8s.py#L401 ]

If i set the key to 'env', i get some kubectl errors instead.

By the way, this is linked to [ https://clearml.slack.com/archives/CTK20V944/p1616398936061300 ]

  
  
Posted 2 years ago

Hi, just wondering if this 'feature: Passing env via the code' is in the works?
https://clearml.slack.com/archives/CTK20V944/p1616677400127900?thread_ts=1616585832.098200&cid=CTK20V944

  
  
Posted 2 years ago

No i didn't indicate this particular issue on the git issue. Only the apply template.yml is on the issue.

  
  
Posted 2 years ago

unfortunately, our security posture is so strict that we cannot have an agent git user that have unfettered read access to all repos.

  
  
Posted 2 years ago

Thanks SubstantialElk6 !
I believe an initial a fix was pushed 😉 A full one (merging Task --env with k8s template) will be added soon

  
  
Posted 2 years ago

Could you please add it, I really do not want to miss it 🙂

  
  
Posted 2 years ago

Hi SubstantialElk6

We can't seem to find a way for the end user to pass in their git credentials when they run their codes in both agent and non-agent scenarios. Any advice here?

The bottom line is the agent needs to have read-only access to all the repositories so it can launch any Task. I would recommend to create an agent git user with read-only credentials and configure the agent to use it. wdyt?

  
  
Posted 2 years ago

Hmm that sounds like the agent needs to access a vault with credentials per user, unfortunately this is not covered in the open-source 😞 I "think" this is supported in the enterprise version as part of the permission management

  
  
Posted 2 years ago

Hi AgitatedDove14 , i was refering to
task.set_base_docker("nvcr.io/nvidia/tensorflow:19.11-tf2-py3 --env TRAINS_AGENT_GIT_USER=git_username_here --env TRAINS_AGENT_GIT_PASS=git_password_here")The above will give error
skipping docker argument TRAINS_AGENT_GIT_USER=git_username_here (only -e --env supported) TRAINS_AGENT_GIT_PASS=git_username_here (only -e --env supported)

  
  
Posted 2 years ago

These are both specific cases of the glue, and yes both need to be fixed.
(1) I think is actually a feature, nonetheless we should support it.
FriendlySquid61 could you verify specifically on (2)

  
  
Posted 2 years ago

The apply.yaml template is not working (E.g. the arguments env is not passed to the container), this is why i tried the code approaach instead.

  
  
Posted 2 years ago

Ohh, I see now, yes that should be fixed as well 🙂

  
  
Posted 2 years ago

I can't think of any hack that will satisfy your IT other than than an actual vault...
wdyt?

  
  
Posted 2 years ago

Thanks!

  
  
Posted 2 years ago

Hi SubstantialElk6
I think you are absolutely correct, it seems the glue pops all the arguments, when in fact it should maybe process them a,d convert the --env/-e
What do you think?
Aloso I assume if these are the default arguments they should actually be part of the k8s apply.yaml template no ?

  
  
Posted 2 years ago

SubstantialElk6 (2) yes definitely will be fixed
Regrading (1), what do you mean by "via the code" ? Do you mean like as a Task docker cmd ?

  
  
Posted 2 years ago

Hi FriendlySquid61 , AgitatedDove14 , the issue and possible fix is in this issue raise. https://github.com/allegroai/clearml-agent/issues/51

  
  
Posted 2 years ago

AgitatedDove14 , will these be fixed?
Passing env via the code Passing env via template yaml

  
  
Posted 2 years ago

Do we have it on the git issue ?

  
  
Posted 2 years ago
521 Views
20 Answers
2 years ago
one year ago
Tags
Similar posts