Ohh I see now the force SSH did not replace the user in the SSH link (only if the original was http), right ?

Hi EagerOtter28
I think the replacement should happen here:
https://github.com/allegroai/clearml-agent/blob/42606d9247afbbd510dc93eeee966ddf34bb0312/clearml_agent/helper/repo.py#L277

should work as well 🙂

You are awesome !!!

I do expect it to 

pip

 install though which doesn’t root access I think

Correct, it is installed on a venv (exactly for that).
It will not fail if the apt-get fails (only warnings)
Let me know if it worked

Kind of as it tries to do "apt-get install"...
what did you have in mind ?

LazyTurkey38 , ohh I think you are correct 😞
it should be:
# patch the Task and actually send it for execution if Task.running_locally(): # this will verify all auto repo detection and python is done. task.close() # so that we can edit the task task.reset() # update the repo task.update_task(task_data={'script': {'branch': 'new_branch', 'repository': 'new_repo'}}) # now to actually enqueue the Task Task.enqueue(task, queue_name='default')wdyt?

But these changes haven’t necessarily been merged into main. The correct behavior would be to use the forked repo.

So I would expect the agent to pull from your fork, is that correct? is that what you want to happen ?

LazyTurkey38
The last part makes sense, not sure I get the "if clone", we are calling execute_remotely, so I'm assuming we do not need to clone ourselves, but send the current Task.
Other than that yes, makes sense (BTW, assuming you have upgraded the server >=1.0 you can just do mark_stopped, no need to reset

is that what you want to happen ?

Hi LazyTurkey38
Configuring these folders will be pushed later today 🙂
Basically you'll have in your clearml.conf
` agent {
docker_internal_mounts {
sdk_cache: "/clearml_agent_cache"
apt_cache: "/var/cache/apt/archives"
ssh_folder: "/root/.ssh"
pip_cache: "/root/.cache/pip"
poetry_cache: "/root/.cache/pypoetry"
vcs_cache: "/root/.clearml/vcs-cache"
venv_build: "/root/.clearml/venvs-builds"
pip_download: "/root/.clearml/p...

I want to keep the above setup, the remote branch that will track my local will be on 

fork

 so it needs to pull from there. Currently it recognizes 

origin

 so it doesn’t work because the agent then can’t find the commit.

So you do not want to push the change set ?
You can basically add the entire change set (uncomitted changes) from the last pushed commit).
In your clearml.conf, set store_code_diff_from_remote: true
https://github.com/allegroai...

Sure LazyTurkey38 here's a nice hack for that:
` # code here
task.execute_remotely(queue_name=None, clone=False, exit_process=False)

patch the Task and actually send it for execution

if Task.running_locally():
task.update_task(task_data={'script': {'branch': 'new_branch', 'repository': 'new_repo'}})
# now to actually enqueue the Task
Task.enqueue(task, queue_name='default') You can also clear the git diff by passing "diff": "" `
wdyt?

yey 🙂 notice that when executed by the agent the call execute_remotely is skipped, and so does the If statement I added (since running_locally will return False when the process is executed by the agent)

Hi LazyTurkey38
What do you mean the git repo is not recognized? When execute_remotely leaves you should see on the task a ref to the git repo with the exact commit ID you have locally pulled, do you see it under the Execution tab?

I’m not sure if 

https

 will work because I want to use ssh keys for creds.

BTW: I was not aware github provide pypi like artifactory, do they ?
Regrading SSH keys, they are passed from the host machine (i.e. in venv mode it will use the SSH keys from the user running the agent, and n docker mode, they are automatically mapped into the container)

LazyTurkey38 configuration pushed to github :)

Unfortunately not yet in venv mode. What would you have put there?

Nice!

-e

:user/private_package.git@57f382f51d124299788544b3e7afa11c4cba2d1f#egg=private_package

Is this the correct link to the repo and a valid commit id ?

Can you post a few more lines from the agent's log ?
Something is failing to install I'm just not sure what

LazyTurkey38 I think this is caused by new versions of pip to report the wrong link:
https://github.com/bwoodsend/pip/commit/f533671b0ca9689855b7bdda67f44108387fe2a9

p.s. you should remove this line 🙂
extra_index_url: ["git@github.com:salimmj/xxxx"]

This means that in your "Installed packages" you should see the line:
Notice that this is not a pypi artifactory (i.e. a server to add to the extra index url for pip), this is a direct pip install from a git repository, hence it should be listed in the "installed packages".
If this is the way the package was installed locally, you should have had this line in the installed packages.
The clearml agent should take care of the authentication for you (specifically here, it should do nothing).
If ...

First let's verify with the manual change, but yes

Try to manually edit the "Installed Packages" (right click the Task, select "reset", now you can edit the section)
and change it to :
-e git+ssh@github.com:user/private_package.git@57f382f51d124299788544b3e7afa11c4cba2d1f#egg=private_package(assuming " pip install -e mailto:git+ssh@github.com :user/... " will work, should solve the issue )

I wonder if using our own containers which should have most the deps will work better than a simpler container.

Why not, it's transparent, just run in --docker mode and provide a default docker image if the Task doesn't specify one.

Can you copy the "Installed Packages" here, and point to the package causing the issue?

I see..
Generally speaking If that is the case, I would think it might be better to use the docker mode, it offers way more stable environment, regardless on the host machine runinng the agent. Notice there is no need to use custom containers, as the agent will basically run the venv process, only inside a container, allowing you to reuse offf the shelf containers.

If you were to add this, where would you put it? I can use a modified version of 

clearml-agent

Yep, that would b...

I want to inject a bash command after the repo has been clone (and maybe even after the venv has been installed).

LazyTurkey38 the created venv inherits from the system environment, so in theory you can do all the installation on the system python and the created venv will just inherit the packages, no?
(btw: just to clarify, there is only one entry point for the custom bash script and that is before everything, so users can configure the container before the agent starts)

Thanks StrongHorse8
Where do you think would be a good place to put a more advanced setup? Maybe we should add an entry for DevOps? Wdyt?