CLEARML_DOCKER_IMAGE=nvidia/cuda:10.1-runtime-ubuntu18.04
How do I pull the image using the agent?

right, of course 🙂 so just to make sure I'm running it correctly. I ran python aws_autoscaler.py --run on my laptop and I see the Task on ClearML. Then took a completed task, cloned it and enqueued to the queue defined on the autoscaler. That should spin up an instance, right? (it currently doesn't, and I'm not sure where to debug)

nope, the experiment is stuck in RUNNING state

yeah, totally. Are there any services OOB like this?

Sure, redacted most of the params as they are sensitive:
` run_experiment {
base_task_id = "478cfdae5ed249c18818f1c50864b83c"
queue = null
parents = []
timeout = null
parameters {
# Redacted the parameters
}
executed = "d1d361d1059c4f0981200f59d7683773"
}
segment_slides {
base_task_id = "ae13cc979855482683474e9d435895bb"
queue = null
parents = ["run_experiment"]
timeout = null
parameters {
Args/param = """
[
#...

Also, tried the continue_pipeline option, didn't work as it couldn't parse the previous step that run...
ValueError: Could not parse reference '${run_experiment.models.output.-1.url}', step run_experiment could not be found

yup, it's there in draft mode so I can get the latest git commit when it's used as a base task

Great, good to know!

` Exception in thread Thread-5:
Traceback (most recent call last):
File "/opt/pyenv/versions/3.6.8/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/opt/pyenv/versions/3.6.8/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/root/.clearml/venvs-builds/3.6/lib/python3.6/site-packages/clearml/automation/controller.py", line 615, in _daemon
if self._launch_node(self._nodes[name]):
File "/root/.clearml/venvs-builds/3.6/lib/pyt...

opened it, works now! Thanks.

AgitatedDove14 is there any update on the open issue you talked about before? I think it's this one: https://github.com/allegroai/clearml/issues/214

python -m http://script.as .a.module first_arg second_arg --named_arg value <- something like that

And for some reason this clone is marked as completed. Not sure why, as it failed

Right... apparently the nvidia-docker wasn't set up. Thanks!

what about using ENV variables? is it possible to override the config file's credentials?

I was thinking about sending the parameters programatically. We have different pipelines that can generate tasks, I would like to be able to tell trains the user who started the pipeline.

legit, I was thinking only about task tracking, less about user based credentials. good point

so no magic "username" key? 😛

I looked there, but couldn't find it. I'm currently experimenting with your free hosted server

Sounds promising, any ETA for the next version?

yeah, maybe as an option in the Task.init

ok, hopefully last question on this subject 🙂
I want to use Jenkins for some pipelines. What I would like to do is have one set of credentials saved on Jenkins. Then whenever a user triggers a pipeline - this is the user that will be marked as the task's user.
If I understand the options you suggested, I'll currently need either to (1) have some mapping between users and their credentials and have all the credentials saved on Jenkins; or, (2) have each user manually add 2 environment varia...

ok cool

I just want to use auth0 (which we already use in the company) in order to manage the users...

I did not, I see that there's a field for extra_trains_conf , but couldn't find clear documentation on how to use it. Is it just a reference to a trains_conf (maybe clearml_conf ?)?

No, I use an SSH connection which worked with the regular clearml-agent , we prefer to work with SSH instead of creating a git user.

nope, only port 22 is open for SSH. Is there anyway to set that as the port for clearml-session?

cool! just to verify - I'll still need to have the credentials created in the server, right?

Update: got the same error while trying to clone a public repo: git@gitlab.com:gitlab-org/gitlab-foss.git

it's in the docker image, doesn't the git clone command run in the container?