See e None @<1523701087100473344:profile|SuccessfulKoala55>

Right, so where can one find documentation about it?
The repo just has the variables with not much explanations.

… And it’s failing on typing hints for functions passed in pipe.add_function_step(…, helper_function=[…]) … I guess those aren’t being removed like the wrapped function step?

If that's the case, wouldn't it apply across the board? This happens in a single task within ray - the other tasks (I have many in a single run) are fine

Odd; switching to virtual environment results in
fatal: could not read Username for ' ': terminal prompts disabledeven though it does earlier show that:
agent.git_user = xxx

Nope, no .netrc defined anywhere, really (+I've abandoned the use of docker for the autoscaler as it complicates things, at least for now)

I just set the git credentials in the clearml.conf and it works out of the box

And actually it fails on quite many tasks for us with this Python 3.6.
I tried to set up a different image ( agent8sglue.defaultContainerImage: "ubuntu:20.04" ) but that did not change much.
I suspect the culprit is agentk8sglue.image , which is set to tag 1.24-21 of clearml-agent-k8s-base . That image is quite very old… Any updates on that? 🤔

The last kind when launching the agents.
The former on task execution.

I think the environment variables path might work for you then?
You'd set your config with
use_credentials_chain: ${CREDENTIALS_CHAIN} Then in Python you could os.environ['CREDENTIALS_CHAIN'] = True/False before you make any calls to ClearML?

Well you can install the binary in the additional start up commands.
Matter of fact, you can just include the ECR login in the "startup steps" offered by the scaler, so no need for this repository. I was thinking these are local instances.

Follow up on this btw, from the WebUI/Server POV, I see there's an "Admin" role, etc. Do those have additional views available, such as users etc?

Following up on that (I don't think the K8s helm chart for 1.7.0 is out yet SlimyDove85 , is it?) - but what's the recommended way to backup the mongodb before upgrading on K8s?

The api.files_server is set to the MinIO endpoint s3://ip:9000/clearml (both locally and remotely) The sdk.development.default_output_uri is set to the MinIO endpoint (both locally and remotely) When we call Task.init I do not set the output_uri at all I get the logger directly with task.get_logger()

It does, but I don't want to guess the json structure (what if ClearML changes it or the folder structure it uses for offline execution?). If I do this, I'm planning a test that's reliant on ClearML implementation of offline mode, which is tangent to the unit test

Hm, that seems less than ideal. I was hoping I could pass some CSV locations. I'll try and find a workaround for that. Thanks!

No, fixed 🙂

I know the ClearML enterprise offers a vault.
If these are static-ish, you can set them directly in the agent's config file.
If not, what we did was that before executing remotely, we uploaded environment variables of interest as parameters, and then loaded them in the remote task.
These can then be overwritten with *** after loading them.

What do you mean 😄 Using logging.config.dictConfig(...)

Hey FrothyDog40 ! Thanks for clarifying - guess we'll have to wait for that as a feature 😁

Should I create a new issue or just add to this one? https://github.com/allegroai/clearml/issues/529

~20

But it is strictly that if condition in Task.init, see the issue I opened about it

What's new in 1.1.6rc0?

1.1.4

The SDK is fine as it is - I'm more looking at the WebUI at this point

JitteryCoyote63 please do not get used to it :D there's an open ticket/feature request to either revert this or let the user/server choose the most comfortable way

AgitatedDove14 Unfortunately not, the queues tab shows only the number of tasks, but not resources used in the queue . I can toggle between the different workers but then I don't get the full image.

We just do task.close() and then start a new task.Init() manually, so our "pipelines" are self-controlled

It's a small snippet that ensures identically named projects are still unique'd with a running number.

I'm using some old agent I fear, since our infra person decided to use chart 3.3.0 😕
I'll try with the env var too. Do you personally recommend docker over the simple AMI + virtual environment?

More complete log does not add much information -
Cloning into '/root/.clearml/venvs-builds/3.10/task_repository/xxx/xxx'... fatal: could not read Username for ' ': terminal prompts disabled fatal: clone of ' ` ' into submodule path '/root/.clearml/venvs-builds/3.10/task_repository/...