The logs are on the bucket, yes.
The default file server is also set to s3://ip:9000/clearml

Sure! It looks like this

… And it’s failing on typing hints for functions passed in pipe.add_function_step(…, helper_function=[…]) … I guess those aren’t being removed like the wrapped function step?

CostlyOstrich36 That looks promising, but I don't see any documentation on the returned schema (i.e. workers.worker_stats is not specified anywhere?)

Right so it uses whatever version is available on the agent.
Yeah it would be nice to have either a poetry_version (a-la https://github.com/allegroai/clearml-agent/blob/5afb604e3d53d3f09dd6de81fe0a494dacb2e94d/docs/clearml.conf#L62 ), rename the latter to manager_version , or just install from the captured environment, etc? 🤔

Could also be related to K8, so pinging JuicyFox94 just in case 😉

Not sure if ClearML has any built in support, but we used the above for a similar issue but with Prefect2 :)

So a missing bit of information that I see I forgot to mention, is that we named our packages as foo-mod in pyproject.toml . That hyphen then get’s rewritten as foo_mod.x.y.z-distinfo .

foo-mod @ git+

Either one would be nice to have. I kinda like the instant search option, but could live with an ENTER to search.

I opened this meanwhile - https://github.com/allegroai/clearml-server/issues/138

Generally, it would also be good if the pop-up presented some hints about what went wrong with fetching the experiments. Here, I know the pattern is incomplete and invalid. A less advanced user might not understand what's up.

I realized it might work too, but looking for a more definitive answer 😄 Has no-one attempted this? 🤔

We're using the example autoscaler, nothing modified

I'm saying it's a bug

It is. In what format should I specify it? Would this enforce that package on various components? Would it then no longer capture import statements?

There's a specific fig[1].set_title(title) call.

Those are for specific packages, I'm wondering about the package managers as a whole

Here's an example where poetry.lock is removed, and still the console reads:
url: .... branch: HEAD commit: 22fffaf8d5f377b7f10140e642a7f6f26b72ffaa root: /.../.clearml/venvs-builds/3.10/task_repository/... Applying uncommitted changes Poetry Enabled: Ignoring requested python packages, using repository poetry lock file! Creating virtualenv ds-platform in /.../.clearml/venvs-builds/3.10/task_repository/.../.venv Updating dependencies Resolving dependencies...

I see that the GUI AutoScaler is only in the paid version, wonder why the GCP driver is not open source?

Can I query where the worker is running (IP)?

Should this be under the clearml or clearml-agent repo?

I'll try that in a bit (that requires some access control changes). Any idea how can I modify the dynamically created virtualenv?

` Poetry Enabled: Ignoring requested python packages, using repository poetry lock file!
The currently activated Python version 3.10.6 is not supported by the project (~3.8.0).
Trying to find and use a compatible version.
Using python3.8 (3.8.16)
Creating virtualenv ... in /root/.clearml/venvs-builds/3.10/task_repository/...git/.venv
Installing dependencies from ...

No it does not show up. The instance spins up and then does nothing.

@<1523701070390366208:profile|CostlyOstrich36> I added None btw

I opened a GH issue shortly after posting here. @<1523701312477204480:profile|FrothyDog40> replied (hoping I tagged the right person).

We need to close the task. This is part of our unittests for a framework built on top of ClearML, so every test creates and closes a task.

Thanks CostlyOstrich36 !
And can I make sure the same budget applies to two different queues?
So that for example, an autoscaler would have a resource budget of 6 instances, and it would listen to aws and default as needed?

So now we need to pass Task.init(deferred_init=0) because the default Task.init(deferred_init=False) is wrong

IIRC, get_local_copy() downloads a local copy and returns the path to the downloaded file. So you might be interested in e.g.
local_csv = pd.read_csv(a_task.artifacts['train_data'].get_local_copy())
With the models, you're looking for get_weights() . It acts the same as get_local_copy() , so it returns a path.

EDIT: I think also get_local_copy() for a model should work 👍

I mean, I see these are defined here https://github.com/allegroai/clearml-agent/blob/master/clearml_agent/definitions.py

But I do not see where an EnvironmentConfig.set() is called...

Yeps

I mean, it makes sense to have it in a time-series plot when one is logging iterations and such. But that's not always the case... Anyway I opened an issue about that too! 🙂

Yeah that works too. So one can override the queue ID but not the worker 🤔