After the task was initialized? 🤔

I mean, it makes sense to have it in a time-series plot when one is logging iterations and such. But that's not always the case... Anyway I opened an issue about that too! 🙂

Does that make sense CostlyOstrich36 ? Any thoughts on how to treat this? For the time being I'm also perfectly happy to include something specific to extra_clearml_conf , but I'm not sure how to set the sdk.aws.s3.credentials to be a list of dictionaries as needed

SuccessfulKoala55 That at least did not work, unless one has to specify wildcard patterns perhaps..?

Using an on-perm clearml server, latest published version

I mean, if I search for "model", will it automatically search for tasks containing "model" in their name?

Then that did not work, but I'll look into it again soon!

Much much appreciated 🙏

I created a new task with the project name internal tests , and no task name (so it's derived by ClearML).
The task was a simple print out.
The project does not appear in the project space and does not turn up on searches (the task does)

I'm not sure, I'm not getting anything (this is the only thing I could fin that's weird about this project).
It has a space in the name, has no subprojects, and it just doesn't show up anywhere 🤔

can I assume these files are reused

A definite maybe, they may or may not be used, but we'd like to keep that option 🙃

Maybe the "old" way Dataset were shown is better suited ?

It was, but then it's gone now 😞

I see your point, this actually might be a "bug"?!

I would say so myself, but could be also by design..?

Awesome, I'll ask Product to reach out

LMK, happy to help out!
I know our use case is maybe a very different one, but...

Looks good! Why is it using an OutputModel and an InputModel?

Yes 😅 I want ClearML to load and parse the config before that. But now I'm not even sure those settings in the config are even exposed as environment variables?

I couldn't find it directly in the SDK at least (in the APIClient)... 🤔

On it! Should I include the additional user filters described above?

I'll have some reports tomorrow I hope TimelyPenguin76 SuccessfulKoala55 !

None, they're unusable for us.

These are per-user. Essentially we log user DB access as well (for various backtracking afterwards), so it's beneficial for us to pass the user DB secrets to the task and not have it configured once on the agent.

It's of course not an MLOps issue so I understand it's not high on the priority list, but would be kinda cool to just have a simple view presenting the content of users.get_all 😄

You could probably either:
Start the task first (using Task.init ), and then set the parameters if needed Attach the dataset to the task itself

The api.files_server is set to the MinIO endpoint s3://ip:9000/clearml (both locally and remotely) The sdk.development.default_output_uri is set to the MinIO endpoint (both locally and remotely) When we call Task.init I do not set the output_uri at all I get the logger directly with task.get_logger()

No I don't this at all

with s3

I thought so too - so I added flush calls just in case, but nothing's changed.
This is somewhat weird since it always happens in the above scenario (Ray + ClearML), and always in the last task/job from Ray

https://clearml.slack.com/archives/CTK20V944/p1652350709961419

We’d be happy if ClearML captures that (since it uses e.g. pip, then we have the git + commit hash for reproducibility), as it claims it would 😅

Any thoughts CostlyOstrich36 ?

Basically you have the details from the Dataset page, why should it be mixed with the others ?

Because maybe it contains code and logs on how to prepare the dataset. Or maybe the user just wants increased visibility for the dataset itself in the tasks view.

why would you need the Dataset Task itself is the main question?

For the same reason as above. Visibility and ease of access. Coupling relevant tasks and dataset in the same project makes it easier to understand that they're...

AgitatedDove14 Basically the fact that this happens without user control is very frustrating - https://github.com/allegroai/clearml/blob/447714eaa4ac09b4d44a41bfa31da3b1a23c52fe/clearml/datasets/dataset.py#L191

Maybe they shouldn't be placed under /tmp if they're mission critical, but rather the clearml cache folder? 🤔

@<1523701205467926528:profile|AgitatedDove14> this