I can't seem to manage the first way around. If I select tasks in different projects, I don't get the bottom bar offering to compare between them

It can also log generate a log file with this method, it does not have to output it to CONSOLE tab.

I tried that, unfortunately it does not help 😞

https://clearml.slack.com/apps/A019MGGA2Q1-around ?

I've been answering there as well 🤕

When is the next release expected? 😄

(in the current version, that is, we’d very much like to use them obviously :D)

Or well, because it's not geared for tests, I'm just encountering weird shit. Just calling task.close() takes a long time

This is with:
Task.set_offline_mode(True) task = Task.init(..., auto_connect_streams=False)

Or if it wasn't clear, that chunk of code is from clearml's dataset.py

I've also followed https://clearml.slack.com/archives/CTK20V944/p1628333126247800 but it did not help

Honestly I wouldn't mind building the image myself, but the glue-k8s setup is missing some documentation so I'm not sure how to proceed

AFAICS it's quite trivial implementation at the moment, and would otherwise require parsing the text file to find some references, right?
https://github.com/allegroai/clearml/blob/18c7dc70cefdd4ad739be3799bb3d284883f28b2/clearml/task.py#L1592

Right so this is checksum based? Are there plans to only store delta changes for files (i.e. store the changed byte instead of the entire file)?

Just because it's handy to compare differences and see how the data changed between iterations, but I guess we'll work with that 🙂
We'll probably do something like:
When creating a new dataset with a parent (or parents), look at immediate parents for identically-named files If those exist, load those with matching framework (pyarrow, pandas, etc), and log differences to the new dataset 🙂

I also tried switching to dockerized mode now, getting the same issue 🤔

I opened a GH issue shortly after posting here. @<1523701312477204480:profile|FrothyDog40> replied (hoping I tagged the right person).

We need to close the task. This is part of our unittests for a framework built on top of ClearML, so every test creates and closes a task.

Yes exactly that AgitatedDove14
Testing our logic maps correctly, etc for everything related to ClearML

Well, -ish. Ideally what we're after is one of the following:
Couple a task with a dataset. Keep it visible in it's destined location. Create a dataset separately from the task. Have control over its visibility and location. If it's hidden, it should not affect normal UI interaction (most annoying is having to click twice on the same project name when there are hidden datasets, which do not appear in the project view)

I'm using 1.1.6 (upgraded from 1.1.6rc0) - should I try 1.1.7rc0 or smth?

Ah I see, if the pipeline controller begins in a Task it does not add the tags to it…

Still failing with 1.2.0rc3 😞 AgitatedDove14 any thoughts on your end?

Yeah I managed to work around those former two, mostly by using Task.create instead of Task.init . It's actually the whole bunch of daemons running in the background that takes a long time, not the zipping.

Regarding the second - I'm not doing anything per se. I'm running in offline mode and I'm trying to create a dataset, and this is the error I get...
There is a data object it, but there is no script object attached to it (presumably again because of pytest?)

Say I upload each of these yamls as a configuration object (as with the above). Once I try to load bar.yaml remotely it will crash, since foo.yaml is missing (and is instead a clearml configuration object).
Does that make sense?

Right and then for text (file path) use some regex or similar for extraction, and for dictionary simply parse the values?

We're using the example autoscaler, nothing modified

There used to be a good example but it's now missing. I'm not sure what does Use only for automation (externally), otherwise use Task.connect_configuration mean when e.g. looking at Task.set_configuration_object , etc.

Could you clarify a bit, CostlyOstrich36 or AgitatedDove14 ?

So the config loading is not deferred until execution 😞

Generally, really. I've struggled recently (and in the past), because the documentation seems:
Very complete wrt available SDK (though the formatting is sometimes off) Very lacking wrt to how things interact with one anotherA lot of what I need I actually find from pluging into the source code.
I think ClearML would benefit itself a lot if it adopted a documentation structure similar to numpy ecosystem (numpy, pandas, scipy, scikit-image, scikit-bio, scikit-learn, etc)

I see, okay that already clarifies some stuff, I'll dig a bit more into this then! Thanks!