I can't seem to manage the first way around. If I select tasks in different projects, I don't get the bottom bar offering to compare between them

It can also log generate a log file with this method, it does not have to output it to CONSOLE tab.

I tried that, unfortunately it does not help 😞

https://clearml.slack.com/apps/A019MGGA2Q1-around ?

I've been answering there as well 🤕

When is the next release expected? 😄

(in the current version, that is, we’d very much like to use them obviously :D)

You don't even need to set the CLEARML_WORKER_ID, it will automatically assign one based on the machine's name

Or well, because it's not geared for tests, I'm just encountering weird shit. Just calling task.close() takes a long time

This is with:
Task.set_offline_mode(True) task = Task.init(..., auto_connect_streams=False)

I'm trying to decide if ClearML is a good use case for my team 🙂
Right now we're not looking for a complete overhaul into new tools, just some enhancements (specifically, model repository, data versioning).

We've been burnt by DVC and the likes before, so I'm trying to minimize the pain for my team before we set out to explore ClearML.

Ah, uhhhh whatever is in the helm/glue charts. I think it’s the allegroai/clearml-agent-k8s-base , but since I hadn’t gotten a chance to try it out, it’s hard to say with certainty which would be the best for us 😛

Or if it wasn't clear, that chunk of code is from clearml's dataset.py

I've also followed https://clearml.slack.com/archives/CTK20V944/p1628333126247800 but it did not help

Honestly I wouldn't mind building the image myself, but the glue-k8s setup is missing some documentation so I'm not sure how to proceed

AFAICS it's quite trivial implementation at the moment, and would otherwise require parsing the text file to find some references, right?
https://github.com/allegroai/clearml/blob/18c7dc70cefdd4ad739be3799bb3d284883f28b2/clearml/task.py#L1592

Right so this is checksum based? Are there plans to only store delta changes for files (i.e. store the changed byte instead of the entire file)?

Just because it's handy to compare differences and see how the data changed between iterations, but I guess we'll work with that 🙂
We'll probably do something like:
When creating a new dataset with a parent (or parents), look at immediate parents for identically-named files If those exist, load those with matching framework (pyarrow, pandas, etc), and log differences to the new dataset 🙂

I also tried switching to dockerized mode now, getting the same issue 🤔

I opened a GH issue shortly after posting here. @<1523701312477204480:profile|FrothyDog40> replied (hoping I tagged the right person).

We need to close the task. This is part of our unittests for a framework built on top of ClearML, so every test creates and closes a task.

Yes exactly that AgitatedDove14
Testing our logic maps correctly, etc for everything related to ClearML

Well, -ish. Ideally what we're after is one of the following:
Couple a task with a dataset. Keep it visible in it's destined location. Create a dataset separately from the task. Have control over its visibility and location. If it's hidden, it should not affect normal UI interaction (most annoying is having to click twice on the same project name when there are hidden datasets, which do not appear in the project view)

I'm using 1.1.6 (upgraded from 1.1.6rc0) - should I try 1.1.7rc0 or smth?

Ah I see, if the pipeline controller begins in a Task it does not add the tags to it…

Still failing with 1.2.0rc3 😞 AgitatedDove14 any thoughts on your end?

Yeah I managed to work around those former two, mostly by using Task.create instead of Task.init . It's actually the whole bunch of daemons running in the background that takes a long time, not the zipping.

Regarding the second - I'm not doing anything per se. I'm running in offline mode and I'm trying to create a dataset, and this is the error I get...
There is a data object it, but there is no script object attached to it (presumably again because of pytest?)

Say I upload each of these yamls as a configuration object (as with the above). Once I try to load bar.yaml remotely it will crash, since foo.yaml is missing (and is instead a clearml configuration object).
Does that make sense?

Right and then for text (file path) use some regex or similar for extraction, and for dictionary simply parse the values?

We're using the example autoscaler, nothing modified

There used to be a good example but it's now missing. I'm not sure what does Use only for automation (externally), otherwise use Task.connect_configuration mean when e.g. looking at Task.set_configuration_object , etc.

Could you clarify a bit, CostlyOstrich36 or AgitatedDove14 ?