Woot! What about clearml-agent 1.2.0? 🙂

No, fixed 🙂

And actually it fails on quite many tasks for us with this Python 3.6.
I tried to set up a different image ( agent8sglue.defaultContainerImage: "ubuntu:20.04" ) but that did not change much.
I suspect the culprit is agentk8sglue.image , which is set to tag 1.24-21 of clearml-agent-k8s-base . That image is quite very old… Any updates on that? 🤔

That is, we have something like:
` task = Task.init(...)
ds = Dataset.create(dataset_name=task.name, dataset_project=task.get_project_name(), use_current_task=True)

upload files

dataset.upload(show_progress=True)
dataset.finalize()

do stuff with task and dataset

task.close() `But because the dataset is linked to the task, the task is then moved and effectively becomes invisible 😕

Any thoughts AgitatedDove14 SuccessfulKoala55 ?

SmugDolphin23 we've been working with this for 2 weeks now, and it creates a lot of junk in our UI. Is there anyway to have better control over this?

Ah right, I missed that in the codebase. It just adds the .dataset convention to the dataset task.

JitteryCoyote63 yes exactly, sorry, I forgot to add the Task.get_task in my response. That's exactly what we do 😅

AgitatedDove14 yeah I see this now; this was an issue because I later had to "disconnect" the remote task, so it can, itself, create new tasks (using clearml.config.remote.override_current_task_id(None) ). I guess you might remember that discussion? 😁

EDIT: It's the discussion we had here, for reference. https://clearml.slack.com/archives/CTK20V944/p1640955599257500?thread_ts=1640867211.238900&cid=CTK20V944
So probably not needed in JitteryCoyote63 's case, we still have some...

I'm not sure; the setup is not unique to Mac.
Each user has their own .env file which is given to the code entry point, and at some point will be loaded with dotenv.load_dotenv() .
The environment variables are not set in code anywhere, but the clearml.conf uses them directly.

The key/secret is also shared internally so that sounds like a nice mitigation actually!
Which environment variable am I looking for? I couldn't spot anything specifically in that environment variables page

TimelyPenguin76 CostlyOstrich36 It seems a lot of manual configurations is required to get the EC2 instances up and running.
Would it not make sense to update the autoscaler (and example script) so that the config.yaml that's used for the autoscaler service is implicitly copied to the EC2 services, and then any extra_clearml_conf are used/overwritten?

UPDATE: Apparently the quotation type matters for furl ? I switched the ' to \" and it seems to work now

Yes and no SmugDolphin23
The project is listed, but there is no content and it hides my main task that it is attached to.

Thanks SuccessfulKoala55 , I made https://github.com/allegroai/clearml-agent/issues/126 as a suggestion.

Do you have any thoughts on how to expose these... manually?
It does so already for environment variables that prefixed with CLEARML_ , so it would be nice to have some control over that.

Visibility, same issue as I had 😁

Eek. Is there a way to merge a backup from elastic to current running server?

So a normal config file with environment variables.

I guess it's mixed. If #340 is resolved, then this initializer task will be a no-op: detach, and init-close new tasks as needed.

Happens pretty much consistently across all our projects -
Have a project with over 15 tasks (i.e. one that needs the Load More button) Click Load More, select a task that's not in the first 15 Let the page "rest" for a while (a couple of hours) Flip back to the page - the task is still active, but you cannot see it in the task list and there is no more Load More button

I am; it seems like maybe a couple of hours?

Yeah I figured (2) would be the way to go actually 😄

Also something we are very much interested in (including the logger-based scatter plots etc)

Another example - trying to validate dataset interactions ends with

` else:
self._created_task = True
dataset_project, parent_project = self._build_hidden_project_name(dataset_project, dataset_name)
task = Task.create(
project_name=dataset_project, task_name=dataset_name, task_type=Task.TaskTypes.data_processing)
if bool(Session.check_min_api_server_version(Dataset.__min_api_version)):
get_or_create_proje...

Coming back to this; ClearML prints a lot of error messages in local tests, supposedly because the output streams are not directly available:
` --- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.10/logging/init.py", line 1103, in emit
stream.write(msg + self.terminator)
ValueError: I/O operation on closed file.
Call stack:
File "/home/idan/CC/git/ds-platform/.venv/lib/python3.10/site-packages/clearml/task.py", line 3504, in _at_exit
self.__shutdown...

At any case, if we were upgrading from e.g. 4.0.0 to 4.1.2, this shouldn't have happened?

Any updates on this? We can't do anything with our K8s since this 404...

yes, a lot of moving pieces here as we're trying to migrate to AWS and set up autoscaler and more 😅

I will! (once our infra guy comes back from holiday and updates the install, for some reason they setup server 1.1.1???)

Meanwhile wondering where I got a random worker from

AgitatedDove14 I will try! I remember there were some issues with it, where I had to resort to this method first, but maybe things have changed since :)

Hm, just a small update - I just verified and it does indeed work on linux:

` import clearml
import dotenv

if name == "main":
dotenv.load_dotenv()
config = clearml.backend_api.Config.load() # Success, parsed with environment variables `