Well you could start by setting the output_uri to True in Task.init .

So basically I'm wondering if it's possible to add some kind of small hierarchy in the artifacts, be it sections, groupings, tabs, folders, whatever.

We're still working these quirks out. But one issue after we changed the AMI is that the VPC (SubnetId?) was missing from the instance so it could not reach the ClearML API server.

I think maybe the autoscaler service is missing some additional settings...

Right and then for text (file path) use some regex or similar for extraction, and for dictionary simply parse the values?

One must then ask, of course, what to do if e.g. a text refers to a dictionary configuration object? 🤔

Maybe they shouldn't be placed under /tmp if they're mission critical, but rather the clearml cache folder? 🤔

Yes it is

I think I may have brought this up multiple times in different ways :D
When dealing with long and complicated configurations (whether config objects, yaml, or otherwise), it's often useful to break them down into relevant chunks (think hydra, maybe).

In our case, we have a custom YAML instruction !include , i.e.
` # foo.yaml
bar: baz

bar.yaml

obj: !include foo.yaml
maybe_another_obj: !include foo.yaml `

AFAICS it's quite trivial implementation at the moment, and would otherwise require parsing the text file to find some references, right?
https://github.com/allegroai/clearml/blob/18c7dc70cefdd4ad739be3799bb3d284883f28b2/clearml/task.py#L1592

That's what I thought @<1523701087100473344:profile|SuccessfulKoala55> , but the server URL is correct (and WebUI is functional and responsive).
In part of our code, we look for projects with a given name, and pull all tasks in that project. That's the crash point, and it seems to be related to having running tasks in that project.

From the traceback ( backend_interface/task/task.py, line 178, in __init__ ), notice it's not Task.init

~20

Thanks!

Hm, I'm not sure I follow 🤔 How does the API server config relate to the file server?

Indeed with ~ the .root call ends with an empty string, so it has a bit of different flow

Now, the original pyhocon does support include statements as you mentioned - https://github.com/chimpler/pyhocon

Yes -- that's what I meant by The title is specified in the plot . I make the plots manually - title, axes labels, ticks, etc. In that sense, the figure is entirely configured. ClearML just saves it as "untitled 00/plot image"

Any simple ways around this for now? @<1523701070390366208:profile|CostlyOstrich36>

I don't think there's a PR issue for that yet, at least I haven't created one.

I could have a look at this and maybe make a PR.
Not sure what would the recommended flow be like though 🤔

CostlyOstrich36 so internal references are not resolved somehow? Or, how should one achieve:

def my_step(): from ..utils import foo foo("bar")

🤔

But there's nothing of that sort happening. The process where it's failing is on getting tasks for a project.

There's a specific fig[1].set_title(title) call.

For the former (static-ish environment variables), just add:

environment {
  VAR1: value1
  VAR2: value2
}

to the agent’s clearml.conf

I can see the task in the UI, it is not archived, and that's pretty much the snippet, but in full I do e.g.

Here's a full description of the layout:
Remote agent + entire ClearML docker suite running on host A. Host A also has a /data/clearml folder mounted to it and to it's docker containers (I've edited the docker-compose to add this mount point) Connect to host A, use StorageManager on the /data/clearml folder for some early troubles (e.g. long .list call) Use the same connection to run a task with execute_remotely and download_folder and see it crash :disapp...

some absolute path, i.e. /data/clearml

On my remote server..?

Bump?

That sounds about right FrothyDog40