Created this for follow up, SuccessfulKoala55 ; I'm really stumped. Spent the entire day on this 🥹
https://github.com/allegroai/clearml-agent/issues/134

It failed on some missing files in my remote_execution, but otherwise seems fine now

Hi AgitatedDove14 !

Ah, thanks! I'll use the artifacts for linking.

We've forgone the "use current task" already because it indeed made things even more difficult (the task that was used is then automatically hidden by this automatic renaming of dataset tasks).
The current implementation (since 1.6.3 I think) creates the issues in the linked comment (with images to visualize).

That's a nice work around of course - I'm sure it works and I'll give it a shot momentarily. I'm just wondering if ClearML could automatically recognize image files in upload_artifact (and other well known suffixes) and do that for me.

Indeed. I'll open an issue, sure!

I guess I'll have to rerun the experiment without tags for this?

So the ..data referenced in the example above are part of the git repository?
What about setting the working_directory to the user working directory using Task.init or Task.create ?

https://clearml.slack.com/archives/CTK20V944/p1652350709961419

I commented on your suggestion to this on GH. Uploading the artifacts would happen via some SDK before switching to remote execution.
When cloning a task (via WebUI or SDK), a user should have an option to also clone these input artifacts or simply linking to the original. If linking to the original, then if the original task is deleted - it is the user's mistake.

Alternatively, this potentially suggests "Input Datasets" (as we're imitating now), such that they are not tied to the original t...

That gives us the benefit of creating "local datasets" (confined to the scope of the project, do not appear in Datasets tabs, but appear as normal tasks within the project)

DeterminedCrab71 not in this scenario, but I do have it occasionally, see my earlier thread asking how to increase session timeout time

I was thinking of using the --volume settings in clearml.conf to mount the relevant directories for each user (so it's somewhat customizable). Would that work?

It would be amazing if one can specify specific local dependencies for remote execution, and those would be uploaded to the file server and downloaded before the code starts executing

Not really, I've only been able to somewhat understand the scope of where it happens, and I'm not sure it's even a ClearML issue (maybe matplotlib)

If relevant, I'm using Chrome Version 101.0.4951.41 (Official Build) (64-bit)

Uhhh, but pyproject.toml does not necessarily entail poetry... It's a new Python standard

Hm, that seems less than ideal. I was hoping I could pass some CSV locations. I'll try and find a workaround for that. Thanks!

I understand, but then the toml file needs to be parsed to ensure poetry is used. It's just a tool entry in the pyproject.toml.

Not really - it will just show the string. A preview would be more like a low-res version of the uploaded image or similar.

I just used this to create the dual_gpu queue:
clearml-agent daemon --queue dual_gpu --create-queue --gpus 0,1 --detached

Let me know if you do; would be nice to have control over that 😁

can I assume these files are reused

A definite maybe, they may or may not be used, but we'd like to keep that option 🙃

Maybe the "old" way Dataset were shown is better suited ?

It was, but then it's gone now 😞

I see your point, this actually might be a "bug"?!

I would say so myself, but could be also by design..?

Awesome, I'll ask Product to reach out

LMK, happy to help out!
I know our use case is maybe a very different one, but...

Sure CostlyOstrich36 , sorry it took me so long to reply. I minimized the window a bit here so everything will fill in nicely. Worth mentioning this happens on all pages of course, but I went to the profile page so you can also see the clearml server version.

Sure! It's a bit intricate as it accommodates many of our different plotting functionalities, but this consists of the important bits (I realize we have some bad naming here, but fig[0] is actually a Figure object, and fig[1] is an Axes object):

` plt.switch_backend('agg')
sns.set_theme(...)
fig = plt.subplots(...)

sns.histplot(data, ax=fig[1], ...)
fig[1].set_xlim(...)
fig[1].set_ylim(...)
fig[1].legend(loc='best')
fig[1].set_xlabel(xlabel)
fig[1].set_ylabel(ylabel)
fig[1].set_...

Any thoughts CostlyOstrich36 ?

Yes, as I wrote above 😄

I mean, I see these are defined here https://github.com/allegroai/clearml-agent/blob/master/clearml_agent/definitions.py

But I do not see where an EnvironmentConfig.set() is called...

The title is specified in the plot (see the example, even if small).
I'm just creating a figure normally with matplotlib and save it to disk.

We're using 1.1.5 at the moment -- I'll make sure everyone updates to 1.1.6 on Monday.

That solution does not work for us unfortunately -- the .env is an argument from argparse, and because we cannot attach non-git files to a remote task (again issue #395), we have to first download CLI arguments for remote execution and ensure they exist on the remote agent.

SuccessfulKoala55 That string was autogenerated by pyhocon and matches their documentation too - https://github.com/lightbend/config/blob/master/HOCON.md#substitutions
The first example won't work (it will treat ${...} as a string literal and won't replace it). The second does work, but as mentioned anyway, these were not hand typed, but rather generated from pyhocon, so I don't think that's the issue 🤔

I'm not sure; the setup is not unique to Mac.
Each user has their own .env file which is given to the code entry point, and at some point will be loaded with dotenv.load_dotenv() .
The environment variables are not set in code anywhere, but the clearml.conf uses them directly.