But it is strictly that if condition in Task.init, see the issue I opened about it

From our IT dept:

Not really, when you launch the instance, the launch has to already be in the right VPC/Subnet. Configuration tools are irrelevant here.

It is. Let me see what else I have set up for MinIO in configs, one moment

Woot! What about clearml-agent 1.2.0? 🙂

SuccessfulKoala55 The changelog wrongly cites https://github.com/allegroai/clearml/issues/400 btw. It is not implemented and is not related to being able to save CSVs 😅

ClearML 1.1.4, Matplotlib 3.3.0 (it's not the latest as we have some backward compatibility issues)

Sure! It's a bit intricate as it accommodates many of our different plotting functionalities, but this consists of the important bits (I realize we have some bad naming here, but fig[0] is actually a Figure object, and fig[1] is an Axes object):

` plt.switch_backend('agg')
sns.set_theme(...)
fig = plt.subplots(...)

sns.histplot(data, ax=fig[1], ...)
fig[1].set_xlim(...)
fig[1].set_ylim(...)
fig[1].legend(loc='best')
fig[1].set_xlabel(xlabel)
fig[1].set_ylabel(ylabel)
fig[1].set_...

It was really easy with the attached code, really 👍

I would only maybe suggest adding in the documentation, that if one uses the default recommended install location, then the script can be run without any command line arguments.
I had to momentarily look at the code to see the default paths match my own (though I could've also looked at --help default values 😛 )

FYI @<1523701087100473344:profile|SuccessfulKoala55> (or I might be doing something wrong), but it seems the python migration code comes with carriage returns, so it fails on linux by default (one has to tr -d '\r' to use it)

EDIT: And also it defaults to /opt/allegro/data rather than the recommended /opt/clearml/data which is suggested when installing the server 🤔

Ah it already exists https://github.com/allegroai/clearml-server/issues/134 , so I commented on it

No need for S3 or other resources?

My code is done, it's hanging waiting for ClearML :E

Hm, this didn't happen until now; I'd be happy to try again with a new version, but something with 1.4.0 broke our StorageManager, so we reverted to 1.3.2

A couple of minutes or so

No worries @<1537605940121964544:profile|EnthusiasticShrimp49> ! I made some headway by using Task.create , writing a temporary Python script, and using task.update in a similar way to how pipeline steps are created.

I'll try and create an MVC to reproduce the issue, though I may have strayed from your original suggestion because I need to be able to use classes and not just functions.

AgitatedDove14 Unfortunately not, the queues tab shows only the number of tasks, but not resources used in the queue . I can toggle between the different workers but then I don't get the full image.

Generally the StorageManager seems a bit slow, even a simple StorageManager.list(...) on a local path seems to take a long time

Any leads TimelyPenguin76 ? I've also tried setting up a minio s3 bucket, but I'm not sure if the remote agent has copied the credentials and host 🤔

Can I query where the worker is running (IP)?

That's probably in the newer ClearML server pages then, I'll have to wait still 😅

Ubuntu 18.04, latest clearml version

I'm not sure what you mean by "entity", but honestly anything work. We're already monkey-patching our way 😄

A follow up question (instead of opening a new thread), is there a way I could signal some files/directories to be copied to the execute_remotely task?

Tada! https://github.com/allegroai/clearml-agent/issues/90

Thanks for your help SuccessfulKoala55 ! Appreciate the patience 🙏

Hi SuccessfulKoala55 !
Could you elaborate on how best to delete these from the database?

You don't even need to set the CLEARML_WORKER_ID, it will automatically assign one based on the machine's name

One more thing that may be helpful SweetBadger76 , I've gone ahead and looked into clearml.storage.helper , and found that at least if I specify the bucket name directly in the aws.s3.credentials configuration for MinIO, then:
In [4]: StorageHelper._s3_configurations.get_config_by_uri(' ` ')
Out[4]: S3BucketConfig(bucket='clearml', host='some_ip:9000', key='xxx', secret='xxx', token='', multipart=False, acl='', secure=False, region='', verify=True, use_credentials_chain=False)...

Here's how it failed for us 😅
poetry stores git related data in poetry.lock , so when you pip list , you get an internal package we have with its version, but no git reference, i.e. internal_module==1.2.3 instead of internal_module @ git+https://....@commit .
Then pip actually fails (our internal module is not on pypi), but poetry suceeds

Yes, exactly! I've added instructions for the users on creating their account and running clearml-init , and then they run the snippet that updates the api and sdk sections.

Or did you mean I can couple a short "mini config" with the package and redirect clearml to use this local one (instead of the one at ~/clearml.conf)?