Reputation
Badges 1
662 × Eureka!The title is specified in the plot (see the example, even if small).
I'm just creating a figure normally with matplotlib and save it to disk.
Without knowing anything, I'm assuming maybe ClearML patches plt.title
and not Axes.set_title
?
ClearML 1.1.4, Matplotlib 3.3.0 (it's not the latest as we have some backward compatibility issues)
Sure! It's a bit intricate as it accommodates many of our different plotting functionalities, but this consists of the important bits (I realize we have some bad naming here, but fig[0]
is actually a Figure object, and fig[1]
is an Axes object):
` plt.switch_backend('agg')
sns.set_theme(...)
fig = plt.subplots(...)
sns.histplot(data, ax=fig[1], ...)
fig[1].set_xlim(...)
fig[1].set_ylim(...)
fig[1].legend(loc='best')
fig[1].set_xlabel(xlabel)
fig[1].set_ylabel(ylabel)
fig[1].set_...
It was really easy with the attached code, really 👍
I would only maybe suggest adding in the documentation, that if one uses the default recommended install location, then the script can be run without any command line arguments.
I had to momentarily look at the code to see the default paths match my own (though I could've also looked at --help
default values 😛 )
FYI @<1523701087100473344:profile|SuccessfulKoala55> (or I might be doing something wrong), but it seems the python migration code comes with carriage returns, so it fails on linux by default (one has to tr -d '\r'
to use it)
EDIT: And also it defaults to /opt/allegro/data
rather than the recommended /opt/clearml/data
which is suggested when installing the server 🤔
Ah it already exists https://github.com/allegroai/clearml-server/issues/134 , so I commented on it
Hm, this didn't happen until now; I'd be happy to try again with a new version, but something with 1.4.0 broke our StorageManager, so we reverted to 1.3.2
No worries @<1537605940121964544:profile|EnthusiasticShrimp49> ! I made some headway by using Task.create
, writing a temporary Python script, and using task.update
in a similar way to how pipeline steps are created.
I'll try and create an MVC to reproduce the issue, though I may have strayed from your original suggestion because I need to be able to use classes and not just functions.
AgitatedDove14 Unfortunately not, the queues tab shows only the number of tasks, but not resources used in the queue . I can toggle between the different workers but then I don't get the full image.
Generally the StorageManager seems a bit slow, even a simple StorageManager.list(...)
on a local path seems to take a long time
Any leads TimelyPenguin76 ? I've also tried setting up a minio s3 bucket, but I'm not sure if the remote agent has copied the credentials and host 🤔
Can I query where the worker is running (IP)?
That's probably in the newer ClearML server pages then, I'll have to wait still 😅
I'm not sure what you mean by "entity", but honestly anything work. We're already monkey-patching our way 😄
A follow up question (instead of opening a new thread), is there a way I could signal some files/directories to be copied to the execute_remotely
task?
I'm aware, but it would be much cleaner to define them in the worker's clearml.conf
and let ClearML expose them locally to running tasks.
EDIT: Also the above is specifically about serving, which is not the target here 🤔 At least not yet 😄
Thanks SuccessfulKoala55 , I made https://github.com/allegroai/clearml-agent/issues/126 as a suggestion.
Do you have any thoughts on how to expose these... manually?
It does so already for environment variables that prefixed with CLEARML_
, so it would be nice to have some control over that.
Thanks for your help SuccessfulKoala55 ! Appreciate the patience 🙏
Thanks SuccessfulKoala55 ! Is this listed anywhere in the documentation?
Could I set an environment variable there and then refer to it internally in the config with the ${...}
notation?
I see https://github.com/allegroai/clearml-agent/blob/d2f3614ab06be763ca145bd6e4ba50d4799a1bb2/clearml_agent/backend_config/utils.py#L23 but not where it's called 🤔
Hi SuccessfulKoala55 !
Could you elaborate on how best to delete these from the database?
You don't even need to set the CLEARML_WORKER_ID, it will automatically assign one based on the machine's name
One more thing that may be helpful SweetBadger76 , I've gone ahead and looked into clearml.storage.helper
, and found that at least if I specify the bucket name directly in the aws.s3.credentials
configuration for MinIO, then:In [4]: StorageHelper._s3_configurations.get_config_by_uri('
` ')
Out[4]: S3BucketConfig(bucket='clearml', host='some_ip:9000', key='xxx', secret='xxx', token='', multipart=False, acl='', secure=False, region='', verify=True, use_credentials_chain=False)...