
Reputation
Badges 1
662 × Eureka!We're using 1.1.5 at the moment -- I'll make sure everyone updates to 1.1.6 on Monday.
That solution does not work for us unfortunately -- the .env
is an argument from argparse, and because we cannot attach non-git files to a remote task (again issue #395), we have to first download CLI arguments for remote execution and ensure they exist on the remote agent.
@<1523701827080556544:profile|JuicyFox94> we have it up and running, hurray ๐
One thing I noticed in the k8s logs is frequent warnings about Python 3.6..? Is the helm chart built with that Python version?
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
from cryptography.utils import int_...
Is there a way to accomplish this right now FrothyDog40 ? ๐ค
Yes, exactly. I have not yet had a chance to try this out -- should it work?
It misses the repository information of course, but the 'configuration/Args' were logged. So something weird in identifying the repository
We're using the example autoscaler, nothing modified
That could be a solution for the regex search; my comment on the pop-up (in the previous reply) was a bit more generic - just that it should potentially include some information on what failed while fetching experiments ๐
Not really, I've only been able to somewhat understand the scope of where it happens, and I'm not sure it's even a ClearML issue (maybe matplotlib)
Sorry, been away for a while!
I have no additional information, since it was a bug in my model that I have since eliminated...
Maybe it was just a matplotlib error and can be dropped for now. I'll let you know if it pops up again!
Just a side note - the 1.1.1 notice keeps popping up even though the server is at 1.1.1 (and I've cleared browser cache etc)
TimelyPenguin76 that would have been nice but I'd like to upload files as artifacts (rather than parameters).
AgitatedDove14 I mean like a grouping in the artifact. If I add e.g. foo/bar
to my artifact name, it will be uploaded as foo/bar
.
It is. Let me see what else I have set up for MinIO in configs, one moment
Either one would be nice to have. I kinda like the instant search option, but could live with an ENTER to search.
I opened this meanwhile - https://github.com/allegroai/clearml-server/issues/138
Generally, it would also be good if the pop-up presented some hints about what went wrong with fetching the experiments. Here, I know the pattern is incomplete and invalid. A less advanced user might not understand what's up.
Unfortunately not, each task defines and constructs its own dataset. I want cloned task to save that link ๐ค
Iโll give the create_function_task
one more try ๐ค
It's a small snippet that ensures identically named projects are still unique'd with a running number.
I cannot, the instance is long gone... But it's not different to any other scaled instances, it seems it just took a while to register in ClearML
@<1539780258050347008:profile|CheerfulKoala77> you may also need to define subnet or security groups.
Personally I do not see the point in Docker over EC2 instances for CPU instances (virtualization on top of virtualization).
Finally, just to make sure, you only ever need one autoscaler. You can monitor multiple queues with multiple instance types with one autoscaler.
There's a specific fig[1].set_title(title)
call.
On an unrelated note, when cloning an experiment via the WebUI, shouldn't the cloned experiment have the original experiment as a parent? It seems to be empty
We're wondering how many on-premise machines we'd like to deprecate. For that, we want to see how often our "on premise" queue is used (how often a task is submitted and run), for how long, how many resources it consumes (on average), etc.
Hey @<1537605940121964544:profile|EnthusiasticShrimp49> ! Youโre mostly correct. The Step
classes will be predefined (of course developers are encouraged to add/modify as needed), but as in the DataTransformationStep
, there may be user-defined functions specified. Thatโs not a problem though, I can provide these functions with the helper_functions
argument.
- The
.add_function_step
is indeed a failing point. I canโt really create a task from the notebook because calling `Ta...