
Reputation
Badges 1
662 × Eureka!I tried that, unfortunately it does not help π
Actually TimelyPenguin76 I get only the following as a "preview" -- I thought the preview for an image would be... the image itself..?
AgitatedDove14
I'll make a PR for it now, but the long story is that you have the full log, but the virtualenv
version is not logged anywhere (the usual output from virtualenv
just says which Python version is used, etc).
Sure, for example when reporting HTML files:
Thanks! That's what I thought, but then I get2021-12-21 22:08:35,376 - clearml.storage - ERROR - Failed uploading: Parameter validation failed: Invalid bucket name "": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$"
Follow up on this btw, from the WebUI/Server POV, I see there's an "Admin" role, etc. Do those have additional views available, such as users etc?
I can see the task in the UI, it is not archived, and that's pretty much the snippet, but in full I do e.g.
Oh! Nice! I'll have a go at it and report back at the PR if it's in a functional state π Thanks AgitatedDove14 !
I'll have a look, at least it seems to only use from clearml import Task
, so unless mlflow changed their SDK, it might still work!
The deferred_init
input argument to Task.init
is bool
by default, so checking type(deferred_init) == int
makes no sense to begin with, and is altering the flow.
I'm saying it's a bug
For now we've monkey-patched it to our usecase:
` Dataset._Dataset__hidden_tag = "active"
def foo(cls, dataset_project, dataset_name):
dataset_project = dataset_project or "Datasets"
return dataset_project, dataset_project.rpartition("/")[0]
Dataset._build_hidden_project_name = foo `
I see that the GUI AutoScaler is only in the paid version, wonder why the GCP driver is not open source?
Happens with the latest version indeed.
I canβt share our code, but the gist of it is:
pipe = PipelineController(name=..., project=..., version=...)
pipe.add_function_step(...) # Many calls
pipe.set_default_execution_queue(...)
pipe.start(queue=..., wait=True)
Would be good if that's mentioned explicitly in the docs π Thanks!
I think you're interested in the Monitor
class:)
StaleButterfly40 what use case are you looking for? I've used environment variables in the config file and then I can overwrite them in os.environ
before ClearML loads the config
Follow-up question/feature request (out of interest) - could the WebUI show the matching commit message?
In which repo?:)
IIRC, get_local_copy()
downloads a local copy and returns the path to the downloaded file. So you might be interested in e.g.local_csv = pd.read_csv(a_task.artifacts['train_data'].get_local_copy())
With the models, you're looking for get_weights()
. It acts the same as get_local_copy()
, so it returns a path.
EDIT: I think also get_local_copy()
for a model should work π
Is it currently broken? π€
@<1523701205467926528:profile|AgitatedDove14> this
Sorry, I misspoke, yes of course, the agents config file, not the queues
(the extra_vm_bash_script
is what you're after)
I mean, if I search for "model", will it automatically search for tasks containing "model" in their name?
@<1539780258050347008:profile|CheerfulKoala77> you may also need to define subnet or security groups.
Personally I do not see the point in Docker over EC2 instances for CPU instances (virtualization on top of virtualization).
Finally, just to make sure, you only ever need one autoscaler. You can monitor multiple queues with multiple instance types with one autoscaler.
We just redeployed to use the 1.1.4 version as Jake suggested, so the logs are gone π