Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
BeefyHippopotamus73
Moderator
4 Questions, 25 Answers
  Active since 10 January 2023
  Last activity one year ago

Reputation

0

Badges 1

25 × Eureka!
0 Votes
17 Answers
493 Views
0 Votes 17 Answers 493 Views
Running this code from inside a docker container locally: models = Model.query_models(project_name="foo-project-name", tags=target_tags, only_published=True)...
one year ago
0 Votes
12 Answers
544 Views
0 Votes 12 Answers 544 Views
I’m a total beginner with Kubernetes and somewhat new to ClearML. I can’t seem to get the server setup on AWS EKS. I’ve created an EKS cluster and node. I ca...
one year ago
0 Votes
6 Answers
553 Views
0 Votes 6 Answers 553 Views
one year ago
0 Votes
15 Answers
502 Views
0 Votes 15 Answers 502 Views
I’m trying to get a copy of a model through ClearML which is stored in S3: from clearml import Model models = Model.query_models(project_name="example_projec...
one year ago
0 Running This Code From Inside A Docker Container Locally:

s3://<bucket>/<foo>/local/<env>/<project-name>/v0-0-1/2022-05-12-30-9-rocketclassifier.7b7c02c4dac946518bf6955e83128bc2/models/2022-05-12-30-9-rocketclassifier.pkl.gz

one year ago
0 Running This Code From Inside A Docker Container Locally:

I wonder if there’s an exception being caught by the final try/except block (below) but because I’m running it in dagster I’m not getting the logger output like I would normally:
` # Don't canonize URL since we already did it
try:
instance = cls(base_url=base_url, url=url, logger=logger, canonize_url=False, **kwargs)
except (StorageError, UsageError) as ex:
cls._get_logger().error(str(ex))
return None
except Exception as ex:
cls._get_logger().error("Failed creating stora...

one year ago
0 Running This Code From Inside A Docker Container Locally:

I had to modify the code a bit because I’m running this in a dagster pipeline but that print line put out: helper None

one year ago
0 Running This Code From Inside A Docker Container Locally:

in case it’s helpful, the url is an S3 URI

one year ago
0 Running This Code From Inside A Docker Container Locally:

For that reason I suspect there is a silent error trying to grab boto3 credentials.

one year ago
0 Running This Code From Inside A Docker Container Locally:

also, I can run this same code (same model) in a separate script (not in a docker container) with the AWS_PROFILE env variable set and load the model properly with joblib

one year ago
0 Running This Code From Inside A Docker Container Locally:

ah, I just ran:
helper = StorageHelper(url=s3_uri, base_url=s3_uri) context.log.info(f"helper: {helper}")and I got:
ValueError: Missing key and secret for S3 storage access ( ` )
File "/opt/venv/lib/python3.7/site-packages/dagster/core/execution/plan/utils.py", line 44, in solid_execution_error_boundary
yield
File "/opt/venv/lib/python3.7/site-packages/dagster/utils/init.py", line 383, in iterate_with_context
next_output = next(iterator)
File "/opt/venv/lib/python3.7...

one year ago
0 Running This Code From Inside A Docker Container Locally:

I just checked for white space at the beginning or end of the url but there are none

one year ago
0 I’M Trying To Get A Copy Of A Model Through Clearml Which Is Stored In S3:

I think I can set AWS_PROFILE . I have theoretically placed the clearml.conf in the correct place in the container: $HOME/clearml.conf . But I get an error that’s actually not very helpful, I still think the issue is getting boto3 credentials but I’m not sure I can prove it at this point. I posted the error in the community channel https://clearml.slack.com/archives/CTK20V944/p1653507863528969

one year ago
0 I’M Trying To Get A Copy Of A Model Through Clearml Which Is Stored In S3:

well, I can get this exact object from my machine, from the same script through boto3, so I’m not sure where the disconnect is AgitatedDove14 SuccessfulKoala55

one year ago
0 I’M Trying To Get A Copy Of A Model Through Clearml Which Is Stored In S3:

so a little more clarity. I get nearly this same error when downloading the same object from boto3 if I try to get at it from an s3 resource like so:
` bad_s3_resource = boto3.resource("s3")
obj = bad_s3_resource.Object(bucket, key)
obj.get()["Body"]

botocore.exceptions.NoCredentialsError: Unable to locate credentials Only when I set the boto3 session with the appropriate profile does it work: sess = boto3.session.Session(profile_name="foo-profile")
s3_resource = sess.resource("s3")
...

one year ago
0 I’M Trying To Get A Copy Of A Model Through Clearml Which Is Stored In S3:

Don’t know why I didn’t think of it earlier, but I set my env variable AWS_PROFILE=<foo-profile> and it copied it successfully.

one year ago
0 I’M Trying To Get A Copy Of A Model Through Clearml Which Is Stored In S3:

yes I did finally resolve this. If my memory serves, I think I needed to do some double checking that the clearml.conf file was in the proper location. It wasn’t where I thought I put it and I had to set all the proper environment variables in the container.

one year ago
0 I’M Trying To Get A Copy Of A Model Through Clearml Which Is Stored In S3:

SuccessfulKoala55 and AgitatedDove14 So setting my environment variable worked for my local runs, but it doesn’t work when running it from a container (in AWS ECS, for example)

one year ago
0 Running This Code From Inside A Docker Container Locally:

I have already successfully printed model.name though I didn’t try the other ones.

one year ago
0 I’M A Total Beginner With Kubernetes And Somewhat New To Clearml. I Can’T Seem To Get The Server Setup On Aws Eks. I’Ve Created An Eks Cluster And Node. I Can Access And Run Helm Commands Against The Cluster. According To

That’s very helpful, thanks!. BTW when I enter that command I get No resources found in clearml namespace. Similarly if I run:
kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services clearml
I get:
Error from server (NotFound): services "clearml" not found I

one year ago
0 I’M A Total Beginner With Kubernetes And Somewhat New To Clearml. I Can’T Seem To Get The Server Setup On Aws Eks. I’Ve Created An Eks Cluster And Node. I Can Access And Run Helm Commands Against The Cluster. According To

I’m happy to use AWS’s ALB I was just even less sure about how to set that up at the time. I assume I need to setup a Target Group Attachement to the clearml-webserver ip address, correct?

one year ago
0 Running This Code From Inside A Docker Container Locally:

looks like I accidentally placed my clearml.conf file in a non-standard place so I had to set the CLEARML_CONFIG_FILE environment variable. Thanks for your help AgitatedDove14 !!

one year ago