Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Am Doing Port Forwarding Of Ports From Localhost Clearml Server In Ec2 Instance To The Ports In Laptop Locally. I Am Able To Login To The Server And Generate The Credentials But I Am Not Able To Create Task

I am doing port forwarding of ports from localhost clearml server in EC2 instance to the ports in laptop locally. I am able to login to the server and generate the credentials but I am not able to create task task = Task.init(project_name="aws_project", task_name="exp-1")

Error I am getting is
Retrying (Retry(total=239, connect=239, read=240, redirect=240, status=240)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb646e46190>: Failed to establish a new connection: [Errno 111] Connection refused')': /auth.login Retrying (Retry(total=238, connect=238, read=240, redirect=240, status=240)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb646e46310>: Failed to establish a new connection: [Errno 111] Connection refused')': /auth.login
My clearml.conf file has below configuration
api { web_server: " " api_server: " " files_server: " " credentials { "access_key" = "...." "secret_key" = "...." } verify_certificate: True }

  
  
Posted 2 years ago
Votes Newest

Answers 19


i guess so. make your tests and please keep us updated if you still encounter issues 🙂

  
  
Posted 2 years ago

Which port should I add?? Of file_server i.e., 8081 ??

  
  
Posted 2 years ago

DepravedSheep68 you could also try to add the port to your uri.
Output_uri: "s3://...... : port"

  
  
Posted 2 years ago

DepravedSheep68 , can you please give a bit more context on the error? Also can you show an example of your usage?

You can try the following:
Configure your ~/clearml.conf with sdk.development.default_output_uri: "s3://<YOUR_BUCKET>" and in code simply using Task.init(..., output_uri=True) . Let's see if that setup works.

  
  
Posted 2 years ago

SweetBadger76 using output_uri="s3://......" in task.init gives me following error:
ValueError: Invalid port ''.

  
  
Posted 2 years ago

SweetBadger76 thanks for heads up on that I will try that

  
  
Posted 2 years ago

CostlyOstrich36 I mean whatever is sent to clearml-server and displayed there, I want to store it to s3 !!

  
  
Posted 2 years ago

Hello DepravedSheep68 ,

In order to store your info into the S3 bucket you will need two things :
specify the uri where you want to store your data when you initialize the task (search for the parameter output_uri in the Task.init function https://clear.ml/docs/latest/docs/references/sdk/task#taskinit ) specify your s3 credentials into the clear.conf file (what you did)

  
  
Posted 2 years ago

Or when running something and uploading to a s3?

  
  
Posted 2 years ago

DepravedSheep68 , do you mean when registering your data?

  
  
Posted 2 years ago

Providing path to s3 bucket is not storing the artifacts and information in the s3 bucket!!

  
  
Posted 2 years ago

Do I need to provide s3 bucket name or I could actually provide path for directory under s3 bucket?

  
  
Posted 2 years ago

yes, from the config
UI will popup a from for buckets' key and secret

  
  
Posted 2 years ago

If I am doing port forwarding then I just need to add s3 bucket name right?? credentials and everything related to aws will be taken automatically!?

  
  
Posted 2 years ago

only you will have access to material upload to the SAAS files server, same security as the web and api has.

you can of course use your S3 bucket to store these resources
https://clear.ml/docs/latest/docs/integrations/storage#configuring-aws-s3

  
  
Posted 2 years ago

DeterminedCrab71 does the artifacts and results send to clearml ec2 server accessible to anyone or it would be perfectly private and confidential??

also it would be great if you could direct me to relevant documentation, where all of the atifacts and plots displayed in cleaml-server are stored to particular s3 bucket and all of the information displayed in the server comes from s3 bucket!!

  
  
Posted 2 years ago

port forwarding is an unconventional setup, and inconvenient 1. you should configure a public address.
https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_aws_ec2_ami has some info and links.

for a more secure option I'd follow this guide to create a user/password environment https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_config/#web-login-authentication
and use AWS ELB to securely expose the 3 services though it.
you can then configure your client with the exposed address
web_server: "http://[expossed web address]:8080" api_server: "http://[expossed API address]:8008" files_server: "http://[expossed file address]localhost:8081"

another simple option would be our hosted SAAS at https://app.clear.ml

  
  
Posted 2 years ago

DeterminedCrab71 could you help me with this!

  
  
Posted 2 years ago

I have removed the keys values just for security purpose

  
  
Posted 2 years ago
900 Views
19 Answers
2 years ago
one year ago
Tags