Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Everyone And Thanks Again For The Help, I Still Have No Success In Running Clearml Agent, It Just Gets Stuck Without Any Output, On Debug Mode For

Hi everyone and thanks again for the help, i still have no success in running clearml agent, it just gets stuck without any output, on debug mode for clearml-agent -d list i get the following output:
WARNING:urllib3.connectionpool:Retrying (Retry(total=238, connect=4, read=240, redirect=240, status=240)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)'))': /auth.login DEBUG:urllib3.connectionpool:Starting new HTTPS connection (3): <here is my internal clearml server>:8008 DEBUG:urllib3.util.retry:Incremented Retry for (url='/auth.login'): Retry(total=237, connect=4, read=240, redirect=240, status=240)any idea how to solve it?

  
  
Posted one year ago
Votes Newest

Answers 24


ChubbyLouse32 could it be the configuration file is not passed to the agent machine itself ?
(were you able to run anything against this internal server? I mean to connect to it from code, clearml/cleamrl-agent) ?

  
  
Posted one year ago

i dont have agent configuration file if this might be the problem
all the machines share the same file system so i managed to do all the things i mentioned from different machines on the system

  
  
Posted one year ago

This makes no sense to me 😞
Both are reading the exact same file, and using the same session / flow ...
Maybe there is an error with the "verify_certificate" on the agent ?

  
  
Posted one year ago

Im not sure what exactly your asking, someone else configured the server, im just using it

  
  
Posted one year ago

Hi ChubbyLouse32 , how did you configure the ClearML Server address?

  
  
Posted one year ago

sdk.development.store_uncommitted_code_diff: false api.verify_certificate : false api { web_server: https://<...>.com:8080 api_server: https://<...>.com:8008 files_server: https://<...>.com:8081 credentials { "access_key" = "OMF..." "secret_key" = "oox..." } }

  
  
Posted one year ago

ChubbyLouse32 does server URL configured in your clearml.conf file start with http:// or https:// ?

  
  
Posted one year ago

So what is the difference ? both running from the same machine ?

  
  
Posted one year ago

python code

  
  
Posted one year ago

And how is your clearml.conf configured?

  
  
Posted one year ago

` sdk.development.store_uncommitted_code_diff: false
api.verify_certificate : false
api {

web_server: <ADDRESS>:8080
api_server: <ADDRESS>:8008
files_server: <ADDRESS>:8081
credentials {
    "access_key" = "OMF..."
    "secret_key" = "oox..."
}

} `

  
  
Posted one year ago

yes i can communicate with the server, i managed to put tasks in the queue and retrieve them as well as running tasks with metrics reporting

Through the UI or python code ?

  
  
Posted one year ago

where address is our server adderss starting with https://.. etc

  
  
Posted one year ago

both the SDK and the Agent use clearml.conf

  
  
Posted one year ago

Let me verify something in the code,

  
  
Posted one year ago

correct. just verified again now.

  
  
Posted one year ago

what do you mean?

  
  
Posted one year ago

Okay found the issue, to disable SSL verification global add the following env variable:
CLEARML_API_HOST_VERIFY_CERT=0(I will make sure we fix the actual issue with the config file)

  
  
Posted one year ago

thanks it fixed it!

  
  
Posted one year ago

thanks

  
  
Posted one year ago

ChubbyLouse32 and this works when running python code and not when the agent is running ?
On the same machine ?

  
  
Posted one year ago

Yey!

  
  
Posted one year ago

yes i can communicate with the server, i managed to put tasks in the queue and retrieve them as well as running tasks with metrics reporting

  
  
Posted one year ago

Or, what appears in <here is my internal clearml server> ? I don't need the exact IP, of course, but is it http ? https ?

  
  
Posted one year ago