Here is one log from the API
[2021-05-24 09:04:46,311] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /

I assumed this was the error when clear-ml tried to connect. Looks like this was the curl.

Try curl -u <access_key>:<secret key> https://myclearmlapi.com/users.get_current_user and see what you get

Cool I'll open an issue and see the response.

Try to see the server log while you're trying to verify the agent credentials (not agent-services)

I'm not sure it's working using curl since you're not doing anything using the credentials - for all I know you might be reaching the demo server 🙂

https://clearml.slack.com/archives/CTK20V944/p1621844719193900?thread_ts=1621840885.183000&cid=CTK20V944
This didn't work it seems getting the same error as before.

Since the traffic are routed through nginx for 3 url (eg,  http://myclearmlfiles.com ,  http://myclearmlapi.com ,  http://myclearmlapp.com ) to different port, what should be the value of CLEARML_HOST_IP

which means this is probably not the same server instance you're seeing in the WebApp (if the credentials indeed appear there)

I used the configuration file provided as example changes the necessary things. Looks like it's somehow not able to hit the api.
clearml_agent: ERROR: Failed getting token (error 400 from https://myclearmlapi.com ): Bad Request

was checking the clearml-apiserver logs there are no events when doing clearml-agent init

I wanted to enable HTTPS. exposing the ports would disable the ssl nginx was providing.

Got it exposing the ports should work, I'm guessing. How do you suggest to secure the port in that case ?

It was

What do you mean? What are you trying to protect against?

I'm not sure - it can be a simple mechanism, but obviously it will take some time to make it work 🙂

if you're configuring CLEARML_WEB_HOST ,  CLEARML_API_HOST  and  CLEARML_FILES_HOST for the agent-services, you don't need to pass CLEARML_HOST_IP

Redirection seems to be working with curl, as well as dashboard ie i'm able to access the dashboard with the URL.

I get this response

So the server you're reaching (whichever server it is) does not have the credentials on record

{"meta":{"id":"e1dbf2fcec864085ae05762a61594e9c","trx":"e1dbf2fcec864085ae05762a61594e9c","endpoint":{"name":"users.get_current_user","requested_version":"2.13","actual_version":"1.0"},"result_code":401,"result_subcode":22,"result_msg":"Unauthorized (invalid credentials) (failed to locate provided credentials)","error_stack":null,"error_data":{}},"data":{}}

What is CLEARML_HOST_IP actually signify ?

In the case of agent-services, it's the base host name used for calculating  CLEARML_WEB_HOST ,  CLEARML_API_HOST  and  CLEARML_FILES_HOST

Here is one log from the API

[2021-05-24 09:04:46,311] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /

For what?

Well, it's possible you're redirecting to the wrong place? I'm not sure the nginx redirection works - we've only used sub-domain rules

No it seems something is still going wrong, my clear-agent-services is not starting, as well as when I install clearml-agent anywhere it's not able to verify the credentials.

Do you thing putting it behind nginx will be an useful feature ? Then i'll open a feature request issue.

Okay, let me try that out.

Yeah, well, that's exactly why we used sub-domains to support HTTPS/SSL

There are no logs in clearml-apiserver for auth.