nvm I fixed it thank you @<1523701087100473344:profile|SuccessfulKoala55> 🙏
Everything works fine when I access the /files endpoint from a browser after logging in via the webserver. However, when I run an experiment using the ClearML Python package and it tries to upload the artifacts to the /files endpoint, it gets blocked with a 401 Unauthorized error.
I’ve checked the ClearML Python package and it seems to be attaching the auth headers correctly. However, the JWT token it’s using is different from the one used by the browser, and it’s not being accepted by the server. I’ve tested this by manually sending requests to the server with both tokens, and only the browser token is accepted.
I’ve gone through the ClearML configuration file and didn't find any settings that could be causing this issue. Perhaps it is because I am using the wrong endpoint, since I am doing this:
local res, err = httpc:request_uri(" http://<my_host_ip>:8008/users.get_current_user ", {
method = "POST",
headers = {
["Authorization"] = "Bearer " .. token
}
})
if not res or res.status ~= 200 then
ngx.exit(ngx.HTTP_UNAUTHORIZED)
return
end
I assume you're taking this token from the cookie header?
Yes @<1523701087100473344:profile|SuccessfulKoala55> , I'm using a Lua script which makes a light api call to /users.get_current_user to check the validity of the JWT, not just its existence, before access the /files endpoint. This way, trying to access before logging in (no presence of token/invalid token) gets blocked, and logging in (presence of valid token in request header) grants access to the files. Works all fine and dandy in the browser, but I'm now realizing the application itself behaves differently and won't have access to these headers.
Yes @<1523701087100473344:profile|SuccessfulKoala55> , basically like this:
local token = ngx.req.get_headers()["Authorization"] or ngx.var.cookie_clearml_token_basic
if not token then
ngx.exit(ngx.HTTP_UNAUTHORIZED)
return
end
But did you set up the proxy to verify the JWT token?
Is there any information on how the ClearML python package authenticates itself with the ClearML server? I am almost certain that the endpoint I am using to check the token validity can only handle the browser session, and fails when the package tries to access files, i.e. the server isn't correctly setup to validate the JWT from the ClearML Python package.
So I read that in the ClearML docs that the file server has no security whatsoever and that you guys recommend to use object storage (s3/azure/etc). I currently do not have resources to use those sadly. However, I do know that the webserver has JWT auth, and so the proxy uses some lua scripting to verify a valid token before access the /files endpoint.
Does ClearML have configuration settings or environment variables that allow you to specify additional headers for HTTP requests?
As for the proxy, what exactly does using it provide you with regards to keeping your files private?
It works fine in the browser when you try to access None , it gets blocked when you haven't logged in via the /login page first. I didn't forsee how the package may need to access this endpoint, meaning it'll also need a valid JWT in it's header.
Hi @<1600661428556009472:profile|HighCoyote66> , there's currently no place where you can add custom headers to the ClearML calls (it's not very hard to add, of course, as we already do that for authorization headers).