[2022-08-18 20:58:49,892] [9] [INFO] [clearml.request_handlers] GET [2022-08-18 20:58:49,895] [9] [INFO] [clearml.request_handlers] Basic NUtORklQWUIzM0tIVUFRNUI4SkM6eU4yUXE4RTNvWDlpYlBRVmF1NEpiQXZQMHlUdHozNUluZVY4ZkpuZXNTRnRYbmRlR3I= [2022-08-18 20:58:49,904] [9] [INFO] [clearml.service_repo] Returned 200 for auth.login in 6ms [2022-08-18 20:58:50,059] [9] [INFO] [clearml.request_handlers] GET [2022-08-18 20:58:50,060] [9] [INFO] [clearml.request_handlers] Basic NUtORklQWUIzM0tIVUFRNUI4SkM6eU4yUXE4RTNvWDlpYlBRVmF1NEpiQXZQMHlUdHozNUluZVY4ZkpuZXNTRnRYbmRlR3I= [2022-08-18 20:58:50,069] [9] [INFO] [clearml.service_repo] Returned 200 for auth.login in 6ms [2022-08-18 20:58:50,102] [9] [INFO] [clearml.request_handlers] GET [2022-08-18 20:58:50,104] [9] [INFO] [clearml.request_handlers] Basic NUtORklQWUIzM0tIVUFRNUI4SkM6eU4yUXE4RTNvWDlpYlBRVmF1NEpiQXZQMHlUdHozNUluZVY4ZkpuZXNTRnRYbmRlR3I= [2022-08-18 20:58:50,114] [9] [INFO] [clearml.service_repo] Returned 200 for auth.login in 8ms [2022-08-18 20:58:50,130] [9] [INFO] [clearml.request_handlers] GET [2022-08-18 20:58:50,132] [9] [INFO] [clearml.request_handlers] Basic dXNlcjpsb2NhbC1iMzE4MjljNmJjOTZkYmFhNGFiOWE5OGIwMGFiZDBlNzliN2I5YzY3 [2022-08-18 20:58:50,138] [9] [WARNING] [clearml.service_repo] Returned 401 for projects.get_all in 3ms, msg=Unauthorized (invalid credentials) (failed to locate provided credentials) [2022-08-18 20:58:50,145] [9] [INFO] [clearml.request_handlers] GET [2022-08-18 20:58:50,147] [9] [INFO] [clearml.request_handlers] Basic NUtORklQWUIzM0tIVUFRNUI4SkM6eU4yUXE4RTNvWDlpYlBRVmF1NEpiQXZQMHlUdHozNUluZVY4ZkpuZXNTRnRYbmRlR3I= [2022-08-18 20:58:50,159] [9] [INFO] [clearml.service_repo] Returned 200 for auth.login in 9ms [2022-08-18 20:58:50,173] [9] [INFO] [clearml.request_handlers] GET [2022-08-18 20:58:50,175] [9] [INFO] [clearml.request_handlers] Basic dXNlcjpsb2NhbC1iMzE4MjljNmJjOTZkYmFhNGFiOWE5OGIwMGFiZDBlNzliN2I5YzY3 [2022-08-18 20:58:50,181] [9] [WARNING] [clearml.service_repo] Returned 401 for projects.get_all in 3ms, msg=Unauthorized (invalid credentials) (failed to locate provided credentials)
If you want to try and patch the server code to get some visibility, I can guide you on how 🙂
So the basic authentication header is different for the problematic call 🙂
SuccessfulKoala55 thanks alot for your help i greatly appreciate it, i think you are right that there is something happening with the request, I did some checking i can get it running on another computer (my non work computer), so i think you are right there must be something happening with my work related computer which is causing this to occur 😕 I'll keep trying to drill down on this issue on my work computer to see what the difference is, but I can atleast proceed now to as i got experiments working on another computer 😄 ! Again thanks a lot for your help SuccessfulKoala55 and AgitatedDove14 i greatly appreciate it!
You basically do this:
In your docker-compose.yaml file, add the following env var to the apiserver service: FLASK_DEBUG: "1"
You'll need to restart the server exec into the clearml-apiserver container CD into /opt/clearml/apiserver edit server.py Around line 35 right before call = self._create_api_call(request)
, add:log(str(request.method)) log(str(request.headers["Authorization"]))
5. Save the file
6. Try running your python code again and than check the clearml-apiserver docker container logs
bugs i understand are better than ones i dont lol
i'm looking at the server.py
file and it doesn't have call =
self.
create_api_call(request)
i see what you are saying I dont thinl there would be anything on my system which might do that i can investigate
We can try and patch the server to print out this info when it receives requests
For those also experiencing Unauthorized (invalid credentials) (failed to locate provided credentials)
and tracking down here:
I solved the issue by replacing the localhost
in ~/clearml.conf
with your actual IP address. This could be result from some firewall issues of the VM: https://github.com/allegroai/clearml/issues/158#issuecomment-653199211
SDK version: 1.9.0
Good luck!
FLASK_DEBUG: "1"
-> should 1 be a string or int?
Failed processing request
http://localhost:8008/auth.login : 'Logger' object is not callable
SuccessfulKoala55 do you mean request_handlers.py
` from argparse import ArgumentParser
from flask import Flask
from apiserver.config_repo import config
from apiserver.server_init.app_sequence import AppSequence
from apiserver.server_init.request_handlers import RequestHandlers
app = Flask(name, static_url_path="/static")
AppSequence(app).start(request_handlers=RequestHandlers())
=================== MAIN =======================
if name == "main":
p = ArgumentParser(description=doc)
p.add_argument(
"--port", "-p", type=int, default=config.get("apiserver.listen.port")
)
p.add_argument("--ip", "-i", type=str, default=config.get("apiserver.listen.ip"))
p.add_argument(
"--debug", action="store_true", default=config.get("apiserver.debug")
)
p.add_argument(
"--watch", action="store_true", default=config.get("apiserver.watch")
)
args = p.parse_args()
# logging.info("Starting API Server at %s:%s and env '%s'" % (args.ip, args.port, config.env))
app.run(
debug=args.debug,
host=args.ip,
port=args.port,
threaded=True,
use_reloader=args.watch,
) `
and contains these values:
username: user
password: local-b31829c6bc96dbaa4ab9a98b00abd0e79b7b9c67
This is not a pattern used by ClearML anywhere 😞
Also, if you have an anti-virus or something similar, I would turn that off first and try before going into changing the server's code