So the basic authentication header is different for the problematic call 🙂

Probably, sorry 😕

yeah

For those also experiencing Unauthorized (invalid credentials) (failed to locate provided credentials) and tracking down here:

I solved the issue by replacing the localhost in ~/clearml.conf with your actual IP address. This could be result from some firewall issues of the VM: https://github.com/allegroai/clearml/issues/158#issuecomment-653199211

SDK version: 1.9.0

Good luck!

should it be http://log.info (...)

bugs i understand are better than ones i dont lol

With pleasure 😊

We can try and patch the server to print out this info when it receives requests

This is not a pattern used by ClearML anywhere 😞

And the 1 should be a string

Failed processing request http://localhost:8008/auth.login : 'Logger' object is not callable

and contains these values:
username: user
password: local-b31829c6bc96dbaa4ab9a98b00abd0e79b7b9c67

i think the call is wrong

[2022-08-18 20:58:49,892] [9] [INFO] [clearml.request_handlers] GET [2022-08-18 20:58:49,895] [9] [INFO] [clearml.request_handlers] Basic NUtORklQWUIzM0tIVUFRNUI4SkM6eU4yUXE4RTNvWDlpYlBRVmF1NEpiQXZQMHlUdHozNUluZVY4ZkpuZXNTRnRYbmRlR3I= [2022-08-18 20:58:49,904] [9] [INFO] [clearml.service_repo] Returned 200 for auth.login in 6ms [2022-08-18 20:58:50,059] [9] [INFO] [clearml.request_handlers] GET [2022-08-18 20:58:50,060] [9] [INFO] [clearml.request_handlers] Basic NUtORklQWUIzM0tIVUFRNUI4SkM6eU4yUXE4RTNvWDlpYlBRVmF1NEpiQXZQMHlUdHozNUluZVY4ZkpuZXNTRnRYbmRlR3I= [2022-08-18 20:58:50,069] [9] [INFO] [clearml.service_repo] Returned 200 for auth.login in 6ms [2022-08-18 20:58:50,102] [9] [INFO] [clearml.request_handlers] GET [2022-08-18 20:58:50,104] [9] [INFO] [clearml.request_handlers] Basic NUtORklQWUIzM0tIVUFRNUI4SkM6eU4yUXE4RTNvWDlpYlBRVmF1NEpiQXZQMHlUdHozNUluZVY4ZkpuZXNTRnRYbmRlR3I= [2022-08-18 20:58:50,114] [9] [INFO] [clearml.service_repo] Returned 200 for auth.login in 8ms [2022-08-18 20:58:50,130] [9] [INFO] [clearml.request_handlers] GET [2022-08-18 20:58:50,132] [9] [INFO] [clearml.request_handlers] Basic dXNlcjpsb2NhbC1iMzE4MjljNmJjOTZkYmFhNGFiOWE5OGIwMGFiZDBlNzliN2I5YzY3 [2022-08-18 20:58:50,138] [9] [WARNING] [clearml.service_repo] Returned 401 for projects.get_all in 3ms, msg=Unauthorized (invalid credentials) (failed to locate provided credentials) [2022-08-18 20:58:50,145] [9] [INFO] [clearml.request_handlers] GET [2022-08-18 20:58:50,147] [9] [INFO] [clearml.request_handlers] Basic NUtORklQWUIzM0tIVUFRNUI4SkM6eU4yUXE4RTNvWDlpYlBRVmF1NEpiQXZQMHlUdHozNUluZVY4ZkpuZXNTRnRYbmRlR3I= [2022-08-18 20:58:50,159] [9] [INFO] [clearml.service_repo] Returned 200 for auth.login in 9ms [2022-08-18 20:58:50,173] [9] [INFO] [clearml.request_handlers] GET [2022-08-18 20:58:50,175] [9] [INFO] [clearml.request_handlers] Basic dXNlcjpsb2NhbC1iMzE4MjljNmJjOTZkYmFhNGFiOWE5OGIwMGFiZDBlNzliN2I5YzY3 [2022-08-18 20:58:50,181] [9] [WARNING] [clearml.service_repo] Returned 401 for projects.get_all in 3ms, msg=Unauthorized (invalid credentials) (failed to locate provided credentials)

i got these extra logs

If you want to try and patch the server code to get some visibility, I can guide you on how 🙂

sure where do i patch it?

Also, if you have an anti-virus or something similar, I would turn that off first and try before going into changing the server's code

FLASK_DEBUG: "1" -> should 1 be a string or int?

A bit hackish but it should work

` from argparse import ArgumentParser

from flask import Flask

from apiserver.config_repo import config
from apiserver.server_init.app_sequence import AppSequence
from apiserver.server_init.request_handlers import RequestHandlers

app = Flask(name, static_url_path="/static")
AppSequence(app).start(request_handlers=RequestHandlers())

=================== MAIN =======================

if name == "main":
p = ArgumentParser(description=doc)
p.add_argument(
"--port", "-p", type=int, default=config.get("apiserver.listen.port")
)
p.add_argument("--ip", "-i", type=str, default=config.get("apiserver.listen.ip"))
p.add_argument(
"--debug", action="store_true", default=config.get("apiserver.debug")
)
p.add_argument(
"--watch", action="store_true", default=config.get("apiserver.watch")
)
args = p.parse_args()

# logging.info("Starting API Server at %s:%s and env '%s'" % (args.ip, args.port, config.env))

app.run(
    debug=args.debug,
    host=args.ip,
    port=args.port,
    threaded=True,
    use_reloader=args.watch,
) `

Oh, sorry, yes 🙂

You basically do this:
In your docker-compose.yaml file, add the following env var to the apiserver service: FLASK_DEBUG: "1" You'll need to restart the server exec into the clearml-apiserver container CD into /opt/clearml/apiserver edit server.py Around line 35 right before call = self._create_api_call(request) , add:log(str(request.method)) log(str(request.headers["Authorization"]))5. Save the file
6. Try running your python code again and than check the clearml-apiserver docker container logs

i would greatly appreciate that!

since its a logger object

SuccessfulKoala55 thanks alot for your help i greatly appreciate it, i think you are right that there is something happening with the request, I did some checking i can get it running on another computer (my non work computer), so i think you are right there must be something happening with my work related computer which is causing this to occur 😕 I'll keep trying to drill down on this issue on my work computer to see what the difference is, but I can atleast proceed now to as i got experiments working on another computer 😄 ! Again thanks a lot for your help SuccessfulKoala55 and AgitatedDove14 i greatly appreciate it!

i see what you are saying I dont thinl there would be anything on my system which might do that i can investigate