Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I Have A Problem Of Showing Images And Artifacts In The Web Ui. We Get Unauthorized (401) When Loading The Plot Page Of A Task. It Almost Like

Hi, I have a problem of showing images and artifacts in the web ui.
we get Unauthorized (401) when loading the plot page of a task.
It almost like this issue , I tried to add the following to my docker-compose.yml:

services:
    webserver:
        environment:
	        WEBSERVER__fileBaseUrl: "
_domain/"
	        WEBSERVER__useFilesProxy: true

but it didn't help, I still get 401 and can't access the images.

We run it on prem, with a self signed certificate. we use a haproxy to expose the different services that runs on the machine:

webapp: clearml.mydoamin
api: clearmlapi.mydomain
fileserver: clearmlfs.mydomain

Thanks for any help!

  
  
Posted 28 days ago
Votes Newest

Answers 27


Unfortunately I have the same error, but just to make sure:
before this addition I only had the following in my apiserver.conf

auth {
    # Fixed users login credentials
    # No other user will be able to login
    fixed_users {
        enabled: true
        pass_hashed: true
        users: [
            {
                username: "user.name"
                password: "blabla"
                name: "User Name"
            },
        ]
    }
}

and now after the addition:

auth {
    # Fixed users login credentials
    # No other user will be able to login
    fixed_users {
        enabled: true
        pass_hashed: true
        users: [
            {
                username: "user.name"
                password: "blabla"
                name: "User Name"
            },
        ]
    }
    cookies {
    httponly: true
    secure: true
    domain: ".my.domain"
    max_age: 99999999999
  }
}

Now I noticed in this conf example that all is wrapped under a top {} which is not what I have.
So I can add those top {} , but before I do it I want to make sure my users will not be overrided or recreated..

  
  
Posted 26 days ago

I'm starting to remember, stuff we wrote a long time ago.
we intended the rewrite only for plot background images, because of the way plotly fetch images. on regular cases, like debug samples the cookie should be sent with requests to the files server because it's on the same sub domain as the api and the app.
can you make sure the cookie domain rule allows for this

  
  
Posted 26 days ago

can you please make sure the configuration got to the right place and see if you have the expected value for fileBaseUrl in

clearml.mydoamin/configuration.json
  
  
Posted 26 days ago

did you specify samesite: Lax ?

  
  
Posted 26 days ago

thank you so much

  
  
Posted 26 days ago

if WEBSERVER__fileBaseUrlvalue = None a replacement should take place, what version of clearml are you using?

  
  
Posted 26 days ago

it seems that config.skipFileServer : true and that's why he skips the convertToReverseProxy(url)) .
where can I change that value?

  
  
Posted 26 days ago

I think yes
image

  
  
Posted 26 days ago

as long as the 3 services are subdomains of .my.domain this should work

  
  
Posted 26 days ago

you can see the cookies on the applications/storage tab of devtools

  
  
Posted 26 days ago

WebApp: 2.1.0-664 • Server: 2.1.0-664 • API: 2.32

  
  
Posted 26 days ago

this is required for some browsers

  
  
Posted 26 days ago

yes

  
  
Posted 26 days ago

I mainly have problem with debug samples as now i manage to see some plots that were created with plt.imshow() . But I still don't see any kind of debug samples (images, audio, txt etc.) with the same 401 error.
So it's weird how some images from the file server can be accessed while other files can't.

  
  
Posted 26 days ago

for the plot background image to work the UI needs to rewrite the file path from clearmlfs.mydomain to clearml.mydoamin/files this is done based on WEBSERVER__fileBaseUrl value
can you check in the browser devtool, on the network tab if the rewrite happened

  
  
Posted 26 days ago

for reference this is the cookie we set on our community server
image

  
  
Posted 26 days ago

you will need to relogin for the new cookie with the updated info get registered

  
  
Posted 26 days ago

It works!!!

  
  
Posted 26 days ago

yes

  
  
Posted 26 days ago

When I ran the following code:

task = Task.init(project_name='DevOps/debug-subproject', task_name='test-task6')
task.logger.report_media(
    title='audio', 
    series='tada', 
    iteration=1,
    local_path='./audio-sample.mp3'
)
task.close()

and then in the UI when I look in developer tools I see that it tried to access:

  
  
Posted 26 days ago

do you mean I should add the following to /opt/clearml/config/apiserver.conf :

auth {
    cookies {
    httponly: true
    secure: true
    domain: ".my.domain"
    max_age: 99999999999
  }
}

based on these docs

  
  
Posted 26 days ago

no didn't use that,
Do you suggest the following?

    cookies {
    httponly: true
    secure: true
    domain: ".my.domain"
    max_age: 99999999999
    samesite: Lax

  }
  
  
Posted 26 days ago

these are hocon files, I don't think you need the surrounding curly braces

  
  
Posted 26 days ago

tried it, nothing has changed.

  
  
Posted 26 days ago

the fact you have X-Amz-Date in the url tells me the code passed the check here but might not pass this condition or something went wrong in the rewrite function
open source should include the source maps, can you try to debug?

  
  
Posted 26 days ago

great 😄

  
  
Posted 26 days ago

Hi @<1853608151669018624:profile|ColossalSquid53>
can you try without the last slash?

  
  
Posted 27 days ago
175 Views
27 Answers
28 days ago
26 days ago
Tags