Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey Hey, I Having Trouble With Clearml And Albs In The Aws. Could Someone Help Me?

Hey hey, I having trouble with ClearML and ALBs in the AWS. Could someone help me? πŸ™‚

I am currently trying to deploy ClearML in the AWS. The Basic Infrastructure has an Application Load Balancer (ALB) and an Autoscaling Group that launches a ClearML AMI. I followed the instructions described in https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_config/#sub-domains-and-load-balancers for setting up the ALB.

The steps I did are:
Edit /opt/clearml/config/apiserver.conf and added the domain of associated with my ALB via Route53 Create one HTTPS listener with host_header conditions ( app. , api. , files. ) that point to the respective target groups Created 3 HTTP Target Groups ( app. , api. , files. ) for the appropriate ports (8080, 8008, 8081) that target the ClearML server in the Autoscaling Group Double checked that the security groups allow access (web -> ALB and ALB -> Instance) Restarted the ClearML Server
Calling the app.<mydomain>.com (or api. or files. ) adress results in a 504 Gateway Time-out error after 10 seconds and all default health checks in the Target Groups are failing due to Request timed out .

Any idea how I could debug where the problem is? Thanks a lot πŸ™‚

  
  
Posted 11 months ago
Votes Newest

Answers 30


can you change the path in ALB healthcheck pls?

  
  
Posted 11 months ago

This gives me a 200 πŸ™‚

  
  
Posted 11 months ago

doubled copy paste

  
  
Posted 11 months ago

the goal is to get healthchecks green so ALB should be able to work

  
  
Posted 11 months ago

These are the seetings for health check now

  
  
Posted 11 months ago

in some second it should became green

  
  
Posted 11 months ago

from / to /debug.ping

  
  
Posted 11 months ago

ops

  
  
Posted 11 months ago

Ok, I think that's been very helpful πŸ™‚ I'll experiment a little, now that I know a Health Check that must work. I'll write here if I find something! Thanks a lot for the awesome support!

  
  
Posted 11 months ago

And it's still unhealthy. I am starting to suspect that somehow the Autoscaling Part in between the ALB and the ClearML server could be causing the problem.

  
  
Posted 11 months ago

usually you can see if you are getting timeouts or wrong http code

  
  
Posted 11 months ago

it’s alongside health checks tab

  
  
Posted 11 months ago

look also at the monitoring tab

  
  
Posted 11 months ago

But I still have one thing I'd like to fix: the health check for the file server on port 8081 gives me unhealthy for path "/". Is there a valid path you know I can use there for health checks? A curl gives me

  
  
Posted 11 months ago

Currently I'm "cheating" and counting a 405 as the success code for the healthcheck.

  
  
Posted 11 months ago

In fact it's the same we are applying to helm charts for k8s

  
  
Posted 11 months ago

Can you pls share all 3 health checks ?

  
  
Posted 11 months ago

I'm going to ask an update to docs

  
  
Posted 11 months ago

atm it’s the way to go

  
  
Posted 11 months ago

You are not cheating πŸ˜‚

  
  
Posted 11 months ago

it can help debugging

  
  
Posted 11 months ago

And I could access the web server even if the health check was failing. So that was not a problem in the end.

  
  
Posted 11 months ago

Just to be sure we are in sync 😁

  
  
Posted 11 months ago

ok, ty very much for your feedback πŸ˜„

  
  
Posted 11 months ago

File Server

  
  
Posted 11 months ago

Web Server

  
  
Posted 11 months ago

Yes!

  
  
Posted 11 months ago

API

  
  
Posted 11 months ago

Thanks a lot for the help debugging!

  
  
Posted 11 months ago

JuicyFox94 I think I found the problem. To my absolute shame, the security group of the ALB had no Outbound rules, i.e. no traffic was allowed out of the ALB πŸ™ˆ . Now I can access the ClearML Webserver!

  
  
Posted 11 months ago