Reverse proxy instance health check (route_offline_thread)

Reverse proxy offline thread Web Server has a thread named route_offline_thread which does the health check of backend instances. Once this thread is started, it finds out which of the backend instances are alive. To find out whether a backend server is alive, it sends a OPTIONS HTTP request to the backend instance. If the server responds back then the server is alive. After the initial phase, the thread wakes up every 60 seconds to do this health check. The thread doesn't send any "OPTIONS" request to those instances which are already alive. It only tries to send "OPTIONS" request to those instances which are offline. This way thread doesn't impose any performance impact to the system and to backend instances. If a backend instance is shutdown or crashed for whatever reason, route_offline_thread will never know about it. Only way Web Server rpp knows about the death/hang of the backend is when an instance is chosen for serving a request. Rpp sends the request to backend instance and if connection to the instance is either broken then it will result in read/send failure. Web Server sets a response timeout which can be configure using http-client-config  ObjectType function in obj.conf.
ObjectType fn="http-client-config" timeout="400"

The above statement tells the rpp to close the connection to backend instance if a connection (read) hangs for more than 400 seconds. Default value is 300 seconds.  If that happends then rpp marks the instance as offline. After this time route_offline_thread will poll for this offline instance every 60 seconds (which is not configurable).  If backend instance recovers from hang or restarted and it responds back to "OPTIONS" HTTP request then route_offline_thread mark the instance back to online. Connect timeout for backend instance is 5 seconds (hardcoded). So if backend instance is hanging (or too busy) and unable to handle connection for next 5 seconds then rpp will treat the instance as offline. 



Comments:

Post a Comment:
Comments are closed for this entry.
About

Basant Kukreja

Search

Top Tags
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today