Install and Setup HAProxy 1.6 on Ubuntu 14.04

Posted on Thursday, May 19, 2016




How to install HAProxy 1.6 on Ubuntu 14.04.  (And start playing with it)

First what is HAProxy and why would you want it?

From Wikipedia

HAProxy is free, open source software that provides a high availability load balancer and proxy server for TCP and HTTP-based applications that spreads requests across multiple servers.[2] It is written in C[3] and has a reputation for being fast and efficient (in terms of processor and memory usage).





So in short a free, open source, highly reliable load balancer.



My Test set up goal


My goal for this test set up is to have an HAProxy box sitting in front of two nginx boxes listening on port 80.



Something like this.  I want to HAProxy to balance the load (traffic) between the two nginx boxes sitting behind it.

With that in mind let me first roll two nginx boxes that listen on port 80.






Nginx box 1 setup



  > sudo apt-get -y install nginx


In a default install nginx listens on port 80 and the root directory is /usr/share/nginx/html

Edit the base html file so we know which server we are hitting.


  > sudo vi /usr/share/nginx/html/index.html


And place the following in it



<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
    span {
        color: red;
    }
    h1 {
        font-size: 75px;
    }
</style>
</head>
<body>
<center>
<h1>This is the nginx server <span>01</span></h1>
</center>
</body>
</html>







Nginx box 2 setup



  > sudo apt-get -y install nginx


In a default install nginx listens on port 80 and the root directory is /usr/share/nginx/html

Edit the base html file so we know which server we are hitting.


  > sudo vi /usr/share/nginx/html/index.html


And place the following in it



<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
    span {
        color: red;
    }
    h1 {
        font-size: 75px;
    }
</style>
</head>
<body>
<center>
<h1>This is the nginx server <span>02</span></h1>
</center>
</body>
</html>








HAProxy install and set up


Now that I have two nginx boxes set up to load balance between let me install and set up HAProxy on my third box.

Doing a quick test to see which version apt-get would install…


  > apt-cache policy haproxy




Looks like it will install 1.4.24, but I want to install the latest 1.6.5

Looking around I found this site showing how to install HAProxy 1.5 https://www.vultr.com/docs/installing-and-configuring-haproxy-on-ubuntu-14-04 [2]  I am going to tweak what they did to install 1.6



Let me add a repository so I can install the newer version.


  > sudo add-apt-repository ppa:vbernat/haproxy-1.6
  > sudo apt-get update
  > sudo apt-get dist-upgrade



Now Install HAProxy



  > sudo apt-get install haproxy


Now tweak the configurations.





Configuration Getting it to load balance…


Open up the HAProxy conf file at /etc/haproxy/haproxy.cfg


  > sudo vi /etc/haproxy/haproxy.cfg


Update it to the following


global
  log         127.0.0.1 syslog
  maxconn     1000
  user        haproxy
  group       haproxy
  daemon

defaults
  log  global
  mode  http
  option  httplog
  option  dontlognull
  option  http-server-close
  option  forwardfor except 127.0.0.0/8
  option  redispatch
  option  contstats
  retries  3
  timeout  http-request 10s
  timeout  queue 1m
  timeout  connect 10s
  timeout  client 1m
  timeout  server 1m
  timeout  check 10s

listen  http_proxy
  bind        *:80
  balance     roundrobin
  server      server1 192.168.0.10:80 maxconn 100
  server      server2 192.168.0.11:80 maxconn 100



Now restart the haproxy service


  > sudo service haproxy restart






Test





Refresh it a few times and watch it go back and forth.




That was a very simple basic example. Let me see if I can do a few more advanced things.





Add HAProxy page


Edit the conf file again adding an Haproxy stats page



  > sudo vi /etc/haproxy/haproxy.cfg


Update it to the following


global
  log         127.0.0.1 syslog
  maxconn     1000
  user        haproxy
  group       haproxy
  daemon

defaults
  log  global
  mode  http
  option  httplog
  option  dontlognull
  option  http-server-close
  option  forwardfor except 127.0.0.0/8
  option  redispatch
  option  contstats
  retries  3
  timeout  http-request 10s
  timeout  queue 1m
  timeout  connect 10s
  timeout  client 1m
  timeout  server 1m
  timeout  check 10s


listen stats
  bind *:8080
  mode  http
  maxconn  10
  stats  enable
  stats  hide-version
  stats  realm Haproxy\ Statistics
  stats  uri /
  stats  auth admin:mypassword


listen  http_proxy
  bind        *:80
  balance     roundrobin
  server      server1 192.168.0.10:80 maxconn 100
  server      server2 192.168.0.11:80 maxconn 100





Now restart the haproxy service


  > sudo service haproxy restart




(login with admin:mypassword)




Now I can see some detailed information on my how my load balancer is doing.




Health Checking



HAProxy is smart enough to stop routing to unhealthy boxes.   For example if I kill nginx on my first nginx box.


  > sudo service nginx stop


Now if I check http://192.168.0.9/

It only shows 02 box





If I look at the stats page



I get no information about it being down.

I can configure a health check and force it to check against a URL to determine if the machine should be part of the load balance.

This is a very useful feature in a live system.  (You can take a machine out of rotation and work on it or replace it)


Edit the conf file again adding an Haproxy stats page



  > sudo vi /etc/haproxy/haproxy.cfg


Update it to the following


global
  log         127.0.0.1 syslog
  maxconn     1000
  user        haproxy
  group       haproxy
  daemon

defaults
  log  global
  mode  http
  option  httplog
  option  dontlognull
  option  http-server-close
  option  forwardfor except 127.0.0.0/8
  option  redispatch
  option  contstats
  retries  3
  timeout  http-request 10s
  timeout  queue 1m
  timeout  connect 10s
  timeout  client 1m
  timeout  server 1m
  timeout  check 10s


listen stats
  bind *:8080
  mode  http
  maxconn  10
  stats  enable
  stats  hide-version
  stats  realm Haproxy\ Statistics
  stats  uri /
  stats  auth admin:mypassword


listen  http_proxy
  bind        *:80
  balance     roundrobin
  option      httpchk GET /check
  server      server1 192.168.0.10:80 maxconn 100 check rise 2 fall 2
  server      server2 192.168.0.11:80 maxconn 100 check rise 2 fall 2



Now restart the haproxy service


  > sudo service haproxy restart








You should now see that they are both down and trying to open up http://192.168.0.9/



Give a 503, since all load balanced servers are offline

So what is going on in the code?


  option      httpchk GET /check


This part sets the check and tells it to use GET and to use the URL /check.




  server      server1 192.168.0.10:80 maxconn 100 check rise 2 fall 2
  server      server2 192.168.0.11:80 maxconn 100 check rise 2 fall 2


This part says use the check and fall 2 is the number of consecutive checks that have to fail before the server is considered down.  Rise 2 is the number of consecutive checks the server has to pass before it is considered up.

So how often does it check?  The default is 2 seconds. See https://www.haproxy.com/doc/aloha/7.0/haproxy/healthchecks.html#check-interval [4]

How do I bring the servers back?   I simply need a reply when I go to this URL http://192.168.0.10/check
So since these are simple nginx servers I can make a file there.

Let me just make a file on the first nginx box and bring that one back up.


  > sudo touch /usr/share/nginx/html/check





And the one server is back up!

Let me run the same command on the second machine.






Health Check 2


What if you want a smarter health check that can check against a returned string?

Let me check for the string "UP"


  > sudo vi /etc/haproxy/haproxy.cfg


Update it to the following


global
  log         127.0.0.1 syslog
  maxconn     1000
  user        haproxy
  group       haproxy
  daemon

defaults
  log  global
  mode  http
  option  httplog
  option  dontlognull
  option  http-server-close
  option  forwardfor except 127.0.0.0/8
  option  redispatch
  option  contstats
  retries  3
  timeout  http-request 10s
  timeout  queue 1m
  timeout  connect 10s
  timeout  client 1m
  timeout  server 1m
  timeout  check 10s


listen stats
  bind *:8080
  mode  http
  maxconn  10
  stats  enable
  stats  hide-version
  stats  realm Haproxy\ Statistics
  stats  uri /
  stats  auth admin:mypassword


listen  http_proxy
  bind           *:80
  balance        roundrobin
  option         httpchk GET /check
  http-check     expect string UP
  default-server inter 3s fall 3 rise 2
  server         server1 192.168.0.10:80 maxconn 100 check
  server         server2 192.168.0.11:80 maxconn 100 check



Now restart the haproxy service


  > sudo service haproxy restart







  http-check     expect string UP
  default-server inter 3s fall 3 rise 2


Now it should check for the string UP and also I pulled out the fall and rise per server into a default-server setting.




Now it's failing again.  Even though we do have a file at /check it is empty.

Let me go update those files




  > sudo su root -c 'echo "UP" > /usr/share/nginx/html/check'




And we are back up again.


Note:

The expect string seems to search the entire text for a match so

UP
it is UP
wow look UP there

all pass


There is another option you could use rstring (for regular expressions)


listen  http_proxy
  bind           *:80
  balance        roundrobin
  option         httpchk GET /check
  http-check  expect rstring ^UP$
  default-server inter 3s fall 3 rise 2
  server         server1 192.168.0.10:80 maxconn 100 check
  server         server2 192.168.0.11:80 maxconn 100 check


That will only match UP.





Frontend Backend


Looking around at a few configurations I see listen, frontend, and backend.  What is the difference between them all.


From the document


    Frontend:


              A "frontend" section describes a set of listening sockets accepting client connections.

    Backend:


            A "backend" section describes a set of servers to which the proxy will connect to forward                    incoming connections.

    Listen:


         A "listen" section defines a complete proxy with its frontend and backend parts combined in one          section. It is generally useful for TCP-only traffic.



OK so for my purposes using a frontend section and a backend section just separates out the responsibilities and I thin makes it a littler clearer as to what is going on.

I am going to edit my file to use a frontend and backend



  > sudo vi /etc/haproxy/haproxy.cfg


Update it to the following


global
  log         127.0.0.1 syslog
  maxconn     1000
  user        haproxy
  group       haproxy
  daemon
 
defaults
  log  global
  mode  http
  option  httplog
  option  dontlognull
  option  http-server-close
  option  forwardfor except 127.0.0.0/8
  option  redispatch
  option  contstats
  retries  3
  timeout  http-request 10s
  timeout  queue 1m
  timeout  connect 10s
  timeout  client 1m
  timeout  server 1m
  timeout  check 10s


###########################################
#
# HAProxy Stats page
#
###########################################
listen stats
  bind *:8080
  mode  http
  maxconn  10
  stats  enable
  stats  hide-version
  stats  realm Haproxy\ Statistics
  stats  uri /
  stats  auth admin:mypassword

###########################################
#
# Front end for www
#
###########################################
frontend www
  bind   *:80
  mode   http
  default_backend  www

###########################################
#
# Back end for www
#
###########################################
backend www
  balance         roundrobin
  option          httpchk GET /check
  http-check      expect string UP
  default-server  inter 3s fall 3 rise 2
  server         server1 192.168.0.10:80 maxconn 100 check
  server         server2 192.168.0.11:80 maxconn 100 check



Now restart the haproxy service


  > sudo service haproxy restart








I can see I know have a two section www for frontend and www for backend www.



As a quick test I removed the check file on one nginx server.


  > sudo rm /usr/share/nginx/html/check




Worked!





Sticky Sessions


What if I am in a situation that I need to keep a session attached to the same server?   In other words when someone hits my URL for the first time I do not care which server they hit.   But when they hit I keep track of which box they hit and keep them talking to that box.  So it if hits nginx box 2 it will keep hitting that same box for all requests.

Of course I would probably want that stickiness to disappear the next day (I don’t need them to hit the same server forever just during their 'session')



Here is a link to some documentation on cookies in haproxy http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#4.2-cookie [7]

Here is my first go at it.





  > sudo vi /etc/haproxy/haproxy.cfg


Update it to the following


global
  log         127.0.0.1 syslog
  maxconn     1000
  user        haproxy
  group       haproxy
  daemon
 
defaults
  log  global
  mode  http
  option  httplog
  option  dontlognull
  option  http-server-close
  option  forwardfor except 127.0.0.0/8
  option  redispatch
  option  contstats
  retries  3
  timeout  http-request 10s
  timeout  queue 1m
  timeout  connect 10s
  timeout  client 1m
  timeout  server 1m
  timeout  check 10s


###########################################
#
# HAProxy Stats page
#
###########################################
listen stats
  bind *:8080
  mode  http
  maxconn  10
  stats  enable
  stats  hide-version
  stats  realm Haproxy\ Statistics
  stats  uri /
  stats  auth admin:mypassword

###########################################
#
# Front end for www
#
###########################################
frontend www
  bind   *:80
  mode   http
  default_backend  www

###########################################
#
# Back end for www
#
###########################################
backend www
  balance         roundrobin
  cookie          SERVERID insert indirect nocache
  option          httpchk GET /check
  http-check      expect string UP
  default-server  inter 3s fall 3 rise 2
  server          server1 192.168.0.10:80 maxconn 100 check cookie nginx-01
  server          server2 192.168.0.11:80 maxconn 100 check cookie nginx-02


Looking over it



cookie          SERVERID insert indirect nocache


Just says insert a cookie named "SERVERID"



  server          server1 192.168.0.10:80 maxconn 100 check cookie nginx-01
  server          server2 192.168.0.11:80 maxconn 100 check cookie nginx-02


This says if route to this server and you are putting in a cookie put the following in the text of the cookie.   So for server1 it would put nginx-01



Now restart the haproxy service


  > sudo service haproxy restart





If you don't have the chrome plugin editthiscookie go and download it so you can easily look at your cookies https://chrome.google.com/webstore/detail/editthiscookie/fngmhnnpilhplaeedifhccceomclgfbg?hl=en









When you have it you should see this little cookie icon.









Reload it several times to see that you stick to the same server.



Now click on the cookie tool



There is the cookie name SERVERID and the value nginx-02.


Now as is I think, as long as someone does not wipe their cookies, they will be routed to the same server today, tomorrow and forever….  I do not like that idea so I tweaked mine a bit.

Here is my tweaked version


global
  log         127.0.0.1 syslog
  maxconn     1000
  user        haproxy
  group       haproxy
  daemon
 
defaults
  log  global
  mode  http
  option  httplog
  option  dontlognull
  option  http-server-close
  option  forwardfor except 127.0.0.0/8
  option  redispatch
  option  contstats
  retries  3
  timeout  http-request 10s
  timeout  queue 1m
  timeout  connect 10s
  timeout  client 1m
  timeout  server 1m
  timeout  check 10s


###########################################
#
# HAProxy Stats page
#
###########################################
listen stats
  bind *:8080
  mode  http
  maxconn  10
  stats  enable
  stats  hide-version
  stats  realm Haproxy\ Statistics
  stats  uri /
  stats  auth admin:mypassword

###########################################
#
# Front end for www
#
###########################################
frontend www
  bind   *:80
  mode   http
  default_backend  www

###########################################
#
# Back end for www
#
###########################################
backend www
  balance         roundrobin
  cookie          SERVERID insert indirect nocache maxidle 30m maxlife 8h
  option          httpchk GET /check
  http-check      expect string UP
  default-server  inter 3s fall 3 rise 2
  server          server1 192.168.0.10:80 maxconn 100 check cookie nginx-01
  server          server2 192.168.0.11:80 maxconn 100 check cookie nginx-02




That should at least reset it every day



Looks like it stores the data here rather than expire the cookie.


It would take a while to test this out so I am going to change the times to 1m and 2m and run some test


  cookie          SERVERID insert indirect nocache maxidle 1m maxlife 2m



Now restart the haproxy service


  > sudo service haproxy restart



I ran several tests where I waited over 1 min to reload the page and I was able to get the other server.   If I hit reload before the 1 min had elapsed I stayed on the server I was on.

If I kept hitting refresh, eventually after two minutes, I switched to the other server.






Last test


If I have a sticky session and the server goes down to I get routed to the other server?


Let me bump my time outs back to 30m and 8 hours

Reload the page



Now go to this box and remove the check file




  > sudo rm /usr/share/nginx/html/check






Confirm it is down.






Confirm the cookie points to nginx-01

Now reload the page




Yep made the jump no hesitiation.




And now my cookie as nginx-02




Wrap up


That gets me pretty far with what I want to do.  I am sure there are a lot more neat little tweaks I will learn over the years but for now that is more than enough to get me going.

Next I am going to do another write up trying to figure out how to add an SSL certificate.




References


[1]        HAProxy Wikipedia page
Accessed 05/2016
[2]        Installing HAProxy on Ubuntu 14.04
[3]        HAProxy Health Checking
[4]        HAProxy Health Check Interval
[5]        HAProxy Configuration Doc sec 4. Proxies
[6]        LOAD BALANCING, AFFINITY, PERSISTENCE, STICKY SESSIONS: WHAT YOU NEED TO KNOW
[7]        HAProxy Configuration Manual sec 4.2 cookie



No comments:

Post a Comment