Setup instructions for HAProxy with HTTP SSL termination on OpenWRT with CloudFlare Origin CA and Authenticated Origin Pulls.
I activated IPv6 on my Pi. Due to the absence of NAT, I needed to update CloudFlare with Pi’s new IPv6 every time it changes and not forgetting to update the white-list of CloudFlare IP addresses in firewall for each Pi.
To ease managing my network, I decided to setup HAProxy as a front-end load balancer on OpenWRT that:
Besides the above, I am also sharing tips and tricks. :)
This tutorial details steps I used for my setup. To follow this tutorial, you need a fair bit of understanding on:
In OpenWRT Chaos Calmer repository, you should be able to find haproxy
package, install it. At time of writing, the version available is 1.5.15-13
.
After installing, edit its configuration at /etc/haproxy.cfg
.
Add a front-end into the configuration file by appending the following lines. Please read the comments:
frontend https-in
# Bind on port 443 only, CloudFlare Page Rule has been setup to redirect HTTP to HTTPS
# cloudflare-origin-ca.pem = CloudFlare Origin CA certificate and key concatenated
# cloudflare-origin-pull-pem = CloudFlare Authenticated Origin Pulls certificate
bind :443 ssl strict-sni crt /etc/ssl/private/cloudflare-origin-ca.pem verify required ca-file /etc/ssl/private/cloudflare-origin-pull-ca.pem
mode http
# Allow CloudFlare IP addresses, deny others
# I have these commented because my firewall is enforcing this
#acl cloudflare_ipv4 src -f /etc/cloudflare/ips-v4
#acl cloudflare_ipv6 src -f /etc/cloudflare/ips-v6
#http-request allow if cloudflare_ipv4
#http-request allow if cloudflare_ipv6
#http-request deny
# Do not need this line since CloudFlare sends it and HAProxy will send all incoming headers to backend
#reqadd X-Forwarded-Proto:\ http
# HTTP Strict Transport Security
rspadd Strict-Transport-Security:\ max-age=15768000
# Define hostnames that we are serving
acl host_www_leowkahman_com hdr(host) -i www.leowkahman.com
acl host_static_leowkahman_com hdr(host) -i static.leowkahman.com
# Determine which backend to use
use_backend backend_raspberrypi if host_www_leowkahman_com
use_backend backend_raspberrypi if host_static_leowkahman_com
Next, add a backend:
backend backend_raspberrypi
mode http
balance leastconn
# Replace [raspberry IP] with your backend server's IP address
server raspberrypi [raspberry IP]:80 check
In case you are interested in seeing statistics, I suggest binding on a different port and one that is not exposed to the outside world:
listen stats
bind :1936
mode http
stats enable
stats uri /
stats hide-version
I hardened my SSL security by appending the following lines into global
section:
global
# set default parameters to the modern configuration
tune.ssl.default-dh-param 2048
ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
ssl-default-server-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
Next, add a default section so that I do not have to put them all over the place:
defaults
# Slowloris protection
timeout http-request 5s
timeout connect 5s
timeout client 30s
timeout server 30s
timeout http-keep-alive 4s
# Close the backend connection
option http-server-close
Do not start/restart the HAProxy yet.
You may want to refer to my post on how to setup HTTP server on Pi. Note that you do not need to setup SSL on it since SSL is terminated at the load balancer.
To configure the Pi firewall to allow inbound traffic from LAN subnet only, follow these steps:
UFW must be installed:
sudo apt-get install ufw
Ensure that IPv6 is enabled in /etc/default/ufw
, you should see IPV6=yes
.
Configure default rules; deny incoming, allow outgoing:
sudo ufw default deny incoming
sudo ufw default allow outgoing
Allow all incoming traffic from your LAN (change the subnet accordingly):
sudo ufw allow from 192.168.0.0/24
Verify list of rules:
sudo ufw status
Enable UFW:
sudo ufw enable
If you are currently forwarding port 443 to your backend web server, you need to remove the rule because HAProxy needs to listen on that port.
By default, OpenWRT denies all incoming connections unless you have explicitly allowed the traffic through in your firewall configuration. I did not want to open that port to all inbound traffic. Instead, limiting to CloudFlare IP addresses only.
To accomplish this, I needed to create bash script download the latest CloudFlare IPv4 and IPv6 lists and subsequently load it into memory to be used by the firewall.
Create a folder to store the script and IP address lists:
mkdir -p /etc/cloudflare
Use your favourite text editor, I use nano
to create a bash script at /etc/cloudflare/cloudflare-whitelist.sh
with the following lines:
#!/bin/sh
set -e
# Pull the latest CloudFlare IP list into memory and overwrite existing copy
# only when changed to minimise non-volatile storage wear
mkdir -p /var/run/cloudflare
wget https://www.cloudflare.com/ips-v4 -O /var/run/cloudflare/ips-v4
wget https://www.cloudflare.com/ips-v6 -O /var/run/cloudflare/ips-v6
rsync --size-only /var/run/cloudflare/ips-v* /etc/cloudflare/
rm /var/run/cloudflare/ips-v*
# Create hash sets to store list of IP
ipset create -exist cloudflareipv4s hash:net family inet
ipset create -exist cloudflareipv6s hash:net family inet6
for cfip in $(cat /etc/cloudflare/ips-v4); do ipset add -exist cloudflareipv4s $cfip; done
for cfip in $(cat /etc/cloudflare/ips-v6); do ipset add -exist cloudflareipv6s $cfip; done
Give it permissions to run:
chmod 755 /etc/cloudflare/cloudflare-whitelist.sh
Try to run it:
/etc/cloudflare/cloudflare-whitelist.sh
Upon successful run, there should be two new files in the same folder:
ips-v4
ips-v6
If you run into issues, you are probably missing some packages that I use. If so, please install them (i.e. wget
, rsync
, ca-certificates
, iptables
, ip6tables
, ipset
) or you will have to find another way around.
Schedule this script to run weekly by putting into Cron:
# Every week, update CloudFlare IP white list
0 0 * * 1 /etc/cloudflare/cloudflare-whitelist.sh
In Firewall custom rules, add the following lines:
# Allow inbound HTTPS from CloudFlare IPs obtained from hash sets
ipset create -exist cloudflareipv4s hash:net family inet
ipset create -exist cloudflareipv6s hash:net family inet6
for cfip in $(cat /etc/cloudflare/ips-v4); do ipset add -exist cloudflareipv4s $cfip; done
for cfip in $(cat /etc/cloudflare/ips-v6); do ipset add -exist cloudflareipv6s $cfip; done
iptables -A INPUT -m set --match-set cloudflareipv4s src -p tcp -m multiport --dport 443 -j ACCEPT
ip6tables -A INPUT -m set --match-set cloudflareipv6s src -p tcp -m multiport --dport 443 -j ACCEPT
For firewall rules to take effect:
/etc/init.d/firewall restart
Previously, I had my Pi serving SSL traffic. Now with SSL terminated at load balancer, WordPress on my Pi thinks that the incoming visitor is browsing without SSL thus giving http://
link prefixes. To correct this, I added the following lines to the top of wp-config.php
:
define('FORCE_SSL_ADMIN', true);
if (strpos($_SERVER['HTTP_X_FORWARDED_PROTO'], 'https') !== false) {
$_SERVER['HTTPS']='on';
$_SERVER['SERVER_PORT'] = 443;
}
Since I am using Dynamic IP address, it will change from time to time. CloudFlare must be notified of my IP changes so I have configured as below.
Refer to my post on how to update DNS on CloudFlare. Note that the script on OpenWRT repository sets CloudFlare cloud colour to grey. It is important that you refer to this link for instructions on how to make it set to orange.
In addition to that, I have added a IPv6 DDNS entry:
config service 'cloudflare_ipv6_leowkahman_com'
option enabled '1'
option use_ipv6 '1'
option service_name 'CloudFlare'
option domain 'leowkahman.com'
option username '[CloudFlare username]'
option password '[CloudFlare API key]'
option use_https '1'
option cacert '/etc/ssl/certs'
option ip_source 'network'
option interface 'wan6'
option ip_network 'wan6'
option use_syslog '2'
option use_logfile '0'
option check_interval '24'
option check_unit 'hours'
option force_interval '0'
Restart HAProxy:
/etc/init.d/haproxy restart
You should be good to go. :)
Now that I have a load balancer in place, I have the pre-requisites for adding more backend nodes in future.