You’ve reached that point where a single server can no longer sustain the traffic that you receive. Perhaps you’ve heard about the ability to scale your service to multiple servers. In this tutorial we’ll go thorough the basics of load-balancing and how to use nginx’s reverse proxy functionality to provide your users with content across multiple servers without having to worry about manually directing them to the appropriate server. You can then offer your users a stable experience and higher fault-tolerance & redundancy.

A Primer to Load Balancing and Reverse Proxy

When we talk about a proxy server, we’re describing a server that performs some action on behalf of some other client. If you’ve heard of proxy forwarding before, you’ll know that it allows one to fetch resources and data from a website via a proxy server. For example, consider the situation where you wish to download a file from a server from a server in Latvia but the administrator of that server doesn’t allow connections from outside Europe. If you own a server in Europe, you could use that server as a proxy server to download the resources to your server in Europe, and then download those resources to your local computer with no restriction.

Now consider a different scenario where you have a number of servers each with their own IP address. If each server is responding with almost the exact same content, it improves the user experience to have one name to identify your service and routing the user to the appropriate server. This is known as a reverse proxy. The idea is that you configure a server as a reverse proxy and allow your domain name to point to that server. The reverse proxy server will handle all requests and direct them to the appropriate server according to a number of different methods. If you’re primarily trying to manage a high volume of users, a round-robin scheduling algorithm (route users to each server one by one to distribute the load evenly) will work well for you. There are also other methods of distributing incoming requests to your servers that you can look into.

We’ll be using nginx as our reverse proxy software. Keep in mind that although we’re using nginx to set up load balancing, we do not require that our other servers (the ones to which our proxy server to redirecting users) run nginx. They could be apache servers, node.js applications listening on a certain port, and even database servers.

Now that we’ve got a good handle on what load balancing is, let’s get started.

Installing and Configuring nginx

We’ll first update our package repositories and install nginx by running the following commands as a superuser. Keep in mind that we perform all these actions on the server we wish to use as a load balancer

sudo apt-get update && sudo apt-get install nginx

We now need to configure our nginx server to pass all requests to our load balancer to our application servers. For illustration purposes, suppose I own the domain ‘myServer.com’ and that I’ve pointed that domain name to my nginx load balancer. Using your favourite command-line text editor, open up your nginx server’s configuration file. For illustration’s purposes I’ll use the default file provided by nginx.

sudo vim /etc/nginx/sites-available/default

We need to add an upstream module. This is a named group of servers you wish to direct certain traffic to. For example, if a had a three Stacks that serve videos on video1.videoservice.com, video2.videoservice.com, and video3.videoservice.com, I would want to logically group these servers together under the name videoServers. I could add the following directive to my nginx configuration file:

upstream videoServers {
	server video1.videoservice.com/id/
	server video2.videoservice.com/id/
	server video3.videoservice.com/id/
}

A simple way to think of upstream modules is to think of which way you wish content to flow down your chain of connections. Video data is upstream and is allowing data to flow into your load balancer, eventually being delivered to the client.

Once we have an upstream module defined, we can reference it in our load balancer configuration.

server {
	location /video {
		proxy_pass  http://videoServers;
	}
	
	location / {
		try_files $uri =404;
	}
	
}

What we’ve done here is passed all requests to routes with the prefix /video to our upstream videoServers module. For example, if someone sends a request to http://myServer.com/video/10012, the load balancer would pass on that request to one of the servers in the videoServers module with the appropriate URI. So, if the request was passed to the second video server, the client would be routed to http://video2.videoservice.com/id/10012. Clearly this is valuable if you are serving raw content to users via a web API or even interacting with an internal API.

Different Scheduling Methods

Round robin scheduling is not the only type of routing algorithm. You can choose from a number of methods by which load is dsitributed across servers. Following the same example above, if you have three video servers where the first and second servers can handle 10 and 5 times more traffic than the third, respectively, you could modify your configuration file as follows:

upstream videoServers {
	server video1.videoservice.com/id/ weight = 10;
	server video2.videoservice.com/id/ weight = 5;
	server video3.videoservice.com/id/ weight = 1;
}

In this configuration, for every 1 request video3.videoservice.com receives, 5 will be sent to video2 and 10 to video1.

Alternatively, you could implement least-connected load balancing. In least-connected load balancing, the server avoids sending requests to very busy servers by distributing new requests to less busy servers. You can activate least-connected load balancing by simply adding the least_conn directive in the upstream module block.

upstream videoServers {
	least_conn;
	server video1.videoservice.com/id/;
	server video2.videoservice.com/id/;
	server video3.videoservice.com/id/;
}

Finally, it might be beneficial to route subsequent requests by a client to the same server every time. For example, if you have clients that upload data to your server and will want to retrieve it at some point, it would be helpful to route the user to the server to which they uploaded their data. We can use IP hashing by simply adding the ip_hash directive to the top of the upstream block.

upstream videoServers {
	ip_hash;
	server video1.videoservice.com/id/;
	server video2.videoservice.com/id/;
	server video3.videoservice.com/id/;
}

Final Words

That’s it! You have all the necessary tools to scale your application horizontally by adding as many servers as you need. You can look further into topics regarding reverse proxies and nginx for even more use cases such as SSL proxies. Happy stacking!