Introduction to Proxies



Introduction

This week I worked on adding a network proxy to our backend architecture. Today we will talk about what a network proxy is, why it is useful, and how we built ours.

What even is a proxy?

A proxy is a service that sits in front of another computer and forwards network messages on behalf of that computer. Similar to how a proxy voter might vote on behalf of a person - a proxy server sends and receives network traffic on behalf of another server.

Proxy servers are typically categorized into two types:

  1. Forward Proxies - Operates on behalf of a client
  2. Reverse Proxies - Operates on behalf of a server

Forward Proxies

Forward proxies exist to separate a particular set of clients from the rest of the internet. Services such as a Virtual Private Network (or VPN) is a good example of a forward proxy. Also, when you’re at work and you can’t access your favorite MMO tutorial series, that’s probably a forward proxy blocking you from YouTube.

Reverse Proxies

Reverse proxies are the more typical type of proxy that we will see when working with backend services (Such as our game server). For almost every backend server that does real work - there’s a proxy (or in some cases multiple tiers of proxies) in front of it that manages the network traffic. In our case we will be constructing a reverse proxy, because our goal is to improve the security, availability, and latency of the MMO service that we provide to our players.

Prior Art

For a few common examples of reverse proxies, you don’t have to look very far.

  1. HAProxy
  2. NGINX
  3. HTTPD

If you read about any of the above proxies, then you may have noticed that the common case for using a proxy is for an HTTP/HTTPS service. Since our game isn’t strictly built on HTTP requests and responses, we can’t leverage many of the useful tools that these proxies provide. That said, if we build our own proxy then we can add custom tools that suit our needs.

Why do we need a proxy?

Whenever we want to build large scale distributed systems, we have to be wary of adding too much complexity. It’s really easy to stick another service over here and fall into situations where we can’t manage the complexity that we created. Additionally, with every new computer we add to our stack, we might introduce another point of failure.
Figure 1: A simple example of multiple clients connecting to a Game Server via a Proxy

As you can see in the figure above, the clients have no knowledge about the actual Game Server (or clusters of GameServers) running in the background. This protects us in a few ways and also provides us the ability to offload some of our non-gameplay related code to our proxy service (thus offloading some of the work). Let’s go through a few advantages and disadvantages of adding a proxy:

  1. Organizational Advantages
  2. Scalability Advantages
  3. Security Advantages
  4. Availability Advantages
  5. Latency Tradeoff

1. Organizational Advantages

The core “service” that we want to provide is a video game. At its most essential level, that’s just a networked physics/world simulation that multiple clients are taking part in. In order to accomplish that feat, there is a large amount of “bookkeeping” that must go on. Clients can connect and disconnect sporadically, and they may even login and logout repeatedly. It would be nice to have some sort of “contract” with our proxy so that our proxy can handle all of that. This way, our game server could worry about the thing it does best: simulating our game. So we can move all of the thread management, connection requests, timeouts, authorization, and authentication work into our proxy. This will help us mentally manage “who does what”.

2. Scalability Advantages

As mentioned in the previous section, moving connections, authorizations, and authentications will reduce strain on our game server. But we can push it a bit further, we can also perform network packet compression, encryption/decryption, and maybe even caching on our proxy. The largest advantage of a proxy server, though, is the fact that it is easier to scale horizontally. This is because there is a very small amount of state held on the proxy server (contrast that to our game server which holds the entire state of our game). Because of this, if we write our proxy server with horizontal scalability in mind, then we can scale to more users by simply launching more nodes in a cluster.

3. Security Advantages

Because clients must access our Game Server by connecting to our proxy, external attackers have no way to directly attack our Game Server. So if an attacker were to mount a DDOS attack against us, they would have to mount it against our proxy, not our game server. Because our proxy is easily scaled, we can counter that attack by launching more proxies to distribute the load.

The fact that our proxy is the only server available to the internet lets us categorize our other servers as “internal”. This can help us maintain a clear network separation so that we can prevent any misconfigurations in our networking layout.

4. Availability Advantages

There is a bit of a tradeoff with availability: On one hand we are adding more servers which can lead to more failure cases (causing a loss in availability); but on the other hand, separating our proxy from our game server lets us easily restart proxies. This would cause clients to temporarily reconnect to a different proxy, but not lose any of their game state on the main game server. I’m interested in exploring this topic more as our backend architecture develops.

5. Latency Tradeoff

Obviously, adding another computer in between our client and server will cause additional latency. I’m interested in investigating how much latency will be added to every request. My hope is that this number will remain <1ms, even when deployed. My assumption for now is that the latency will be small enough to not outweigh any of the aforementioned benefits of having a proxy. There may also be some chance to improve latency by doing some caching, but I haven’t pursued this yet.

Design and Implementation

Right now we handle two message types:

  1. Each client will send its Input to the server
  2. The server will send a WorldUpdate to each client

As shown below, the architecture for this is basically the same as a client connecting directly to a server, except that there is now a proxy fielding the connection. To implement this proxy, there is a thread ClientToServer which passes messages from a client to a server. On the opposite side, there is another thread ServerToClient which passes messages from a server to a client. Notably, the number of threads is dependent on the numbers of clients and servers: If there are N Clients then there will be N ClientToServer threads. If there are M Servers, then there will be M ServerToClient threads.
Figure 2: High level architecture of the full system

Let’s look a little bit more in-depth, because nothing in life is that easy. See the below figure for a more convoluted diagram that explains how our proxy works internally. The main takeaway from Figure 3 is that there needs to be some way to correlate a specific user (via a UserId) to the connection that the user is available on. To implement this, I created a data type (Called Room) which uses a thread-safe HashMap with key as UserId and value as socket connection.
Figure 3: In depth look at the proxy implementation

In the ClientToServer thread, there are quite a few use-cases to care for:

  1. If a user logs in via their client, then we add their UserId and socket connection to the Room struct
  2. If a user logs out or disconnects, then we remove their UserId and socket connection from the Room struct
  3. If a user sends a new InputUpdate, then the proxy will prepend the appropriate UserId to the InputUpdate message and forward it to the Game Server.

In the ServerToClient thread, things are a bit more straightforward:

  1. If the server sends a WorldUpdate message, they will direct it at a specific UserId. The proxy will check the Room to find the socket connection for that UserId, and if found, will forward the message to the client.

Conclusion

Proxies can provide a lot of advantages in organization, scalability, security, availability, and maybe even latency. As you saw, the general architecture is not that difficult to follow. That said, what I’ve presented here isn’t a production-ready proxy by any means. This was an introduction to what a first-draft proxy might look like. In the future, I’m hoping to ruggedize this a bit more and hopefully provide better insights into performance losses and gains.

This entry is related to my YouTube series: Making an MMO

You can find code related to this project here: Github