On Application Routers/Gateways


The discussion below is about Routing & Application Gateways. It is about a component a wrote the code for and is published here https://github.com/khenidak/Router along with the documentation on design & how to use. The below is ranting on how things in this component are made in a certain way.

“Application Gateways are hard to build, but should not be hard to use”

Why Build an application Gateway or a Router

First: Why Application Routers/Gateways?

Your application will have requests coming to your external (or even internal) end points. These requests are not meant to be fulfilled by the application that received the requests but rather via backend of some sort. Most large scale (and especially those that are built for hyper-scale) have this requirement. Take the following examples:

  1. A SaaS application where users’ requests are routed to specific tenants. These requests might be web pages, Web Sockets, REST API Calls or even TCP/IP packets.
  2. An application that distribute the loads/requests types among multiple backend systems.
  3. An application that went through massive transformation (take M&A for example scenarios) where external API have changed, you will need a gateway in the middle to upgrade or bridge the protocol without the need to disrupt your existing external clients.
  4. Application Zoning/Containment/Partitioning/Segregation, where connections from the external world are authenticated, authorized then terminated at the gateway. The gateway then establishes connections to the backend systems.
  5. A microservices application deployed on top of a cluster managed by a cluster management platform such as Apache Mesos or Azure Service Fabric will need a gateway. While some examples refer to using 2 tiers (Web UX deployed on all nodes routing to compute in the backend such as Azure Service Fabric’s WordCount (https://github.com/Azure/servicefabric-samples/tree/master/samples/Services/VS2015/WordCount). This approach does not deal with situations where Web UX might be subject to routing per request (take a SaaS application, A/B Testing, or an application that provision mesos jobs or Azure Service Fabric App instances to scale to meet the load or isolation requirements).
  6. Specific platform requirements such as offering Azure Service Fabric Actor framework to external – to the cluster – applications (where Actor clients cannot be used).

And many others.

Is Routing a New Thing?

No, as a matter of fact every time you used a web server internal routing happens on the OS – typically by a kernel component – to route requests to different processes responsible for different address. At a higher level, frameworks such as ASP.NET Web API or WCF preform per request routing (dispatching in case of WCF) to the target controller/method (Service/Interface in case of WCF).

But This Is at the Platform Level, What about the Application Level Are This Requirement New? Is It a Cloud Thing?

No & No. Application gateways has been a thing since forever. I remember building an Http application gateway 1998. And there are 10+ years old products out that perform various parts of Application Gateway logic. The Cloud came in to fulfill “hyper-scale” requirements. Where the application itself can be provisioned multiple times to support load or isolation (hence the stress on routing).

The Problem With Routing

If your requirements are about from point A to point B and both are the same semantics (say Web API) with fairly static URL then you don’t need a custom gateway and I would strongly recommend looking for existing solutions such as Azure API Management (https://azure.microsoft.com/en-us/documentation/articles/api-management-get-started/). Or build a single purpose gateway. The problem is most requirements comes in the following forms:

  • Routing in multitenant solutions

If request is on https://<some-tenante>.<hist>.com/crm route it to http://node:port/<some-tenante>/crmapp (node is backend server) and add custom authorization header, then get the response and put in the original downstream payload while adding a custom header.

  • Routing in multi version and or A/B testing scenarios

If request is on https://www.<host>.com/api/customer and authorization header claim contains usergroup = “default” or usertype != “admin” then route it to http://node:port/api-v2/customer else route to http://node:port/api /customer

  • Routing in microservices like environments

if request is on https://www.<host>.com/api/customer then resolve microservices address list and perform round robin load balancing between them however if the request type is Post or Put or Delete then route only to primary partition (in case of Service Fabric).

  • Routing in protocol bridges

If request came on Web Sockets address ws://www.host.com/sockets/customer then route based to http://node:port/<some-tenante>/api/customer and set Http Method based on messagePayload.Type MT, MT = “add” then Method = Post, MT = “update” method = post etc.

Sounds complex enough? those are typical requirements for an application gateway.

The problem is scary enough but relatively easy to solve. If you split the representing the logic from the actual execution

Routing Logic, Simplifying the Complex

You can easily represent the logic by a linked list where each node represents a condition and/or logic and is only executed if the node.next executed successfully. In my code I called nodes Matcher (not the sexiest name I know). Consider representing them as the following:


//pseudo code 

// If request on bing then route it as Get to &lt;a href="http://www.microsoft.com"&gt;http://www.microsoft.com&lt;/a&gt; and add a custom header “CUSTOM_HEADER” with value “Hello, World!”;

var head = new SetMethod(“Get”)

head.chain(new SetAddress(“http://www.microsoft.com”), new AddHeader(“CUSTOM_HEADER, “Hello, World!”), new MatchAddress(“bing”, MatchType.MatchHostNameOnly);

// for more concrete samples and implementation check https://github.com/khenidak/Router

The above code describes routing and processing logic in an easy to understand fashion, and more importantly an easy extend framework. You can extend to add whatever matcher types you want. You will end up with something that looks like this (all images below are from the repo):


But what about my ANDs and ORs? This can also be represented by matchers, consider the following:

// pseudo code 

// If request on bing and user type is “dev” then route it as Get to &lt;a href="http://www.,msdn.com"&gt;http://www.,msdn.com&lt;/a&gt; else &lt;a href="http://microsoft.com"&gt;http://microsoft.com&lt;/a&gt; and add a custom header “CUSTOM_HEADER” with value “Hello, World!”;

// If request on bing then route it as Get to &lt;a href="http://www.microsoft.com"&gt;http://www.microsoft.com&lt;/a&gt; and add custom header “CUSTOM_HEADER” with value “Hello, World!”;

var head = new SetMethod(“Get”)

var msdn = IsUserType(“Dev”)
msdn.chain(new SetAddress(“http://www.msdn.com”))

   new OrMatcher(
             new SetAddress(“http://www.microsoft.com”)
   new AddHeader(“CUSTOM_HEADER, “Hello, World!”),
   new MatchAddress(“bing”, MatchType.MatchHostNameOnly));

// for more concrete samples and implementation check https://github.com/khenidak/Router

Because of this type of branching the matchers are represented in memory as a tree not just a linked list.



Because Matcher is a .NET type you can subclass it in new types of matching that suits your application (the code published here https://github.com/khenidak/Router contains most of the common stuff) or You can extend the existing ones with new capabilities specific to your application.

Executing the logic becomes a matter of mechanics as described in the documentation here https://github.com/khenidak/Router (the basic idea a “Routing Context” is pushed through the linked list).



I have chosen to use this space to describe what and why we need application routers / gateways (the what and how along with source code is published on GitHub). I have also chosen to cover just one aspect of the complexity that usually comes in building them. Check the documentation to get an idea about the rest of the problems that a typical application router/gateway have to solve and how they were solved.

till next time