Using Service Fabric Partitioning to Concurrently Receive From Event Hub Partitions

Prelog

The classic tale is you have Azure EventHub with a number of partitions, you run your receiver process in a singleton Azure Web Jobs or a single instance Azure Worker Role cloud service both will use “Event Processor Host” to pump events into your process. The host externalize offset state storage to Azure Blob Storage (to survive processor crashes and restarts). This works fine yet it has the following problem. You can not scale it out easily. You have to create more separate cloud services or web jobs each handling some of your event hub partitions. Which will complicate your upgrade and deployment processes.

This post is about solving this problem using a different approach.

Service Fabric Azure Event Hub Listener

Azure Service Fabric partitions are somehow logical processes (for the purists I am talking about primary replicas). They can exist separably each on a server, or co exist on a node. depending on how big your cluster is and your placement approach.

Using this fundamental knowledge we can create a service that receives events from Azure Event Hub as the following:

  • If the number of Service Fabric Partitions is greater than Event Hub Partitions. Then each Service Fabric partition should receive from multiple Event Hub partitions at once.
  • If the number of Service Fabric Partitions is equal Event Hub Partitions then each Service Fabric partition should receive from one Event Hub Partition.

Because we have absolute control over how the above mapping is done, we can also add 1:1 mapping (irrespective of the number of partitions here and there). or even custom funky logic where some service fabric service partitions are doing the receive the rest are doing something else.

The second piece of the new tale is state. Service Fabric has it is own state via the use of stateful services I can save few network I/Os by using them to store the state – reliably – internally on the cluster via the use of Reliable Dictionary. The state management for our receiver is plug-able. As in you can replace it with whatever custom storage you want (one of the scenarios I had in mind is running the listener in a stateless service and use a another packing stateful service).

The third piece of the new tale is usage. Event Hub Processor is great but it requires a few interfaces to be implemented (because it is a general purpose component). Ours is highly specialized single purpose component. Hence we can simplify the entire process by just implementing one interface (that represents what happened to events and when to store the offset).

The last piece of our new tale is the entire code wrapped in Service Fabric ICommnuicationListener implementation allowing you to use just as any other Service Fabric listener.

The code for the above is on github here https://github.com/khenidak/service-fabric-eventhub-listener

till next time

@khnidk

On Application Routers/Gateways

Prelog

The discussion below is about Routing & Application Gateways. It is about a component a wrote the code for and is published here https://github.com/khenidak/Router along with the documentation on design & how to use. The below is ranting on how things in this component are made in a certain way.

“Application Gateways are hard to build, but should not be hard to use”

Why Build an application Gateway or a Router

First: Why Application Routers/Gateways?

Your application will have requests coming to your external (or even internal) end points. These requests are not meant to be fulfilled by the application that received the requests but rather via backend of some sort. Most large scale (and especially those that are built for hyper-scale) have this requirement. Take the following examples:

  1. A SaaS application where users’ requests are routed to specific tenants. These requests might be web pages, Web Sockets, REST API Calls or even TCP/IP packets.
  2. An application that distribute the loads/requests types among multiple backend systems.
  3. An application that went through massive transformation (take M&A for example scenarios) where external API have changed, you will need a gateway in the middle to upgrade or bridge the protocol without the need to disrupt your existing external clients.
  4. Application Zoning/Containment/Partitioning/Segregation, where connections from the external world are authenticated, authorized then terminated at the gateway. The gateway then establishes connections to the backend systems.
  5. A microservices application deployed on top of a cluster managed by a cluster management platform such as Apache Mesos or Azure Service Fabric will need a gateway. While some examples refer to using 2 tiers (Web UX deployed on all nodes routing to compute in the backend such as Azure Service Fabric’s WordCount (https://github.com/Azure/servicefabric-samples/tree/master/samples/Services/VS2015/WordCount). This approach does not deal with situations where Web UX might be subject to routing per request (take a SaaS application, A/B Testing, or an application that provision mesos jobs or Azure Service Fabric App instances to scale to meet the load or isolation requirements).
  6. Specific platform requirements such as offering Azure Service Fabric Actor framework to external – to the cluster – applications (where Actor clients cannot be used).

And many others.

Is Routing a New Thing?

No, as a matter of fact every time you used a web server internal routing happens on the OS – typically by a kernel component – to route requests to different processes responsible for different address. At a higher level, frameworks such as ASP.NET Web API or WCF preform per request routing (dispatching in case of WCF) to the target controller/method (Service/Interface in case of WCF).

But This Is at the Platform Level, What about the Application Level Are This Requirement New? Is It a Cloud Thing?

No & No. Application gateways has been a thing since forever. I remember building an Http application gateway 1998. And there are 10+ years old products out that perform various parts of Application Gateway logic. The Cloud came in to fulfill “hyper-scale” requirements. Where the application itself can be provisioned multiple times to support load or isolation (hence the stress on routing).

The Problem With Routing

If your requirements are about from point A to point B and both are the same semantics (say Web API) with fairly static URL then you don’t need a custom gateway and I would strongly recommend looking for existing solutions such as Azure API Management (https://azure.microsoft.com/en-us/documentation/articles/api-management-get-started/). Or build a single purpose gateway. The problem is most requirements comes in the following forms:

  • Routing in multitenant solutions

If request is on https://<some-tenante>.<hist>.com/crm route it to http://node:port/<some-tenante>/crmapp (node is backend server) and add custom authorization header, then get the response and put in the original downstream payload while adding a custom header.

  • Routing in multi version and or A/B testing scenarios

If request is on https://www.<host>.com/api/customer and authorization header claim contains usergroup = “default” or usertype != “admin” then route it to http://node:port/api-v2/customer else route to http://node:port/api /customer

  • Routing in microservices like environments

if request is on https://www.<host>.com/api/customer then resolve microservices address list and perform round robin load balancing between them however if the request type is Post or Put or Delete then route only to primary partition (in case of Service Fabric).

  • Routing in protocol bridges

If request came on Web Sockets address ws://www.host.com/sockets/customer then route based to http://node:port/<some-tenante>/api/customer and set Http Method based on messagePayload.Type MT, MT = “add” then Method = Post, MT = “update” method = post etc.

Sounds complex enough? those are typical requirements for an application gateway.

The problem is scary enough but relatively easy to solve. If you split the representing the logic from the actual execution

Routing Logic, Simplifying the Complex

You can easily represent the logic by a linked list where each node represents a condition and/or logic and is only executed if the node.next executed successfully. In my code I called nodes Matcher (not the sexiest name I know). Consider representing them as the following:

 

//pseudo code 

// If request on bing then route it as Get to &lt;a href="http://www.microsoft.com"&gt;http://www.microsoft.com&lt;/a&gt; and add a custom header “CUSTOM_HEADER” with value “Hello, World!”;

var head = new SetMethod(“Get”)

head.chain(new SetAddress(“http://www.microsoft.com”), new AddHeader(“CUSTOM_HEADER, “Hello, World!”), new MatchAddress(“bing”, MatchType.MatchHostNameOnly);

// for more concrete samples and implementation check https://github.com/khenidak/Router

The above code describes routing and processing logic in an easy to understand fashion, and more importantly an easy extend framework. You can extend to add whatever matcher types you want. You will end up with something that looks like this (all images below are from the repo):

matching-frx

But what about my ANDs and ORs? This can also be represented by matchers, consider the following:


// pseudo code 

// If request on bing and user type is “dev” then route it as Get to &lt;a href="http://www.,msdn.com"&gt;http://www.,msdn.com&lt;/a&gt; else &lt;a href="http://microsoft.com"&gt;http://microsoft.com&lt;/a&gt; and add a custom header “CUSTOM_HEADER” with value “Hello, World!”;


// If request on bing then route it as Get to &lt;a href="http://www.microsoft.com"&gt;http://www.microsoft.com&lt;/a&gt; and add custom header “CUSTOM_HEADER” with value “Hello, World!”;

var head = new SetMethod(“Get”)

var msdn = IsUserType(“Dev”)
msdn.chain(new SetAddress(“http://www.msdn.com”))


head.chain(
   new OrMatcher(
              msdn
              ,
             new SetAddress(“http://www.microsoft.com”)
             ),
   new AddHeader(“CUSTOM_HEADER, “Hello, World!”),
   new MatchAddress(“bing”, MatchType.MatchHostNameOnly));

// for more concrete samples and implementation check https://github.com/khenidak/Router

Because of this type of branching the matchers are represented in memory as a tree not just a linked list.

matching-frx-tree

 

Because Matcher is a .NET type you can subclass it in new types of matching that suits your application (the code published here https://github.com/khenidak/Router contains most of the common stuff) or You can extend the existing ones with new capabilities specific to your application.

Executing the logic becomes a matter of mechanics as described in the documentation here https://github.com/khenidak/Router (the basic idea a “Routing Context” is pushed through the linked list).

 

Epilog

I have chosen to use this space to describe what and why we need application routers / gateways (the what and how along with source code is published on GitHub). I have also chosen to cover just one aspect of the complexity that usually comes in building them. Check the documentation to get an idea about the rest of the problems that a typical application router/gateway have to solve and how they were solved.

till next time

@khnidk

 

On October & Azure Service Fabric IoT Sample

Azure Service Fabric public preview is out today. Check the announcement here http://blogs.msdn.com/b/azureservicefabric/archive/2015/11/18/service-fabric-enters-public-preview.aspx. over most of October I have been busy – as in you wouldn’t get my usual ranting – putting together a reference architecture and sample for IoT on top of Service Fabric the sample is published here https://github.com/Azure-Samples/service-fabric-dotnet-iot
The sample in brief – quoting the docs – is:
“This sample is a reference architecture & implementation for hyper scale IoT event processing. The sample uses compute and storage on Azure Service Fabric and integrates with Azure Storage, Azure Power BI, and Event Hubs.”
Follow the link above for the sample. for the rest of the samples check http://aka.ms/servicefabricsamples be sure to check the “Party Clusters” 🙂
Till next time
@khnidk

Azure Service Fabric: Multiple Communication Listeners Per Service

So I was off for a week, then got elbow deep on one of the projects. Slowly recovering to normal work rhythm. Until the work I am doing is completed – expected by end of October, stay tuned for future announcements 🙂 – I thought I’d share something of interest.

I am working with a Service Fabric service, where each service is expected to have:

  1. 0..n number of event hubs listeners. The event hub number can change as in add, remove hubs etc. Needless to say the service doesn’t need to stop, start, reset or upgrade in case of event hub
  2. 1 – Only one – REST control endpoint, where you can interact with the service to do:
    • Add Hubs.
    • Remove Hubs.
    • Get # of processed messages and other relevant telemetry data.
    • Stop/Start/Pause processing.

Service Fabric allows you to create one listener per service (for purest folks, per replica). More details here (http://henidak.com/2015/07/service-fabric-partitions/). In order to allow this to happen I created a composite listener, basically sets on top a list of listeners, listeners can be added, removed without affecting service lifecycles. The composite listener ensure that newly added listeners status matches the composite listener. As in if the composite listener has been opened-async then the newly added listeners will be opened-async-ed as well.

The composite listener can work with any class that implements ICommunicationListener interface (including those provided in the public samples).

I published it as a gist on github here: https://gist.github.com/khenidak/21f8de349a460ab90408

Till next time @khnidk

 

Azure for SaaS Providers

So I kicked off yet another side project. Azure for SaaS providers is a projet aims is to build a reference literture to be used by those who are using Azure to build SaaS solutions. This a long term project (probably ending by Feb 2016). I will be adding frequently to this content.

While the target the primary audience is SaaS providers the information can also be for solutions that needs to support massive scale.  And the core concepts can be used to support private cloud hosting or even different cloud providers.

The content is in OneNote format hosted here http://1drv.ms/1Ni2geR (no you don’t need to install OneNote)

Below is the content introduction page for a quick read:

What is this?

This is collection of literature and reference materials that help those planning or currently using Azure to build or host SaaS solutions. While the primary audience is those who are planning to use Azure in SaaS context there is nothing stopping you from using this for other applications, specifically applications with high scale requirements. Typically applications that require partitioning, isolation and containerization.

Why?

Frankly speaking, because there is not a lot of literature out there covering this topic. And those which cover it either too narrow, too old or both. The purpose is to liberate SaaS vendors’ developers to build more great features and less platform components (for architects this represents a wealth of options to consider while thinking about your SaaS solutions running on Azure).

What should I expect here?

Design options with tradeoffs, diagrams, code samples, code components and reference to external sources.

Why using this format?

  1. SaaS vendors come in all shapes and colors and there is no one solution that fits all. Hence the recipes approach. You mix, match and modify recipes as you see fit.
  2. Microsoft Azure is currently (and will always be) in fluid state. New services/components will be added and existing ones will be updated. This format allows us to revisit each recipe without having to change a lot of other recipes.
  3. OneNote is easy to use, free and with web frontend which can be viewed irrespective of the device you choose to use to view this content.

What if I want to contribute?

Please do reach out and let us have a discussion on which areas you can cover.

Questions/comments

@khnidk