Service Fabric: The Anatomy of a Service Fabric Application Package

Prelog

Before we begin I believe a quick introduction to what is service fabric is due. Check https://azure.microsoft.com/en-us/documentation/articles/service-fabric-overview/ for an architecture overview and https://azure.microsoft.com/en-us/documentation/articles/service-fabric-technical-overview/ for technical overview.

So what is a Service Fabric application package anyway? A service fabric application package is the smallest deploy-able unit to a service fabric cluster. The content of an application package describe a service or a set of services that will be deployed on a Service Fabric cluster. The content and structure of the package is so important to understand in context of creating & updating your micro- services. Hence a dedicated post on this topic.

Service Fabric Application Package Contents

Package Directory Physical Structure

The following is what VS.NET actually produces when you Right Click->Package on your Service Fabric Application project. (Go to %solution directory%/%Service Fabric Application project%/pkg/Debug to see it). Which what gets eventually deployed to Service Fabric cluster using PowerShell cmdlets.

  • One ApplicationMainfest.xml file: contains the description of
    1. What are the services included in the package (with reference to Service Manifest files)
    2. How are they are deployed to the cluster. Such as whether they are stateful or stateless and how each service is partitioned and number of replicas per partition.
  • Number of directories – each is for a service within the application package. Each will contain the following
    • One directory named “Code” where your .NET assembly & and referenced assemblies will be copied to.
    • One or more directory mapped to configuration and data packages (more on this below)
    • One ServiceManifest.xml contains description of:
      1. What are the services in your assembly and their .NET types.
      2. What are the code, configuration & data packages.
      3. Resources used by your services (more on that in later posts).

folderStructure

 

For example the above is the contents of my application package. It contains one service called “SFPkg.StatefulSvcPkg” which contains “Code” directory. A configuration package named “Config” and a data package named “SvcData”.

What is “Configuration” & “Data” Packages?

There is nothing to say about them beyond naming. Configuration is whatever you use in services such as API addresses, Throttling values and so forth. Data is whatever static data you use in your service. for instance the example i have in my repo right now uses “States” and “Zones” as a static data. As you probably figured by now. Configuration & Data are shared between replicas of “Services” not with other services even those which are members of the same Application Package.

Notes on Configuration & Data Packages Versioning (& Updates)

Everything in the application package is versioned. These versions allows you to update the package on the cluster using Rolling Update. Rolling Update is a mechanism that allows you to perform live updates on the application without taking the services down.

And yes you guessed it right you can update each component of your service without affecting the rest. As in update code only, configuration only, data only or combination of them. We will be covering Rolling Updates in later posts.

How Service Fabric Application Package Relates to Your Visual Studio Solution

1-VSToPackage

//You know the drill, Click to enlarge.

 

The above is a screen shot of Visual Studio Solution (the same one produced the directory structure presented earlier).

Logical View of a Single Service Fabric Application Package Contents

  1. Application Manifest
  2. Services (for-each)
    1. Service Manifest: which describes how your service should run on top of Service Fabric cluster.
    2. Code Packages: One or more .NET assemblies.
    3. Data Packages: One or more data packages. Each data package is a directory in your visual studio project. A directory represent one data package and can contain multiple data files. Service Fabric treats data as opaque binary files and avail them to your services.
    4. Configuration Packages: One or more configuration packages. Each configuration package is a directory in your visual studio project. A directory represent one configuration package and can contain multiple files. Except for Settings.XML Service Fabric treats configuration file as opaque files and does not process them.

There are two types of configurations associated with your service

    1. Settings.xml: by default added to your Service Fabric Service project by Visual Studio tooling. And can be accessed using
      var configPkg = ServiceInitializationParameters.CodePackageActivationContext.GetConfigurationPackageObject("Config");
      
       // working with standard configuration file settings.xml
      // settings file has to be XML as it is surfaces in Settings member below 
      // (fabric takes care of loading it for you)
      var customSection = configPkg.Settings.Sections["<Section name here>"];
      

      Then parameters collection each will have a name and value (strings)

    2. Custom Configuration: (discussed below)

Configuration & Data Packages

Service Fabric enables you to deploy configuration and data in a non-specific format (except for Settings.xml as described below) with your service code. The configuration and data can later be updated independently from the service code. Updating the configuration and data will not trigger tear down/recreation replicas.  Note: App.Config is part of the service code and updating it will require code update.

Data packages works exactly like configuration packages. Data packages are opaque to Service Fabric. And can be in any format. It can also be updated independently from service code without triggering tear down/recreation of replicas. They are read only objects; all replicas will get the same copy of the Data Packages.

Typically when designing a micro-service you have 3 types of data:

  1. Old with very low volatility typically generated from request processing or OLTP like operations. Those are externalized in external storage such as Azure Tables/Blobs or SQL.
  2. Current highly volatile typically used and generated from request processing or OLTP like operations. Those are partitioned and kept with your services using reliable queues and reliable dictionaries.
  3. Lookup data that is almost static and used during request processing or OLTP like operations. This is your Data Packages.

For both types of packages Service Fabric allows you to subscribe to “Add”, “Modified” & “Deleted” events in order to update the internal state of your service accordingly. These events are typically fired during Rolling Updates.

Adding and Using Configuration Packages to Your Application Package

Using Default Settings.xml

Settings.xml is added by default in the ServiceMainfest.xml by Visual Studio (and is used by the classes your service class is inhriting from (more on this later). Visual Studio adds it as the following

  <!—
Config package is the contents of the Config directoy under PackageRoot that contains an independently-updateable and versioned set of custom configuration settings for your service. 
-->
  <ConfigPackage Name="Config" Version="1.0.0.0" />

You can add custom sections to Settings.xml as the following

<Section Name="customSettingsSection">
    <Parameter Name="p1" Value="v1" />
  </Section>

To use custom configuration sections in settings.xml in service code:

var configPkg = ServiceInitializationParameters.CodePackageActivationContext.GetConfigurationPackageObject("Config");

// working with standard configuration file settings.xml
// settings file has to be XML as it is surfaces in Settings member below (fabric takes // care of loading it for you)
var customSection = configPkg.Settings.Sections["customSettingsSection"];

foreach (var p in customSection.Parameters)
{
Debug.WriteLine(string.Format("*** Custom Settings Section Param:{0} -> {1}", p.Name, p.Value));
}

Adding Custom Configuration (New file or New Package)

If setings.xml is not enough for your needs you have the following options

  1. Adding a custom file to “Config” package by using the following steps:
    1. Add a file to “Config” directory and name it for example customConfig.json
    2. Reference the file in your code as the following
      // working with none standard configuration file other than settings.xml
         // this are treated as blobs by fabric so we can use JSON or any other format of //choice
      var customConfigFilePath = configPkg.Path + @"\CustomConfig.json";
       // ConfigMaster is a POCO object where we are keeping the configuration
      ConfigMaster configMaster = JsonConvert.DeserializeObject<ConfigMaster>(File.ReadAllText(customConfigFilePath));
                  Debug.WriteLine(string.Format("*** Custom Config File ConfigKey -> {0}", configMaster.ConfigKey));
      
  2. Add a an entirely new configuration package using the following steps:
    1. Create a directory under “PackageRoot”. For example name it “APIEndPoints”.
    2. Add your configuration files under it. For example one file can be named PurchaseEndPoints.json
    3. Use it as the following

      var ApiConfig = ServiceInitializationParameters.CodePackageActivationContext.GetConfigurationPackageObject("APIEndPoints");
      
      // fabric doesn't load data it is just manages for you. data is opaque to Fabric
      var ApiConfigFilePath = ApiConfig.Path + @"\PurchaseEndPoints.json";
      
      // read the content & use your favorite json tool for this. 
      File.ReadAllText(ApiConfigFilePath); 
      

Adding and Using Data Packages

To add a Data Package use the following steps:

  1. Create a directory under “PackageRoot” directory. For example name it “SvcData”.
  2. Add your data file in your format of choice to the directory. For this example I have used json.
  3. In your service manifest add a data package reference to your directory as the following:
    <!-- name has to match your directory name-->
      <DataPackage Name="SvcData" Version="1.0.0.0"/>
    
  4. To use the data in your service code
    // get the data package
    var DataPkg = ServiceInitializationParameters.CodePackageActivationContext.GetDataPackageObject("SvcData");
    
     // fabric doesn't load data it is just manages for you. data is opaque to Fabric
    var customDataFilePath = DataPkg.Path + @"\StaticDataMaster.json";
    
    
    // Config Master is a POCO object where we are keeping the configuration
    StaticDataMaster dataMaster = JsonConvert.DeserializeObject<StaticDataMaster>(File.ReadAllText(customDataFilePath));
    

Structuring Your Configuration & Data Packages

When thinking about packages keep the following in mind:

  1. When performing an update. Service Fabric updates the entire package not a single file. Packages are updated using the versions (more on this in later posts).
  2. Updating a package requires a copy to the cluster which might be hosted remotely such as Azure Service Fabric. Application Packages are managed by “ImageStore” service on the cluster (part of the “System” application). And yes they follow the same semantics of committing to quorum before returning a successful write. As in your upload is actually a multiple writes to cluster nodes hence uploading large data (gigabytes for example) might not be the best approach.
  3. You will get notification in your service code when a package is updated, you will not be able to identify which file has been updated. Accordingly you will have to reload the entire content of the package to reflect the change on the internal service state.

In essence break your packages according to volatility and size keep in mind that if two or more files has to be updated together then they will have to belong to the same package. This is to avoid situation where the internal state of your services were updated to reflect a partial change (yielding into broken internal state).

Epilog

This article is the first of a miniseries where I talk about Service Fabric. Expect more to come. once we have a few i will create a master index of all Service Fabric articles in one place for your quick reference

The code for this article is hosted on https://github.com/khenidak/ServiceFabricSeries

Till next time

@khnidk

Leave a Reply

Your email address will not be published. Required fields are marked *