Service Fabric Remoting: Interception & Custom Headers
#azure#servicefabric#microservices#csharp
Peter Bons

Peter Bons @expecho

About: Software Architect, C# lover with a strong focus on architecture, backend & Azure Cloud development. Love to learn & share my knowledge using StackOverflow, GitHub and knowledge sessions.

Location:
The Netherlands
Joined:
Apr 16, 2018

Service Fabric Remoting: Interception & Custom Headers

Publish Date: Jan 9 '19
6 3

Service Fabric is a distributed systems platform for packaging, deploying, and managing stateless and stateful distributed applications and containers at large scale. Service Fabric runs on Windows and Linux, on any cloud, any datacenter, across geographic regions, or on your laptop.

Introduction

In this post we will learn two things that can make life easier to support advanced scenarios when using Service Remoting for communication between services and actors. We will do this by making use of a simple library I created. It provides you with the ability to:

  • Intercept service fabric remoting messages so you can take action when the call is about to be made and after it has been delivered.
  • Add custom headers to the remoting messages. This can be used to, for example, add a trace identifier for call tracing & logging purposes.

Message Interception

Messages can be intercepted on both the sending side and the receiving side

Client-side message interception

On the receiving side messages can be intercepted using the BeforeHandleRequestResponseAsync and AfterHandleRequestResponseAsync extension points when creating a service listener:

protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
    yield return new ServiceInstanceListener(context =>
        new FabricTransportServiceRemotingListener(context,
            new ExtendedServiceRemotingMessageDispatcher(context, this)
            {
                // Optional, log the call before being handled
                BeforeHandleRequestResponseAsync = reqInf =>
                {
                    var sw = new Stopwatch();
                    sw.Start();
                    ServiceEventSource.Log.ServiceMessage(Context, 
                            $"BeforeHandleRequest {reqInf.Service} {reqInf.Method}");
                    return Task.FromResult<object>(sw);
                },
                // Optional, log the call after being handled
                AfterHandleRequestResponseAsync = respInf =>
                {
                    var sw = (Stopwatch) respInf.State;
                    ServiceEventSource.Log.ServiceMessage(Context, 
                            $"AfterHandleRequest {respInf.Method} took {sw.ElapsedMilliseconds}ms");
                    return Task.CompletedTask;
                }
            }));
}
Enter fullscreen mode Exit fullscreen mode

Server-side message interception

On the sending side messages can be intercepted using the BeforeSendRequestResponseAsync and AfterSendRequestResponseAsync extension points when creating the ExtendedServiceRemotingClientFactory on constructor of the ServiceProxyFactory:

var proxyFactory = new ServiceProxyFactory(handler => // or ActorProxyFactory in case of actors
    new ExtendedServiceRemotingClientFactory(
        new FabricTransportServiceRemotingClientFactory(remotingCallbackMessageHandler: handler), 
            customHeadersProvider)
    {
        // Optional, log the call before being handled
        BeforeSendRequestResponseAsync = reqInf =>
        {
            var sw = new Stopwatch();
            sw.Start();
            Console.WriteLine($"BeforeSendRequest {reqInf.Method}");
            return Task.FromResult<object>(sw);
        },
        // Optional, log the call after being handled
        AfterSendRequestResponseAsync = respInf =>
        {
            var sw = (Stopwatch)respInf.State;
            var duration = sw.ElapsedMilliseconds;
            Console.WriteLine($"AfterSendRequest {respInf.Method} took {duration}ms");
            return Task.CompletedTask;
        }
    });
Enter fullscreen mode Exit fullscreen mode

Custom Headers

Custom headers can be used to pass data between the sender and the receiver like tracing information or security context data. Using the BeforeHandleRequestResponseAsync and AfterHandleRequestResponseAsync actions additional logging can be applied monitor the flow between remoting calls.

Prepare the Reliable Service

Modify the service and create a listener that can handle the requests:

protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
    yield return new ServiceInstanceListener(context =>
        new FabricTransportServiceRemotingListener(context,
            new ExtendedServiceRemotingMessageDispatcher(context, this)));
}
Enter fullscreen mode Exit fullscreen mode

Adding custom headers (the caller) 

var customHeaders = new CustomHeaders
{
    {"Header1", DateTime.Now},
    {"Header2", Guid.NewGuid()}
};

var serviceUri = new Uri("fabric:/ServiceFabric.Remoting.CustomHeaders.DemoApplication/DemoService");
var proxyFactory = new ServiceProxyFactory(handler => 
                    new ExtendedServiceRemotingClientFactory(
                        new FabricTransportServiceRemotingClientFactory(remotingCallbackMessageHandler: handler), 
                            customHeaders));
var proxy = proxyFactory.CreateServiceProxy<IDemoService>(serviceUri);
Enter fullscreen mode Exit fullscreen mode

There is an overload of the Create method that accepts a Func. This is useful in scenarios where the created proxy factory or proxy is reused. Since creating a proxy factory is expensive this is the preferred way if you need dynamic header values. The func is invoked on every request made using the proxy.

Reading custom headers (the callee)

The receiving service or actor can extract the values in the custom headers using the RemotingContext class:


public async Task<string> SayHello()
{
    var remotingContext =
        string.Join(", ", RemotingContext.Keys.Select(k => $"{k}: {RemotingContext.GetData(k)}"));

    ServiceEventSource.Log.ServiceMessage(Context, 
            $"SayHelloToActor got context: {remotingContext}");
    return Task.FromResult($"Got the following message headers: {remotingContext}")
}
Enter fullscreen mode Exit fullscreen mode

What is next?

More documentation and a demo Service Fabric application can be found at this repository:

GitHub logo Expecho / ServiceFabric-Remoting-CustomHeaders

This package allows injecting custom message headers into remoting messages (Actors and Reliable Services, V2 remoting only) at runtime. The headers are available client side to read. It also provides message interception using BeforeHandleRequestResponseAsync and AfterHandleRequestResponseAsync to act on remoting events.

ServiceFabric.Remoting.CustomHeaders

This package allows injecting custom headers into remoting messages (Actors and Reliable Services, V2 remoting only) at runtime. The headers are available client side to read It also provides message interception using BeforeHandleRequestResponseAsync and AfterHandleRequestResponseAsync to act on remoting events.

Common used classes:

NuGet

Download the NuGet package NuGet Status

Examples

This repository includes a Service Fabric application for demonstration purposes. A Console Application is used to access the application and shows the usage of the package.

Usage scenarios

Custom headers can be used to pass data between the sender and the receiver like tracing information or security context data. Using the BeforeHandleRequestResponseAsync and AfterHandleRequestResponseAsync actions additional logging can be applied monitor the flow between remoting calls.

How to use

Prepare Reliable Services

Modify the service and create a listener that can handle the requests

protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
    yield return new ServiceInstanceListener
…
Enter fullscreen mode Exit fullscreen mode
View on GitHub

Comments 3 total

  • JustinKaffenberger
    JustinKaffenbergerJun 24, 2019

    This is great, as it opens up some opportunities to extend the functionality of a given service without having to necessarily update the contract. Could also add some opportunities for a more 'aspect-oriented' approach to adding general purpose functionality around services without cluttering the service contract. Obviously overuse of that as a technique would make the code quite ugly, but everything in moderation 👍

    • Peter Bons
      Peter BonsJun 25, 2019

      Yeah I mainly used it for tracing purposes. I wouldn't use it to pass all kind of information to services instead of using proper variables for instance.

  • Alex Marshall
    Alex MarshallJun 26, 2019

    This is a great piece of functionality, just POC'd the Remoting piece in my own sandbox. Any chance you could split the Reliable Services and Actor pieces out into separate libraries with a lower dependency level ? I've gotten the code working as early as SDK 3.3.664

Add comment