Category name:SOA

BizTalk Server and the ESB SOA design pattern

In January this year I participated in a SOA architect workshop and after 5 exams I officially became certified SOA architect. The certification was issued by SOA schools which is an initiative of the well known SOA author Thomas Erl.

It was a very interesting course which discussed concepts I already knew, but wasn’t really aware of. During the course I more and more was able to map the concepts of SOA to Microsoft technology. One of the SOA characteristics is vendor neutrality so the course didn’t really discuss implementations.

Two course modules discussed design patterns (http://www.soapatterns.org/) and one of them is the ESB compound design pattern. It is called a compound pattern because it is build of other design patterns.

I started to recognize and map parts of BizTalk Server onto the ESB design pattern, but I didn’t have the full picture yet. In this blog post I take the ESB design pattern as a starting point to find out how and if BizTalk Server can be called an ESB based on this design pattern.

 

The ESB compound design pattern consists of the base pattern with these design patterns:

  • Asynchronous Queuing
  • Service Broker
    • Data Model Transformation
    • Data Format Transformation
    • Protocol Bridging
  • Intermediate Routing

and the extended version is:

  • Reliable Messaging
  • Policy Centralization
  • Rules Centralization
  • Event-Driven Messaging

BizTalk Server (together with the ESB Toolkit) is Microsofts product for implementing an ESB. To what extend does the vendor neutral design pattern map to the functionality in BizTalk Server?

Let’s start with the base pattern.

Asynchronous Queuing

In the SOA world this means that a service should be able to exchange messages with its consumers via an intermediary buffer, allowing service and consumers to process messages independently by remaining temporally decoupled.

BizTalk Server is asynchronous by nature, because all messages flowing through BizTalk are stored in the SQL Server message box thereby disconnecting consumers from services.

This SOA design pattern is explained here.

Service Broker

The service broker design pattern covers some of the most basic concepts necessary for application integration. This is about conversion of messages and the capability to receive messages via various types of protocols.

  • Data Model Transformation
    • A data transformation technology should be incorporated to convert data between disparate schema structures.
  • Data Format Transformation
    • Intermediary data format transformation logic needs to be introduced in order to dynamically translate one data format into another
  • Protocol Bridging
    • Services using different communication protocols or different versions of the same protocol should be able to exchange data.

Both types of transformations are solved in BizTalk Server using maps. Model transformation is about transforming one XML Schema to another and format transformation is transforming one message format to another, for example flat file or CSV to an XML Schema.

Protocol bridging is implemented in BizTalk Server using adapters and is about the ability of communication between different communication protocols like different versions of SOAP or something totally different like FTP.

Intermediate Routing

From a SOA perspective this is about the ability to dynamically determine message paths though the use of routing logic. Intermediate routing is typically implemented using service agents. A service agent is an event-driven program that doesn’t require explicit invocation, it just intercepts the messages. There are two types of service agents, active and passive. Active service agents not only read the message but also modify the message contents. Passive service agents only read the message and will or will not take action on the message content but doesn’t change it.

In BizTalk Server intermediate routing is implemented by means of a message agent. This agent is monitoring the message box to handle subscriptions to messages. In BizTalk everything is routed via the message box, so it can be implemented with only promoted properties or there can be complete routing logic in orchestrations or via business rules.

Reliable Messaging

This is about assuring that a message is actually received to the service. In a SOA world this is a technology concept to enable reliable communication with services. Commonly this is solved by a combination of the WS-ReliableMessaging standard and guaranteed delivery extensions like a persistent repository.

In BizTalk Server this is called guaranteed delivery, which assures to the caller that once a message is received in BizTalk Server it never will be lost. This is the persistent repository in the section above. The WS-ReliableMessaging standard is not supported by BizTalk due to its architecture.

Policy Centralization

In the context of SOA a service contract can be comprised of one or more XML Schemas, the WSDL, one or more Policies and an SLA. A policy adds information to the service commonly applied using the WS-Policy standard. This information can for example consist of the service window of a service or that it is only capable of handling so many requests an hour. Policy centralization is about applying centrally maintained policies to services.

Looking at BizTalk Server it seems like the ESB Toolkit supports some kind of policy centralization, but that appears to be related to itinerary centralization. Policy centralization probably should be maintained by a 3rd party product and cannot be maintained in BizTalk Server or with the ESB Toolkit.

Rules Centralization

Business rules are critical to companies and having the rules spread across a company is not a sustainable solution. The rules centralization design pattern is about centrally accessing, storing and managing business rules.

Business rules are part of BizTalk Server by means of the Business Rules Engine(BRE). This is even a separate component that can be used without other BizTalk components. Rules can be centrally accessed, stored and maintained in there. In a SOA approach the BRE would be wrapped into a service so it’s functionality is accessible in a standardized way from other technologies besides BizTalk Server. Wrapping such a component into a service raises other issues like introducing a single point of failure and capacity issues. These issues can be solved by applying other SOA design patterns

Event-Driven Messaging

From a SOA perspective this is about notifying customers of events. Commonly this is solved by subscribing customers to service events via one-way and asynchronous data transfer.

This is one of the most known concepts in BizTalk Server, because it refers to the publish-subscribe pattern as it is used in BizTalk Servers messaging architecture with the SQL Server message box as the heart of the system.

      Conclusion

      In this blog post we’ve been discussing the official ESB compound design pattern as defined by Thomas Erl. At least one of the requirements cannot be fulfilled by BizTalk Server, and that is Policy Centralization. So officially BizTalk Server is not an ESB product according to these requirements.

      On the other hand, policy centralization is one of the least used requirements in an ESB implementation at the moment. And besides that, BizTalk Server implements many more design patterns than just the ones mentioned in the ESB design pattern.

      On of the most recognizable design patterns for BizTalk Server is the Orchestration compound design pattern.

      Although this compound pattern is targeted at synchronous processing, it does consist of interesting design patterns we can also find in BizTalk Server:

      • Process Abstraction
        • This is about abstracting logic into a parent processes (task services). In BizTalk Server processes can be constructed of combined (agnostic) services using orchestrations. An orchestration is an abstraction of such parent processes which can be called by services without knowledge of the underlying logic.
      • Process Centralization
        • This is to prevent process logic to be spread across the company. All processes in BizTalk Server are centralized into one repository thereby fulfilling this requirement.
      • State Repository
        • This is about offloading state to a repository to release resources when they are not necessary at that time. BizTalk Server supports this by means of hydration and dehydration of processes.
      • Compensating Service Transaction
        • To prevent system locks during long running processes, compensating measures can be taken in case of an exception. BizTalk Server supports compensating service transactions as part of orchestration building blocks.
      • Atomic Service Transaction
        • To be able to rollback changes if one of the services in a chain of service calls fail, atomic service transactions can be used. BizTalk Server supports atomic service transactions in it’s WCF adapter.
      • Rules Centralization
        • Is also part of the ESB compound design pattern
      • Data Model Transformation
        • Is also part of the ESB compound design pattern

      Didago IT Consultancy