Category name:BizTalk

BizTalk Server 2013 SharePoint Adapter Walkthrough

The SharePoint adapter has been redesigned in BizTalk 2013. It now supports for example SharePoint 2013 and SharePoint online, but it can still communicate with the current common versions like SharePoint 2007 and 2010.

If SharePoint 2013 is your platform, the adapter is utilizing the new SharePoint Client side Object Model (SCOM). For older versions of SharePoint it still uses the WSS adapter service component which needs to be installed on the SharePoint server itself.

I wanted to see how easy it has become to use the SharePoint adapter with SCOM so I decided to do some testing. The adapter is obviously capable of reading and writing to SharePoint document libraries and I was surprised how easy it actually is.

My scenario was pretty simple. I wanted to read a message from file and write it to a SharePoint document library. Next I want to read this message from the document library again and write it to disk. For this I installed a SharePoint 2013 foundation edition (by accident in Dutch) on my BizTalk 2013 environment, which is equipped with Windows Server 2012 and SQL Server 2012.

So first I created the Visual Studio solution with a source schema, a target schema and a mapping between the two. No rocket science. From the Visual Studio point of view nothing has changed. You create artifacts like you’re used to in BizTalk 2013.

After deploying the artifacts to BizTalk I created a receive port and location to read from file (with a simple map to transform from source to target), and more interesting a send port to send the message to SharePoint. After selecting ‘Windows SharePoint Services’ in the transport type of the send port, the following dialog is displayed.

What can be configured here:

  • Adapter Web Service Port – The HTTP port of the IIS website where the WSS adapter web service is installed
  • Timeout – Value in milliseconds the adapter waits to finish the send action
  • Use Client OM – Specify if the adapter can use the new SCOM or should use the older WSS adapter service component. You can also set this property in an orchestration.
  • Destination Folder URL – Location of the list to write the messages to, relative to the SharePoint site URL.
  • Filename (optional) – This is pretty cool, you can specify a fixed filename or construct it based on an Xpath expression to get a value from the message.
  • Namespaces (optional) – define namespaces used for the Xpath expressions like in the filename just mentioned.
  • Overwrite – Like in the FILE adapter, allow the message to be overwritten with the same name, but here also rename is possible.
  • SharePoint Site URL – Location of the SharePoint site to use .
  • Microsoft Office Integration – Specify here values to integrate solutions with Microsoft Office InfoPath, by adding processing instructions.

Then continued with this, to cover the second half of the transport properties dialog:

  • SharePoint Online – For the first time it is possible to connect with the SharePoint online cloud solution!
  • Windows SharePoint Services Integration – Allows to fill up to 16 columns by means of name/value pairs in the destination SharePoint list. In my scenario I added two additional columns to test this; ‘CategoryFixed’ with a fixed value and ‘NameFromXpath’ to figure out how Xpath expressions work. The columns defined must exist in the SharePoint list, it won’t create them if they are missing.

An example of an Xpath expression used as column value could be this:

%XPATH=/*[local-name()=’Target’ and namespace-uri()=’http://Didago.SharePointAdapter.Target’]/*[local-name()=’Fullname’ and namespace-uri()=”]%

It should start and end with the ‘%’ symbol and the rest is copied from the schema editor in Visual Studio. It is clear I have a target schema containing a (single) ‘Fullname’ element.

The SharePoint site I used is plain and simple and looks like this (unfortunately in Dutch):

There are two document lists named ‘Processed’ and ‘BizTalkList’ and that list has the two in the configuration mentioned columns ‘CategoryFixed’ and ‘NameFromXpath’. With all this configured and setup we can start testing by dropping a test file into the folder.

The test message is also simple:

The mapping between the source and target is just a concatenation of the first and last name into a full name. After dropping the test message in the folder, it is picked up by BizTalk and send to the SharePoint list.

The message is nicely written to the SharePoint list, the fixed column is filled with ‘Target’ and the concatenated full name is extracted from the message. This was very straight forward. One thing to remind is the host instance account must have write permissions on the list.

The second scenario is reading from the SharePoint list, therefor we first need to create a receive location configured for SharePoint.

What can be configured here:

  • Adapter Web Service Port – The HTTP port of the IIS website where the WSS adapter web service is installed.
  • Timeout – Value in milliseconds the adapter waits to finish the send action.
  • Use Client OM – Specify if the adapter can use the new SCOM or should use the older WSS adapter service component. You can also set this property in an orchestration.
  • Archive Filename (optional) – Name of the file when archived, fixed or via Xpath expressions. I used Xpath here.
  • Archive Location URL – List relative to SharePoint Site URL where to archive the processed messages.
  • Archive Overwrite – Allow to overwrite existing messages in the archive.
  • Batch Size – Maximum number of messages read in one batch.
  • Error Threshold – Maximum number of consecutive polling failures before the receive location is disabled.
  • Namespaces (optional) – Define namespaces used for the Xpath expressions like in the archive filename mentioned before.
  • Polling Interval – Interval in seconds between polling for new messages.
  • SharePoint Site URL – Location of the SharePoint site to use.
  • Source Document Library URL – List relative to SharePoint Site URL where to read the messages from.
  • View Name – Name of the view used to filter the messages to read, leave empty to process all messages.
  • Microsoft Office Integration – (optionally try to) remove the Microsoft Office InfoPath processing instructions on the message.
  • SharePoint Online – For the first time it is possible to connect with the SharePoint online cloud solution!

So with this scenario messages will be read from the view named ‘CategorySource’, which has a filter on the ‘CategoryFixed’ column. It shows only messages where that column has the text ‘Source’ as value. Because the first scenario only writes messages to the list with text value ‘Target’, those aren’t picked up by the adapter and I needed to add a document manually to the list with the required field value. In the screenshot below you’ll find the manually added message in the ‘BizTalkList’ SharePoint list and the ‘CategorySource’ is the selected view.

To my surprise both the message from scenario 1 and the manually added message were picked up and not only the one that shows up on the ‘CategorySource’ view. I’m not sure if this is a bug? I couldn’t find anything about it and after all it is a beta.

In the configuration of the receive location I specified to archive the message, which is correctly done by the adapter.

Some nice behavior I experienced is caused by the ‘Archive Overwrite’ setting, which dictates what to do when a message with a certain name already exists in the archive document list. In that case the adapter performs a check-out on the file to process and then waits. This prevents the message from being removed without being processed.

To conclude I must say it is very easy to use the new SharePoint adapter in BizTalk 2013, but I’m going to do some more research on the view name issue I ran into.

Didago IT Consultancy

Dutch BTUG meeting with the BizTalk product team

Yesterday we had a special Dutch BizTalk User Group meeting. The guys from the BizTalk product team were in The Netherlands because of the BizTalk summit tour and they came by to talk about BizTalk.

Three presentations were given about the upcoming release of BizTalk Server 2013: one overview and two deep dives. Much of the overview stuff was already known. If you’re curious about the features and want to download the BizTalk 2013 beta, you can take a look here. One important take-away is that BizTalk will be available as Azure virtual image, and pay-per-use will bring BizTalk closer to smaller companies because the licensing costs will no longer be a barrier.

The other presentations were deep-dives about BizTalk 2013 adapters and the Azure BizTalk Services.

The adapters talk discussed the new SharePoint 2013 adapter, which is using the SharePoint Client Object Model (COM Winking smile ). No longer do we need to install something on the SharePoint server if we want to use the WSS adapter. Also the cloud adapters were discussed and demoed. More details can be found in this blog post. It is clear that the focus will be more and more on cloud connectivity.

The next deep-dive covered Azure BizTalk Services. This was formerly known as Servicebus EAI/EDI. Like discussed in this blog post it is the next step to bring BizTalk to the cloud. It is clear that BizTalk and Servicebus still are separate products. Both have transformation capabilities, but BizTalk is focused on XML-to-XML transformations using XSLT, where the Servicebus is capable of much more. It is likely that both products will merge.

The evening ended with Q&A. Some nice discussions about the direction of BizTalk. I asked a question about a long time rumor of replacing BizTalk’s XLANG/s engine with Workflow Foundation. Answer: currently there is nothing that you can’t do with the XLANG/s engine that you can do with WF, but the developer experience of WF is much better. It is unlikely that the XLANG/s engine will be replaced by WF soon, but the product team is working on enhancing the developer experience in future versions of BizTalk. Another question from the audience was about low latency scenarios. Because every message travels through the messagebox, it is difficult to implement a low latency scenario. The response of the product team was clear. BizTalk is designed with guaranteed delivery in mind, not low latency. Of course they’re working on improving performance, in BizTalk 2013 and SQL 2012 the team managed to improve performance of for example ordered delivery with 700%, but the motto is guaranteed delivery over guaranteed performance.

It was great talking to the guys that actually build the product and know every edge of the product!

Didago IT Consultancy

Installing and Configuring ESB Toolkit 2.2

With the announcement of BizTalk Server 2013 Beta Microsoft also announced the next minor version of the ESB Toolkit: v2.2

My experience with the previous versions of the ESB Toolkit was that the installation procedure was complex hence preventing  people from using it. Microsoft already announced that the installation of the ESB Toolkit would be simplified and with this blog post I would like to see what has changed. By the way the procedure is documented here.

If you start the setup of BizTalk, the bottom option allows for installation of the ESB Toolkit 2.2.

Also in the ESB Toolkit we have to accept the license agreement.

Next we find the first new screen, which shows that Microsoft keeps its promise. The installation of the ESB Toolkit is very easy, just a matter of selecting the components you need.

Next the regular summary screen before installation starts.

And as always we end with the progress and result screens.

Plain and simple! The components have been installed in C:Program Files (x86)Microsoft BizTalk ESB Toolkit

That’s all that needs to be done installing the ESB Toolkit onto the system, but if you open the BizTalk Administration Console you’ll see nothing has been installed in BizTalk yet.

Installing in BizTalk will be done via configuration. Different from BizTalk is that it’s not possible to start the configuration from the ‘Installation Completed’ screen. The tool needs to be started separately. Once started we see some familiar parts in the tool as well as something new, but first some status-check action is performed.

Then the configuration screen as we know it is started, although I got an error I could bypass it clicking ‘Continue’. I haven’t read anything about this error in other blog posts, so I’m not sure what’s causing it.

This initial configuration screen doesn’t seem to be very much different from v2.1, but if you take a closer look you’ll notice an additional configuration option at the bottom: “ESB BizTalk Applications”.

This is the most interesting part, because the option to enable components will install the ESB application in BizTalk. If you enable it and apply configuration the core components get installed and the BizTalk Administration Console shows a new application has been added.

Besides the application, the ESB also needs policies in the Business Rules Engine. Which have been added.

By the way, it is clear this is a beta, because the BRE version is already changed to the BizTalk Server 2013 version but the picture still shows BizTalk Server 2010.

So the installation and configuration procedure is extremely simplified, which is a great advantage over the previous versions of the ESB Toolkit. This won’t be a showstopper anymore.

But there is also a disappointment: the ESB Toolkit 2.2 still uses Enterprise Library 4.1 where I would expect Microsoft to upgrade it at least to the latest version 5.0 which is around since April 2010. The components used from Enterprise Library are “Microsoft.Practices.EnterpriseLibrary.Common” and “Microsoft.Practices.EnterpriseLibrary.Caching”. Like mentioned in a lot of blog posts (here, here and here for example), this will still be a problem when developers use Enterprise Library 5.0 on their system for other applications while the ESB Toolkit uses 4.1.

Didago IT Consultancy

Installing and Configuring BizTalk Server 2013 beta

This week Microsoft released the first public downloadable beta of BizTalk Server 2013, which is scheduled to be released H1 2013. Curious about the changes in the installation procedure, I decided to grab a Windows Server 2012 VHD, Visual Studio 2012 and SQL Server 2012 to create a base setup for BizTalk Server 2013 beta. I expect not much change in the installation and configuration procedure because it hasn’t really changed the past 6 years and nothing shocking has changed in BizTalk itself.

If you unzip the downloaded BizTalk Server 2013 beta bits, you can start the installation by running the setup.

I picked “Install BizTalk Server 2013 Beta”, and the consumer information screen is shown.

Next the license agreement, which we accept of course. Smile

The following screen is the question if we want to participate in the customer experience improvement program, which we also want. The more people use the beta, the more bugs and issues are found.

Next the components we would like to install, where we select almost everything.

Finally a question about where the installer should get the prerequisites.

Then the summary and the installation process can start.

First the redistributable components.

Then BizTalk Server 2013 beta itself.

After this screen a new screen appears, although this could also be a one-time screen to enable Microsoft Update. At least with me after installing BizTalk Server 2013 beta this screen popped up.

The strange thing is that this step isn’t mentioned in the installation guide and I’m not sure what this means. I hardly can imagine that Microsoft Update would auto-update BizTalk.

After this screen, the installation is finished.

So far nothing really new in the procedure, as expected.

Next the configuration of the BizTalk Server 2013 beta installation. As with the install procedure I don’t expect a lot of change compared to the configuration procedure the past 6 years.

I created a BizTalk service account to be used by the configuration wizard and choose ‘Basic configuration’.

The next screen is also familiar.

As well as the configuration process itself.

Including the SSO warning not to use the admin account.

Now BizTalk Server 2013 beta is installed and configured, we need to perform some standard post-installation steps like mentioned here. Next we can take a look at the new features, which will be the subject of a next blog post.

One final interesting thing is the new BizTalk product version:

With this number all kinds of tools can be updated to recognize the latest offspring.

Didago IT Consultancy

BizTalk Server 2013 beta announced

Tonight Microsoft announced the public download of BizTalk Server 2013 beta.

I’m not going to relist the features because, they are perfectly described here on the blog of the Microsoft BizTalk server team.

Formerly this release was known as BizTalk Server 2010 R2, but it was rebranded with a more future proof name.

Since a few months it was already available for testing purposes on the Windows Azure platform as a Virtual Machine. Testing with Azure Virtual Machines is still possible and the fastest and easiest way to try out the bits.

For our on-premise friends: you can download BizTalk Server 2013 beta here.

The system requirements are picked from the download page and show a platform alignment with the new 2012 series of products.

Supported operating systems: Windows Server 2008 R2 SP1, Windows Server 2012

To run BizTalk Server 2013 Beta you need:

    • 32-bit (x86) platforms: Computer with an Intel Pentium-compatible CPU that is 1 GHz or faster for single processors; 900 MHz or faster for double processors; or 700 MHz or faster for quad processors
    • 64-bit (x64) platforms: Computer with a CPU that is compatible with the AMD64 and Extended Memory 64-bit Technology (EMT64T), 1.7 GHz or faster processor recommended for BizTalk Server 2013 Beta
    • 2 GB of RAM minimum (more recommended)
    • 10 GB of available hard-disk space
    • VGA monitor (1024 x 768) or higher-resolution monitor
    • Microsoft Mouse or compatible pointing device

To use BizTalk Server 2013 Beta you need the following software:

    • SQL Server 2012 or SQL Server 2008 R2 SP1
    • Microsoft .NET Framework 4.5
    • Microsoft Visual Studio 2012 [Required for selected features only]
    • Microsoft Office Excel 2010 [Required for selected features only]
    • SQL Server 2005 Notification Service [Required for selected features only]
    • SQLXML 4.0 with Service Pack 1 [Required for selected features only]
    • Internet Information Services (IIS) Version 7.5 and 7.0 [Required for selected features only]

Interesting to see you can install BizTalk on SQL Server 2012, but still need 2005 Notification Services. Also there is a new enhanced SharePoint adapter (for SharePoint 2013?), but BAM still needs Excel 2010. I assume this will also be aligned with the RTM version.

Another interesting thing is that in previous versions you can install BizTalk on a client version of Windows like Windows XP or Windows 7. Although not recommended, it is possible. In this list only server products are mentioned, does that mean that there won’t be support for Windows 8? In the installation documentation which can be found here, is stated the following Windows versions are supported: Windows Server 2008 R2 SP1, Windows Server 2012, Windows 7 SP1, Windows 8.

If you need a Windows Server 2012 machine to install BizTalk on, you can pick a 180-day trial version on VHD from here. An evaluation version of SQL Server 2012 can be found here.

Have fun!

Didago IT Consultancy

Don’t use the BizTalk 2010 Assembly Checker (BTSAssemblyChecker.exe)!

I’m working on some system engineering documentation for my current customer and of course I’m using the BizTalk Server 2010 Operations Guide as a reference for this. One of the tools is the BTSAssemblyChecker tool which is mentioned here "Checklist: Performing Monthly Maintenance Checks".

One of the steps in the checklist is this one:

“Ensure that the correct version of a set of assemblies is installed on each BizTalk machine (integrity check).”

“Use the BizTalk Assembly Checker and Remote GAC tool (BTSAssemblyChecker.exe) to check the versions of assemblies deployed to the BizTalk Management database and to verify that they are correctly registered in the GAC on all BizTalk Server computers. You can use this tool to verify that all the assemblies containing the artifacts of a certain BizTalk application are installed on all BizTalk nodes. The tool is particularly useful in conjunction with a solid versioning strategy to verify that the correct version of a set of assemblies is installed on each BizTalk machine, especially when side-by-side deployment approach is used. The tool is available with the BizTalk Server 2010 installation media at SupportToolsx86BTSAssemblyChecker.exe.”

If you use the tool, it looks in the wrong GAC (like mentioned in this MSDN forum thread) and therefore is kind of useless, since the job of this tool is to verify the assemblies deployed in the management database are also in the GAC. Besides that, the readme documentation still mentions BizTalk 2006 in several places!

What would be the reason for Microsoft to ship this tool which doesn’t work and with out-dated documentation?

The risk of having assemblies in the management database but not in the GAC still exists, but I’m not aware of an alternative tool to check this.



Didago IT Consultancy

SSO Config Cmd Tool

Every experienced BizTalk developer knows that the SSO store is a good place to safely store configuration information. It is secure, distributed and can be updated without having to dive into configuration files (but too bad the host instances have to be restarted after a change before it is picked up). Accessing the SSO used to be difficult until Richard Seroter wrote a tool for it.

In 2009 Microsoft acknowledged the problems and introduced an MMC snap-in to basically do the same thing, but then with a nice user interface. However so far no command line version appears to be available to support automated deployment. Although the Microsoft package contains MSBuild support, I couldn’t find a commandline version so I decided to write/create one myself.

The tool is no rocket science, most of the code is borrowed from the code the MMC snap-in executes to perform tasks (long live ILSpy).

The tool supports importing, exporting, deleting, retrieving details and listing SSO applications.

With the MMC snap-in there now are two SSO tools available, because also the good old ENTSSO Administration tool is present. The weird thing is that the ENTSSO Administration tool displays more SSO applications than the MMC snap-in does. The reason for this is the strange fact that the MMC snap-in only retrieves SSO applications where the contact information equals .com”>“BizTalkAdmin@<company_specified_in_sso>.com”. This is not always an issue because creating an SSO application automatically sets “BizTalkAdmin@<yourcompany>.com” as the contact email address. It will become an issue if you have to change the email address for example because you don’t have a .com domain…..

Filtering still serves a purpose because there are some default SSO applications that shouldn’t show up in the tool, like adapter configuration settings. In the SSO Config Cmd Tool this potential problem is solved by taking the opposite approach, namely to exclude the contact email addresses “” and “” because they are related to the build-in SSO applications.

Let’s take a look at the tool and go over the actions one by one.

If you open the SSO snap-in, you’ll get this user interface. For demo purposes I already created a ‘TestApp2’ with two keys.

If you run the SSOConfigCmdTool without arguments, you’ll get help about the usage of the tool.

To list all SSO applications, where the contact info is not “” or “”:

To retrieve the details of this ‘TestApp2’ application (as you can see the tool is case insensitive):

To export an SSO application, you need a encryption/decryption key and of course a file to export to. If you omit the extension automatically ‘.sso’ is appended. After the export a file is present at the specified location.

To be able to demonstrate the import, first the delete:

After delete, as expected the SSO snap-in doesn’t display the SSO application anymore:

Next we run the import to reinstall the SSO application. The nice thing is you can supply a different name for it (here ‘ImportedTestApp’ is used’).

Now we run into the issue mentioned above. The SSO snap-in doesn’t display the newly created application, while the good old ENTSSO Administration tool does. This is caused by the filter I previously discussed. The SSOConfigCmdTool doesn’t know your SSO company name, so it imports (creates) the SSO application with contact information ‘’, which will be filtered out by the SSO snap-in.

How to solve this?

One option is not to solve it, because the SSO application is actually there despite the fact that it isn’t displayed.

Another option is to go into the ENTSSO Administration tool and change the contact info to .com”>“BizTalkAdmin@<company_specified_in_sso>.com”, but that would involve manual intervention.

The most easy and sustainable way is to grab the source code from this tool and change “YourCompany” to <company_specified_in_sso>, which is in fact just a single line of code in the Program class.

Just compile it after the change and you’re good to go.

Anyhow, after setting the contact information correctly, the imported SSO application shows up.


The executable and the source code of the tool can be downloaded here at Github.

If you run into an issue, please let me know so I can improve the tool. And of course I’m curious if this tool already existed…..


Didago IT Consultancy

Installing HL7 Accelerator for BizTalk Server 2010

Application integration is all about exchanging messages and messages can only be exchanged if both the sender and receiver know what to expect from each other.

In some branches the standardization of messages and their format is already going on for a while. Examples are:

  • Financial (SWIFT)
  • Automotive (EDI)
  • Supply chain (RosettaNet)
  • Medical (HL7)

If plain BizTalk would be used in those environments it would take quite some time to define the message types according to the existing branch standard. In case of EDI these schemas are already supplied by default when installing BizTalk. For the other standardized schemas, they’re available as a separate accelerator install.

In versions prior to BizTalk 2010 the accelerators were distributed as a separate download (from the MSDN subscriber downloads). In BizTalk 2010 it is no longer a separate download, but part of the BizTalk developer or enterprise edition. This blog post focuses on the installation of the HL7 accelerator, which contains standards used by a lot of medical companies and hospitals to exchange messages.

Want to know more about HL7?

The installation of the HL7 accelerator results in one or more of these items, depending on the selection during installation:

  • Schemas
    • Contains the XSD representation of HL7 messages which are in flat file format in version v2.x
  • Pipelines
    • Converts HL7 messages in flat file format into XML on receive and XML to flat file when sending messages
    • Validates the HL7 message
  • Adapter
    • Minimal Lower Layer Protocol (MLLP) adapter enables BizTalk to receive or send HL7-based messages, which BizTalk Server typically transports using the MLLP protocol. The MLLP adapter ensures that BizTalk Server and BTAHL7 are interoperable with HL7-based messaging applications.
    • Generates acknowledgements for received messages
  • Tools and Utilities
    • Configuration Explorer
    • MLLP Test Tool
    • SDK
    • Logging framework (for auditing and logging purposes)

If you unpack the BizTalk Server install package (developer edition in this case), you’ll see the BizTalk installer and the accelerator items.

The accelerators folder contains the installer for SWIFT, RosettaNet(RN) and HL7.

Double click ‘Setup.exe’ to start the installation procedure.

The account you’re using during installation must be member of the BizTalk Server Administrators group, even if you’re using the administrator account you have to add this account to this group. Otherwise you’ll get the error message below (which is pretty clear by the way).

If that is fixed, you can move to the next screens, which show the license agreement and user information. Next the choice between typical and custom installation should be made.

In the installation guide from Microsoft, there is an overview of which features get installed for which option. The table below is copied from that document.

The custom type install shows this dialog, with only the items selected according to the table above.

For demo and development purposes, I selected all items. This is including the ‘Logging Framework’ and if selected, the next dialog will ask for an account to run the logging service under.

After specifying an account, the “logon as a service” right will be granted to the account to be able to work unattended.

The Logging Framework will install a Windows service named ‘HL7 Logging Service’ (in previous accelerator versions called ‘Auditing and Logging Service’), which runs in the end “<install folder>Microsoft BizTalk 2010 Accelerator for HL7BinAuditingLoggingService.exe”. HL7 data flowing through BizTalk is most of the time sensitive medical and/or private information. The logging framework takes care of logging errors, events and messages for auditing purposes. Logging data can be written to the Windows Event Log, WMI or a SQL Server database. After the summary dialog and install path dialog has been passed, the SQL server must be specified.

This is the last dialog of the installation procedure. The next dialog shows a button to start the installation, which will install and configure the accelerator.

After a few minutes the installation is finished and the artifacts, tools and utilities are installed.

There is one interesting button on this dialog: “Launch Tutorial’”.

This button launches a script which prepares your machine with an end-to-end tutorial, by installing schemas and orchestrations including configuring send and receive ports. Of course you won’t use this button when installing the accelerator on a production machine, but for development purposes this is quite handy to get a head start.


At the end of the installation you’ll find a set of files at the chosen installation location (by default C:Program Files (x86)Microsoft BizTalk 2010 Accelerator for HL7).


Have fun with HL7!


Didago IT Consultancy

BizTalk Server and the ESB SOA design pattern

In January this year I participated in a SOA architect workshop and after 5 exams I officially became certified SOA architect. The certification was issued by SOA schools which is an initiative of the well known SOA author Thomas Erl.

It was a very interesting course which discussed concepts I already knew, but wasn’t really aware of. During the course I more and more was able to map the concepts of SOA to Microsoft technology. One of the SOA characteristics is vendor neutrality so the course didn’t really discuss implementations.

Two course modules discussed design patterns ( and one of them is the ESB compound design pattern. It is called a compound pattern because it is build of other design patterns.

I started to recognize and map parts of BizTalk Server onto the ESB design pattern, but I didn’t have the full picture yet. In this blog post I take the ESB design pattern as a starting point to find out how and if BizTalk Server can be called an ESB based on this design pattern.


The ESB compound design pattern consists of the base pattern with these design patterns:

  • Asynchronous Queuing
  • Service Broker
    • Data Model Transformation
    • Data Format Transformation
    • Protocol Bridging
  • Intermediate Routing

and the extended version is:

  • Reliable Messaging
  • Policy Centralization
  • Rules Centralization
  • Event-Driven Messaging

BizTalk Server (together with the ESB Toolkit) is Microsofts product for implementing an ESB. To what extend does the vendor neutral design pattern map to the functionality in BizTalk Server?

Let’s start with the base pattern.

Asynchronous Queuing

In the SOA world this means that a service should be able to exchange messages with its consumers via an intermediary buffer, allowing service and consumers to process messages independently by remaining temporally decoupled.

BizTalk Server is asynchronous by nature, because all messages flowing through BizTalk are stored in the SQL Server message box thereby disconnecting consumers from services.

This SOA design pattern is explained here.

Service Broker

The service broker design pattern covers some of the most basic concepts necessary for application integration. This is about conversion of messages and the capability to receive messages via various types of protocols.

  • Data Model Transformation
    • A data transformation technology should be incorporated to convert data between disparate schema structures.
  • Data Format Transformation
    • Intermediary data format transformation logic needs to be introduced in order to dynamically translate one data format into another
  • Protocol Bridging
    • Services using different communication protocols or different versions of the same protocol should be able to exchange data.

Both types of transformations are solved in BizTalk Server using maps. Model transformation is about transforming one XML Schema to another and format transformation is transforming one message format to another, for example flat file or CSV to an XML Schema.

Protocol bridging is implemented in BizTalk Server using adapters and is about the ability of communication between different communication protocols like different versions of SOAP or something totally different like FTP.

Intermediate Routing

From a SOA perspective this is about the ability to dynamically determine message paths though the use of routing logic. Intermediate routing is typically implemented using service agents. A service agent is an event-driven program that doesn’t require explicit invocation, it just intercepts the messages. There are two types of service agents, active and passive. Active service agents not only read the message but also modify the message contents. Passive service agents only read the message and will or will not take action on the message content but doesn’t change it.

In BizTalk Server intermediate routing is implemented by means of a message agent. This agent is monitoring the message box to handle subscriptions to messages. In BizTalk everything is routed via the message box, so it can be implemented with only promoted properties or there can be complete routing logic in orchestrations or via business rules.

Reliable Messaging

This is about assuring that a message is actually received to the service. In a SOA world this is a technology concept to enable reliable communication with services. Commonly this is solved by a combination of the WS-ReliableMessaging standard and guaranteed delivery extensions like a persistent repository.

In BizTalk Server this is called guaranteed delivery, which assures to the caller that once a message is received in BizTalk Server it never will be lost. This is the persistent repository in the section above. The WS-ReliableMessaging standard is not supported by BizTalk due to its architecture.

Policy Centralization

In the context of SOA a service contract can be comprised of one or more XML Schemas, the WSDL, one or more Policies and an SLA. A policy adds information to the service commonly applied using the WS-Policy standard. This information can for example consist of the service window of a service or that it is only capable of handling so many requests an hour. Policy centralization is about applying centrally maintained policies to services.

Looking at BizTalk Server it seems like the ESB Toolkit supports some kind of policy centralization, but that appears to be related to itinerary centralization. Policy centralization probably should be maintained by a 3rd party product and cannot be maintained in BizTalk Server or with the ESB Toolkit.

Rules Centralization

Business rules are critical to companies and having the rules spread across a company is not a sustainable solution. The rules centralization design pattern is about centrally accessing, storing and managing business rules.

Business rules are part of BizTalk Server by means of the Business Rules Engine(BRE). This is even a separate component that can be used without other BizTalk components. Rules can be centrally accessed, stored and maintained in there. In a SOA approach the BRE would be wrapped into a service so it’s functionality is accessible in a standardized way from other technologies besides BizTalk Server. Wrapping such a component into a service raises other issues like introducing a single point of failure and capacity issues. These issues can be solved by applying other SOA design patterns

Event-Driven Messaging

From a SOA perspective this is about notifying customers of events. Commonly this is solved by subscribing customers to service events via one-way and asynchronous data transfer.

This is one of the most known concepts in BizTalk Server, because it refers to the publish-subscribe pattern as it is used in BizTalk Servers messaging architecture with the SQL Server message box as the heart of the system.


In this blog post we’ve been discussing the official ESB compound design pattern as defined by Thomas Erl. At least one of the requirements cannot be fulfilled by BizTalk Server, and that is Policy Centralization. So officially BizTalk Server is not an ESB product according to these requirements.

On the other hand, policy centralization is one of the least used requirements in an ESB implementation at the moment. And besides that, BizTalk Server implements many more design patterns than just the ones mentioned in the ESB design pattern.

On of the most recognizable design patterns for BizTalk Server is the Orchestration compound design pattern.

Although this compound pattern is targeted at synchronous processing, it does consist of interesting design patterns we can also find in BizTalk Server:

  • Process Abstraction
    • This is about abstracting logic into a parent processes (task services). In BizTalk Server processes can be constructed of combined (agnostic) services using orchestrations. An orchestration is an abstraction of such parent processes which can be called by services without knowledge of the underlying logic.
  • Process Centralization
    • This is to prevent process logic to be spread across the company. All processes in BizTalk Server are centralized into one repository thereby fulfilling this requirement.
  • State Repository
    • This is about offloading state to a repository to release resources when they are not necessary at that time. BizTalk Server supports this by means of hydration and dehydration of processes.
  • Compensating Service Transaction
    • To prevent system locks during long running processes, compensating measures can be taken in case of an exception. BizTalk Server supports compensating service transactions as part of orchestration building blocks.
  • Atomic Service Transaction
    • To be able to rollback changes if one of the services in a chain of service calls fail, atomic service transactions can be used. BizTalk Server supports atomic service transactions in it’s WCF adapter.
  • Rules Centralization
    • Is also part of the ESB compound design pattern
  • Data Model Transformation
    • Is also part of the ESB compound design pattern

Didago IT Consultancy

Azure Service Bus EAI/EDI April 2012 Refresh, What’s new?

Back in December 2011 Microsoft released the first CTP of the Azure Service Bus EAI/EDI. It showed what direction Microsoft is heading with bringing BizTalk like functionality to the cloud. Or to quote Microsoft:

“Windows Azure Service Bus EAI and EDI Labs provides integration capabilities for the Windows Azure Platform to extend on-premises applications to the cloud, provides rich messaging endpoints on the cloud to process and transform the messages, and helps organizations integrate with disparate applications, both on cloud and on-premises. In other words, Service Bus EAI and EDI Labs provides common integration capabilities (e.g. bridges, transforms, B2B messaging) on Windows Azure Service Bus”

If you want to read more about this, please take a look at my two previous blog posts about this topic:

BizTalk in the Cloud, one step closer (part 1)

BizTalk in the Cloud, one step closer (part 2)

Now Microsoft has released the second preview:

Service Bus EAI and EDI Labs – April 2012 Release

Download the April 2012 Release

So what’s new in this April 2012 release?

Regarding the EAI Bridges, we’ve got the following enhancements. BizTalk developers will recognize parts that currently only are available in BizTalk….

  • FTP Source Component. In the December 2011 release it was already possible to connect WCF endpoints via the publish/subscribe pattern. In this refresh another endpoint type has been added: FTP, which can be used as a source component on the configuration bridge. In the BizTalk world this would be the FTP adapter, but so far it is reading only.
  • Flat File Support. This is also a familiar concept in the BizTalk world. Combined with the FTP source component, flat files for example can be read and converted into XML now. But also via HTTP flat files can be read and processed. The available flat file wizard looks quite the same as currently can be found in BizTalk Server.
  • Schema Editor. Previously only available in BizTalk, now also part of the Service Bus EAI/EDI release. Used to create and edit XSD and flat file schemas
  • Service Consuming Wizard. Also straight from BizTalk Server, the possibility to generate the necessary artifacts to be able to consume WCF services. The difference with EAI is that it only generates an XSD and no other output files like binding info and an orchestration.
  • Operational Tracking of messages processed by the bridge. This is also something from BizTalk Server. Within a bridge, a message undergoes processing under various stages and can be routed to configured endpoints. Specific details of the message such as transport properties, message properties, etc. can be tracked and queried separately by the bridge developers to keep a track of message processing.

For Service Bus Connect, there are the following enhancements.

  • Support for Windows Authentication while connecting to on-premise SQL Servers
  • User Interface enhancements
  • Support for SSL

The transforms also have been improved.

  • New map operations. In BizTalk this is called ‘Functoids’ but are named operations here. Added are:
    • Number format operation, to format an input number using a given format
    • Date/Time operation, to manipulate date/time
    • Generate ID operation, to generate a unique ID of a given length
  • Runtime behavior configuration. This is also interesting for BizTalk developers, because like Microsoft says:”In this release, users can configure the runtime behavior for how the transform handles exceptions, errors, null values, and empty data input. This configuration can be done for each transform”. This sounds exciting and probably something we’ll be seeing in the next version of BizTalk Server.

Finally the EDI or Business to Business Messaging, which now has support for:

  • Processing batched EDI messages. You can now use the TPM portal to configure agreements that can process batched messages coming from trading partners
  • Tracking and archiving EDI messages. You can now use the TPM portal to track messages as they are processed using the agreements

I’m looking forward dive into the details! BizTalk Server in the cloud again one step closer!

Didago IT Consultancy

  • Recent Posts
  • Recent Comments
  • Archives
  • Categories
  • Meta