Category name:ESB

Installing and Configuring ESB Toolkit 2.2

With the announcement of BizTalk Server 2013 Beta Microsoft also announced the next minor version of the ESB Toolkit: v2.2

My experience with the previous versions of the ESB Toolkit was that the installation procedure was complex hence preventing  people from using it. Microsoft already announced that the installation of the ESB Toolkit would be simplified and with this blog post I would like to see what has changed. By the way the procedure is documented here.

If you start the setup of BizTalk, the bottom option allows for installation of the ESB Toolkit 2.2.

Also in the ESB Toolkit we have to accept the license agreement.

Next we find the first new screen, which shows that Microsoft keeps its promise. The installation of the ESB Toolkit is very easy, just a matter of selecting the components you need.

Next the regular summary screen before installation starts.

And as always we end with the progress and result screens.

Plain and simple! The components have been installed in C:Program Files (x86)Microsoft BizTalk ESB Toolkit

That’s all that needs to be done installing the ESB Toolkit onto the system, but if you open the BizTalk Administration Console you’ll see nothing has been installed in BizTalk yet.

Installing in BizTalk will be done via configuration. Different from BizTalk is that it’s not possible to start the configuration from the ‘Installation Completed’ screen. The tool needs to be started separately. Once started we see some familiar parts in the tool as well as something new, but first some status-check action is performed.

Then the configuration screen as we know it is started, although I got an error I could bypass it clicking ‘Continue’. I haven’t read anything about this error in other blog posts, so I’m not sure what’s causing it.

This initial configuration screen doesn’t seem to be very much different from v2.1, but if you take a closer look you’ll notice an additional configuration option at the bottom: “ESB BizTalk Applications”.

This is the most interesting part, because the option to enable components will install the ESB application in BizTalk. If you enable it and apply configuration the core components get installed and the BizTalk Administration Console shows a new application has been added.

Besides the application, the ESB also needs policies in the Business Rules Engine. Which have been added.

By the way, it is clear this is a beta, because the BRE version is already changed to the BizTalk Server 2013 version 3.10.171.0 but the picture still shows BizTalk Server 2010.

So the installation and configuration procedure is extremely simplified, which is a great advantage over the previous versions of the ESB Toolkit. This won’t be a showstopper anymore.

But there is also a disappointment: the ESB Toolkit 2.2 still uses Enterprise Library 4.1 where I would expect Microsoft to upgrade it at least to the latest version 5.0 which is around since April 2010. The components used from Enterprise Library are “Microsoft.Practices.EnterpriseLibrary.Common” and “Microsoft.Practices.EnterpriseLibrary.Caching”. Like mentioned in a lot of blog posts (here, here and here for example), this will still be a problem when developers use Enterprise Library 5.0 on their system for other applications while the ESB Toolkit uses 4.1.

Didago IT Consultancy

Installing and Configuring BizTalk Server 2013 beta

This week Microsoft released the first public downloadable beta of BizTalk Server 2013, which is scheduled to be released H1 2013. Curious about the changes in the installation procedure, I decided to grab a Windows Server 2012 VHD, Visual Studio 2012 and SQL Server 2012 to create a base setup for BizTalk Server 2013 beta. I expect not much change in the installation and configuration procedure because it hasn’t really changed the past 6 years and nothing shocking has changed in BizTalk itself.

If you unzip the downloaded BizTalk Server 2013 beta bits, you can start the installation by running the setup.

I picked “Install BizTalk Server 2013 Beta”, and the consumer information screen is shown.

Next the license agreement, which we accept of course. Smile

The following screen is the question if we want to participate in the customer experience improvement program, which we also want. The more people use the beta, the more bugs and issues are found.

Next the components we would like to install, where we select almost everything.

Finally a question about where the installer should get the prerequisites.

Then the summary and the installation process can start.

First the redistributable components.

Then BizTalk Server 2013 beta itself.

After this screen a new screen appears, although this could also be a one-time screen to enable Microsoft Update. At least with me after installing BizTalk Server 2013 beta this screen popped up.

The strange thing is that this step isn’t mentioned in the installation guide and I’m not sure what this means. I hardly can imagine that Microsoft Update would auto-update BizTalk.

After this screen, the installation is finished.

So far nothing really new in the procedure, as expected.

Next the configuration of the BizTalk Server 2013 beta installation. As with the install procedure I don’t expect a lot of change compared to the configuration procedure the past 6 years.

I created a BizTalk service account to be used by the configuration wizard and choose ‘Basic configuration’.

The next screen is also familiar.

As well as the configuration process itself.

Including the SSO warning not to use the admin account.

Now BizTalk Server 2013 beta is installed and configured, we need to perform some standard post-installation steps like mentioned here. Next we can take a look at the new features, which will be the subject of a next blog post.

One final interesting thing is the new BizTalk product version: 3.10.171.0

With this number all kinds of tools can be updated to recognize the latest offspring.

Didago IT Consultancy

BizTalk Server 2013 beta announced

Tonight Microsoft announced the public download of BizTalk Server 2013 beta.

I’m not going to relist the features because, they are perfectly described here on the blog of the Microsoft BizTalk server team.

Formerly this release was known as BizTalk Server 2010 R2, but it was rebranded with a more future proof name.

Since a few months it was already available for testing purposes on the Windows Azure platform as a Virtual Machine. Testing with Azure Virtual Machines is still possible and the fastest and easiest way to try out the bits.

For our on-premise friends: you can download BizTalk Server 2013 beta here.

The system requirements are picked from the download page and show a platform alignment with the new 2012 series of products.

Supported operating systems: Windows Server 2008 R2 SP1, Windows Server 2012

To run BizTalk Server 2013 Beta you need:

  •  
    • 32-bit (x86) platforms: Computer with an Intel Pentium-compatible CPU that is 1 GHz or faster for single processors; 900 MHz or faster for double processors; or 700 MHz or faster for quad processors
    • 64-bit (x64) platforms: Computer with a CPU that is compatible with the AMD64 and Extended Memory 64-bit Technology (EMT64T), 1.7 GHz or faster processor recommended for BizTalk Server 2013 Beta
    • 2 GB of RAM minimum (more recommended)
    • 10 GB of available hard-disk space
    • VGA monitor (1024 x 768) or higher-resolution monitor
    • Microsoft Mouse or compatible pointing device

To use BizTalk Server 2013 Beta you need the following software:

  •  
    • SQL Server 2012 or SQL Server 2008 R2 SP1
    • Microsoft .NET Framework 4.5
    • Microsoft Visual Studio 2012 [Required for selected features only]
    • Microsoft Office Excel 2010 [Required for selected features only]
    • SQL Server 2005 Notification Service [Required for selected features only]
    • SQLXML 4.0 with Service Pack 1 [Required for selected features only]
    • Internet Information Services (IIS) Version 7.5 and 7.0 [Required for selected features only]

Interesting to see you can install BizTalk on SQL Server 2012, but still need 2005 Notification Services. Also there is a new enhanced SharePoint adapter (for SharePoint 2013?), but BAM still needs Excel 2010. I assume this will also be aligned with the RTM version.

Another interesting thing is that in previous versions you can install BizTalk on a client version of Windows like Windows XP or Windows 7. Although not recommended, it is possible. In this list only server products are mentioned, does that mean that there won’t be support for Windows 8? In the installation documentation which can be found here, is stated the following Windows versions are supported: Windows Server 2008 R2 SP1, Windows Server 2012, Windows 7 SP1, Windows 8.

If you need a Windows Server 2012 machine to install BizTalk on, you can pick a 180-day trial version on VHD from here. An evaluation version of SQL Server 2012 can be found here.

Have fun!

Didago IT Consultancy

BizTalk Server and the ESB SOA design pattern

In January this year I participated in a SOA architect workshop and after 5 exams I officially became certified SOA architect. The certification was issued by SOA schools which is an initiative of the well known SOA author Thomas Erl.

It was a very interesting course which discussed concepts I already knew, but wasn’t really aware of. During the course I more and more was able to map the concepts of SOA to Microsoft technology. One of the SOA characteristics is vendor neutrality so the course didn’t really discuss implementations.

Two course modules discussed design patterns (http://www.soapatterns.org/) and one of them is the ESB compound design pattern. It is called a compound pattern because it is build of other design patterns.

I started to recognize and map parts of BizTalk Server onto the ESB design pattern, but I didn’t have the full picture yet. In this blog post I take the ESB design pattern as a starting point to find out how and if BizTalk Server can be called an ESB based on this design pattern.

 

The ESB compound design pattern consists of the base pattern with these design patterns:

  • Asynchronous Queuing
  • Service Broker
    • Data Model Transformation
    • Data Format Transformation
    • Protocol Bridging
  • Intermediate Routing

and the extended version is:

  • Reliable Messaging
  • Policy Centralization
  • Rules Centralization
  • Event-Driven Messaging

BizTalk Server (together with the ESB Toolkit) is Microsofts product for implementing an ESB. To what extend does the vendor neutral design pattern map to the functionality in BizTalk Server?

Let’s start with the base pattern.

Asynchronous Queuing

In the SOA world this means that a service should be able to exchange messages with its consumers via an intermediary buffer, allowing service and consumers to process messages independently by remaining temporally decoupled.

BizTalk Server is asynchronous by nature, because all messages flowing through BizTalk are stored in the SQL Server message box thereby disconnecting consumers from services.

This SOA design pattern is explained here.

Service Broker

The service broker design pattern covers some of the most basic concepts necessary for application integration. This is about conversion of messages and the capability to receive messages via various types of protocols.

  • Data Model Transformation
    • A data transformation technology should be incorporated to convert data between disparate schema structures.
  • Data Format Transformation
    • Intermediary data format transformation logic needs to be introduced in order to dynamically translate one data format into another
  • Protocol Bridging
    • Services using different communication protocols or different versions of the same protocol should be able to exchange data.

Both types of transformations are solved in BizTalk Server using maps. Model transformation is about transforming one XML Schema to another and format transformation is transforming one message format to another, for example flat file or CSV to an XML Schema.

Protocol bridging is implemented in BizTalk Server using adapters and is about the ability of communication between different communication protocols like different versions of SOAP or something totally different like FTP.

Intermediate Routing

From a SOA perspective this is about the ability to dynamically determine message paths though the use of routing logic. Intermediate routing is typically implemented using service agents. A service agent is an event-driven program that doesn’t require explicit invocation, it just intercepts the messages. There are two types of service agents, active and passive. Active service agents not only read the message but also modify the message contents. Passive service agents only read the message and will or will not take action on the message content but doesn’t change it.

In BizTalk Server intermediate routing is implemented by means of a message agent. This agent is monitoring the message box to handle subscriptions to messages. In BizTalk everything is routed via the message box, so it can be implemented with only promoted properties or there can be complete routing logic in orchestrations or via business rules.

Reliable Messaging

This is about assuring that a message is actually received to the service. In a SOA world this is a technology concept to enable reliable communication with services. Commonly this is solved by a combination of the WS-ReliableMessaging standard and guaranteed delivery extensions like a persistent repository.

In BizTalk Server this is called guaranteed delivery, which assures to the caller that once a message is received in BizTalk Server it never will be lost. This is the persistent repository in the section above. The WS-ReliableMessaging standard is not supported by BizTalk due to its architecture.

Policy Centralization

In the context of SOA a service contract can be comprised of one or more XML Schemas, the WSDL, one or more Policies and an SLA. A policy adds information to the service commonly applied using the WS-Policy standard. This information can for example consist of the service window of a service or that it is only capable of handling so many requests an hour. Policy centralization is about applying centrally maintained policies to services.

Looking at BizTalk Server it seems like the ESB Toolkit supports some kind of policy centralization, but that appears to be related to itinerary centralization. Policy centralization probably should be maintained by a 3rd party product and cannot be maintained in BizTalk Server or with the ESB Toolkit.

Rules Centralization

Business rules are critical to companies and having the rules spread across a company is not a sustainable solution. The rules centralization design pattern is about centrally accessing, storing and managing business rules.

Business rules are part of BizTalk Server by means of the Business Rules Engine(BRE). This is even a separate component that can be used without other BizTalk components. Rules can be centrally accessed, stored and maintained in there. In a SOA approach the BRE would be wrapped into a service so it’s functionality is accessible in a standardized way from other technologies besides BizTalk Server. Wrapping such a component into a service raises other issues like introducing a single point of failure and capacity issues. These issues can be solved by applying other SOA design patterns

Event-Driven Messaging

From a SOA perspective this is about notifying customers of events. Commonly this is solved by subscribing customers to service events via one-way and asynchronous data transfer.

This is one of the most known concepts in BizTalk Server, because it refers to the publish-subscribe pattern as it is used in BizTalk Servers messaging architecture with the SQL Server message box as the heart of the system.

      Conclusion

      In this blog post we’ve been discussing the official ESB compound design pattern as defined by Thomas Erl. At least one of the requirements cannot be fulfilled by BizTalk Server, and that is Policy Centralization. So officially BizTalk Server is not an ESB product according to these requirements.

      On the other hand, policy centralization is one of the least used requirements in an ESB implementation at the moment. And besides that, BizTalk Server implements many more design patterns than just the ones mentioned in the ESB design pattern.

      On of the most recognizable design patterns for BizTalk Server is the Orchestration compound design pattern.

      Although this compound pattern is targeted at synchronous processing, it does consist of interesting design patterns we can also find in BizTalk Server:

      • Process Abstraction
        • This is about abstracting logic into a parent processes (task services). In BizTalk Server processes can be constructed of combined (agnostic) services using orchestrations. An orchestration is an abstraction of such parent processes which can be called by services without knowledge of the underlying logic.
      • Process Centralization
        • This is to prevent process logic to be spread across the company. All processes in BizTalk Server are centralized into one repository thereby fulfilling this requirement.
      • State Repository
        • This is about offloading state to a repository to release resources when they are not necessary at that time. BizTalk Server supports this by means of hydration and dehydration of processes.
      • Compensating Service Transaction
        • To prevent system locks during long running processes, compensating measures can be taken in case of an exception. BizTalk Server supports compensating service transactions as part of orchestration building blocks.
      • Atomic Service Transaction
        • To be able to rollback changes if one of the services in a chain of service calls fail, atomic service transactions can be used. BizTalk Server supports atomic service transactions in it’s WCF adapter.
      • Rules Centralization
        • Is also part of the ESB compound design pattern
      • Data Model Transformation
        • Is also part of the ESB compound design pattern

      Didago IT Consultancy

      ESB Toolkit 2.1

      September 10, 2010 / No Comments on ESB Toolkit 2.1

      Via a blog post it came to my attention that the BizTalk user groups of Australia and New Zealand organise a combined event around BizTalk Server 2010. It is called BizTalk Saturday. It looks like a very nice event, so if your in the neighbourhood you should attend.

      One of the presentations I found scheduled is about the ESB Toolkit 2.1. Off course I know about the ESB Toolkit 2.0, but 2.1 hasn’t had so much attention yet.

      It is a release especially for the platforms supported with BizTalk Server 2010 like Visual Studio 2010. Information can be found here.

      So, what is new in this release? From the Microsoft site:

      The ​Microsoft BizTalk ESB Toolkit 2.1 extends the capabilities of BizTalk Server 2010. The following list summarizes the additional support extended in ​Microsoft BizTalk ESB Toolkit 2.1:

      • Added support for Visual Studio 2010 Visualization and Modeling SDK for the Itinerary designer.
      • Support both .NET Framework version 4 and .NET Framework 3.5.
      • Supports the use of itineraries developed in ​Microsoft BizTalk ESB Toolkit 2.0.
      • The Itinerary designer now supports copying and pasting itinerary shapes.
      • A new itinerary project template, BizTalk ESB Itinerary designer, provides an easy option to create itinerary projects under BizTalk Projects.
      • Opening a new project in the Itinerary designer sets the export mode to Strict by default.
      • Added a new itinerary model property, Require encryption certificate, in the Itinerary designer to enable and disable encryption certificate while validating the itinerary. This flag is set to True by default.

      The picture below shows the ESB Toolkit 2.1 core engine components. Although I haven’t spend a lot of time on this release yet, one thing that caught my eye was the mentioning of Enterprise Library 4.0 while Enterprise Library 5 has been released for some months. Also the UDDI 2.0 is in the picture, while the toolkit requires UDDI 3.0 according to this documentation.

      So althought the ESB Toolkit 2.1 download isn’t mentioned as a beta download (while BizTalk 2010 is), the documentation is marked as prerelease and inconsistent in some places.

      BizTalk Server 2009 beta and ESB Guidance 2.0 CTP are public!

      Last night the good news of BizTalk Server 2009 beta being public received me, finally!

      It consists of the following items:

      12/8/2008BizTalk Server 2009 Beta – Release NotesDocumentBizTalk Server 2009 Beta – Release Notes
      12/8/2008BizTalk Server 2009 Beta – Installation and Configuration GuideDocumentBizTalk Server 2009 Beta – Installation and Configuration Guide
      12/5/2008BizTalk Server 2009 Accelerators Beta Evaluation DownloadBuildBizTalk Server 2009 Accelerators Beta Evaluation Download
      12/4/2008BizTalk Server 2009 Public Beta Evaluation DownloadBuildBizTalk Server 2009 Public Beta Evaluation Download

      Also the ESB Guidance 2.0 CTP was released, with system requirements like Microsoft BizTalk Server 2009 and Microsoft Visual Studio 2008 SP1  🙂 I wrote in a previous post about the features of this 2.0 release.

      To be complete, I also noticed a new release of BizUnit 3.0 now with support for creating test cases via an object model and reading test cases from an Excel sheet.

      So how about that? A lot of cool BizTalk news! I know I have something to do the coming weeks.

      HowTo: Using XLANGMessages in Custom Exceptions in BizTalk

      On the project I’m working on at the moment we’re using custom exceptions in BizTalk. We have adopted the exception handling part of the ESB guidance and so the custom exception consists of a string indicating what to do next and an XLANGMessage object to store messages in.

      There was something not working as expected so I had to debug it. I wasn’t very familiar with custom exceptions in BizTalk so I searched the internet for some information. On this blog by Tomas Restrepo I found some interesting tips and information:

      Tip #1: Exceptions are not serializable by default. If you create a custom exception class, it is your responsibility to ensure it is serializable.

      Tip #2: Marking an exception class with [Serializable] is not enough. System.Exception implements ISerializable, so it forces you to do so as well.

      Here’s what you should keep in mind when writing an exception class:

      • Mark the exception type with the [Serializable] attribute.
      • Add an empty protected serialization constructor that simply delegates to the base class: protected MyException(SerializationInfo info, StreamingContext ctxt) : base(info, ctxt)

      For most exception classes, this will be enough, since most don’t actually add new properties and simply rely on the message.

      However, if your custom exception class has custom properties, then it MUST override ISerializable.GetObjectData() (and don’t forget to call the base one), and you MUST unpack those properties in your serialization constructor.

      While testing the solution I found out that something went wrong after the custom exception was returned to the orchestration. All information was present in the object but the messages could not be retrieved. I was puzzled. It looked like something in the serialization/deserialization process of the XLANGMessage object failed, but if one object should be serializable it should be this object. Right?

      Further research showed that the XLANGMessage object was serialized/deserialized ok, but the XLANGPart object wasn’t. This resulted in a correct XLANGMessage structure, with the correct number of parts in it, but with empty content. As a last resort I posted a question on the MSDN forum.

      Within a day I had the solution. It turned out that the XLANGMessage property containing the XLANGParts data wasn’t public so with binary serialization it was skipped. That at least explained this behavior, but how to solve it? In fact that was also in the reply (thanks Douglas Marsh) and in the Tomas Restrepo blog. You have to manually get the parts that aren’t serializable and put them in the SerializationInfoStore in the GetObjectData method. Like this:

               public override void GetObjectData(SerializationInfo info, StreamingContext context)
              {
                  base.GetObjectData(info, context);
                  if (info != null)
                  {
                      info.AddValue("Fault", this.Fault, typeof(XLANGMessage));
                      // Get the message parts, they aren't serializable so we have to do it by hand
                      // Get the contents of the parts, convert them to a byte[] and add them to the store
                      if (this.Fault != null)
                      {
                          info.AddValue("NumberOfFaultParts", this.Fault.Count, typeof(Int32));
                          for (int i = 0; i < this.Fault.Count; i++)
                          {
                              info.AddValue("FaultMsgPart" + i, Helper.XmlDocumentToByteArray((XmlDocument)this.Fault[i].RetrieveAs(typeof(XmlDocument))), typeof(byte[]));
                          }
                      }
                  }
              }

      When deserializing you can get them back in the serialization constructor, like this:

              protected MyCustomException(SerializationInfo info, StreamingContext ctxt)
                  : base(info, ctxt)
              {
                  if (info != null)
                  {
                      // get all the public members
                      this.Fault = (XLANGMessage) info.GetValue("Fault", typeof(XLANGMessage));
                      // Get the message parts, they aren't serializable
                      // This has to be in a try block, because the 'NumberOfFaultsParts' isn't always there and there is no way of knowing in advance
                      try
                      {
                          int faultPartsCnt = info.GetInt32("NumberOfFaultParts");
                          for (int i = 0; i < faultPartsCnt; i++)
                          {
                              XmlDocument doc = Helper.ByteArrayToXmlDocument((byte[])info.GetValue("FaultMsgPart" + i, typeof(byte[])));
                              this.Fault[i].LoadFrom(doc);
                          }
                      }
                      catch (Exception ex)
                      {
                          Trace.WriteLine("XLANGPart serialization error: ", ex.Message);
                      }
                  }
              }

      So after all it wasn’t that difficult at all, but isn’t that always the case?? Thanks to everybody helping me.

      Importing ESB BAM definitions without Excel

      I was working on a PC on which I wanted to install the BAM definitions that come with the ESB guidance for exception handling.


      So I browsed to the <installpath>ESB Exception Handling InstallInstallBAM folder to execute “deploy_exception_tracking.cmd”.


      Unfortunately I was unable to run it because it requires Excel to be present to process the Excel file containing the definitions and that wasn’t installed on that machine. On the other hand I wasn’t allowed to install Excel so I had to convert the XLS to XML which also can be used to import the definitions. On my external drive I found an VPC with Biztalk 2006 and Excel installed and used that to let Excel generate the XML.


      With renewed energy I changed the ESB deployment file “bmdeploy.cmd” to use the XML instead of the XLS. If you want be able to undeploy it again, also change the “bmundeploy.cmd”.


      It ran some time and then returned the following error:


      Updating Activity… ERROR: The BAM deployment failed.
      Encountered error while executing command on SQL Server “VPC-BT2006”.
      New request is not allowed to start because it should come with valid transaction descriptor.


      After some searching I found that it had to do with a bug in ADO.NET that would be fixed in SP2 of SQL Server 2005. So I installed it and tried again. Again the same error. Further research was necessary to find the cause.


      At the end it turned out to be a SQL Hotfix fixing it because SP2 didn’t.


      The XML file with the representation of the BAM definitions in XML can be found here.


      Some other thing that I rediscovered is the location of the Excel plugin for BAM. I was used to BizTalk 2004 where BAM.XLS was the Excel sheet to use. Now that one is gone and a plugin is used. Microsoft tells you where to look for it:
      If Excel 2003 is not installed, the file BAM.xla is installed to its default location at %System Drive%Program FilesMicrosoft BizTalk Server 2006Exceldir. If Excel is installed, it can directly be added as plugin using the Tools menu.