So now that you know how to solve the problem of
receiving messages out of order, you canproperly receive messages,
resequence them, and send them in the correct order to their
destination. But what about batch scenarios—what happens in the case
where you need to combine the received messages, order them, and then
aggregate them into a batch? This need is quite common in the EDI world
where batching has existed for years. Also, in many legacy system
integration projects, large aggregated data files are needed to load
downstream systems with data. For example, let's look at the company ABC
example. Assume that orders received from the FTP bulk upload are
compressed and need to be sent as a batch to the fulfillment system,
which only understands flat text files. Luckily you have all the pieces
you need to solve this except one. To recap, you can
Use the Unzip component to decompress the files.
Use the resequencing pipeline and pipeline component from the resequencing example to add the sequence information.
Use the resequencing orchestration to order them properly.
However, you still have no
way to A) combine the messages into one message, and B)turn the sorted
list of the messages received into a flat file. Luckily, with BizTalk
2006 this becomes a trivial task. There are, however, two possible
solutions to this problem depending on how the messages are received. If
the messages are not received as described, but arereceived as
independent Interchanges, you need to use the Resequencer pattern
described previously. If they are indeed received as a batch, a more
elegant solution is available.
Solution 1: Status Quo—Messages Received As Independent Interchanges
If the messages are
received independently, a simple change is required to the
resequencingorchestration to allow the pattern to work. All you need to
do is not send
the messages as they arrive and wait to receive all messages before
sending them. Once all the messages in the batch are received, you can
use the new Call Pipeline from Orchestration code that ships with
BizTalk 2006 to send the SortedList
to a send pipeline that contains a flat-file Assembler component. So
long as the messages are defined with a schema that uses the flat-file
extensions of BizTalk, the pipeline will return a text message with all
the messages inside aggregated in their proper order. The orchestration
will look something like Figure 6-9.
The code for the Call Send Pipeline Expression shape will look like this:
//Initialize pipeline output
AggregatedOutMessage = null;
//Add the messages to the list via a component
PipelineHelper.AddSortedMessages(SortedList, SendPipeMsg)
Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteSendPipeline
(typeof(ABC.BizTalk.Pipelines.FlatFileSendPipeline),SendPipeMsg,
AggregatedOutMessage);
You will need to use a Construct shape, since the shape is creating new messages via thecomponent. The SendPipeMsg is of type Microsoft.XLANGs.Pipeline.SendPipelineInputMessages. The reason why you need to add these messages to the SendPipeMsg
via a component is because within the component you will need to loop
through each of the messages stored in the sorted list, and add them one
at a time to the SendPipeMsg to ensure
that the order is properly maintained. Alternatively, you could use a
Loop shape in the orchestration, but this is much easier done in code.
Once this is done, the send pipeline is called, the messages are
aggregated, and the complete message is sent to the send port.
Solution 2: Not So Status Quo—Messages Received As a Batch
In the
resequencing aggregator outlined earlier, the messages are received as a
completeInterchange. They may be out of order when stored inside the
ZIP file, but they are completely batched as a whole. In this case, you
actually don't need a resequencing orchestration and can implement this
solution entirely within a pipeline and use content-based routing.
Let's reexamine our list of pipeline components from previous examples:
Unzip component: The Unzip component extracts the ZIP file contents and createsa multipart message based on the files included.
Resequencing Decoder: The component examines a message and determines whereit fits within a unique sequence.
What if you modified
the Resequencing Decoder to handle multipart messages? In thisscenario,
the component would have additional smarts to know that multipart
messages can also be resequenced. In this case, the component would
simply loop through each of the message parts, examine its data,
determine what number it is in the sequence, and promote that property
along with the unique sequence ID and the last message in the sequence
property. Also in this case, you need to use another message part
feature—they have a property bag. So instead of promoting the values to
the message context of the message, you simply write these values as
key/value pairs to the parts property bag. Once all the message parts
are resequenced properly, the message can flow through BizTalk like any
other message. The send port hosting the FlatFileAssembling component
would be subscribed for messages of this type, parse the incoming
messages, and output one aggregated message, which is sent to the
downstream system.
|
While the preceding
example is technically easier to implement, it has some drawbacks.
First, as stated earlier, since pipeline execution is stateful, should
the pipeline fail for whatever reason, processing stops. You will
manually need to have some error-handling logic and subscribe to the
failed message for processing. This is explored in the next section.
Also, care should be taken to properly performance test the solution. If
the ZIP file is large and contains a high number of individual
transactions, the CPU on this machine could be affected. Make sure that
you test this scenario properly before implementing it. |
