Introduction to Reliable Messaging
SOAP is a high-level protocol in the overall Web services protocol stack whose main job is to provide a package for XML data, the metadata needed by the recipient, and a binding to a transport. Therefore, SOAP relies on lower-level protocols, such as HTTP, SMTP, and TCP to get the SOAP message and its data payload from endpoint to endpoint. When you send a SOAP message using TCP, HTTP, or any other potential transport protocol for SOAP, these transports see the message path only as a pair of endpoints. Unlike the protocols to which they’re bound, SOAP messages need a more holistic view of where they’re headed. Certain Web service implementations may require a SOAP message be modified by a specific sequence of Web servers before reaching its ultimate destination. A Web service may also need to be able to delegate the processing of an incoming message to other services. There might even be a need during this message routing process for a SOAP message to be transferred from one transport to another, say from HTTP to TCP. Of course, the core SOAP specification doesn’t provide a mechanism for reliably addressing SOAP messages through this kind of complex message routing.
While you generally think of Web services using a two-way, request-response messaging behavior, Web services also support other types of messaging behaviors, including asynchronous messaging. The Web Services Description Language specification (WSDL) identifies the following Web service messaging behaviors:
Request-response in this synchronous messaging scenario, the client sends a request message to the Web Service and receives response message back from the service.
One-way a type of asynchronous messaging, the client sends a message, but a response message is not required or expected. This might be used when a client needs to send period status updates to a Web service.
Solicit-response The Web Service sends a message to the client, and the client then sends back a correlated reply message. This might be used when a Web service needs to poll clients for status information.
Notification another type of asynchronous messaging; the Web Service sends a message to one or more clients without requiring a response. This might be used for clients who have subscribed to event notifications from a Web service.
An equally important concern in SOAP messaging is the issue of message reliability. If we cannot guarantee that messages will be transmitted reliably between endpoints, all of the expected benefits of Web services cannot be realized. Since we do live in a world of unreliable networks, there needs to be a mechanism to enable a message sender to verify that one or a series of SOAP messages have indeed reached the intended recipient. Therefore, reliability for SOAP messaging must address the following issues:
Message ordering The need to order a series of related messages regardless of when they actually arrive.
Message acknowledgment The need to be able to send a response back to the sender stating that one or more messages in a series have been received.
Message delivery policies The need for an endpoint to be able to request specific reliability behaviors for requests, such as the number of times that a failed message should be re-sent and how long to wait before resending.
is transport independent, and WS-ReliableMessaging, which defines mechanisms for message acknowledgment and ordering and for declaring messaging policies.
in real- world applications, your Web service might need to implement a more complex message-routing scenario, such as dynamic, decision-based routing; routing through firewalls or intermediaries; or even using multiple transport protocols. For example, consider a Web service application that has been scaled out to run on multiple Web servers and behind a firewall. In such a scenario, a load- balancing application could be used to route request messages to whichever Web server was best able to handle the load. For example, such an application might determine, based on relative loads, whether to route a request to a processing center located on the East Coast or one located on the West Coast. In a multiple-message transaction, it would then be important that all future requests be sent directly to the application center where the original request was being handled. If this level of specificity for addressing is not supported in the message itself, then the routing application would need to do extra work to determine which processing center should handle all subsequent incoming messages from the client and would likely require the expense of having to maintain a server-side process that includes state information.
In a scenario like this, it is important to have an addressing mechanism that transcends the simple point-to-point functionality provided by the message transports. Among other benefits, such a mechanism would enable any intermediary Web services, such as routers, to have all of the addressing information that they need to ensure that the messages end up in the right place. To provide this consistent addressing for Web services, the WS-Addressing specification defines two constructs for conveying message addressing information: endpoint references and message headers. In this definition, an endpoint is simply an addressable resource to which a SOAP message can be sent; the resource can be a Web service or client application, a SOAP router, or any other SOAP-aware entity.
Addressing Endpoint References
In the reliable messaging model, SOAP messages are sent between and via endpoints. To address a messaging endpoint, we need to be able to explicitly, uniquely, and consistently reference such endpoints. As an additional hurdle, endpoint references must be discoverable so that messages can be addressed dynamically. Of course, as with Web Services Description Language (WSDL), before an endpoint reference can be consumed it must first be described. WS- Addressing introduces a mechanism for describing endpoint references.
From a WSDL standpoint, endpoint description information would most logically be defined within the Service element of a WSDL document; however, WSDL 1.1 does not allow for the extensibility of this element. To get around this limitation, WS-Addressing defines a new typed EndpointReference element that can be used within a Web service description document, such as WSDL, to describe a referenced endpoint. The EndpointReference element supports the following child elements:
Address Provides the Uniform Resource Identifier (URI) of the endpoint, which can be a network or logical address.
ReferenceProperties Provides application-specific information that must be supplied when addressing messages to a particular endpoint.
PortType Specifies the WSDL port type of the referenced endpoint.
ServiceName Specifies the WSDL service that defines the referenced endpoint. This element can also include a PortName attribute that references the name of the WSDL port definition for the endpoint.
Policy Can contain one or more policy assertions, which can be used to describe the policies that are enforced at an endpoint.
The following is an example of an endpoint reference for the CollectionService Web service:
In this example, the Policy element would contain a policy assertion element, which I’ll describe along with a detailed discussion of Web service policies in Chapter 6. By adding this endpoint reference to the WSDL file that describes the CollectionService Web service, you can dynamically assign a policy behavior to the endpoint. In addition to being used in description documentation, the EndpointReference element is also the base type used when describing an endpoint in one of the SOAP message headers defined by WS- Addressing.
Addressing Message Headers
In addition to defining a mechanism for referencing message endpoints, WS- Addressing also specifies a set of message headers that can be used during transport to define the endpoints on the message path. When endpoint information is transported in SOAP headers, this information can be accessed by messaging intermediaries when they need to make intelligent routing decisions. The following SOAP message header elements, which are child elements of the Header element, are defined by WS-Addressing:
To Contains information about the immediate message recipient in the form of a URI.
From Contains information about the message sender in the form of the endpoint reference type.
ReplyTo Contains information about the endpoint to which a reply message should be sent in the form of the endpoint reference type.
Recipient When a message is being routed via one or more intermediaries, can be used to specify an intended ultimate message recipient and is in the form of an endpoint reference type.
FaultTo Contains information about the endpoint to which any fault response messages generated by the request should be sent, in the form of the endpoint reference type.
Action A URI that represents the action for the message, which can be the same as the value of SOAPAction in an HTTP message header.
MessageID A URI that is used to uniquely identify a message.
RelatesTo Used to relate a message to another message by relationship type. This optional element contains a URI that is the MessageID value of the related message. The relationship type is specified using the element’s RelationshipType attribute, where at this point the only accepted value is Response.
The following SOAP request message to the DocumentService shows the use of WS-Addressing message headers.
In this example, the URI value in the From element that defines the endpoint of the requesting party is http://schemas.xmlsoap.org/ws/2003/03/addressing/role/anonymous . This URI is used when a fixed URI cannot be assigned to the Address value of an endpoint. Of course, this anonymous URI value cannot be used with the To element, and if the value is specified in the ReplyTo or FaultTo element, there must be an alternative mechanism for communicating this information.
WSE supports the headers defined in WS-Addressing and, by default, generates a set of addressing headers for all outbound SOAP messages. Also, when incoming messages contain addressing headers, the WSE input filters read these headers and generate the relevant programming objects. As I’ll show later in this chapter, the WSE API can be used to add or change the values of these message headers or to access the addressing information for incoming messages.
Message Reliability for Web Services
While WS-Addressing helps to solve the challenges of consistently describing and accessing Web service endpoints, the general issues of getting messages reliably between message endpoints are being addressed in another specification. A Web service needs to be able to determine when messaging has become unreliable so that it can adjust accordingly, which might involve resending certain messages or delaying the processing of messages until other messages arrive. To that end, IBM, BEA Systems, and Microsoft have again teamed up to draft WS-ReliableMessaging, a specification that attempts to guarantee the reliability of SOAP messaging by defining message acknowledgments, message ordering, and reliability policies
Message Ordering
When a Web services–based interaction comprises a series of SOAP messages, it’s conceivable that because of transmission on unreliable networks these messages might not be received in the order in which they were sent. When messages in a series are lost or delayed, the interaction can become unreliable. This can cause problems that require advanced application logic for handling cases when the message sender assumes that as a result of an earlier message the Web service is in a certain state, but when messages are received out of order, this might not be the case.
To track the order of Web service requests, WS-ReliableMessaging introduces the Sequence SOAP header element. When a client communicates with a Web service implementing WS-ReliableMessaging, the Sequence element is used to number the messages in a sequence, set a time-out period for a sequence, and to declare the end of a sequence. The Sequence element implements the following child elements:
Identifier A unique identifier for the sequence, which leverages the Identifier element defined in the Web Service Security Addendum specification and which is qualified against the http://schemas.xmlsoap.org/ws/2002/07/utility namespace.
MessageNumber A value that defines the sequence number for the message, wherein numbers are assigned as consecutive unsigned integers starting at 1 and increasing by 1 throughout the lifetime of the conversation.
LastMessage Declares that a message is the last in the sequence.
Expires Declares a time in the future when the sequence expires, whether or not the LastMessage element has been used.
The following is an example of a message containing the Sequence header; this message is the third in a sequence, which is defined by the GUID in the Identifier element:
Message Receipt Acknowledgments
Since the Web service architecture enables asynchronous messaging, there is no requirement that a response message be returned for any given request message, as there would be in two-way messaging. However, from the standpoint of message reliability, such an acknowledgment in essence closes the loop on the request process. In asynchronous messaging and without receipt acknowledgments, the requesting party has no way of knowing whether the request message was received. In this case, if there was unreliability on the network, it will likely be manifested later in the interaction as failures that result from having both parties in a different application state. For example, an application might send a one-way request to open a “shopping cart” session, with items to be added in later requests. If this request was not received, subsequent requests to add items to the “cart” will fail since the Web service never received the first request to create the cart session.
One solution might be simply to return a response message that a new cart session was created, essentially making it a two-way messaging scenario. While this dedicated acknowledgment message solves this reliability issue, it is inefficient to generate such a response message, particularly in situations in which both parties are regularly exchanging messages. In such cases, it would be more efficient to piggyback the acknowledgment in the header of an otherwise unrelated message. WS-ReliableMessaging proposes the use of a SequenceAcknowledgment header element that is used to return a receipt acknowledgment for one or more messages in a given sequence, either in an arbitrary response message or in a response created exclusively to return the acknowledgment. A SequenceAcknowledgment element supports the following child elements:
Identifier A unique identifier for the sequence, which leverages the Identifier element defined in the Web Service Security Addendum specification and is qualified against the http://schemas.xmlsoap.org/ws/2002/07/utility namespace.
AcknowledgmentRange Defines a range of one or more sequence numbers that identifies messages that have been received, where the range is defined by the integer values of the Upper and Lower attributes.
An acknowledgment for a single message in a sequence will have the same sequence number for both Upper and Lower attributes. If the range of messages in the sequence being acknowledged is not contiguous, more than one AcknowledgmentRange element can be used. The following example shows the inclusion of a SequenceAcknowledgment element in an acknowledgment response message:
This acknowledgment indicates that message number 3 in the sequence was not received and that the requester should resend this message.
A Web service can wait for a negotiated amount of time before returning an acknowledgment, which gives it time to attach it to another outgoing message. However, once this limit has been reached, the service must generate a response message solely to return the acknowledgment, as in the preceding example. Of course, at any point the AckRequested header element can be sent with any message to the Web service to explicitly request that an acknowledgment for the sequence be returned. This element supports the following child element:
IdentifierA unique identifier for the sequence, which leverages the Identifier element defined in the Web Service Security Addendum specification and which is qualified against the http://schemas.xmlsoap.org/ws/2002/07/utility namespace.
The following message includes a request for a series acknowledgment:
Reliable Messaging Policies
In addition to the functionalities just discussed that use reliability headers, WS- ReliableMessaging also defines a set of policy assertions that can be published by a Web service to declare which reliable messaging functionalities are required when accessing the endpoint. These policy assertions include the following general categories:
Specification version The version of WS-ReliableMessaging supported by the endpoint.
Delivery assurance The number of times a message can be delivered and whether message sequences must be delivered in order.
Sequence expiration The lifetime of a messaging sequence.
Inactivity time-out The time-out period after which a sequence is considered terminated.
Retransmission interval The time that an application should wait for an acknowledgment before resending the message.
Acknowledgment interval The time that a Web service should wait for a response message to be sent back to the requester before generating a stand-alone acknowledgment message.
WSE Support for Reliable Messaging
In version 2.0, WSE supports the WS-Addressing specification, but WS-ReliableMessaging is not yet supported. WSE support for WS-Addressing consists of a set of input and output message filters that read and write the SOAP headers defined in WS-Addressing. WSE automatically adds the appropriate addressing headers to all outgoing SOAP messages. In addition, WSE provides a full API for programmatically creating and accessing addressing objects, and you can explicitly define the addressing objects for outgoing messages to override the values that are automatically generated by WSE.
Creating Custom Address Headers
One of the things that you can do with WSE is to explicitly define the ReplyTo and FaultTo headers in an outbound message. This lets you redirect this type of response message to a different endpoint than the one from which the request was originally sent. When using the WSE, this can be accomplished programmatically by instantiating both a ReplyTo and a FaultTo object in the Microsoft.Web.Services.Addressing namespace, defining appropriate address URI values for each, and adding them to the SoapContext for the outgoing message, as in the following example:
// Create a new SoapContext for the request messageSoapContext myContext = myService.RequestSoapContext;
// Define a different reply address for the requestReplyTo replyAddress = new ReplyTo(new Uri("http://example.com/DocumentService/ResponseHandler.asmx"));
// Define a reply address for fault messagesFaultTo faultAddress = new FaultTo(new Uri("http://example.com/DocumentService/FaultHandler.asmx"));
// Add the custom ReplyTo and FaultTo addressing headers to the SoapContext myContext.Addressing.ReplyTo = replyAddress;myContext.Addressing.FaultTo = faultAddress;
// Security headers added here
// Call the GetDocument methodmyService.GetDocument(docID);
Based on this code, WSE generates the following SOAP request message:
In the same way, the values of address headers in an incoming message can be accessed by reading the relevant property from the AddressingHeaders object, which is accessed from the SoapContext.Addressing property of the incoming message.
Implementing Customized Message Routing Using WSE
In addition to backing WS-Addressing and WS-ReliableMessaging, Microsoft also initially floated a pair of Web service messaging specifications, WS-Routing and WS-Referral, as part of their initial vision of the overall Web services architecture. The likelihood of these two specifications becoming Web service standards seems to be in doubt as no one besides Microsoft has stepped up to support them. Also, much of the functionality described in WS-Routing is now being provided by WS-Addressing, which has the benefit of a much broader base of support.
WSE version 1.0 supported both WS-Routing and WS-Referral, and these functionalities have been carried forward in WSE 2.0. However, since WSE 2.0 also supports WS-Addressing, it is unclear whether this support will continue. As this book goes to press, the WSE team is deciding whether or not to remove support for WS-Routing and most of WS-Referral that was included in WSE 1.0 from the final release of the 2.0 version, so don’t be surprised to find changes in these messaging functionalities in the final version of the product. Since the pre-Beta version that I am working from still supports WS-Routing and WS-Referral, I will discuss these specifications and their WSE implementation because if you are looking to implement a decision-based message routing topology, WSE likely provides the best implementation for message routing without having to write it all from scratch. Remember, however, that this type of routing topology will prove useful only between endpoints that implement WSE.
The WS-Routing Model
Rather than the endpoint-based mechanism of WS-Addressing, WS-Routing defines a routing model based on a message path. In this model, when a Web service creates a new message to be sent using a WS-Routing–based topology, the message has a path element block in the message header that contains all of the information required to route messages. The initial sender and ultimate receiver in a routing topology are represented by the from and to elements, respectively. The following example shows a message created by an endpoint named Tom and sent directly to an endpoint named Fred:
As you can see in this example, the required action element is analogous to the SOAPAction field in the HTTP header and is used to specify a URI that represents the intended action for the SOAP message. Of course, the from and to elements show that this message was sent from Tom to Fred. The id element specifies a GUID to uniquely reference the message. The to, from, action, and id elements are created by the initial sender of the message and are not modified by any other intermediary endpoints along the message path. The other two elements (which in this example are empty) are fwd and rev, which represent the forward and reverse message paths.
When sending messages along a complex message path, you can specify both the forward path that the message should take to the ultimate receiver and the reverse path that the response message should take back to the initial receiver. In addition, SOAP routers along the message path modify the forward and reverse message paths as needed by the routing topology. The following example assumes that when a Web service router, named Bettie, forwards the message on to Fred, a new via element is created in the reverse path so that the message will return via Bettie:
Also, note in this example that Bettie removed the topmost via element in the forward path and added it to the reverse path before resending the message.
In the preceding example, you might have noticed that all of the URI values in the to, from, and via elements are prefixed with soap rather than http. This is because WS-Routing introduces a new URI scheme to use when addressing Web services that are being used as SOAP routers and that are implementing WS-Routing. This scheme prefixes the URI in the routing path block with soap, which is used regardless of the underlying transport. Of course, this addressing mechanism is applicable only to WS-Routing and does not extend to any other reliable messaging specifications.
In WS-Routing, any intermediaries that move a message along the message path are specified using the via element, wherein the order of the via elements in the message’s fwd element indicates the order of the intermediaries. If a message does not have a to element, the last via element denotes the ultimate destination of the message.
The complete forward message path can be created by the initial sender of a WS-Routing message, or it can be constructed and modified by the various SOAP routers as the message moves along the message path. In fact, in many cases the exact message path isn’t known to a sender or routing endpoints dynamically modify the message path based on the contents of the message, and there’s no reason why the service that the sender thinks is the ultimate receiver can’t continue to send the message along paths unknown to the sender.
After a SOAP router validates the routing information for the incoming message, the first change that the router must make to the message header is to remove the topmost via element that contains its own URI. A Web service can add more via elements when there is a need to forward the message to additional Web services. It can also add a via element that represents a recommended return path for the response message as the topmost via element in the reverse path described by the rev element. After this, the service must send the message on to the URI listed in the topmost via element in the forward path or to the URI listed in the to element if no more via elements exist in the forward path.
The WS-Referral Model
WS-Routing describes a mechanism for a Web service acting as a SOAP router to modify via elements in the message routing header for the purpose of providing dynamic message routing capabilities. The question that WS-Routing doesn’t answer, or any other existing specification for that matter, is how individual SOAP routers negotiate these message paths and how routing topologies might be constructed. WS-Referral describes a mechanism by which a Web service examines the message path for endpoints that match a set of referrals the service knows about. When a match is found, the service can change the message path to send the message to the preferred endpoint instead. The set of referrals that a SOAP router needs to know about can be maintained locally, although WS-Referral does specify how to do this. WS-Referral also enables a SOAP router to query for referral statements at other routers as well as to request that a router accept a new referral statement.
WS-Referral clearly builds upon WS-Routing, but the two specifications are designed to be orthogonal, and WS-Referral can be implemented with any scheme that defines a SOAP actor or endpoint. The end result of WS-Referral is that it enables a routing topology to dynamically route messages more easily to support advanced routing behaviors such as failover, load balancing, progressive path discovery, and outsourcing of service hosting. Perhaps the primary benefit of WS-Referral is the ability to separate the external endpoint for a Web service from the actual services that perform the work, thus insulating the client from any knowledge of the back-end architecture of the routing topology and providing a fixed address that’s independent of any activity at the back end.
Referral Statement
At the core of the WS-Referral specification is the notion of a referral statement. Unlike the routing header, which is contained in the SOAP message being routed, the referral statement is maintained locally at the SOAP router, where it’s used to provide the necessary logic for conditional message routing. The following referral statement present on the SOAP router named Tom dictates that if a message is received that’s intended for a SOAP router named Fred, it should first be forwarded to a SOAP router named Bettie on its way to Fred.
When Tom processes an incoming message in which the value of one of the via elements or the to element is equal to soap://fred.example.com, the service adds a via element equal to soap://bettie.example.com to the forward message path in the header and forwards the message to Bettie. When Bettie receives the message, that service will continue to follow the routing path in the header and send the message on to Fred. As long as Tom persists this referral statement, all messages received by Tom on their way to Fred will be forwarded through Bettie.
The way that a SOAP router uses a referral statement is that when a message is received, the Web service checks the referral statements that it knows about for a match between a URI specified in the for element of the referral statement and the to element or any via elements in the forward message path in the routing header. When a match is found, the logic in the if element is checked. If both are satisfied, the message is forwarded to one of the addresses specified in the go statement. In addition, two SOAP endpoints can communicate to negotiate new referrals through a process of referral querying and registration.
Referral Query
Rather than rely on a static message routing topology in which referrals never change or must be updated manually at a machine, WS-Referral specifies a mechanism through which individual SOAP routers can communicate to coordinate referral information and better route messages. The first part of this mechanism is a referral query, which is a two-way messaging exchange by which one router queries for referral statements that are registered at a second SOAP router. If Web service Fred queries Tom for any referrals relating to messages headed for Bettie, they might look like the following request message:
In this example, Fred uses the for child of the query element in the message body to ask Tom whether any referral statements for handling messages are currently on the way to Bettie.
In a two-way exchange, each request must have a response. Upon receiving the previous request, Tom checks the referrals in the referral cache for any referrals that match the value of the for element. If any referrals match, they are returned in the query response message. If no matching referrals exist, an empty response is returned. The following is an example referral query response to the previous message:
Notice that the entire referral statement that matches the supplied for value is returned inside a queryResponse block in the message body. If multiple referral statements match the query, each is returned in a separate ref block. Referral queries provide a way to get a referral statement from a SOAP router. The second part of the referral negotiation mechanism involves registering a referral statement at a second SOAP router.
Referral Registration
A Web service router can request that another router either accept or reject a referral statement. Like a referral query, referral registration is a two-way message exchange. A registration request from Fred to Tom might look like this:
In this example, Tom is requested to register the referral to Bettie that we saw earlier. If Tom accepts the referral statement, the statement is appended to the referral cache and a positive registration response is generated as follows:
The presence of the registrationResponse element indicates that the registration succeeded. If Tom fails to accept the referral, a fault is generated as follows:
Using query and registration requests, it’s possible to dynamically maintain a routing topology by adding or removing referrals when necessary. The invalidates element is used to dynamically remove a referral statement from the cache. When an accepted referral contains an invalidates element, its child vid element references another referral that should no longer be used.
Unfortunately, dynamic referral registration is an ideal mechanism for staging man-in-the-middle attacks. If a malicious user can trick a router into accepting a counterfeit referral, that router can then be used to send all subsequent messages to a malicious service. For this reason, before a SOAP router accepts a referral from another SOAP router, a secure level of trust must first be established between the two routers. At a minimum, both parties should be using digital signatures and a trusted mechanism to establish mutual authentication between both services, such as are described by WS-Security and WS- SecureConversation, which I discuss in Chapters 5 and 8, respectively.
As an alternative to the referral query and registration exchange processes, WS-Referral also permits referral statements to be transported in the SOAP message header, which lets you essentially piggyback referrals with regular SOAP messages. When you do this, the collection of referrals is sent within the referrals block in the SOAP header. In this type of exchange, good security practices should still be observed.
WSE Support for WS-Routing and WS-Referral
The support for WS-Routing and WS-Referral in WSE is implemented in the routing and referral pipeline filters. When you send a Web service request using WS-Routing, WSE lets you add one or more Path objects to the SoapContext for the outgoing message. Within the Path object, you can define a collection of Via objects in the Fwd object that defines the endpoints along the forward message path for each desired destination on the message path. When the message is processed by the routing output filter, this information is used to generate the prescribed routing header, which conforms to WS-Routing.
Likewise, the routing input filter processes the routing header information for incoming messages, generates a Path object related to the SoapContext object for the incoming message, and removes the routing header information. This routing information persists in the SoapContext for this message until all other message processing is complete and the message is ready to be forwarded to the next Web service in the message path. When the message is sent, it’s processed by the routing output filter, where the routing header is re-created using the persisted path information. The new request message is then sent on to the next SOAP router in the message path.
WSE manages referral statements locally using a referral cache, which is denoted by the referrals element and contains a referral element for each referral that the local service knows about. Before an incoming message hits the routing input filter, it’s handled by the referral input filter. This filter checks the incoming message for referral headers and referral query or registration requests. If referrals exist, the filter generates a ReferralCollection object to contain the referral header information. The filter also scans the message routing path for any matches with referral statements stored locally in the referral cache. When matches are found, the values of the via elements are modified according the rules in the referral statement.
No comments:
Post a Comment