steven :: Release 2.0

Tuesday, August 22, 2006

Xml Schema Validation in WCF and Web Services - a Discussion

I recently read the excellent Windows Communication Foundation HandsOn by Craig McMurty - it is an excellent read as it is practical and goes through the code at a level that lets you fill in the detail and concentrates on the code that makes WCF an absolutely amazing product. The more i have read, more more interested i have got with one of the most exciting developments to come out of Redmon for a while. If you have ASMX and particularly WSE experience, then you'll be set to start, but WCF offers a full solution and is even extensible to allow you to add many customizations.

However, one particular part of the discussion caught my attention. It came in chapter 3 and was under the title "Xml Fetish". I do agree that it is easy to have everything in Xml and go way beyond the actual requirements, but Craig (or whoever wrote that particular chapter!) says:

"... control over how data is represented in XML is generally of no use to software developers.."

The preceding sections portray a view of Schema based contract development in a relatively negative light, with a preference for software developers caring only about the interface for communication. There is also the mention of interoperability, with DataSet being a particular example, but the .Net DataSet type is in no way related to Xml Schema or even contract first development - indeed many a software developer in the past (and future) adds a DataSet as a parameter and never worries about its representation on the wire or interop - hence hitting problems that wouldn't not necessarily happened with an Xml Schema standards based definition of the types in the contract (i even remember being at Chris Sells Web Services DevCon in 2003 where a standards based DataSet was loosely discussed).

I also read an MSDN article that hinted about these issues without really mentioning any [1] :

"There are several more sophisticated XML Schema concepts you can apply when using XmlSerializer, but using these more advanced features can often lead to interoperability problems across frameworks."

I'm not sure whether these issues relate to Xml Schema (a W3C standards), or some proprietary data embedded in the Schema. I'm sure there are

issues as the author of the article writes excellent articles, but there is just no detail of what these are. Further, in the same article, one can read:

"The result is great flexibility. You're free to define new data contracts in either XML Schema or .NET code and you can easily convert to the other. If you're working in a situation where the XML Schema definitions already exist and you need to support them, you can simply start with svcutil.exe. If you're defining new contracts and need to quickly, start by writing class definitions."

There is an obvious grey area in here - where you don't "need to start quickly", but don't have existing schemas. If you could go back and forwards between .Net and Xml Schema, then one may argue "whatever you are most confortable with". However, Xml Schema is inherently more powerful than the .Net types for defining contracts, mainly because you get some level of business validation on all platforms - in .Net you really only get the Interface.

As a simple example, consider a user ID. Perhaps you have a restriction that the user id is at least 3 charcaters and at most 20 characters. Your contract may accept a .Net "int" type, but that leaves the possible values that can be sent to be ANY valid .Net integer. With Xml Schema, you easily get validation that the value is within the contraints defining the length of the user id. Xml Schema can provide quite complex validation rules and through patterns, regular expressions can be used.

So why do this? Well, if you have a number of clients calling your services, it would be good to ensure information has at least gone through some basic validation before it hits your server. It's very similar to how client scripting helps in web browsers - not as a last resort as validation is still done on the server, but exceptions for example are expensive to raise on the server, so you can get a performance improvement here alone. There are a host of reasons why this would also be useful at the server too, such as providing an external validation before hitting your core internal services and so on.

If Microsoft allowed you to decorate your properties and so on with some equivalent of the configuration validation attributes [2], you may be able to get something close to what is available in Xml Schema - at least at a primitive level - the DataContractSerializer MaxItemsInObjectGraph uses exactly this technique.

At a basic level, would it not make sense for Microsoft's WCF to provide a behavior that could be switched on to says "validate against schema"?

My view at the moment (and hopefully i can get some feedback on this) is that using Xml Schema for contract based development offers some fairly hefty benefits, with pretty much zero work. In fact, the only reason there IS work is that the Microsoft web services frameworks require you to do quite a bit of work to USE your Schemas for validation (certainly outwith the skills of your casual web service developer). Please see my net post for some implementation details and samples in WSE 3.0 and WCF.

At the moment i am working with a set of services within a financial organization that is called from a Java service and a Win32 C++ application, the latter passing pure soap messages. In both these cases Xml Schema validation is useful as the actual call is expensive only to discover your "city" is not a valid entry (and so on).

I'd like to hear arguments and examples of why Schema should be so difficult to turn on and then perhaps i will start to understand these problems - in my *simple* examples, other than the code to perform the validaition itself, i ran into no problems, so i'm open to suggestions!

[1] http://msdn.microsoft.com/msdnmag/issues/06/08/ServiceStation/default.aspx
[2] http://msdn2.microsoft.com/en-us/library/system.configuration.configurationvalidatorbase.aspx

# posted by weblivz @ 10:19:00 pm Comments:

This comment has been removed by a blog administrator.

# posted by

Anonymous : Wednesday, August 23, 2006 8:31:00 am

Thanks for the kind words about the book. I did indeed write the passage to which your post refers.

You make a good point about the capability of being able to express constraints on values in schema. And, at least for me, its being able to express them that is the key point--I would not want to actually do schema validation, because in practice that turns out to be a bottleneck. However, to convey the nature of my constraints to the consumers of my service, schema would certainly be an option, and let us agree that it is a better one that writing prose documentation.

Here is a snag though: I lovingly craft all the constraints into my schema, incorporate the schema in my WSDL, and publish that. The consumers walk up and grab the WSDL. Their next step is to generate code from the WSDL. I'll bet the constraints just disappeared. To cite your example, they'll see an int, not an int between 0 and 100.

What just happened? Well, we got let down by the tooling around XML again. I vividly remember sitting in an office in North York City Center in Ontario, reading about XML for the first time in an article on MSDN. The author talked about how we were all going to be unindated with great XML tools. Well, that didn't happen on any platform. Surely the will was there, because the tool vendors had to see the market opportunity. I think that anyone that starts writing an XML tool quickly runs up against the sheer excessive complexity of spec, which of course looks simple compared to its forbears, but is still too much for the job for which it was intended.

But I digress. How would I communicate my validation constraints to my consumers? I would use fault contracts--those should show up in generated code as possible exceptions that can be thrown by the operations. And I would not do schema validation for the reason I cited and a couple more that I didn't--I would do the validation I need (which I think is minimal for most folk in practice) in code (which is easier anyway), and throw back a nice, meaningful fault if my tests fail.

But you still have a good point. And I didn't consider it when I wrote the chapter.

# posted by

Anonymous : Wednesday, August 23, 2006 3:27:00 pm

Let's accept that we lack XML tooling when it comes to .NET/JAVA and do not make any excuse about it.
This is really frustrating to see lack of XML schema support in .NET even in its third revision. The point that when client generate the code and will loose the constraints is not valid because here you are making an assumption that all the clients or server use some form of programming language. But what about integration products like webMethods, TIBCO, BizTalk do they lack the tooling too? Not able to completely convert the XML schema in C# is my greatest frustration with .NET, especially when you are working in industry like Insurance where messages are predefined through standard body like ACORD. I feel so helpless when I loose simple constraints like patterns, key/keyref/unique and end up writing those validation in C# classes manually. Why should I write validation code if my schema already define those constraints? Wouldn't it be wonderful my code generator generates that code for me or better put some .net attributes on top of serialized classes?
- Gaurav

# posted by

Anonymous : Tuesday, October 03, 2006 4:23:00 pm