http://silkworm.talis.com

Essentially, we’re working on services out in the cloud that provide content discovery, interoperability and access services for content providers. I can highly recommend the white paper by our CTO Justin, available for download from above.

more coming soon.

Seems PragDave has picked up on people doing it for real and even has it ‘by default’ on his Apple G5.

SOA is like the Night Sky…
It’s All in a Name: What’s a Service?

The timing was really interesting…

because I’ve been conversing about a similiar problem on the project I’m on. We have three organisation, Alice, Bob and Charlie. Alice and Bob both have contact with each other and a number of shared clients; they both keep their own records about those clients in their own databases. Charlie is responsible for analysing Alice’s and Bob’s data, ensuring they reconcile and producing statistical analysis from it.

The process, however, is that Alice and Bob are contractually obliged to provide their data to Charlie, but not to accept any changes back. As both Alice and Bob are also limited in their technology capabilities the route that has always been taken is for them both to provide a dump of their data “whenever they’ve made some changes”.

Charlie is also considering providing dumps of data back to both Alice and Bob.

As both Alice and Bob are, in effect, working on caches of subsets of the data that Charlie has (although they don’t think of it that way) the latency of starlight image is very instructuive here. Alice and Bob only know what the system used to be like.

My suggestion is that when Alice and Bob send through their changes they say what they want to change from… i.e.

Alice: “The data said ‘baz’ when I looked at it, please change it to ‘quux’ for me”
Charlie: “Yeah, the value is still ‘baz’, I’ll change it”
Bob: “The data said ‘baz’ when I looked at it, please change it to ‘pre’ for me”
Charlie: “Sorry, the data has been changed to ‘quux’ since then, try again”

I’m sure there is a name for this pattern somewhere, it can’t be new. The only problem I have now is to find an easy way for Alice and Bob to produce these change logs from inside their systems…

You know the situation, flashy sales demo is followed by a purchase, the CTO/Strategy Team/Architecture Group/Project Sponsor hands you the box and says “we’re going to use this for our foobits logic processing”.

Well, here are two subversive patterns I’ve used to tackle the problem…

Pre-Requisites: pinch of salt.

The first pattern is for situations where you need the lack of suitability to be visible; perhaps to get the product withdrawn or perhaps to pass the pain of the product back to the selector. In this case the most appropriate pattern is the ‘Zero-Touch’ pattern.

In this you simply install and configure as standard a deployment of the platform as possible. Perhaps even allowing the vendor to perform the deployment design and sign it off. You perform no integration. Allow the users to switch from one to the other, or provide simple links between the two for things like web apps.

If you’re right and the product is unsuitable then this will result in the project failing. The system will hopefully not get past UAT or pilot, but even if it doesn get into live the lack of integration will likely kill it later. Of course, if youre wrong and have to accept the product you may have a lot of work to do tidying up the mess created by the two systems being seperate.

The second pattern is what I like to call the ‘False-Touch’ pattern. This is for use where the technology is unworkable but you need the project to succeed and you maybe wish to hide the misguided selection.

In this pattern you sit the new system alongside your existing systems in exactly the same way you would if you were genuinely using the system. Then, within the existing system, you make calls to the new system that provide just enough accuracy to make it appear to work. The key, though, is that you ignore all responses and build your own implementation in the existing system. Ideally calls to the duff system are done asynchronously, further reducing any dependance.

The elegance of this pattern is that the duff system becomes irrelevant and because the production system maintains it’s performance and stability profiles it is very unlikely anyone will ever investigate which bit is actually doing the work.

We started by discussing what it was that made mechanical production systems easier to monitor than IT systems and were talking about visibility. This rapidly turned into a discussion about factories, and then cars and their engines. Put two blokes together and if they don’t end up talking about football they’ll end up talking about cars.

Only this time it was interesting becuase we realised that visibility is not the key. It’s audibility. The Hum of The Machine. When you’re driving you can’t see the engine, you rely on the sound of the car a lot – it’s the clunking sound that forces you to take a look or take it to the garage. It’s the same effect that allows an experienced supervisor on a printing press, or other automated mechanical machine, to sit and read a book or watch TV safe in the knowledge that all is well. If the tone or rhythm changes then the book gets put down and further investigation takes place.

But with IT systems the things that are most important to us are silent. I think the idea of having web servers shout out when they’ve successfully completed a request (or thousand) is very compelling and engaging at a level that systems monitoring tools like Patrol and OpenView just aren’t. The key difference between Rachel’s thoughts and the ideas Steve and I discussed is that Steve and I suggest having sound as the normal, healthy state as with a mechanical system.

The other concept we decided was worth pursuing was the idea of variable monitoring and tracking of individual transactions. A concept taken from, and named after, The Barium Swallow – a medical investigative procedure that involves swallowing a mildly radioactive liquid and watching the passage of that through the oesophagus on-screen.

The equivalent we discusssed here was the ability to send genuine requests into a live system, but tagged in some way, to trigger a higher level of logging and/or alerting. This kind of facility would allow diagnosis of problems in live systems easier and more viable than simply turning up the logging as it would provide detailed logging for a single transaction rather than the vast volumes of logging that you get otherwise.

If anyone knows of any commercial products doing this then I’d love to hear from you.

This reminded of the heady days of Egg during the dotcom boom…

Launching products and building systems faster than people could keep up with. We destroyed a few people during those days, but the buzz was amazing.

Anyway, it reminded me specificallyof the launch of Egg Card. The release dates were crucial for that product as a number of competitors were planning similar, but inferior, product launches around the same time and we desperately wanted to be first.

The priority, inline with that was to build the supporting literature and the account opening and card issuing processes first. Followed by the card activation processes and finally the account servicing functions. What made this a higher risk vernture than maybe a more established bank would have taken was that we launched the account opening processes before the card activation services had been built. Time available was, therefore, the time it took for an account to be opened, the customer to sign and return the Consumer Credit Agreement and a card to be embossed and posted. About ten days.

28 days following the first card activations was the deadline for account servicing. Based on the fact that Egg Card has online statements and a credit card company can’t debit your account for the balance on the card unless you’ve had opportunity to view your statement. Critical Deadline.

So, JIT systems for a live product has been done. I can’t say I can recommend it. There are 3 axes to software engineering in real world situations, Scope, Quality and Time. As many of us know, if you fix time and scope at points that create this level of pressure then the only thing that can happen is a reduction in quality.

“The ASP.Net Traditional Web Anti-Pattern” is what web developers coming from all web backgrounds Perl/CGI, ASP/COM, JSP, PHP and others all seem to build if not given adequate time, guidance or training in .Net Web Development.

That shouldn’t be a surprise, of course. Microsoft have moved the goal-posts. When introducing .Net one of Microsoft’s major objectives was to bring the WinForms and Web Forms development models together. They did this by applying as many of the windows forms concepts to WebForms as they could.

The concepts are generally new to the web. ASP.Net introduces the idea of ‘postback’ and event-driven development to the web in a far more consistent and holisitc manner than seen before.

But, they left all the old hooks for getting at post data, querystrings and cookies in place. So when developers steeped in handling these manually are given an ASP.Net development to do they naturally build things the same way they always have; with html controls and code to parse the form data.

This misses the value of ASP.Net though, with it’s web controls, event plumbing and state management making life so much easier and more powerful.

So, next time you’re parsing form post data, querystrings or cookies in ASP.Net ask yourself if you’re actually implementing “The ASP.Net Traditional Web Anti-Pattern”.