Microsoft storage: a litany of failure

Internally they all had the same structure, except for the binary streams of each document component itself. Long-term readers might remember the Office Binder, which let you build super-Structured-Storage containers that could hold multiple office documents.

The problem with the traditional file is that you have just one lock on it, at the file system level, so simultaneous multi-user editing was fraught with difficulty.

By breaking the OLE doc structure into multiple NTFS streams, it was possible for multi-user editing to be done in a much more controlled fashion.

At the time HFS seemed to be sexier, so the doc-file streams engine had to die

Obviously, Office didn’t know about this magic, so there needed to be a special chunk of code that sat inside the file system engine, called a file system filter, which did the breakup and reconstruction on the fly.

So why did this useful feature get canned? Well, the Office team had their client-side solution and didn’t really worry too much about the server side of things.

But the server team had a problem: the Office doc-file Structured Storage streams engine clashed with the hierarchical file system (HFS) N-tier storage management engine, which had been licensed from a third party. At the time HFS seemed to be sexier, so the doc-file streams engine had to die.

WinFS debacle

Then we have the debacle that was WinFS. Designed to be a clever file system for Windows Vista, it had some spectacular capabilities.

Filenames were deprecated down to mere marker points: when looking for something you could search by any of the logical Metatags that were available, so it was possible to search for a document last edited by me in the last fortnight, where you were the originator and it was a letter to the bank about Project Frog. Or it was possible to do joins across data types and across the whole network.

It was based on the SQL Server code base, and indeed, around that time I was told that the SQL Server build was done daily in both a WinFS-enhanced and raw SQL Server versions.

So why was this one cancelled? Well, as you no doubt remember, large chunks of the advanced features of Vista never made it to the final release, and WinFS was one of those casualties.

At the time, Microsoft claimed the technology worked well but the user interface was difficult to refine. Other internal voices suggested that the SharePoint Server group took one look and refused to have anything to do with it.

At that time SharePoint Server was – and still is – ruling the roost for complex server-side document storage, despite it being something of a hack that sits inside SQL Server on top of NTFS. Just enquire how many people have successfully migrated a large document store out of SharePoint Server into something else, and expect a very small answer.

The here and now

Which brings us up to the present day. If you want a complex, clever document store for Windows you have to use SharePoint Server. Windows Home Server’s Drive Extender worked well enough in version 1, but version 2 has now tripped and fallen.

The cold reality is that Microsoft has no storage vision at all

Meanwhile, the corporate world has moved on, into the world of active engine Storage Area Networks that look after volume management by themselves, while in the home market we should be looking to Drobo devices or NAS boxes on our networks.

This despite the fact that we need a platform for the home server market that lets third parties develop add-ons and additional capabilities. And if it isn’t based on Windows Server but on some potentially limited NAS engine, what hope do we have of lifting our home network’s capabilities any time soon?

The cold reality is that Microsoft has no storage vision at all. It simply doesn’t believe that anything beyond NTFS is really necessary, or at least necessary enough to require a company-wide push to solve these problems.