Kinect: taking control of computing

Microsoft’s real work, however, lay in the software. Voice recognition was easy, relatively speaking – Microsoft had been working on the technology for years as part of Office and Windows.

Making full-body motion tracking work, however, was more of an issue. To work in high-speed games, Kinect needs to track the movement of 20 or more joints in full 3D at a rate of 30 frames per second, while adjusting to the shape of different human bodies and working from a wide range of start positions. This is no trivial feat.

Kinect doesn’t simply watch how your arm or head moves – it’s constantly guessing where they’re going to move next

The solution came from work undertaken at Microsoft Research in Cambridge back in 2002. Leading researchers Andrew Blake and Kentaro Toyama had created a new model for tracking human movement, using a probability based system of “exemplars” to anticipate how, if a part of the body starts moving one way, it’s most likely to move next.

Research conducted at a later date by Andrew Fitzgibbon and Jamie Shotton refined the technique, ensuring Kinect could recognise specific body parts and evaluate trillions of potential body configurations at 30 frames per second.

Kinect doesn’t simply watch how your arm or head moves – it’s constantly guessing where they’re going to move next, even when it can’t see them completely.

Games developers – led by Microsoft’s Kudo Tsunoda and including Rare, the legendary British games developer – went to work on producing software for the controller, and the first Project Natal units were ready for a public demonstration by August 2009. Little more than a year later, the product arrived with a new name: Kinect.

Enter the hackers

It was inevitable that Kinect would find its own following of keen DIY-ers.

“I think it’s so appealing because of the fact that it’s an extremely cool new technology available for a very reasonable price,” said John Simons, founder of one Kinect-hacking project, KinEmote.

“I immediately thought: ‘this is going to be big’. A widely available, cheap 3D camera means that a lot of people will know about it and find awesome uses for it.”

Kinect attaches to the Xbox 360 through a standard USB 2 port, so all a PC needs is the software to run it. No sooner was Kinect released than Adafruit Industries, the open-source hardware firm, announced a $3,000 Open Kinect bounty for the first programmer who could write the drivers.

Within a month, the bounty had been collected by Hector Martin and days later the first Kinect hacks followed. From virtual hand puppets controlled by gestures to video-capture tools that transformed users into clouds of particles, the community found imaginative new ways of harnessing the sensor.

Microsoft’s initial response wasn’t overwhelmingly positive. “Microsoft doesn’t condone the modification of its products,” a spokesperson told CNET, warning that the device had “safeguards designed to reduce the chances of product tampering”.

Within days, it softened its stance. First, it noted that Kinect hadn’t actually been modified, merely supported by new drivers. Then, Microsoft Game Studios manager Shannon Loftis explained that it had made her “very excited” to see people “so inspired, that… they had started creating and thinking about what they could do” within a week of Kinect’s launch.