Kinect, the motion sensing device Microsoft uses on the ever popular Xbox 360, allows gamers to play games and operate their Xbox using movements, gestures, or voice commands. This system, which was recently hacked by a team from Microsoft Research Cambridge in the UK, can even be applied to more practical, serious uses.
The team was able to configure it in such a way to show what a user’s brain looks like under their skull, in 3D. Using the newest Kinect application programming interface (API), a touch screen tablet, some duct tape, and brain scans, this new augmented reality system is aimed at helping professional brain surgeons and doctors.
Ben Glocker, a post-doc researcher at Microsoft Research Cambridge, recently showed a prototype of the setup during Microsoft TechFest 2013. He discussed how the team duct-taped a USB-powered touchscreen device to the Kinect Fusion, which allows them to 3D model not just brain scans, but just about anything.
As Glocker demonstrated, the system captures the patient’s skull in 3D and then incorporates that info with two-dimensional MRI brain scans that doctors took of the patient in preparation for surgery. Those combined images form an augmented reality: When surgeons look at the patient’s head in the operating room, they also see images of the brain underneath.
The team hopes this could help doctors visualize what they need to do. For example, they might see the location of a tumor they need to remove more clearly. A live-action MRI scan in 3D does seem like it might be more useful than 2D MRI slices.
Obviously duct-taping components together is not the way to go in precision surgical work, but the idea itself is just the stepping stone to the integration of 3D mapping technology in the surgery room. The team also pointed out how the Kinect Fusion system can be a useful tool for teaching. Whether it catches on or not, it’s nice to see Microsoft apply its technology to more serious applications.