A routefinder for brain surgeons
External << The Whole Brain Atlas << URCS Augmented Reality Home Pages - Introduction << Brigham and Women's Hospital: Clinical Imaging System << Mixed Reality Systems Laboratory	Augmented reality is like virtual reality without the escapist fantasy. Michael Fitzpatrick looks at a new imaging technology that promises to take the guesswork out of surgery and to stop builders cutting through the mains cable Brain surgeons in Boston, Massachusetts, have eliminated some of the last vestiges of guesswork from their work, using technology borrowed from virtual reality. Instead of switching his gaze between the patient's head and a brain scan, a surgeon removing a tumour watches a video monitor showing the scalp with a three-dimensional magnetic resonance image of the brain superimposed on it. Using a laser scanner, doctors at the Brigham and Women's Hospital first establish reference points on the patient's scalp. A rough initial alignment is made, and then a computer fine-tunes the process to ensure that the scans obtained from the magnetic resonance imager (MRI) are superimposed precisely over the head. In effect, the surgeon now has "X-ray" vision. The technique comes under the heading of "Augmented Reality" (AR). This is a spin-off from virtual reality, but unlike VR, which seeks to persuade you that you really are in a fantasy place, AR enhances your perception of the real world. Using a see-through head-mounted display, it is possible to make computer-generated, ghost-like images appear in a way that corresponds to what is seen beyond the visor or glasses. The computer-generated image can be text, 2D graphics, or 3D graphics. Despite the wide range of practical applications envisaged for AR, only a few research centres are devoted to it. However, more and more researchers and corporate backers are getting excited by its possibilities. "It sounds futuristic, but this technology's a lot closer than you think," says James Vallino, a graduate student at America's Rochester University who has been working on AR with assistant professor Kiriakos Kutulakos. "It's been evolving slowly but steadily, led by military, design and medical projects." The most exciting work seems to be in the medical field. The Boston research team, in conjunction with a team from the Massachusetts Institute of Technology, is working on surgical instrument tracking. Neurosurgeons need extraordinary precision to ensure they do not cause collateral damage. AR can show them the path and location of probes whose tips are not directly visible. The navigation system has been used at the Brigham hospital for more than 30 neurosurgical cases. Surgeons who have used the system say it cuts time "under the knife" from eight hours to five. Doctors performing other types of surgery would benefit enormously if colour-coded images of vital organs were superimposed on the body of the patient in a similar fashion. The same computer could also superimpose an ultrasound image of a developing foetus on its mother's tummy, giving doctors an unprecedented view of life as it develops in the womb. Other research centres are examining AR goggles to mix and match medical images from endoscopy, ultrasound, MRI scans, CT scans and X-rays - giving them the best views for treatment. Engineers, like doctors, would also benefit from AR's ability to assist with the repair of complex mechanisms. Aero engineers have been quick to spot the potential. "Both Boeing and McDonnell Douglas aircraft manufacturers have prototype AR systems built with academic partners. One system is for the manufacture of cabling harnesses and the other assists with the manufacture of airframes. Each of these augments the view of the workspace to identify where the next operation is to be performed," says Vallino. In Britain, too, manufacturers are beginning to realise that AR could be the next wonder tool. Charles Grimsdale, managing director of VR software specialist Division, is helping them to develop such systems. "AR would be very useful to an aerospace maintenance engineer," he says. "He could put on see-through glasses while looking at a Rolls Royce engine. While standing in a specific place in the engine, maintenance information, tasks to perform, and up-to-the-minute information would be superimposed on that engine." AR also promises to make repairs to buildings much simpler: if a wall's innards are stored as a computerised blueprint, that blueprint can be superimposed on a repairman's view of the wall to pinpoint where hidden ducts, beams and wiring lie. The builder would then have no excuses for wrecking vital plumbing or wiring. However, one Japanese research body, the Mixed Reality Systems Laboratory, which has developed a computer game that uses AR, admits it is some way from applying the technology to the workplace. The major difficulty is co-ordinating the video and computer graphic images as the viewer changes his field of view and line of sight. The Rochester group has found a unique solution, using what's called an "affine set of axes" to augment reality with minimal calibration, placing computer graphics into videos using visible natural landmarks as reference points. "You could get the computer to precisely place a three-dimensional computer-generated character on top of a train," Vallino says. "When the train moves, the resulting motion of landmarks around the train is detected to keep the character correctly positioned in the image. "Augmented reality through such tracking is much easier to manage than the laborious frame-by-frame manipulation now used in Hollywood to make scenes like computerised Martians wandering through the White House."
Next report: Groceries slipping through the Net

mailto: Etcetera

etcetera | A-Z/Help | Search | Classified | Electronic Telegraph Arts & Books | Connected | Expo | Fashion | Food & Drink | HomeFront Horoscopes | Lingua Franca | Motoring | Obituaries | Opinion | Science | Travel © Copyright Telegraph Group Limited 1997. Terms & Conditions of reading. Information about Telegraph Group Limited and Electronic Telegraph. "Electronic Telegraph" and "The Daily Telegraph" are trademarks of Telegraph Group Limited. These marks may not be copied or used without permission. Information for webmasters linking to Electronic Telegraph. Email Electronic Telegraph.