Image Overlay System: Computer-generated Integral PhotographyImage OverlayWhat is Image Overlay?ApparatusRequirements of a 3D navigational systemImportant Factors to ConsiderNecessary backgroundPrinciple of Integral Photography (IP)Computer IP AlgorithmSlide 10RegistrationGeometric AccuracyAccuracy from the location of registered image to the objectMy AssessmentPossible next stepsImage Overlay System:Computer-generated Integral PhotographyAnand ViswanathanCIS II 600.446Image OverlayPicture taken from http://www.mrcas.ri.cmu.edu/projects/overlay/system.htmlWhat is Image Overlay?•The ability to see bone/tissue/target superimposed onto the patient during surgery.•Can be used for surgical planning.•Can be used for surgical navigation.•Using 3D marker information and creating an accurate 2D representation.•Superimposing this 2D representation with video input.Apparatus•Optical Tracking system (ie: Optotrak, POLARIS)•PC collecting data from tracking system•Lens array•LCD display•Half silvered mirrorPicture taken from http://www.mrcas.ri.cmu.edu/projects/overlay/medical.htmlRequirements of a 3D navigational system•Surgeon should recognize 3D position and orientation of any surgery device.•High Image quality.•Minimal eyestrain and fatigue•3D representation should include the area surrounding the target.Important Factors to Consider•Mechanical Accuracy/ Geometric Accuracy•Depth Perception•DistortionNecessary background•Three Dimensional Slice Image Overlay System with Accurate Depth Perception for Surgery. Ken Masamune, Yoshitaka Mautani, Susumu Nakajima, ichiro Sakuma, Takeyoshi Dohi, Hiroshi Iseki and Kintomo Takakura MICCAI pgs 395-402•Surgical Navigation System with Intuitive Three-Dimensional Display Susumu Nakajima, Sumihisa Orita, Ken Masamune, Ichiro Sakuma Takeyoshi Dohi, and Kozo Nakamura MICCAI pgs 404-411Principle of Integral Photography (IP)•Captures and reproduces 3D objects using a lens array and film.•Computer generated IP: 3D info -> 2D coordinates replace film with LCDComputer IP Algorithm•Previous computer generated IP algorithm was too ‘costly’ and could not render medical objects in real time.•Novel approach: Consider each point in 3D space as a light source that will be reconstructed.•3D object -> a group of reconstructed light sourcesComputer IP Algorithm•Main Calculation:•Computes the distance of the ith furthest point for the reference plane by intersection of 2 light rays.Registration•Most Important Problem for display•Manual Adjustment and then more precise Image Transformation to align coordinates of markersGeometric Accuracy•Most important consideration for medical imaging•Tested by trying to display a 100 mm square at a distance of 10 to 40 mm from the surface of the LCD•Mean differences were less than 3.8 mmAccuracy from the location of registered image to the object•50mm vitamin capsule w/ 4 MRI markers used to test accuracy•Tried to superimpose 3D rendered version on the actual capsule•Magnitude of the errors = radius of the markersMy Assessment•Image overlay is a very powerful tool for a surgery.•Significantly accurate within an error of 3-4 mm, but can be improved to let surgeon rely on the navigation system.•Computer-generated IP allows real-time integration of video and graphics.Possible next steps•Better, faster rendering algorithms•Better LCD screens: image resolution,pixel pitch.•Improved Accuracy•Multiple trackers•Network