Panoramas CS 178 Spring 2011 Marc Levoy Computer Science Department Stanford University What is a panorama 2 a wider angle image than a normal camera can capture any image stitched from overlapping photographs an extreme aspect ratio on a normal shot Marc Levoy Outline 3 capturing panoramas stitching together a panorama perspective versus cylindrical projection Marc Levoy Panoramic cameras flatback panoramic camera 4 swing lens panoramic camera Marc Levoy Swing lens panoramic images San Francisco in ruins 1906 5 101 Ranch Oklahoma circa 1920 Marc Levoy Panoramic cameras to avoid parallax errors rotate around center of perspective flatback panoramic camera 6 SLR on panning clamp swing lens panoramic camera motorized pan tilt head Marc Levoy Parallax errors Photoshop aligned on the background so only foreground objects exhibit severe parallax errors 7 Marc Levoy Fuji GX 617 Lee Frost Val D Orcia Tuscany Italy Lee Frost Volubilis Morocco Lee Frost Vertical Panoramas Santorini Matthew Scott Cuernos del Paine Chile gigapan org Scanning Electron Micrograph SEM of barnacle gigapan org Scanning Electron Micrograph SEM of barnacle Stitching images together to make a mosaic 14 Marc Levoy What kind of transformation do we need translation rotation 15 perspective Marc Levoy ro Quick review of perspective projection p p p p pinhole camera p p photographic camera Albrecht D rer s drawing glass center of perspective c p projection of feature in scene onto picture plane p p 16 these three image formation methods will produce the same perspective view on the p p except for the size of the view all that matters is position of c p and orientation of p p Marc Levoy Reprojecting an image onto a different picture plane the sidewalk art of Julian Beever 17 the view on any picture plane can be projected onto any other plane in 3D without changing its appearance as seen from a common center of projection Marc Levoy Reprojecting panoramic images to a common picture plane common p p of the mosaic 18 the common picture plane of the mosaic replaces having had a wide angle non fish eye camera in the first place Marc Levoy Homography p p 2 p p 1 19 perspective mapping between two p p s using the same center of projection is called a homography input and output x y positions are related by a 3 3 matrix Marc Levoy Stitching images together to make a mosaic 20 step 1 find corresponding features in a pair of image step 2 compute perspective from 2nd to 1st image step 3 warp 2nd image so it overlays 1st image step 4 blend images where they overlap one another repeat for 3rd image and mosaic of first two etc Marc Levoy Stitching images together to make a mosaic Take CS 223B Computer Vision Win step 1 find corresponding features in a pair of image Take CS 148 nd to 1st image to Computer step 2 compute perspective from 2Introduction step 3 warp 2nd image so it overlays Graphics Aut 1st image step 4 blend images where they overlap one another Also CS 448A rd repeat for 3 image and mosaicComputational of first two etc Photography Win 21 Marc Levoy Example the Matterhorn common picture plane of mosaic image 22 perspective projection Marc Levoy Using 4 shots instead of 3 23 perspective projection Marc Levoy Cylindrical panoramas even works for 360 panorama y x 24 mosaic image project each image onto a cylinder a cylindrical image can be stored as a rectangular image Marc Levoy Flash demo Cylindrical panoramas http graphics stanford edu courses cs178 applets projection html even works for 360 panorama y x project each image onto a cylinder a cylindrical image can be stored as a rectangular image to view without distortion reproject part of the cylinder onto a picture plane representing the display screen 25 mosaic image if your FOV is narrow this view won t be too distorted Marc Levoy Back to the Matterhorn surface of cylinder 26 cylindrical projection Marc Levoy Back to the Matterhorn surface of cylinder 27 blended Marc Levoy Example 28 As mentioned in class Photoshop does not perform the perspective reprojection shown here Instead it leaves you with the raw cylindrical image shown in the previous slide On this image straight lines are not straight and the edges of the original photographs appear as cur ves This is not a correct linear perspective However some panorama viewing soft ware does perform this perspective reprojection e g Microsoft s HDView Google for it perspective reprojection onto a plane Marc Levoy Example 29 As I mentioned in class the sequence of 1 projecting one or more images to a cylindrical surface and 2 reprojecting that cylindrical image back to a planar surface produces an image with no distortion i e it is a correct linear perspective However it might be rather wide angle as shown in the previous slide Unless you view this image with your face close to the display which would be the correct viewpoint for such a wide angle perspective it will seem distorted We covered this issue in the first lecture of the course To reduce the requirement that you must place yourself so close to the display you should crop the reprojected panorama as is done in this slide The resulting image is not so wideangle and can be viewed from a normal viewing distance without seeming distorted cropped to avoid distortion Marc Levoy Spherical panoramas 30 projections are to a sphere instead of a cylinder can t store as rectangular image without distortion Marc Levoy Recap panoramas can be captured by a camera with a wide planar back a cylindrical back and a moving slit or a rotating camera rotate around the center of perspective to avoid parallax errors to assemble panoramas from a rotating camera use corresponding features to compute a perspective warp that projects the images to a common picture plane then blend them together for very wide angle or 360 panoramas project the images to a common cylindrical surface which can be stored as an ordinary wide rectangular image 31 reproject them to a picture plane for display spherical panoramas are possible but cannot be stored as rectangular images without distortion Que s t ions Marc Levoy Slide credits 32 Fredo Durand Alyosha Efros Steve Seitz Rick Szeliski Frost Lee Panoramic Photography F W Publications 2005 Marc Levoy