On-Demand Sharing of a High-Resolution Panorama Video fromNetworked Robotic CamerasNi Qin and Dezhen SongAbstract—Due to their flexibility in coverage and resolution,networked robotic cameras become more and more popular inapplications such as natural observation, security surveillance,and distance learning. Equipped with a high optical zoomlens, a networked robotic camera can generate a giga-pixel-level panorama to cover its viewable region. As new liveframes enter the system, this panorama can be updated asa panorama video. User requests are usually not limited to thecurrent camera frame. A user may request a specific regionassociated with a specific time window. To satisfy differentspatiotemporal requests for multiple concurrent users, wepresent systems and algorithms to allow the on-demand sharingof the high-resolution panorama video. The high-resolutionpanorama video is encoded into a patch-based representationto allow efficient storage and on-demand content delivery. Wepresent system architecture, user interface, data representation,and encoding/decoding algorithms. In the experiment, we haveimplemented the system using the MPEG-2 codec. Experimentalresults show that our system can not only satisfy different spa-tiotemporal queries but also significantly reduce computationtime and communication bandwidth requirement.I. INTRODUCTIONConsider a high-resolution pan-tilt-zoom camera installedin a deep forest. Connected to the Internet through a long-range wireless network communication, the robotic cameraallows scientists and/or the general public to continuouslyobserve nature remotely. Equipped with robotic pan-tiltactuation mechanisms and a high-zoom lens, the cameracan cover a large region with very high spatial resolutionand allow for observation at a distance without disturbinganimals. For example, a Panasonic HCM 280A pan-tilt-zoomcamera has a 22x motorized optical zoom. The camera canreach a spatial resolution of 500 megapixel per steradian atthe highest zoom level. Since the camera has a 350◦panrange and a 120◦tilt range, the full coverage of the viewableregion is more than 3 gigapixels if represented as a panorama.As the camera patrols the viewable region, the giga-pixelpanorama is also continuously updated.As illustrated in Fig. 1, there are many concurrent scien-tists and other online users who want to access the camera(or cameras if multiple cameras are installed for better cov-erage). Transmitting the full-sized ever-changing giga-pixelpanorama video to every user is unnecessary and expensivein the bandwidth requirement. Each user may want to observea different sub-region and time window of the panoramavideo. For example, an ornithologist is often interested inThis work was supported in part by the National Science Foundationunder IIS-0534848/0643298.N. Qin and D. Song are with Computer Science Department, Texas A&MUniversity, College Station, TX 77843, USA (email: [email protected] [email protected]).Client i Sphere wrapping Image alignment 1 N I B B B B B B I B B B B B B I B B B B B B 1 I B B B i1 I B B B B B B I B B B B B B N … … Requst i I B B B B B B I B B B B B B I B B B B B B … … User i Decoding and composing panorama Rendering and display Request i {area i, time [ts~te]} im … … Live video Time B B B ts te Video frames Live video Fig. 1. Evolving panorama video system diagram. The left hand sideillustrates the server side. The right hand side is a user at the client side.The grid at server represents a patch-based high-resolution panorama videosystem that allows multiple users to query different parts of the videoconcurrently. I’s and B’s indicate the I-frame and the B-frame used inMPEG-2 compression. A user sends a spatiotemporal request to server sideand to retrieve the part of his/her interests in the panorama.bird video data when the camera is aimed at the top of theforest in the morning.Since that both camera coverage and user requestshave spatiotemporal constraints, how to efficiently orga-nize the frames captured by the camera and satisfy var-ious and concurrent user requests becomes a challengingproblem. An analogy to this problem is the Google Earth(http://earth.google.com) system where each user requestsa view of different regions of the planet Earth. While theimage-sharing aspect of Google Earth is similar to oursystem, the primary difference is that the satellite imagedatabase of the google Earth system is relatively static anduser requests do not involve the time dimension whereasour system has to be run in near real time and satisfiesspatiotemporal user requests.In this paper, we present systems and algorithms that allowon-demand sharing of a high-resolution panorama video. It isthe first panorama video system that is designed to efficientlydeal with multiple different spatiotemporal requests. Wepropose a patch-based approach in a spherical coordinatesystem to organize data captured by cameras at the serverend. Built on an existing video-streaming protocol, the patch-based approach allows efficient on-demand transmission ofthe request regions. We present a system architecture, userinterface, data representation, and encoding/decoding algo-rithms followed by experiments. We begin with the relatedwork.II. RELATED WORKOur system builds on the existing work of networked tele-operation [1] and panorama video systems [2].Our system is designed to allow multiple online users toshare access to robotic cameras. In the taxonomy proposedby Chong et al. [3], these are Multiple Operator SingleRobot (MOSR) systems or Multiple Operator Multiple Robot(MOSR) systems. An Internet-based MOSR system is de-scribed by McDonald, Cannon, and their colleagues [4], [5].In their work, several users assist in waste cleanup usingPoint-and-Direct (PAD) commands. Users point to cleanuplocations in a shared image and a robot excavates eachlocation in turn. In [6] Goldberg et al. propose the “SpatialDynamic Voting” (SDV) interface for a MOSR system.Existing work on MOSR and MOMR systems providesstrategies to efficiently coordinate the control of the sharedrobot. Users are usually forced to share the same feedbackfrom the robot. However, users may not be interested inthe same event at the same time even when they accessthe system at the same time. This becomes more obviouswhen the shared robot is a robotic camera. Time and spaceof interests may vary for different online users. This