1 Introduction1.1 Energy efficiency as a design metric 1.2 Energy evaluation during early design 1.3 Roadmap 2 Energy and Mobile User Interaction 2.1 Interfacing hardware 2.2 Interfacing software 2.3 Interfacing human 2.4 Summary3 Energy Adaptive Display 3.1 User study on screen usageThe user study in this section is conducted by HP Labs collaborators prior to the joint work. 3.1.1 Method 3.1.2 Results 3.1.3 Summary of user study3.2 Energy-adaptive display sub-systems3.2.1 System design3.2.2 Implementation3.2.3 Evaluation method 3.3 Experimental results 3.3.1 Power benefits3.4 User acceptance evaluation3.4.1 Method3.4.2 Results3.5 Discussion 3.6 Summary4 Ambient Display4.1 System architecture4.1.1 Remote display system4.1.2 Discovery protocol4.1.3 Software architecture4.1.4 Lightweight user interface4.1.5 Ambient display and display protocol4.1.6 Limitation4.2 Experimental results 4.2.1 Network usage analysis4.2.2 Power consumption analysis4.3 Summary 5 Predicting Mobile User Performance 5.1 Modeling user performance 5.1.1 Cognitive engineering models5.1.2 The GOMS model5.1.3 The Keystroke-Level Model5.1.4 Modeling parallel activities5.2 Verifying KLM on pen-based interfaces 5.2.1 Task definition5.2.2 Model creation5.2.3 User study5.2.4 Result analysis5.3 Estimate system response time 5.4 Summary6 Keystroke Level Energy Modeling 6.1 Extending KLM for energy prediction 6.1.1 Overview of KLEM 6.1.2 Modeling process 6.1.3 Energy characterizing process 6.1.4 Mapping process 6.2 Verification 6.3 Comparing design alternatives using KLEM 6.4 Summary7 Related Work7.1 Energy optimization 7.1.1 Activity adjustment 7.1.2 Mode switching7.1.3 Using alternatives7.2 Energy characterization 7.2.1 Measurement based approach7.2.2 Analysis based approach7.3 Energy and mobile user interaction 7.3.1 Human factors in energy optimization7.3.2 User interface energy optimization7.3.3 Performance of mobile user interaction 7.3.4 Applications of KLM 8 Conclusion8.1 The prospect of mobility 8.2 Contributions8.3 Future workDesigning Energy and User Efficient Interactionswith Mobile SystemsLu LuoCMU-ISR-08-102April 2008School of Computer ScienceInstitute for Software ResearchCarnegie Mellon UniversityPittsburgh, PA 15213Thesis Committee:Daniel P. Siewiorek, ChairBonnie E. JohnPriya NarasimhanDiana Marculescu, Department of Electrical and Computer EngineeringSubmitted in partial fulfillment of the requirementsfor the degree of Doctor of Philosophy.Copyright © 2008 Lu LuoThis research was sponsored by the Defense Advanced Project Agency (DARPA) under Contract No. NBCHD030010,the National Science Foundation (NSF) under Grant Nos. EEEC-540865 and 0205266, the Office of Naval Research(ONR), N00014-03-1-0086, and HP Labs. The views and conclusions contained in this document are those of theauthor and should not be interpreted as representing the official policies or endorsements, either express or implied,of DARPA, the NSF, ONR, HP Labs, Carnegie Mellon University, the U.S. Government, or any other entity.Keywords: Mobile computing, energy efficiency, user interaction, user interface design,cognitive modeling, ubiquitous computingAbstractMobile computing has thrived to provide unprecedented user experiences beyond the boundary ofdesks and wires. The increasing demand for mobility faces two major challenges: reduced formfactor and limited energy supply. Although each challenge has been addressed by significantresearch efforts, the correlation between them is underexplored.Energy efficiency should be integrated as an important metric of user interaction design inmobile systems. By carefully considering the specific requirements of the user and the context ofusage, energy efficiency can be achieved without sacrificing user performance and satisfaction,and interaction design that facilitates user efficiency can also promote energy efficiency.This dissertation starts with a detailed study of typical workload and screen usage that showsthat users seldom use the entire screen on most workloads. Hence, Dark Windows, an energyefficient display design is presented and implemented to optimize display power consumption byadjusting the color and illumination of different screen regions according to the user’s workload.This dissertation next presents AstroRDS, a mobile computing system and network infras-tructure that displays documents and information from user’s mobile devices on ambient ubiqui-tous display resources, thus facilitates much better viewing experience of the user with compara-tively ease of use. The energy consumption and network usage of AstroRDS is several orders ofmagnitudes less than VNC remote desktop protocol on viewing and controlling tasks, and withsimilar initial loading overhead.A common challenge in these two efforts is the high cost of measuring user experience andenergy consumption. This dissertation further presents KLEM, a quantitative methodology basedon cognitive modeling techniques that can predict both user performance and system energyconsumption from story boards of an interactive task during early stages of design. KLEM can beused as a convenient tool to compare and make early decisions among different design options,as well to resolve potential design issues before investing in actual user testing and iterativedevelopment. While KLEM is presented with a focus on handheld devices, the methodology canbe applied on any interactive mobile system.iiiivAcknowledgementsMore than once I was warned against writing a PhD dissertation whilst working full-time, I amglad that I managed it in three months. This dual-task period has been very demanding indeed,but I found it exciting to get continuously inspired by new ideas from work for my writing, aswell as to see how my dissertation research may be further extended and applied in the real world.Because of this I wish to express my sincere appreciation to my advisor, Prof. Dan Siewiorekfor encouraging me to reach out beyond graduate school and collaborate with industrial labson various projects that I was interested in, meanwhile keeping me focused on the right trackof my dissertation work. Dan’s insights, methods and knowledge not only made invaluablecontributions to this dissertation, but also influenced my development as a research professional.Most importantly, I have been fortunate to work with an advisor like Dan, who was always therefor my questions and concerns, who never failed to correct even the tiniest typos in my writings,and who