How this report is structured1. Executive Summary2. Introduction2.1. Study Purpose2.2. Study Methods and Context2.3. Study Summary2.4. User Profile3. Methodology3.1. End-user Test Method3.1.1. Participants3.1.2. Procedure3.1.3. Task Scenarios4. Results4.1. Performance Data4.2. Satisfaction Data4.3. Usability Findings4.3.1. Positive Findings4.3.2. Problematic Findings5. For More InformationAppendix A – Welcome and InstructionsAppendix B - Background QuestionnaireAppendix C – Task DescriptionsAppendix D – Post-test QuestionnaireAppendix E – Post-test Questionnaire DataAppendix F – Performance DataFinal Usability ReportElbuen.org, the website of El Buen Samaritano Episcopal MissionHenry Stokes/Prentiss RiddleINF 385P4/21/2005NOTE: This report was developed for a class project at the School of Information, UT Austin. It was not developed in consultation with El Buen Samaritano. Please do not redistribute. 1STOKES/RIDDLE INF 385P 2STOKES/RIDDLEHow this report is structuredThe usability findings within this report are offered in three levels of detail. For the reader interested in only the overall impressions, a one-page Executive Summary is offered on the next page. For the reader interested in summaries of all the findings, there is the body of the paper, in particular the specific usability findings in Section 5.3. Finally, all the detail of the methods and the raw data are provided in the Appendices, for only the reader interested in the smallest detail.The sections are numbered as follows:1. Executive Summary2. Introduction3. Methodology4. Results5. For more informationAppendices:A. Welcome and InstructionsB. Background QuestionnaireC. Task DescriptionsD. Post-Test QuestionnaireE. Post-Test Questionnaire DataF. Performance DataINF 385P 3STOKES/RIDDLE1. Executive SummaryThis report describes a study of the usability of elbuen.org, the website of the El Buen Samaritano EpiscopalMission. The study was conducted by two graduate students in an introductory course in usability at the Schoolof Information of the University of Texas at Austin in April, 2005.The study tested the use of the site by five representative users, including both English and Spanish speakers.The participants were asked to carry out a set of tasks representative of the needs of clients, donors andvolunteers at El Buen Samaritano. Both their comments and test monitors' observations were recorded as theyperformed the tasks.Participants were able to complete most of the tasks and reported that they found the site easy to use, but theydid report some problems, and there were some tasks which they could not complete. Specific problemsinclude:- Unclear navigational labeling- Unclear placement of information- Confusion regarding services branded in Spanish- Inconsistencies between English and Spanish contentWe make recommendations about how to address each of these problems.INF 385P 4STOKES/RIDDLE2. Introduction2.1. Study PurposeThe purpose of this study was to test the usability of the elbuen.org website, using representative users. 2.2. Study Methods and ContextIn this study we employed an “end-user test” method, wherein individual representative users were tested, one at a time. Test participants were seated at a computer displaying the elbuen.org website, were given some representative tasks to perform, and were observed as they carried out these tasks. This was a “find-and-fix” test, more interested in identifying potential usability problems than in establishing baseline performance data against which later releases could be compared. 2.3. Study SummaryFive representative users were tested one at a time on the elbuen.org website. We wished to collect performance data (error rates), satisfaction data (via a questionnaire), and, most importantly, particular areas of potentially poor usability. Through meetings with the El Buen Samaritano team we agreed on a set of tasks concerned with locating factual information on the website which would be of interest to the organization's client,donors and volunteers. Questions we hoped to answer by studying these tasks include:- Will users in all segments be able to navigate the site and find what they are looking for?- Will users in all segments be able to understand the content once they find it?- Will information presented in both languages be consistent?- Will bilingual users be able to switch easily among equivalent content in both languages?Although we collected general satisfaction data, we did not attempt to measure the website's effect on users' attitudes toward El Buen Samaritano and its services. We recommend a separate marketing study for that purpose.INF 385P 5STOKES/RIDDLE2.4. User ProfileMembers of the El Buen Samaritano community come from the widest possible range of socioeconomic backgrounds. Some speak only Spanish, others only English, and many speak both with varying degrees of mastery. Many have only a primary education or no formal education at all. Many have never used a computer or the Internet and find information online second-hand with the help of friends or family. Some are recent graduates of El Buen Samaritano's own computer literacy programs.The organizational roles of users of the elbuen.org site are varied as well. For the purposes of this study we segmented them into clients, donors and volunteers.Because some categories of users would be hard to recruit in the general population of Austin (in particular, low-income Spanish speakers of limited education who nevertheless have sufficient computer skills to use the web) we relied on El Buen Samaritano to recruit participants for the study. We stressed to their team the importance of finding representative users and to the users the importance of being frank about any problems they found with the website. In order to ensure that participants represented a cross-section of potential users, our background questionnaire asked about languages, educational background and computer usage. In order not to raise privacy concerns we did not inquire in detail about other socioeconomic factors beyond noting participants' roles with El Buen Samaritano (i.e., clients vs. others).INF 385P 6STOKES/RIDDLE 3. Methodology3.1. End-user Test Method3.1.1. ParticipantsAs can be seen in figure 1 below, we tested five participants. Two were volunteers, one was a donor and two were clients of El Buen Samaritano.