Evaluating Your Prototype with Users Class 30 Agenda 3 00 3 05 Announcements 3 05 3 50 Evaluation with Users Overview of Evaluation Methods Without tasks With users Without users With tasks 1 Testing with Users Task specific think alouds wizard of oz Task independent alpha and beta testing Bugs vs usability Conducting a User Study Planning Preparing Doing Analyzing The Challenge Within 2 miles there are no playgrounds safe areas to run around or supermarkets But there are fried food joints Within 5 miles there is a supermarket but no playgrounds or YMCAs 5 mi 2 mi The schools in the immediate area got rid of gym music and art to get the resources needed to fulfill the No Child Left Behind requirements Recently they cut down on recess time for more classroom time Children under 5 have 3 of the 10 factors that go into heart disease risk obesity high blood pressure and tobacco use second hand The single parents have 7 of the 10 factors 2 How can we solve this challenge How can we solve this challenge Obesity Exercise High Blood Pressure Nutrition Tobacco Use Education Low Tech Walking in regular life Pedometer information cards pamphlets Informed decisions about what you eat High Tech Just in time messages instead of lecture nag zzzz Pedometer or Mobile Phone Possible Ideas My food phone Bones in Motion 3 Planning Developing a Task Describes what the user wants to accomplish not how Describes a complete job Describes who the user is Very specific Example Task for our Challenge Tammy wants to find out how many miles she walked today and see if she could increase her step count for tomorrow She wants to see a map of where she walked and see if there are safe alternatives to increase her step count o What o Complete Job o Who o Specific Example Task for our Challenge To calculate how far you walked today turn on your phone and walk throughout the day To see how far you walked in total go to the menu screen select BMI and view your results on the main BMI screen o What o Complete Job o Who o Specific 4 Develop a task for My Food Phone o What o Complete Job o Who o Specific Planning 80 of usability problems are uncovered with 5 participants Proportion of Problems Uncovered 1 0 9 0 8 0 7 0 6 0 5 0 4 0 3 0 2 0 1 0 1 5 10 20 Number of Participants in Study Virzi R A 1990 Streamlining the design process Running fewer subjects Proceedings of the Human Factors Society 34th Annual Meeting 291 294 Santa Monica CA HFES Virzi R A 1992 Refining the test phase of usability evaluation How many subjects is enough Human Factors 34 4 457 468 Be mindful Environmental Factors Know the rules regulations Visit the site Be mindful of safety Developing Relationships Trusting relationship Dress appropriately Participant s Ability Be aware of comfort Take sensitivities into account Time Management Design adaptable studies Multitask Lessons Learned Conducting User Studies in a Dialysis Ward Katie A Siek and Kay H Connelly In Extended Abstracts of CHI 2006 Workshops Reality Testing 5 How could this affect your data How could this affect your data How much useful data will you draw out from a hospital visit I was just in the hospital for 2 months and it was not a good time for me I was disoriented and confused most of the time Hospitals are not a good place for people to reflect Older Adult How could this affect your data How much useful data will you draw out from a hospital visit I was just in the hospital for 2 months and it was not a good time for me I was disoriented and confused most of the time Hospitals are not a good place for people to reflect Older Adult Our entire healthcare system is based on the idea assuming that everyone receiving care in a hospital unit are 100 coherent and understanding of their condition even if this may not be the case Doctor 6 How could this affect your data How much useful data will you draw out from a hospital visit I was just in the hospital for 2 months and it was not a good time for me I was disoriented and confused most of the time Hospitals are not a good place for people to reflect Older Adult Our entire healthcare system is based on the idea assuming that everyone receiving care in a hospital unit are 100 coherent and understanding of their condition even if this may not be the case Doctor Compromise For this particular study we visit the patients while they are in the hospital to see how care can be improved and 2 4 weeks after their hospital visit to give them time to reflect How should information be given Did you say 4th grade reading SMOG 30 sentences Wd ct Grade of Sent Conversion 1 Count the total number of sentences in the material 0 2 4 29 1 03 3 6 5 28 1 07 2 Count the number of words with 3 or more syllables 11 2 7 10 3 0 3 Find the total number of sentences and the corresponding conversion number in SMOG Conversion Table II 4 Multiply the total number of words with 3 or more syllables by the conversion number Use this number as the word count to find the correct grade level from Table I Harvard Health Literacy Studies http www hsph harvard edu healthliteracy materials html MedLine Plus How to Write Easy to Read Health Materials http www nlm nih gov medlineplus etr html Doak and Doah http www hsph harvard edu healthliteracy doak html 7 Did you say 4th grade reading SMOG 30 sentences Hello My name is and I am a computer science Master s student I am currently conducting a study for a Medical Informatics class project on smoking factors and what would motivate people 2 Count the number of words with 3 or more to quit I would like to thank you for participating syllables in this study and would like to express my 3 Find the total number of sentences and the gratitude with this chocolate For the following corresponding conversion number in SMOG hour I would like to ask you some questions about Conversion Table II your smoking history and experience After the question answer session we will be doing a small 4 Multiply the total number of words with 3 or 5 minute exercise Please feel free to stop me at more syllables by the conversion number Use any time you wish to There are no correct or this number as the word count to find the incorrect answers to the questions so please do correct grade level from Table I not worry and express everything you want If you have any questions please do not hesitate to ask You are helping me a lot by contributing your experience to my study Any information you give me will only be shared between my teacher and I and …