USABILITY EVALUATION SCRIPT OBSERVER Team 4 Michael Lavaveshkul MLAVAVES UCI EDU Cameron Bambaeerow CBAMBAEE UCI EDU Jan Ignacio JRIGNACI UCI EDU Alton Chislom ACHISLOM UCI EDU Team Soul Calibur presents ARGUS ALERT You will be evaluating two user interfaces that are being considered for implementation into the Argus Alert system The first presented UI for this evaluation is meant to inform executive and department head level employees at pharmaceutical companies about the status of their company s products and various compliance protocols The UI is meant to provide an easily understandable visual presentation to its users such that one can assess the status of their company by simply looking or with little interaction The second presented UI for this evaluation is meant for database administrators working for pharmaceutical companies This UI is meant to manage the backend component for a desktop widget that is utilized by pharmaceutical executives and department heads for informational purposes Through this admin UI database administrators should be able to manage the critical functions and features offered by Argus Alert including user subscription data access and processing and risk calculation plug in configuration TERMS Some key terms to keep in mind Risk Refers to the degree of serious circumstances in which an executive or department head must address quickly otherwise such circumstances may result in losses to the company financial and or otherwise Adverse Event Describes any reported incident with a drug or treatment that resulted in medically negative circumstances such as injury sickness or death Category Unit of organization that company products compliance reports and similar items are classified under for the sake of calculating a common risk assessment for that classification to display to the user INSTRUCTIONS Observer Please lead the user through each task deliberately allowing the user to perform to the best of his her ability Each task is meant to measure the user s experience with a system that relies heavily on easy of understanding and cognition please pay close attention to the user s reactions methods actions employed to complete the task and time taken to complete the task Please read each task and its instructions aloud to the user TASK 1 ASSESS RISK This task tests the ease of use with checking the status of a company released drug 1 2 3 4 5 6 7 Open the Firefox Internet browser on a computer with Internet connection Navigate to http omm ics uci edu 191grp04 demos sdlw 20v1 index html Input the username tester Input the password tester Once the admin UI loads up check the status of the drug Selebrecks Say out loud whether you should be worried about how Selebrecks is doing or not Say out loud why you came to such a decision Observers should note down any difficulties experienced by the user in navigating through the UI as well as note down the average time and mouse clicks taken to find the drug Selebrecks Note down whether the user was successful as well as any frustrations experienced Success condition User clicks Selebrecks views the pertinent information on the right and can make a judgment about the drug s performance This task should not take more than 2 minutes to complete TASK 1 2 DETERMINE NUMBER OF ADVERSE EVENTS This task tests the ease of doing rudimentary data analysis behind a category s risk assessment 1 2 Check the status of the drug Lippitore Say aloud how many serious adverse event reports this drug has Observers Note if the user experiences problems finding the adverse event report count which is signified by Total Reported Cases Success Condition User should verify that Lippitore has 7 adverse event reports This task should not take more than 1 minute to complete TASK 2 IDENTIFY TREND This task tests the ease of use in understanding trend activity of a specific drug category 1 2 Just as you did in Task 1 access the data for the category Single Use Case Compliance Looking at the data say aloud how the category has performed over time Observers Do NOT tell users to utilize the graphical trend graph to complete this task users should intuitively utilize the graph without being told and out of their own volition Observers should note if the user experiences frustration in commenting on the category s performance over time Note down the time screens and mouseclicks needed to complete the task This task should not take more than 3 minutes to complete Success condition User navigates to Single Use Case Compliance information and can verbally comment on the performance of the category over time TASK 3 VIEW OVERALL SYSTEM HEALTH This task tests the ease of accessing an overall view of the company s risk health 1 2 Attempt to view the overall status of the company s drugs and categories Say out loud how the overall status is for the company and why Observers As in the preview task do NOT direct the user click Main Status to complete the task users should intuitively utilize the link by himself herself Note down the time screens and mouse clicks needed to complete the task This task should not take more than 3 minutes to complete Success condition User navigates to Main Status can view overall status information and comments that the overall status is normal positive okay or anything similar POST EVALUATION TASKS At this point the user is done with all tasks for this first UI Please ask for and note down the user s feedback on his her experience including reactions impressions and comments Afterwards compile your notes and prepare to evaluate the backend administrator UI At this point the evaluation will switch to the backend administrator user interface TASK 4 VIEW LIST OF CURRENT USERS This task tests the ease of use with configuring the list of Argus Alert users 1 2 Navigate to http omm ics uci edu 191grp04 demos admin 20v1 index html Once the admin UI loads up try to find the list of users for Argus Alert Observers Observers should note down any difficulties experienced by the user in navigating to the admin UI as well as note down the average time and mouse clicks taken to find the list of users Note down whether the user was successful as well as any frustrations experienced This task should not take more than 1 minute to complete Success condition User navigates to User Mgmt and can successfully identify the users of Argus Alert TASK 5 ACCESS ADMIN LOGS This task tests the ease of use in understanding the state of the system 1 Suppose