1University of TorontoDepartment of Computer Science© Easterbrook 20042Lecture 9:Eliciting Requirements Basics of elicitation Why info collection is hard Dealing with Bias A large collection of elicitation techniques: Background Reading Hard data collection Interviews Questionnaires Group Techniques Participant Observation Ethnomethodology Knowledge Elicitation TechniquesUniversity of TorontoDepartment of Computer Science© Easterbrook 20043Difficulties of Elicitation Thin spread of domain knowledge The knowledge might be distributed across many sources It is rarely available in an explicit form (I.e. not written down) There will be conflicts between knowledge from different sources Remember the principle of complementarity! Tacit knowledge (The “say-do” problem) People find it hard to describe knowledge they regularly use Limited Observability The problem owners might be too busy coping with the current system Presence of an observer may change the problem E.g. Probe Effect; Hawthorne Effect Bias People may not be free to tell you what you need to know People may not want to tell you what you need to know The outcome will affect them, so they may try to influence you (hidden agendas)University of TorontoDepartment of Computer Science© Easterbrook 20044Example Loan approval department in a large bank The analyst is trying to elicit the rules and procedures for approving a loan Why this might be difficult: Implicit knowledge: There is no document in which the rules for approving loans are written down Conflicting information: Different bank staff have different ideas about what the rules are Say-do problem: The loan approval process described to you by the loan approval officers is quitedifferent from your observations of what they actually do Probe effect: The loan approval process used by the officers while you are observing isdifferent from the one they normally use Bias: The loan approval officers fear that your job is to computerize their jobs out ofexistence, so they are deliberately emphasizing the need for case-by-casediscretion (to convince you it has to be done by a human!)University of TorontoDepartment of Computer Science© Easterbrook 20045Bias What is bias? Bias only exists in relation tosome reference point can there ever be “no bias”? All views of reality are filtered All decision making is basedpartly on personal values. Types of bias: Motivational bias expert makes accommodations toplease the interviewer or someother audience Observational bias Limitations on our ability toaccurately observe the world Cognitive bias Mistakes in use of statistics,estimation, memory, etc. Notational bias Terms used to describe a problemmay affect our understanding of itExamples of Bias Social pressureresponse to verbal and non-verbal cues frominterviewer Group thinkresponse to reactions of other experts Impression managementresponse to imagined reactions of managers, clients,… Wishful thinkingresponse to hopes or possible gains. AppropriationSelective interpretation to support current beliefs. Misrepresentationexpert cannot accurately fit a response into therequested response mode Anchoringcontradictory data ignored once initial solution isavailable Inconsistencyassumptions made earlier are forgotten Availabilitysome data are easier to recall than others Underestimation of uncertaintytendency to underestimate by a factor of 2 or 3.2University of TorontoDepartment of Computer Science© Easterbrook 20046Elicitation Techniques Traditional techniques Introspection Reading existing documents Analyzing hard data InterviewsOpen-endedStructured Surveys / Questionnaires Meetings Collaborative techniques Focus GroupsBrainstormingJAD/RAD workshops Prototyping Participatory Design Contextual (social) approaches Ethnographic techniquesParticipant ObservationEnthnomethodology Discourse AnalysisConversation AnalysisSpeech Act Analysis Sociotechnical MethodsSoft Systems Analysis Cognitive techniques Task analysis Protocol analysis Knowledge Acquisition TechniquesCard SortingLadderingRepertory GridsProximity Scaling TechniquesUniversity of TorontoDepartment of Computer Science© Easterbrook 20047Background Reading Sources of information: company reports, organization charts, policy manuals, job descriptions,reports, documentation of existing systems, etc. Advantages: Helps the analyst to get an understanding of the organization beforemeeting the people who work there. Helps to prepare for other types of fact finding e.g. by being aware of the business objectives of the organization. may provide detailed requirements for the current system. Disadvantages: written documents often do not match up to reality. Can be long-winded with much irrelevant detail Appropriate for Whenever you not familiar with the organization being investigated.University of TorontoDepartment of Computer Science© Easterbrook 20048“Hard Data” and Sampling Hard data includes facts and figures… Forms, Invoices, financial information,… Reports used for decision making,… Survey results, marketing data,… Sampling Sampling used to select representative set from a population Purposive Sampling - choose the parts you think are relevant without worryingabout statistical issues Simple Random Sampling - choose every kth element Stratified Random Sampling - identify strata and sample each Clustered Random Sampling - choose a representative subpopulation and sample it Sample Size is important balance between cost of data collection/analysis and required significance Process: Decide what data should be collected - e.g. banking transactions Determine the population - e.g. all transactions at 5 branches over one week Choose type of sample - e.g. simple random sampling Choose sample size - e.g. every 20th transactionUniversity of TorontoDepartment of Computer Science© Easterbrook 20049Example ofhard dataQuestions:What does this data tellyou?What would you do withthis data?3University of TorontoDepartment of Computer Science© Easterbrook 200410Interviews Types: Structured - agenda of fairly open questions Open-ended - no pre-set
View Full Document
Unlocking...