Department of Computer Science University of Toronto Difficulties of Elicitation Lecture 7 Eliciting Requirements Thin spread of domain knowledge It is rarely available in an explicit form i e not written down distributed across many sources with conflicts between knowledge from different sources Basics of elicitation Why info collection is hard Dealing with Bias Department of Computer Science University of Toronto A large collection of elicitation techniques Tacit knowledge The say do problem People find it hard to describe knowledge they regularly use Background Reading Hard data collection Interviews Questionnaires Group Techniques Participant Observation Ethnomethodology Knowledge Elicitation Techniques Limited Observability The problem owners might be too busy coping with the current system Presence of an observer may change the problem E g Probe Effect E g Hawthorne Effect Bias People may not be free to tell you what you need to know People may not want to tell you what you need to know The outcome will affect them so they may try to influence you hidden agendas 2004 5 Steve Easterbrook This presentation is available free for non commercial use with attribution under a creative commons license 2 2004 5 Steve Easterbrook This presentation is available free for non commercial use with attribution under a creative commons license Department of Computer Science University of Toronto Elicitation Techniques Loan approval department in a large bank Why this might be difficult Implicit knowledge Introspection Reading existing documents Analyzing hard data Interviews Participant Observation Enthnomethodology Discourse Analysis Conversation Analysis Speech Act Analysis Surveys Questionnaires Meetings Conflicting information Different bank staff have different ideas about what the rules are Say do problem The loan approval process described to you by the loan approval officers is quite different from your observations of what they actually do Collaborative techniques Focus Groups Brainstorming JAD RAD workshops Probe effect The loan approval process used by the officers while you are observing is different from the one they normally use Prototyping Participatory Design Bias The loan approval officers fear that your job is to computerize their jobs out of existence so they are deliberately emphasizing the need for case by case discretion to convince you it has to be done by a human 4 Contextual social approaches Ethnographic techniques Open ended Structured There is no document in which the rules for approving loans are written down 2004 5 Steve Easterbrook This presentation is available free for non commercial use with attribution under a creative commons license Traditional techniques The analyst is trying to elicit the rules and procedures for approving a loan Department of Computer Science University of Toronto Example 3 Sociotechnical Methods Soft Systems Analysis Cognitive techniques Task analysis Protocol analysis Knowledge Acquisition Techniques Card Sorting Laddering Repertory Grids Proximity Scaling Techniques 2004 5 Steve Easterbrook This presentation is available free for non commercial use with attribution under a creative commons license 5 1 Department of Computer Science University of Toronto Background Reading Hard Data and Sampling Sources of information Advantages Helps you get an understanding of an organization before meeting the people who work there Helps to prepare for other types of fact finding Purposive Sampling choose the parts you think are relevant without worrying about statistical issues Simple Random Sampling choose every kth element Stratified Random Sampling identify strata and sample each Clustered Random Sampling choose a representative subpopulation and sample it may provide detailed requirements for the current system Disadvantages Sample Size is important written documents often do not match up to reality Can be long winded with much irrelevant detail balance between cost of data collection analysis and required significance Process Appropriate for Whenever you not familiar with the organization being investigated 2004 5 Steve Easterbrook This presentation is available free for non commercial use with attribution under a creative commons license University of Toronto Example of hard data Sampling Sampling used to select representative set from a population e g by being aware of the business objectives of the organization Hard data includes facts and figures Forms Invoices financial information Reports used for decision making Survey results marketing data company reports organization charts policy manuals job descriptions reports documentation of existing systems etc Department of Computer Science University of Toronto 6 Decide what data should be collected e g banking transactions Determine the population e g all transactions at 5 branches over one week Choose type of sample e g simple random sampling Choose sample size e g every 20th transaction 2004 5 Steve Easterbrook This presentation is available free for non commercial use with attribution under a creative commons license Department of Computer Science 7 Department of Computer Science University of Toronto Interviews Source Adapted from Goguen and Linde 1993 p154 Questions Types Structured agenda of fairly open questions Open ended no pre set agenda What does this data tell you What would you do with this data Advantages Rich collection of information Good for uncovering opinions feelings goals as well as hard facts Can probe in depth adapt followup questions to what the person tells you Disadvantages Large amount of qualitative data can be hard to analyze Hard to compare different respondents Interviewing is a difficult skill to master 2004 5 Steve Easterbrook This presentation is available free for non commercial use with attribution under a creative commons license 8 Watch for Unanswerable questions how do you tie your shoelaces Tacit knowledge and post hoc rationalization Removal from context Interviewer s attitude may cause bias e g variable attentiveness 2004 5 Steve Easterbrook This presentation is available free for non commercial use with attribution under a creative commons license 9 2 Department of Computer Science University of Toronto Interviewing Tips Questionnaires Source Adapted from Goguen and Linde 1993 p154 Simplistic presupposed categories provide very little context No room for users to convey their real needs Ask easy questions first perhaps personal information e g
View Full Document
Unlocking...