Refined Micro-analysis of Fluency Gains in a Reading Tutor that ListensResearch questions and approachProject LISTEN’s Reading Tutor: Rich source of guided oral reading dataReading speeds up with practice: exampleLearning curve for mean reading time of first 20 encounters, excluding top 50 wordsFour types of word encountersPredictor variablesExponential model of word reading timeAnalysisOverall resultsEffects of proficiencyConclusion: type of practice matters!Predictive models of word reading in textOutcome variable1CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Refined Micro-analysis of Fluency Gains in a Reading Tutor that Listens Jack Mostow* and Joseph BeckProject LISTEN (www.cs.cmu.edu/~listen)Carnegie Mellon University* Consultant and Scientific Advisory Board Chair, Soliloquy LearningSociety for the Scientific Study of Reading13th Annual Meeting, July, 2006Funding: National Science Foundation, Heinz Endowments2CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Research questions and approachGuided oral reading builds fluency [NRP 00]Typically repeated oral reading… but how do its benefits vary?How good is repeated vs. wide reading?How good is massed vs. spaced practice?How do the answers vary with student proficiency?Approach: micro-analyze oral reading dataMassive: hundreds of childrenLongitudinal: entire school yearFine-grained: word by word3CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Project LISTEN’s Reading Tutor: Rich source of guided oral reading dataMassive650 students age 5-14Mostly grades 1-4Longitudinal2003-2004 school year55,000 sessionsFine-grained6.9 million words“Heard” by recognizerVideo at www.cs.cmu.edu/~listen4CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Initial encounter of muttered:I’ll have to mop up all this (5630 ms) muttered Dennis to himself but how 5 weeks later (different word pair in different sentence):Dennis (110 ms) muttered oh I forgot to ask him for the moneyWord reading time = latency + production time 1/fluencyHow does word reading time change in general?Reading speeds up with practice: example5CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Learning curve for mean reading time of first 20 encounters, excluding top 50 words946.0401.021.1timeReading2)494.0(Rex Do some types of encounters help more than others?6CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Four types of word encountersNew context?First time today?1. Read muttered in a new story. Wide Spaced2. Read muttered in another sentence. Wide Massed3. On a later day, reread sentence 1. Reread Spaced4. Then reread sentence 2. Reread MassedPredict reading time for 770,858 type 1 encountersfrom prior encounters of all 4 types.7CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Predictor variablesNumber of word encounters so far of each typeWide vs. rereadSpaced vs. massedWord difficulty# of letters# of past help requests (controls for difficulty for that student)Student proficiencyWRMT Word Identification grade-equivalent score, e.g. 2.3Interpolated for each encounter from pre- and post-test scores8CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Exponential model of word reading time= L * # letters + (P * proficiency + constant A) * e - learning rate B *ExposureDefine weights for each type of encounterr for rereading vs. 1 for wide readingm for massed vs. 1 for spacedh for help requestsExposure = weighted sum of # of word encounters so far1 * # of wide, spaced encounters+ r * # of reread, spaced encounters+ m * # of wide, massed encounters+ r * m * # of reread, massed encounters+ h * # of help requests[Beck, J. Using learning decomposition to analyze student fluency development. ITS2006 Educational Data Mining Workshop, Taiwan.]9CarnegieMellonMostow & Beck, Project LISTEN 01/14/19AnalysisUse SPSS non-linear regression to fit parametersCaveat: 770,858 trials are not independentSo be conservative:Split 650 students into 10 groupsFit r, m, … for each groupFrom the 10 estimates of each parameter, compute:Mean ± standard errorDiffers significantly from 1?10CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Overall resultsWide reading beats rereadingr = .68 ± .13r < 1 (p = .007)2 new stories ≈ 3 old storiesSpaced beats massed practicem = .67 ± .13m < 1 (p = .007)2 spaced encounters ≈ 3 massed encountersDo these results vary by proficiency?11CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Effects of proficiencyBottom third Middle third Top thirdWord ID GE 1.8 (0-2.3) 2.7 (2.3-3.1) 4.5 (3.1-10.2)Reread (r) .93 ± .23 .99 ± .23 .79 ± .25 Massed (m) 1.73 ± .46 * .41 ± .08 ** .41 ± .21 **When does wide reading beat rereading?Maybe only for high readers?Seeing a word again the same dayMay help low readers more than waiting (p = .058)Helps higher readers less than seeing it later (p < .01)12CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Conclusion: type of practice matters!Wide reading beats rereadingAt least for higher readersAdvantage of spaced practice varies with proficiencyLow readers: seeing a word again the same day may help moreHigher readers: better to waitFluency growth is slow (learning curve is gradual)So differences in practice quality are hard to detectBut possible by micro-analysis of massive, longitudinal, fine-grained dataFuture workClarify interaction with proficiencyRefine model of fluency practiceTest correlational results experimentallyThank you! Questions?See papers & videos at www.cs.cmu.edu/~listen13CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Predictive models of word reading in textSSSR2005 SSSR2006Predict Growth from word encounter i to i+1Performance at encounter i+1Outcome Reading time speedup Reading time, errors, help requestsPredictor Encounter i of word Encounters 1..iModel Linear Exponential14CarnegieMellonMostow & Beck, Project LISTEN 01/14/19Outcome variableCombine reading time, errors, help requestsCap reading time at 3 seconds (0.1% of data)Treat error as 3 secondsTreat help request as 3
or
We will never post anything without your permission.
Don't have an account? Sign up