A Review of StatisticsSTATISTICSConfidence IntervalsBinomial Random VariableGaussian Random VariableGoodness-of-fit not satisfactoryNot satisfactoryData Reduction PrinciplesNotationWORKSHOP ON THE ANALYSIS OF NEURAL DATA 2001MARINE BIOLOGICAL LABORATORYWOODS HOLE, MASSACHUSETTSA REVIEW OF STATISTICSPART 2: STATISTICSEMERY N. BROWNNEUROSCIENCE STATISTICS RESEARCH LABORATORYDEPARTMENT OF ANESTHESIA AND CRITICAL CAREMASSACHUSETTS GENERAL HOSPITALDIVISION OF HEALTH SCIENCES AND TECHNOLOGYHARVARD MEDICAL SCHOOL / MITAUGUST 20, 2001Workshop on the Analysis of Neural Data August 2001Page 2STATISTICSA. THE STATISTICAL PARADIGMB. DATA REDUCTION PRINCIPLESC. ESTIMATION THEORYD. [HYPOTHESIS TESTING]E. CONFIDENCE INTERVALSWorkshop on the Analysis of Neural Data August 2001Page 3Definition. A family ofpdf’s and pmf’s is called an exponential family if it can be expressedas1(|) ()()exp ()(),kiiifx hxc w txθθ θ==∑where 1() 0, (), , ()khx t x t x>… are real-valued functions of x, not depending on θ. () 0cθ≥ and1(), , ()kwwθθ… are real-valued functions of θ, not depending on x.This family will play a central role in our discussions. The binomial, Poisson, exponential,gamma and Gaussian probability models are members of the exponential family.Binomial Random Variable(|) (1 )(1 )1(1 ) exp log .1xnxxnnnfxp p pxnppxpnppxxp−=−=−−  −−Take 11() , () (1 ) () log , () .1nnphx cp p w p t x xxp==−= =− Hence the binomial model belongs tothe exponential family.Gaussian Random Variable121222222222221()(|, ) exp221exp exp .222xfxxxµµσπσ σµµπσ σ σ σ−=−  =−−+   Workshop on the Analysis of Neural Data August 2001Page 4Take 122222 2 212122221() 1, (, )(2 ) exp , (, ) (, ) () , () .22xhx c w w t x t x xµµµσ πσ µσ µσσσσ−=−===−=The Gaussian is in the exponential family.Exercise: Is the inverse Gaussian probability model in the exponential family?II. STATISTICSThe science of making decisions under uncertainty using mathematical models derivedfrom probability theory.A. THE STATISTICAL PARADIGM (Box, Tukey)QuestionPreliminary Data (Exploration Data Analysis)ModelsExperiment (Confirmatory Analysis)Model FitGoodness-of-fit not satisfactoryAssessment SatisfactoryMake an InferenceMake a DecisionWorkshop on the Analysis of Neural Data August 2001Page 5Example: NeuroninhibQuestion: Does neuroinhibin (a new GABA – a agonist, potentialnew general anesthetic) decrease neuronal spikingactivity in neurons in constant conditions in isolatedcultures?Preliminary Data: Recorded spiking activity from individual neurons inculturePreliminary Model: 1. Poisson process with constant λ2. Gamma model with parameters ,αλExperiments: Several days of recording with and without neuroinhibinModel Fit: 1. Poor fit2. Better fitGoodness-of-FitAssessment: SatisfactoryNot satisfactoryMake an Inference: When neuroinhibin was applied to neurons in culture,there was a 60% (statistically significant) decrease inmean spiking activityMake a Decision: Neuroinhibin is a potential inhibitor of neural spikingactivityA. Data Reduction PrinciplesNotationObservations: 1,,nxxx=….Workshop on the Analysis of Neural Data August 2001Page 6Probability Model: 1(|) 1,, (|) (|).nkkkfx k n fx fxθθθ===∏… The parameters of the probabilityare denoted by .θ Let ()Tx= an arbitrary function of the data.Definition: A statistic is any function of a set of data.1. Sufficient StatisticsDefinition: A statistic ()Tx is a sufficient statistic for θ if the conditional distribution of thesample x given the value of ()Tx does not depend on θ.This statement says that once the statistic is computed, it summarizes all the information in thedata sample about the parameter. To find a sufficient statistic we can use the FactorizationTheorem.Factorization Theorem: Let (|)fxθ be the joint pdf or pmf of a sample x. A statistic ()Tx issufficient for if and only if these exist functions (| )gtθ and ()hx such that for all sample points xand all parameter points θ,( | ) (()| )().fx gTx hxθθ=The dimension of the sufficient statistics equals the dimension of θ.Example: Let 1,,nxx…be sample from a Poisson distribution with parameter λ.1111 1(|) ( |) exp(log ) ( !)!knn nnxkkkkkkk kefx fx x n xxλλλλ λλ−−=== ==== −∑∏∏ ∏.Take 1( ( ) | ) exp(log )nkkgTx x nλλλ==−∑ and 11() ( !)nkkhx x−==∏ and we conclude that the sum of theobservations (sample mean) is the sufficient statistic for estimating λ.Example: 21,, (,)nxxNµσ…∼ with µ and 2σ unknownWorkshop on the Analysis of Neural Data August 2001Page 7{}1222222122212212()11(|, ) exp22(2 ) exp ( ( ) ( 1) )/2(, | , )(),nnikxfxnt n tgt t hxµµσπσ σπσµσµσ−=−−=−=−−+−=∏where 1tx= and 221()/(1),nkktxxn==−−∑ where () 1.hx =If 1,,nxx… are iid observation from a pdfor pmf, (|)fxθ. Suppose (|)fxθ belongs to anexponential family given by11(|) ()()exp ()().kiifx hxc w txθθ θ==∑Then 111() ( ), , ( )nnjkjjjTx t x t x===∑∑….2. Likelihood PrincipleDefinition: Let (|)fxθ denote the joint pdf or pmf of the sample 1(, , )nxx x=…. Then givenXx= is observed, the function of θ defined as(|) (|)Lxfxθθ= ,is called the likelihood function.Likelihood Principle. If x and y are two samples points such that (|)Lxθ is proportional to(|)Lyθ, that is, there exists a constant (, )cxy such that (|) (,)(|)L x cxyL yθθ= for all θ then theconclusions drawn from x and yshould be identical.Workshop on the Analysis of Neural Data August 2001Page 8Remarks: The Likelihood Principle states how the likelihood should be used as a data reductiondevice. Likelihoods that are proportional contain the same information. It depends critically on thespecification of a parametric model. Hence it requires diagnostics. Information comes only fromthe current data sample and prior knowledge may not be “formally” used in the estimation andinference process.ESTIMATION