This preview shows page 1-2-3-4-5-6-7-8-52-53-54-55-56-57-58-106-107-108-109-110-111-112-113 out of 113 pages.
Lecture 14: Regression•Sprawling places•Decision trees are powerful tools for data analysis -- Consider one use of your CDC data -- A researcher at JHU wanted to see if there was a connection between urban sprawl obesity•The original CDC BRFSS survey was augmented by a researcher at JHU with a variable that indicates how sprawling a city is; the researcher then applied a technique called logistic regression to study the effect of sprawling places•While that is beyond our reach at a technical level, we can still get a sense of the relationships between obesity and these variables using a tree modeQuestionRegression today•Regression has is a powerful tool in many quantitative disciplines -- In many cases, a regression model acts as a kind of social probe, providing researchers with a glimpse into the workings of some larger phenomenon•OK, that’s generous. It’s also a highly abused tool, one for which the elegant mathematics breaks down rather quickly once you hit modern practice -- Researchers often choose between many competing models, often through exhaustive searchers; data go missing and some form of imputation is often required; the underlying functional form is rarely linear and must also be estimated...•But here’s what regression looks like in various fields...The contested origins of least squares•Stephen Stigler, a well-known statistician who writes extensively on the history of our field, begins a 1981 article on least squares with the sentence “The most famous priority dispute in the history of statistics is that between Gauss and Legendre, over the discover of the method of least squares.”•Legendre is undisputedly the first to publish on the subject, laying out the whole method in an article in 1805 -- Gauss claimed to have used the method since 1795 and that it was behind is computations of the “Meridian arc” published in 1799•In that paper, Gauss used a famous data set collected to define the first meter -- In 1793 the French had decided to base the new metric system upon a unit, the meter, equal to one 10,000,000th part of the meridian quadrant, the distance from the north pole to the equator along a parallel of latitude passing through Paris...Least squares•The relationships between the variables in question (arc length, latitude, and meridian quadrant) are all nonlinear -- But for short arc lengths, a simple approximation holds•Having found values for and , one can estimate the meridian quadrant via•Label the four data points in the previous table •and apply the method of least squares -- That is, we identify values for and such that the sum of squared errors is a minimum•meridian quadrant = 90(β + α/2)αβ4!i=1(ai− α − β sin2Li)2a =(S/d)=α + β sin2L(a1, L1), (a2, L2), (a3, L3) and (a4, L4)αβLeast squares•Given a set of predictor-response pairs , we can write the ordinary least squares (OLS) criterion (as opposed to a weighted version that we’ll get to) as•argminα,βn!i=1(yi− α − βxi)2(x1, y1),...,(xn, yn)●●●●●●●●●●●●●●●●●●●●Least squares•Graphically, in this simple case, we are doing nothing more than hypothesizing a linear relationship between the x and y variables and choosing that line that minimizes the (vertical) errors between model and dataGauss and least squares•Stigler attempts to reproduce Gauss’s calculations, but cannot given the simple linearization (and a couple not-so-simple linearizations) on the previous slide•Ultimately, he reckons that because Gauss was a mathematician and not a statistician, he might have derived a more elaborate expansion -- No matter what form was used, Stigler seems convinced that something like least squares was required•Gauss eventually publishes on least squares in 1809, and his account of the method is much more complete than Legendre’s -- Linking the method to probability and providing computational approachesGalton and regression•While least squares, as a method, was developed by several people at around the same time (often ideas are “in the air”), regression as we have come to understand it, was almost entirely the work of one man •Stigler writes “Few conceptual advances in statistics can be as unequivocally associated with a single individual. Least squares, the central limit theorem, the chi-squared test -- all of these were realized as the culmination of many years of exploration by many people. Regression too came as the culmination of many years’ work, but in this case it was the repeated efforts of one individual.”Galton and regression•Francis Galton (1822-1911) was at various points in his career an inventor, an anthropologist, a geographer, a meteorologist, a statistician and even a tropical explorer -- The latter gig paid quite well as his book “The art of travel” was a best seller•Among his many innovations, was the first modern weather map, appearing in The Times in 1875 -- To draw it, Galton requested data from meteorological stations across Europe •He also developed the use of fingerprints as a means of identification -- This work is just one small part of his larger interest how human characteristics (physical or even mental) varied across populationsGalton and regression•Galton was also half-cousins with Charles Darwin (sharing the same grandfather) and took a strong interest in how physical and mental characteristics move from generation to generation -- Heredity•His work on regression started with a book entitled Hereditary Genius from 1869 in which he studied the way “talent” ran in families -- The book has lists of famous people and their famous relatives (great scientists and their families, for example)•He noted that there was a rather dramatic reduction in awesomeness as you moved up or down a family tree from the great man in the family (the Bachs or the Bernoullis, say) -- And thought of this as a kind of regression toward mediocrityGalton and regression•In some sense, his work builds on that of Adolphe Quetelet -- Quetelet saw normal distributions in various aggregate statistics on human populations•Galton writes “Order in Apparent Chaos -- I know of scarcely anything so apt to impress the imagination as the wonderful cosmic order expressed by the Law of Frequency of Error. The law would have been personified by the Greeks and deified,
View Full Document
Unlocking...