ÚLTIMOS DOCUMENTOS DE TRABAJO DE FEDEAUsing machine learning algorithms to find patterns in stock prices*by Pedro N. Rodríguez**Simón Sosvilla-Rivero***DOCUMENTO DE TRABAJO 2006-12 March 2006 * Pedro N. Rodríguez thanks CONACYT (Mexico) for financial support (Fellowship: 170328). ** Universidad Complutense de Madrid. *** FEDEA and Universidad Complutense de Madrid. Los Documentos de Trabajo se distribuyen gratuitamente a las Universidades e Instituciones de Investigación que lo solicitan. No obstante están disponibles en texto completo a través de Internet: http://www.fedea.es. These Working Paper are distributed free of charge to University Department and other Research Centres. They are also available through Internet: http://www.fedea.es.Depósito Legal: M-13926-2006FEDEA – DT 2006-12 by Pedro N. Rodríguez and Simón Sosvilla-Rivero 1Abstract We use a machine learning algorithm called Adaboost to find direction-of-change patterns for the S&P 500 index using daily prices from 1962 to 2004. The patterns are able to identify periods to take long and short positions in the index. This result, however, can largely be explained by first-order serial correlation in stock index returns. JEL Classification Numbers: C45, G11, G14 Keywords: Direction-of-change predictability, Machine learning algorithms, AdaboostFEDEA – DT 2006-12 by Pedro N. Rodríguez and Simón Sosvilla-Rivero 21. – Introduction Is a move upward or downward in stock prices predictable? A considerable amount of work has been devoted to examining whether or not this is feasible. Even though the presence of linear predictable components in stock returns is nowadays widely accepted [see, e.g., Fama (1965), Lo and MacKinlay (1988), Conrad and Kaul (1988), Jegadeesh (1990), and Kaul (1996)], the existence of a function (or formula) which expresses the likelihood of a market fluctuation is not. However, recent advances in both analytic and computational methods have helped empirical investigation on the behavior of security prices. Particularly, direction-of-change (or sign) predictability is currently evaluated via either supervised learning techniques or machine learning algorithms or classifier induction techniques [see, e.g., Apte and Hong (1995), Tsaih, Hsu, and Lai (1998), Zemke (1999), Chen, Leung, and Daouk (2003), Kim (2003), and Rodriguez and Rodriguez (2004)]. Although this branch of research provides evidence in support of the existence of a function that discriminates up from down movements, it is not clear whether or not machine learning algorithms are extracting information beyond that contained in autocorrelation patterns. In this paper, we reexamine the sample evidence of direction-of-change predictability in weak-form tests. In particular, we use an algorithm that is among the most popular and most successful algorithms for classification tasks called Adaboost. One of the main properties that make the application of Adaboost to financial data bases interesting is its relative (although not complete) robustness to over-fitting. When we apply Adaboost to S&P 500 daily data, one main conclusion emerges about stock return predictability. We show that periods characterized by high first-order serial correlation in stock returns allow both in-sample and out-of-sample direction-of-change predictability. In essence, the lack of autocorrelation in stock returns does not permit Adaboost to discover a function that discriminate future upwards from downwards movements better than random. Indeed, simple random classifiers (i.e., coin-toss classifiers) are able to explain the apparent predictability in such periods. In Section 2, we provide a brief review of machine learning algorithms and describe in detail the specific machine learning algorithm we use in our analysis: Adaboost. We apply this algorithm to the daily returns of the S&P 500 stock index from 1962 to 2004 and report the results in Section 3. To check the accuracy of ourFEDEA – DT 2006-12 by Pedro N. Rodríguez and Simón Sosvilla-Rivero 3predictions, we estimate several random classifiers and autoregressive models and the results are also given in Section 3. Finally, in Section 4 we offer some concluding remarks. 2. – Machine Learning Algorithms and Adaboost The starting point for any study of stock return predictability is the recognition that prices, or more specifically, returns develop in either linear or nonlinear fashion over time and that the behavior contain certain stable patterns. In order to obtain those patterns, we start by declaring that stock price movements{satisfy an expression like the following: }y ( )yf x (1) where x is a set of (lagged) “inputs” or “explanatory” variables 1{ ,..., }pxx x and is the “output” or “response” variable y,y Cwhere C is the set of class labels. When stock price movements are expressed as in Equation (1), it is evident that quantitative patterns may emerge from the application of machine learning algorithms, in which the goal is to find a function that maps x to y, such that over the joint distribution of all pairs the expected value of some specified loss function is minimized. But how useful is this function to discriminate financial movements? *( )f x(, ),yx To answer this question empirically, we must test the in-sample and out-of-sample discriminatory accuracy of machine learning algorithms trained to understand specific movements. Moreover, the performance must be compared against coin-toss classifiers to assess the statistical significance of such functions. In Section 2.A, we provide a brief review of the Adaboost algorithm, while Section 2 examines tree-based models, which are the cornerstone of such algorithm. A. Adaboost Boosting was created from the desire to transform a collection of weak classifiers into a strong ensemble or weighted committee. It is a general method for improving the performance of any learning algorithm. Boosting was proposed in the computational learning theory literature by Schapire (1990) and Freund (1995).FEDEA – DT 2006-12 by Pedro N. Rodríguez and Simón Sosvilla-Rivero 4Freund and Schapire (1997)