1 Introduction2 Why Now? The Rise of Full Stack Bottlenecks in ML3 MLSys: Building a New Conference at the Intersection of Systems + Machine Learning4 ConclusionarXiv:1904.03257v3 [cs.LG] 1 Dec 2019MLSys: The New Frontier of Machine Learning SystemsAlexander Ratner1,2,3Dan Alistarh4Gustavo Alonso5David G. Andersen6,7Peter Bailis1,8Sarah Bird9Nicholas Carlini7Bryan Catanzaro10Jennifer Chayes9Eric Chung9Bill Dally1,10Jeff Dean7Inderjit S. Dhillon11,12Alexandros Dimakis11Pradeep Dubey13Charles Elkan14Grigori Fursin15,16Gregory R. Ganger6Lise Getoor17Phillip B. Gibbons6Garth A. Gibson18,19,6Joseph E. Gonzalez20Justin Gottschlich13Song Han21Kim Hazelwood22Furong Huang23Martin Jaggi24Kevin Jamieson2Michael I. Jordan20Gauri Joshi6Rania Khalaf25Jason Knight13Jakub Koneˇcný7Tim Kraska21Arun Kumar14Anastasios Kyrillidis26Aparna Lakshmiratan22Jing Li27Samuel Madden21H. BrendanMcMahan7Erik Meijer22Ioannis Mitliagkas28,29Rajat Monga7Derek Murray7Kunle Olukotun1,30Dimitris Papailiopoulos27Gennady Pekhimenko31Christopher Ré1Theodoros Rekatsinas27AfshinRostamizadeh7Christopher De Sa32Hanie Sedghi7Siddhartha Sen9Virginia Smith6Alex Smola12,6Dawn Song20Evan Sparks33Ion Stoica20Vivi enne Sze21Madeleine Udell32Joaquin Vanschoren34Shivaram Venkataraman27Rashmi Vinayak6Markus Weimer9Andrew Gordon Wilson32Eric Xing6,35Matei Zaharia1,36Ce Zhang5Ameet Talwalkar∗ 6,331Stanford,2University of Washington,3Snorkel AI,4IST Austria,5ETH Zurich,6Carnegie Mellon University,7Google,8SisuData,9Microsoft,10NVIDIA,11University of Texas at Austin,12Amazon,13Intel,14University of California San Diego,15cTuning Foundation,16Dividiti,17UC Santa Cruz,18Vector Institute,19Univerrsity of Toronto,20UC Berkeley,21MIT,22Facebook,23University of Maryland,24EPFL,25IBM Research,26Rice University,27University of Wisconsi n-Madison,28Mila,29University of Montreal,30SambaNova Systems,31University of Toronto,32Cornell University,33Determined AI,34Eindhoven University of Technology,35Petuum,36DatabricksMay 2, 2019AbstractMachine learning (ML) techniques are enjoying rapidly increasing adoption. However, designing and imple-menting the systems that support ML models in real-world deployments remains a significant obstacle, in largepart due to the radically different development and deployment profile of modern ML methods, and the rangeof practical concerns that come with broader adoption. We propose to foster a new systems machine learning re-search community at the intersection of the traditional systems and ML communities, focused on topics such ashardware systems for ML, software systems for ML, and ML optimized for metrics beyond predictive accuracy.To do this, we describe a new conference, MLSys, t hat explicitly targets research at the intersection of systemsand machine learning with a program committee split evenly between experts in systems and ML, and an explicitfocus on topics at the intersection of the two.1 IntroductionOver the last few years, machine learning (ML) has hit an inflection point in terms of ado ption and results. Largecorporations have invested b illions of dollars in reinven ting themselves as “AI-ce ntric”; swaths of academic dis-ciplines have flocked to incorporate machine learning into their research; and a wave of excitement about AIand ML has pr oliferated thro ugh the broader public sphere. T his has been due to several factors, central amongstthem new deep learning approaches, increasing amounts of data and compute resources, and collective invest-ment in open-source f rameworks like Caffe, Theano, MXNet, TensorFlow, and PyTorch, which have effectivelydecoupled model design and specification f rom the systems to implement these models. The resulting wave oftechnical advances and practical results seems poised to transform ML from a bespoke solution used on certainnarrowly-defined tasks, to a commodity technolo gy deployed nearly everywhere.Unfortu nately, while it is easier th a n ever to run state-of-the-art ML models on pre-packaged datasets, de sig ningand impleme nting the systems that support ML in re al-world applications is increasingly a major bottleneck. Inlarge part this is because ML-based applications require distinctly n ew types o f software, hardware, and en gineer-ing systems to support them. Indeed , modern ML applications have been referre d to by some as a new “Software2.0” [ 5] to emphasize the radical shift they r epresent as comp ared to traditional computing applications. They∗Corresponding author, [email protected] increasingly developed in different ways than traditional software—for example, by collecting, preprocessing,labeling, and reshaping training datasets rath e r than writing code—an d also deployed in different ways, for exam-ple utilizing sp e cialized hardware, new types of quality assurance methods, and new end-to-end workflows. Thisshift ope ns up exciting research cha llenges and opportunities around high-level interfaces for ML development,low-level systems for executing ML m odels, and inte rfaces for embedding learned compon e nts in the middle oftraditional computer systems code.Modern ML approaches also require new solutions for the set of concerns that naturally arise a s these techniqu e sgain broa der u sage in diverse real-world settings. These include cost and other efficiency metrics for small andlarge organizations alike, inclu ding e.g. computational cost at training and prediction time, enginee ring cost, andcost of e rrors in real-world settings; acce ssibility and automation, for the expanding set of ML users that do nothave PhDs in machine learning, or PhD time scales to invest; latency and other run-time constraints, for a wideningrange of computational deployment environments; and con cerns like fairness, bias, robustness, security, privacy,interpretability, and causality, which arise as ML starts to be applied to critical settings where impactful humaninteractions are involved, like driving, m edicine, fina nce, and law enforcement.This c ombination of radically different application requirements, increasingly-prevalent systems-level conc erns,and a rising tide of interest and adoption, collectively point to th e need for a c oncerted research focus on thesystems aspects of machine learning. To acc elerate these research efforts, our g oal is to help foster a new systemsmachine learning community de dicated to these issues. We envision focusing on broad, full-stack questions thatare complementary to those