Aurora: a new model and architecture for data stream managementABOUT MEWHAT ARE MONITORING APPLICATIONS??TRADITIONAL DBMS ASSUMPTIONSREALITY FOR MONITORING APPLICATIONSSYSTEM MODELQUERY MODELAURORA QUERY MODELAURORA QoS GRAPH TYPESOPTIMIZATIONSlide 11AURORA RUNTIME ARCHITECTURESUMMARYQUESTIONS???Aurora: a new model and Aurora: a new model and architecture for data stream architecture for data stream managementmanagementDaniel J. AbadiDaniel J. Abadi11, Don Carney, Don Carney22, Ugur , Ugur CetintemelCetintemel22, Mitch Cherniack, Mitch Cherniack11, Christian , Christian ConveyConvey22, Sangdon Lee, Sangdon Lee22, Michael , Michael StonebrakerStonebraker33, Nesime Tatbul, Nesime Tatbul22, Stan Zdonik, Stan Zdonik221 Department of Computer Science, Brandeis University2 Department of Computer Science, Brown University3 Department of EECS and Laboratory of Computer Science, M.I.T.Presenter: Saurin KadakiaPresenter: Saurin KadakiaABOUT MEABOUT MEMS CS STUDENTMS CS STUDENTGRADUATING IN DEC 08GRADUATING IN DEC 08INTERESTED IN DATABASES AND INTERESTED IN DATABASES AND WEB TECHNOLOGYWEB TECHNOLOGYWHAT ARE MONITORING WHAT ARE MONITORING APPLICATIONS??APPLICATIONS??MONITORING APPLICATIONS ARE MONITORING APPLICATIONS ARE APPLICATIONS THAT MONITOR APPLICATIONS THAT MONITOR CONTINUOUS STREAMS OF DATA.CONTINUOUS STREAMS OF DATA. EXAMPLES??EXAMPLES??MILITARY APPLICATIONSMILITARY APPLICATIONSFINANCIAL ANALYSIS APPLICATIONSFINANCIAL ANALYSIS APPLICATIONSTRACKING APPLICATIONSTRACKING APPLICATIONSTRADITIONAL DBMS ASSUMPTIONSTRADITIONAL DBMS ASSUMPTIONSHUMAN ACTIVE, DBMS PASSIVE MODELHUMAN ACTIVE, DBMS PASSIVE MODELONLY CURRENT VALUE IMPORTANTONLY CURRENT VALUE IMPORTANTTRIGGERS/ASSERTIONS ARE SECONDARYTRIGGERS/ASSERTIONS ARE SECONDARYQUERIES MUST HAVE EXACT ANSWERSQUERIES MUST HAVE EXACT ANSWERSNO REAL TIME SERVICE REQUIREMENTSNO REAL TIME SERVICE REQUIREMENTSREALITY FOR MONITORING REALITY FOR MONITORING APPLICATIONSAPPLICATIONSDBMS ACTIVE, HUMAN PASSIVE MODELDBMS ACTIVE, HUMAN PASSIVE MODELHISTORY OF VALUES REQUIREDHISTORY OF VALUES REQUIREDTRIGGER ORIENTED APPLICATIONSTRIGGER ORIENTED APPLICATIONSAPPROXIMATE ANSWERS TO QUERIESAPPROXIMATE ANSWERS TO QUERIESREAL TIME REQUIREMENTSREAL TIME REQUIREMENTSSYSTEM MODELSYSTEM MODELExternaldata sourceUser applicationOperatorboxesdata flowContinuous& ad hoc queriesHistoricalStorageAuroraSystemQoS specQuery specApplicationadministratorQUERY MODELQUERY MODELTraditionalTraditionalSStructured tructured QQuery uery LLanguageanguageDeclarative query on static dataDeclarative query on static dataAuroraAuroraData flow model for data streamData flow model for data streamApplication manager will construct queries using GUIApplication manager will construct queries using GUISStream tream QuQuery ery AlAlgebragebraQueries are processed by SQuAl operators on the data Queries are processed by SQuAl operators on the data stream stream Some of the operators are filter, map, union, Some of the operators are filter, map, union, aggregate, join bsort, resample.aggregate, join bsort, resample.AURORA QUERY MODELAURORA QUERY MODELb1 b2 b3b4b5 b6b7b8 b9appappQoS specQoS specQoS speccontinuous queryviewad-hoc queryConnectionpointdata inputAURORA QoS GRAPH TYPESAURORA QoS GRAPH TYPESOPTIMIZATIONOPTIMIZATIONFilter BSortFilterUnionAggregateJoinAggregateMapHoldHoldpull dataContinuous queryFilterMapAd hoc queryJoinStatic storageOPTIMIZATIONOPTIMIZATIONDynamic continuous query Dynamic continuous query optimizationoptimizationInserting projectionsInserting projectionsCombining boxesCombining boxesReordering boxesReordering boxesAURORA RUNTIME ARCHITECTUREAURORA RUNTIME ARCHITECTUREBuffer managerStorageManagerPersistent StoreQ1Q2QmQ1Q2QnSchedulerLoadShedderQoSMonitorCatalogBox ProcessorsσμRouterinputs outputsSUMMARYSUMMARYSolution approach itselfSolution approach itselfRethink about everything for the Rethink about everything for the requirementsrequirementsQuery modelQuery modelData flow style query specificationData flow style query specificationOptimizationOptimizationDynamic runtime optimizationDynamic runtime optimizationQoS specification based resource QoS specification based resource