1Publish/SubscribeHans-Arno JacobsenBell University Laboratory Chair in Software EngineeringMiddleware Systems Research GroupUniversity of TorontoMIDDLEWARE SYSTEMSRESEARCH GROUPPublish/Subscribe Lecture2MIDDLEWARE SYSTEMSRESEARCH GROUPAmazon to Chapters to you ....Monday, October 10th in CyberspaceYour book “...”is availableat .... $10 offThursday, November 15th,in TorontoPublish/Subscribe Lecture3MIDDLEWARE SYSTEMSRESEARCH GROUPBusiness Process Execution & Web Service CompositionBrokerBrokerBrokerBrokerWSAgentAgentBroker……DatabaseWSClientPick InvokeWaitScopeReceiveAssignFlowReplyBusiness ProcessScopeReplyBusiness ProcessScopeReceiveSwitchClientPublish/Subscribe Lecture4MIDDLEWARE SYSTEMSRESEARCH GROUPOther “Similar” Applications Selective information dissemination Location-based services Personalization Alerting services Application integration Job scheduling Monitoring, surveillance, and control  Network and distributed system management Workforce management (Scientific) workload management Business activity monitoring Business process management, monitoring, and executionPublish/Subscribe Lecture5MIDDLEWARE SYSTEMSRESEARCH GROUPWhat Relates All These Applications? Asynchronous state transitions captured as events that  drive and underlay all applications and infrastructures implementing these applications Require middleware support for event processing Publish/Subscribe is ideally suited to fulfill these requirementsPublish/Subscribe Lecture6MIDDLEWARE SYSTEMSRESEARCH GROUPPublish/SubscribePublish/Subscribe Lecture7MIDDLEWARE SYSTEMSRESEARCH GROUPPublish/SubscribePublisher PublisherSubscriberSubscriberSubscriptionsPublicationsNotificationNotificationIBM=84MSFT=27INTC=19JNJ=58ORCL=12HON=24AMGN=58Stock marketsNYSENASDAQTSXSubscriptions:IBM > 85ORCL < 10JNJ > 60Broker(s)Publish/Subscribe Lecture8MIDDLEWARE SYSTEMSRESEARCH GROUPData(a lot of)Subscriptions(a lot of)querypublicationQueryand subscriptionis very similar.Set of tuplesand publicationis very similar.However, the two problem statements are inverse.That’s Like Data Base Querying /!!Sets of tuplesMatching subscriptionsAbout pastAbout futurePublish/Subscribe Lecture9MIDDLEWARE SYSTEMSRESEARCH GROUPKey Benefit of Publish/Subscribe Decoupling of publishers and subscribers Publishers do not need to know subscribers Publishers and subscribers do not need to be up simultaneously Amenable for physical distributionPublish/Subscribe Lecture10MIDDLEWARE SYSTEMSRESEARCH GROUPBenefits of Publish/Subscribe independence of participants lends itself well to distributed system development de-coupled development & processing (dynamic) system evolution interaction with large number of entities facilitated naturally supports non-continuous operations potential for scalability & fault-tolerance open for (legacy) system integration on either endOf course it is not a one size fits allparadigm, but a good solution for certain kinds of problems.Publish/Subscribe Lecture11MIDDLEWARE SYSTEMSRESEARCH GROUPPublish/Subscribe Matching Problem Given a set of subscriptions, S, and a publication, e, return all s in S matched by e.  e is referred to as event or publication Splitting hairs Event is a state transition of interest in the environment Publication is the information about e submitted to the publish/subscribe system Simple problem statement, widely applicable, and lots of open questionsPublish/Subscribe Lecture12MIDDLEWARE SYSTEMSRESEARCH GROUPProblem Instantiations I Text / search strings (information filtering) Semi-structured data / queries attribute-value pairs / attribute-operator-value-predicates XML, HTML Tree-structured data / path expressions XML ./ XPath expressions Graph-structured data / graph queries RDF / RDF queries (e.g., SPARQL) Regular languages / regular expressions Tables / SQL queriesPublish/Subscribe Lecture13MIDDLEWARE SYSTEMSRESEARCH GROUPProblem Instantiations II Different matching semantics Crisp Approximate,  Similar n-of-m (n of m predicates match) Probability of matchPublish/Subscribe Lecture14MIDDLEWARE SYSTEMSRESEARCH GROUPProblem Instantiations III Centralized and distributed instantiation Networking architecture Internet (as overlay network) Peer-to-peer style interface (DHT, table-lookup) With mobile publishers, subscribers, brokers Ad hoc networkPublish/Subscribe Lecture15MIDDLEWARE SYSTEMSRESEARCH GROUPPublish/Subscribe Models Channel-based model Subscribe & publish to a channel Topic-based model … topics and topic hierarchy Type-based model … typed objects Content-based model … to content of messages Subject-spaces (State-based model) Maintain state in publications and subscriptionsPublish/Subscribe Lecture16MIDDLEWARE SYSTEMSRESEARCH GROUPChannel-basedPublisherBroadcast ChannelPublisherSubscriber Subscriber SubscriberPublish/Subscribe Lecture17MIDDLEWARE SYSTEMSRESEARCH GROUPTopic-based publish/subscribepublicationnewsCanadapolitics sportssoccerUSpolitics sportssoccerPublish/Subscribe Lecture18MIDDLEWARE SYSTEMSRESEARCH GROUPThe Content-based Model Language and Data model Conjunctive Boolean functions over predicates Predicates are attribute-operator-value triples [class,=,trigger] Subscriptions are conjunctions of predicates [class,=,trigger],[appl,=,payroll],[gid,=,g001] Publications are sets of attribute-value pairs [class,trigger],[appl,printer],[gid,g007] Matching semantic A subscription matches if all its predicates are matchedPublish/Subscribe Lecture19MIDDLEWARE SYSTEMSRESEARCH GROUPContent-based routing Distributed publish/subscribe Network of publish/subscribe brokers Subscriptions & publications are injected into network at closest edge broker Routing protocol distributes subscriptions throughout network Network routes relevant publications to interested subscribers Routing is based on content; it is not based on addresses, which are not available Subscriptions may change dynamicallyMIDDLEWARE SYSTEMSRESEARCH GROUPContent-based RoutingPublisherSubscriber1. Advertise2. Subscribe3. PublishEvent-BasedDecoupledFlexibleResponsiveContent RoutingDeclarativeA: [class, =, stock], [name, =, HP], [price, >, 50]S: [class, =, stock], [name, =, *], [price, >, 50]P: [class, stock], [name, *], [price, 50]Publish/Subscribe