CS433/533 Computer NetworksAdminRecap: Async Network ServerRecap: Java Async I/O Basis: OS State PollingRecap: Dispatcher StructureRecap: Event Handler OrganizationRecap: State Machine of Each ConnectionComparing v2 and v3 FSMsRecap: Problem of v2Extensible Dispatcher DesignDispatcher InterfaceHandler Design: AcceptorHandler Design: ReadWriteHandlerClass HierarchyV3: Fixing v2 ProblemsClass Diagram of v3Discussion on v3Extending v3Extending Dispatcher InterfaceQuestionSlide 21Slide 22Summary: ArchitectureProblems of Event-Driven ServerAnother viewState ManagementSummary: High-Performance Network ServersBeyond Class: Design PatternsSome QuestionsOperational AnalysisUnder the Hood (An example FSM)Operational Analysis: Resource Demand of a RequestOperational QuantitiesUtilization LawDeriving Relationship Between R, U, and S for one DeviceForced Flow LawBottleneck DeviceBottleneck vs System ThroughputExample 1Example 1 (cont.)Example 2Interactive Response Time LawSlide 43Bottleneck AnalysisProofIn Practice: Common BottlenecksSummary: Operational LawsSummary: High-Perf. Network ServerOutlineWhy Multiple Servers?Slide 51Slide 52Problems Caused by Multiple ServersSlide 54Load Direction: OverviewLoad Direction: Basic ArchitectureLoad DirectionBackup SlidesAsynchronous Multi-Process Event Driven (AMPED)CS433/533Computer NetworksLecture 11Asynchronous Network Server, Operational Analysis2/14/20121AdminProgramming assignment 1 questions?2Recap: Async Network ServerBasic idea: non-blocking operationsasynchronous initiation (e.g., aio_read) and completion notification (callback)peek system state to issue only ready operations3Recap: Java Async I/O Basis: OS State PollingExample: a system call to check if sockets are ready for ops.4serverTCP socket spacestate: listeningaddress: {*.6789, *:*}completed connection queue: C1; C2 sendbuf:recvbuf:128.36.232.5128.36.230.2state: listeningaddress: {*.25, *:*}completed connection queue:sendbuf:recvbuf:state: establishedaddress: {128.36.232.5:6789, 198.69.10.10.1500}sendbuf: recvbuf:state: establishedaddress: {128.36.232.5:6789, 198.69.10.10.1500}sendbuf:recvbuf:Completed connectionrecvbuf empty or has datasendbuf full or has spaceRecap: Dispatcher Structure//clients register interests/handlers on events/sourceswhile (true) {- ready events = select() /* or selectNow(), or select(int timeout) is call to check the ready events from the registered interest events of sources */ - foreach ready event { switch event type: accept: call accept handler readable: call read handler writable: call write handler }}Recap: Event Handler Organization6Accept ClientConnectionReadRequestFindFileSendResponse HeaderRead FileSend DataWe often use a finite state machine to manage the invocation of processing stepsRecap: State Machine of Each Connection7!RequestReady!ResponseReadyRequestReady!ResponseReadyInitInterest=READRequest complete(find terminatoror client request close)Interest=-RequestReadyResponseReadyGeneratingresponseResponseReadyInterest=WriteCloseRead from client channelWriteresponseResponseSentComparing v2 and v3 FSMsV2: Mixed read and writeV3: stagedFirst read request and then write responseExact design depends on application, e.g.,HTTP/1.0 channel can use stagedChat channel may use mixed8Recap: Problem of v2Software structureHandler•The program designed for simple echo, but typical servers may not only simply echo the content, thus in Buffer and out Buffer can be differentAttachment•Attaching a ByteBuffer to each channel is a narrow design•A better design can use the attachment to store a callback that indicates not only data (state) but also the handler (function)9Extensible Dispatcher DesignAttachment: generic event handlersDefine interfaces•IAcceptHandler and •IReadWriteHandlerRetrieve handlers at run time10if (key.isAcceptable()) { // a new connection is ready IAcceptHandler aH = (IAcceptHandler) key.attachment(); aH.handleAccept(key);}if (key.isReadable() || key.isWritable) { IReadWriteHandler rwH = IReadWriteHandler)key.attachment(); if (key.isReadable()) rwH.handleRead(key); if (key.isWritable()) rwH.handleWrite(key);}Dispatcher InterfaceRegister a channel to be selected and its handler objectUpdate interest of a selectable channelDeregister11Handler Design: AcceptorWhat should an accept handler object know?ServerSocketChannel (so that it can call accept)•Can be derived from SelectionKey in the call backDispatcher (so that it can register new connections)•Need to be passed in constructor or call backWhat ReadWrite object to create (different protocols may use different ones)?•Pass a Factory object: SocketReadWriteHandlerFactory12Handler Design: ReadWriteHandlerWhat should a ReadWrite handler object know?SocketChannel (so that it can read/write data)•Can be derived from SelectionKey in the call backDispatcher (so that it can change state)•Need to be passed in constructor or in call back13Class Hierarchy14DispatcherregisterNewSelection();deregisterSelection();updateInterests();…IChannelHan dlerhandleException();IAcceptHan dlerhandleAccept();IReadWriteHandlerhandleRead();handleWrite();getInitOps();AcceptorimplementsEchoReadWriteHandlerhandleRead();handleWrite();getInitOps();ISocketReadWriteHandlerFactorycreateHandler();1EchoReadWriteHandlerFactorycreateHandler();V3: Fixing v2 ProblemsSee AsyncEchoServer/v3/*.java15Class Diagram of v316DispatcherregisterNewSelection();deregisterSelection();updateInterests();…IChannelHan dlerhandleException();IAcceptHan dlerhandleAccept();IReadWriteHandlerhandleRead();handleWrite();getInitOps();AcceptorimplementsEchoReadWriteHandlerhandleRead();handleWrite();getInitOps();ISocketReadWriteHandlerFactorycreateHandler();1EchoReadWriteHandlerFactorycreateHandler();NewReadWriteHandlerhandleRead();handleWrite();getInitOps();NewReadWriteHandlerFactorycreateHandler();Discussion on v3In our current implementation (Server.java)171. Create dispatcher2. Create server socket channel and listener3. Register server socket channel to dispatcher4. Start dispatcher threadCan we switch 3 and 4?Extending v3A production network server often closes a connection if it does not receive a complete request in TIMEOUTOne way to implement time out is that the read handler registers a timeout event with a timeout watcher thread