CS 636 InternetworkingCS 636 InternetworkingRamana KompellaEND NODE ALGORITHMICSLecture 8: Control1CS 636 InternetworkingWeb server sending a file2Web server applicationTCP/IP File systemWrite()read()kernelServer bufferSocket bufferFile cache bufferCPUMemoryDISKAdaptorNICMEMORY BUSI/O BUSNew abstraction: Web serverCS 636 Internetworking 3TCP/IP File systemWrite()read()CPUBUSMemoryDiskAdaptornetworkTracking active files & conns.findActive()Client 1 … Client 50Client 51 … Client 74Client 75Packet ReceivedO1: Interrupt (2us)O2: Scheduling (10us)O3: System call (4us)kernelSources of delays: TLB flushes, cache locality problems, memory accessesCS 636 InternetworkingControl issues Sources of control overhead Avoiding process scheduling overhead by structuring network and application code Improving the select() system call used by event driven servers Avoiding system call overhead Reducing interrupt overhead4Conventional approach Modularity vs Performance◦ Modular design allows contain complexity◦ Performance takes a hit if too modularCS 636 Internetworking 5Display-receivetransportreceiveNet-receiveNet-dispatchTransportget portNet-sendtransportsendDisplayGet datatransportarm to sendkeyboardreceiveKeyboardsenderRx interrupthandlerreceiverinterruptUpcalls [Clark85] Only one context switch compared to two in the traditional approach. Used in X-Kernel operating systemCS 636 Internetworking 6Display-receivetransportreceiveNet-receiveNet-dispatchTransportget portNet-sendtransportsendDisplayGet datatransportarm to sendkeyboardreceiveKeyboardsenderRx interrupthandlerreceiverinterruptwakeUser-level protocol implementations Idea: Use protocol upcalls to implement protocol handlers in the same process as applications Two advantages◦ Bypass kernel directly◦ Protocol debugging is easier But we need a mechanism for demuxingpackets to the applicationCS 636 Internetworking 7De-multiplexing Security concerns (sandboxing) Reentrant code to avoid duplicatingCS 636 Internetworking 8Kernel(driver)demuxApp 3App 2App 1Naïve wayTwoContextswitchesKernelF1 F2 … FnFilter for P1Filter for PnApp 1App nPacket filter wayE.g., CSPF,BPFCS 636 Internetworking 9“There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.” --- C.A.R. HoareAvoiding context switching From single-packet view to application-wide view Exploit concurrency within application◦ CPU vs disk I/O◦ CPU vs network I/O Naïve web server◦ One process per client◦ Rely on OS scheduler to juggle resources◦ Maximum concurrencyCS 636 Internetworking 10One process per client Simplicity comes with cost◦ Too many context switches◦ Each context switch expensive (10us) Spawning a new process per client expensive as well!◦ Avoid creating/destroying processes by caching a pool of processes◦ Matchmaking problem (use accept())CS 636 Internetworking 11Accept connectionParse requestLocate fileSend headerRead file send data ProcessOne thread per client Even after precomputation, process context switching too painful◦ 6000 concurrent clients  Threads are light-weight processes◦ Share address space & trust each other◦ So, replace processes with threads Apache uses one thread per clientCS 636 Internetworking 12Accept connectionParse requestLocate fileSend headerRead file send data ThreadThreads still inefficient Two main problems:◦ Context switching (not as heavy as process)◦ Lots of memory usage to store thread contexts (takes away from file cache) If something is too general, remove it! Let the application juggle between clients Kernel typically sees everything Export using FindActive() [e.g., select()]CS 636 Internetworking 13Event-driven server Classic OS debate : Threads vs Events In theory, events should provide same amount of concurrency as threads◦ UNIX do not provide suitable support◦ Eg. Non-blocking reads on network i/o but blocking on file systemsCS 636 Internetworking 14Accept connectionParse requestLocate fileSend headerRead file send data Event DispatcherCS 636 InternetworkingMultiple processes vs. eventsMultiple processes/threads Easier to implement Context switch costs Process creation costs Cannot share caches of file name translationsEvent driven One process (per CPU) Keeps track of all clients No blocking sys calls Uses select() to find out when I/O completes15CS 636 InternetworkingEvent driven variants16Accept connectionParse requestLocate fileSend headerRead file send data Event DispatcherHelper 1Helper 2Helper 4Helper 3Event driven with helper processesUsed in Flash Web ServerCS 636 InternetworkingEvent driven variants17Accept connectionParse requestLocate fileSend headerRead file send data Staged Event DrivenAccept connectionParse requestLocate fileSend headerRead file send dataTomorrow Fast select () implementation for event driven serversCS 636 Internetworking