Evaluating the Performance of PhotonicInterconnection NetworksRoger Chamberlain, Ch’ng Shi Baw, Mark Franklin, Christopher Hackmann,Praveen Krishnamurthy, Abhijit Mahajan, and Michael WrightonComputer and Communications Research CenterWashington University, St. Louis, MissouriABSTRACTThis paper describes the design and use of the In-terconnection Network Simulator (ICNS) framework.ICNS is a modular, object-oriented simulation systemthat has been developed to investigate performance is-sues in multiprocessor interconnection networks thatexploit photonic te chnology in their design. We de-scribe the ICNS infrastructure, present two distinctphotonic inter conne ction networks that have been mod-eled using ICNS, and give performance results for eachof these networks.1 IntroductionWith the advent of optical fiber, photonic technol-ogy has become an indispensable component in the de-sign and deployment of the world’s long-distance com-munications infrastructure. The bandwidth capacityof long-distance fiber links is enormous, and the tech-nical and economic advantages of photonic technol-ogy in this arena are indisputable. What we haven’tyet seen is the exploitation of photonic technology forshort-distance communications (e.g., chip-to-chip andboard-to-board links within a single system). The rea-sons for this are multifold. Generally, however, al-though photonic interconnection networks have signif-icantly increased bandwidth, the complexity and costof such systems, coupled with the inability of proces-sor interfaces to cope with high photonic data rates,usually negates any expected bandwidth advantages.It is a misconception that merely replacing an existingelectronic interconnect with an optical fiber equivalentwill result in a viable architectural design. To trulytake advantage of photonic technology, the total sys-tem design must be rethought with an understandingof the strengths and weaknesses of photonics.1Research reported herein is supported in part by NSF grantMIP-9706918 and DARPA contract DAAL01-98-C-0074.The Interconnection Network Simulator (ICNS)framework was developed to help evaluate candidatearchitectural alternatives that exploit photonic tech-nology in their design. Our specific interest is the useof photonics in the processor-to-processor interconnec-tion network that is an integral part of any multicom-puter system. The design of ICNS was not limited tothis application, however, and we have modeled bothmulticomputer systems and switching fabrics for inter-net routers.The use of photonic technologies as building blocks for multicomputer interconnects is not, as yet, awell-studied subject. Photonics posses strengths andweaknesses different from electronics from an inter-connection network standpoint. Thus, fundamentaldesign space parameters such as slotted-time versusasynchronous transmission, buffered versus unbufferedswitching, packet-based versus message-based trans-port, etc. need to be reconsidered when designing aphotonic interconnect.ICNS was developed using the MODSIM III lan-guage in a modular, object-oriented manner. It is de-signed to be extended as new architectures are pro-posed and new performance questions are raised. Forthis reason, the simulation framework must be flexi-ble enough to allow various network components withvaried characteristics to be modeled faithfully.As applications demand higher performance fromthe interconnection network, the design of the networkwill be more closely guided by the specific targeted ap-plications. Hence it is important to consider both theapplication and the interconnection network togetherin the design process. ICNS takes special care to en-sure that application-level simulation modules can beeasily integrated with the interconnect modules. Thisallows application issues to be fully explored in tandemwith interconnect issues in the design process.At a very high level, an interconnection networkProceedings of the 35th Annual Simulation Symposium (SS’02) 1080-241X/02 $17.00 © 2002 IEEEterminalterminalterminalterminalterminalterminalterminalterminallinks linksswitchesfabricswitchlinks andFigure 1: A Generic Interconnection Network.can be abstracted as a system composed of terminalsthat generate and consume messages, and links andswitches that facilitate the transportation of messagesfrom one terminal to another. Figure 1 shows a genericinterconnection network.To achieve the desired flexibility and extensibility,a modular, object-oriented approach is adopted as theprinciple design and development methodology for thesimulator. For example, a component that models asimple bufferless switching element can be enhanced tomodel a switching element with a simple FIFO buffer.The FIFO buffer component can be easily extended tomodel a prioritized multiqueue buffer. A switching ele-ment with prioritized multiqueue buffer can further beenhanced to model a switching element with schedul-ing capability. The scheduler module can be easilymodified to perform scheduling using different policies.We have used ICNS to model a pair of systems. Thefirst is the Gemini interconnect, a parallel photonicand electronic network that utilizes lithium niobateoptical switches to construct a circuit-switched high-bandwidth data path in the switching fabric. The sec-ond is a photonic multiring interconnect, in which 2-Darrays of Vertical Cavity Surface Emitting Lasers (VC-SELs) and photodetectors are used to provide high-bandwidth I/O to/from CMOS chips. The variety inphotonic technologies used, as well as the distinct ar-chitectures that result, point to the flexibility of theICNS framework.Section 2 describes the ICNS implementation en-vironment, as well as the base classes and basic ob-ject types that form the core of ICNS. Section 3 de-scribes the model used to simulate such objects aslinks, switches, processing nodes, etc. Section 4 de-scribes the usage of the simulator. Section 5 providesa description of the architectures simulated to date us-ing ICNS, including a description of the photonic com-ponents that are enabling technology for these archi-MessageObjMTMessageObj MHMessageObj MCMessageObjGMessageObjNodeObjMCLinkObjLinkObj GeOPObjGoOPObjGSwitchObjSwitchNodeObj GCPUObj TerminalNodeObjMQBufferObjGTerminalObjGSwitch2x2ObjOutPortRecObjGSwitch2x2CCObjGGeneratorObjGoOPVOQObjNetworkObjGTerminalVOQFQObjGTerminalVOQObjFigure 2: ICNS partial class diagram.tectures. It also gives some performance results thathave been