CalRENCalREN 2 1998 to 2003 conceived as a state wide high performance advanced services network serving California s research and higher education communities charter members UC CSU Stanford CalTech USC CalREN 2 network was supposed to OC 48 2 5Gbps but what got delivered was two OC 12 622 Mbps SONET rings should have asked for OC 12c oops UCB External Connectivity ken lindahl communication and network services uc berkeley lindahl berkeley edu implemented as one POS Packet Over SONET ring and one ATM ring EE122 7 May 2004 EE122 7 May 2004 CalRENCalREN 2 northern gigaPOP gigaPOP ca 11 98 distributed gigaPOP with backbone equipment SONET MUXes ATM switches IP routers located in campus facilities 2 CalRENCalREN 2 southern gigaPOP gigaPOP ca 11 98 commercial SONET service for connections between campuses EE122 7 May 2004 3 CalRENCalREN 2 backbone circa 11 98 EE122 7 May 2004 4 CalRENCalREN 2 1998 to 2003 UCB initially connected to CalREN 2 with a single OC 12 POS link subsequently added one then two OC 3 155 Mbps ATM links for ISP traffic separate path required in order to control campus ISP costs diverse routing provided a measure of redundancy for protection against failures but Evans Hall Machine Room was still single point of failure EE122 7 May 2004 5 EE122 7 May 2004 6 1 UCB external connectivity ca 5 2003 1 CENIC ISP Level3 Qwest shortcomings of the original CalRENCalREN 2 network campus facilities often not hardened against power outages flooding rats read Evans Hall Cogent SETI home ISP Internet2 CalREN 2 100 Affiliates Affiliates Affiliates 155 155 622 very little redundancy in network failures tended to have wide spread significant effects PAIX 100 inr 666 inr 667 inr 000 rate limiting devices Border Routers inr 668 1000 campus support staff often not funded for 24x7 support 100 100 1000 1000 1000 100 100 1000 commercial SONET circuits very expensive especially after initial 3 year contract expired 100 100 inr 201 inr 202 1000 Seti Home 1000 mixing research and ISP traffic on a single backbone had an unexpected result reliability and availability for ISP became more important than flexibility for research it became very difficult to deploy new features on the backbone 1000 1000 inr 205 100 Tier 2 Backbone EECS Network inr 281 ResHall inr 299 inr 220 100 inr 120 inr 280 inr 230 inr 240 inr 260 inr 270 inr 203 100 100 100 100 100 inr 130 inr 140 inr 160 inr 170 inr 100 inr 150 Tier 1 Backbone inr 102 inr 101 EE122 7 May 2004 100 7 CENIC next generation backbone considerations 1 running out of bandwidth on OC 12 backbone ciruits vs recurring high costs for commercial circuits provide redundant connections to campuses development of wave division multiplexing technology need to separate production and research networks avoid change whenever possible run well tested code let somebody else find the bugs very low tolerance for outages capacity planning based on expected long term utilization 8 CENIC next generation backbone considerations 2 over investment in fiber plants causing many carriers to sell or lease fiber at low cost eliminate dependence on flakey campus facilities Production Networks EE122 7 May 2004 providing multiple high speed 2 5 Gbps 10 Gbps 40 Gbps channels over a single fiber pair single fiber strand in some cases Advanced Services Networks constantly changing researchers requesting private pipes between campuses run new code as soon as it becomes available outages are undesirable but acceptable capacity planning based on large amounts of headroom needed for occasional use experiment with non IP unroutable packets experiment with routing protocols experiment with dangerous packets small number of researchers requesting dark fiber between campuses need for true separation of ISP traffic for rate limiting EE122 7 May 2004 9 EE122 7 May 2004 the pyramid 10 CalREN Optical Backbone ca 3 2004 plus ISP sub layer within CalREN DC EE122 7 May 2004 11 EE122 7 May 2004 12 2 Wave Division Multiplexing sort of demystified multiple channels of data carried over single fiber pair between WDM terminals CalREN backbone waves CalREN DC HPR DC each channel has its own wavelength lambda or wave HPR Teragrid UC Davis Sacramento optical multiplexers combine multiple wavelengths into single composite signal optical demultiplexers split composite signal into individual wavelengths Triangle Court Oakland amplification and regeneration needed for longer distances Fergus Sunnyvale Soledad Fresno San Luis Obispo Bakersfield Los Angeles Santa Barbara Tustin San Diego EE122 7 May 2004 13 Bay Area Metro Ring EE122 7 May 2004 14 CalRENCalREN DC 10 2003 to present OC 48c 2 5 Gbps backbone over private DWDM fiber Six GE links to the commodity Internet Qwest Level3 Cogent no cost private peerings at PAIX LAAP SDNAP Serving California State University all campuses California Community Colleges all campuses University of California all campuses 80 of K 12 schools in California Estimated user base is 10 million faculty staff and students AOL 26 million users CalREN DC 10 million users MSN 9 million users Earthlink 5 million users Netzero Juno 5 million users Comcast ATT 3 6 million users SBC Internet Services 2 2 million users EE122 7 May 2004 15 CalRENCalREN HPR 2 2004 to present 10Gigabit Ethernet backbone over private DWDM fiber EE122 7 May 2004 UCB connectivity to CalREN 1 Total of six GE links Cisco 12400 routers 10GE connection to Abilene connection to CUDI Mexico s version of Internet2 Jumbo frames 9180 and IPV6 enabled Serving University of California all campuses Stanford University of Southern California CalTech Jet Propulsion Laboratory JPL EE122 7 May 2004 GE to CalREN HPR in CEV near Recreational Sports Facility GE to CalREN DC in CEV near Recreational Sports Facility GE to CalREN ISP in CEV near Recreational Sports Facility GE to CalREN HPR in Evans Hall GE to CalREN DC in Evans Hall GE to CalREN ISP in Evans Hall share a single 1 Gbps channel over SBC EGM not yet in service expect service June 2004 Redundancy two campus locations connected to different CalREN POPs via diverse fiber paths OAK POP Emeryville via dark fiber leased from MFN SFO POP via SBC Extended GigaMan EGM service future 10GE to campuses 12 18 months from now future PacificWave distributed exchange point with POPs between LA and Seattle ideal for landing transPacific fiber links 16 17 EE122 7 May 2004 18 3 new UCB border routers existing border routers were not up to the task UCB connectivity to CalREN 2 inr 666 a
View Full Document
Unlocking...