UCB External Connectivity ken lindahl communication and network services uc berkeley lindahl berkeley edu EE122 7 May 2004 CalREN 2 1998 to 2003 conceived as a state wide high performance advanced services network serving California s research and higher education communities charter members UC CSU Stanford CalTech USC CalREN 2 network was supposed to OC 48 2 5Gbps but what got delivered was two OC 12 622 Mbps SONET rings should have asked for OC 12c oops implemented as one POS Packet Over SONET ring and one ATM ring EE122 7 May 2004 2 CalREN 2 northern gigaPOP ca 11 98 distributed gigaPOP with backbone equipment SONET MUXes ATM switches IP routers located in campus facilities commercial SONET service for connections between campuses EE122 7 May 2004 3 CalREN 2 southern gigaPOP ca 11 98 EE122 7 May 2004 4 CalREN 2 backbone circa 11 98 EE122 7 May 2004 5 CalREN 2 1998 to 2003 UCB initially connected to CalREN 2 with a single OC 12 POS link subsequently added one then two OC 3 155 Mbps ATM links for ISP traffic separate path required in order to control campus ISP costs diverse routing provided a measure of redundancy for protection against failures but Evans Hall Machine Room was still single point of failure EE122 7 May 2004 6 UCB external connectivity ca 5 2003 1 Cogent SETI home ISP Internet2 CENIC ISP Level3 Qwest CalREN 2 100 Affiliates Affiliates Affiliates 155 155 622 PAIX 100 inr 666 inr 667 inr 000 rate limiting devices Border Routers inr 668 1000 100 100 1000 1000 1000 100 100 1000 100 100 inr 201 inr 202 1000 Seti Home 1000 1000 1000 inr 205 100 Tier 2 Backbone EECS Network inr 281 ResHall inr 299 inr 220 100 inr 120 inr 280 inr 230 inr 240 inr 260 inr 270 inr 203 100 100 100 100 100 inr 130 inr 140 inr 160 inr 170 inr 100 inr 150 Tier 1 Backbone inr 102 inr 101 EE122 7 May 2004 100 7 shortcomings of the original CalREN 2 network campus facilities often not hardened against power outages flooding rats read Evans Hall very little redundancy in network failures tended to have wide spread significant effects campus support staff often not funded for 24x7 support commercial SONET circuits very expensive especially after initial 3 year contract expired mixing research and ISP traffic on a single backbone had an unexpected result reliability and availability for ISP became more important than flexibility for research it became very difficult to deploy new features on the backbone EE122 7 May 2004 8 CENIC next generation backbone considerations 1 running out of bandwidth on OC 12 backbone ciruits eliminate dependence on flakey campus facilities provide redundant connections to campuses need to separate production and research networks Production Networks avoid change whenever possible run well tested code let somebody else find the bugs very low tolerance for outages capacity planning based on expected long term utilization Advanced Services Networks constantly changing run new code as soon as it becomes available outages are undesirable but acceptable capacity planning based on large amounts of headroom needed for occasional use need for true separation of ISP traffic for rate limiting EE122 7 May 2004 9 CENIC next generation backbone considerations 2 over investment in fiber plants causing many carriers to sell or lease fiber at low cost vs recurring high costs for commercial circuits development of wave division multiplexing technology providing multiple high speed 2 5 Gbps 10 Gbps 40 Gbps channels over a single fiber pair single fiber strand in some cases researchers requesting private pipes between campuses experiment with non IP unroutable packets experiment with routing protocols experiment with dangerous packets small number of researchers requesting dark fiber between campuses EE122 7 May 2004 10 the pyramid plus ISP sub layer within CalREN DC EE122 7 May 2004 11 CalREN Optical Backbone ca 3 2004 EE122 7 May 2004 12 Wave Division Multiplexing sort of demystified multiple channels of data carried over single fiber pair between WDM terminals each channel has its own wavelength lambda or wave optical multiplexers combine multiple wavelengths into single composite signal optical demultiplexers split composite signal into individual wavelengths amplification and regeneration needed for longer distances EE122 7 May 2004 13 CalREN backbone waves CalREN DC HPR DC HPR Teragrid UC Davis Sacramento Triangle Court Oakland Fergus Sunnyvale Soledad Fresno San Luis Obispo Bakersfield Los Angeles Santa Barbara Tustin San Diego EE122 7 May 2004 14 Bay Area Metro Ring EE122 7 May 2004 15 CalREN DC 10 2003 to present OC 48c 2 5 Gbps backbone over private DWDM fiber Six GE links to the commodity Internet Qwest Level3 Cogent no cost private peerings at PAIX LAAP SDNAP Serving California State University all campuses California Community Colleges all campuses University of California all campuses 80 of K 12 schools in California Estimated user base is 10 million faculty staff and students AOL 26 million users CalREN DC 10 million users MSN 9 million users Earthlink 5 million users Netzero Juno 5 million users Comcast ATT 3 6 million users SBC Internet Services 2 2 million users EE122 7 May 2004 16 CalREN HPR 2 2004 to present 10Gigabit Ethernet backbone over private DWDM fiber Cisco 12400 routers 10GE connection to Abilene connection to CUDI Mexico s version of Internet2 Jumbo frames 9180 and IPV6 enabled Serving University of California all campuses Stanford University of Southern California CalTech Jet Propulsion Laboratory JPL future 10GE to campuses 12 18 months from now future PacificWave distributed exchange point with POPs between LA and Seattle ideal for landing transPacific fiber links EE122 7 May 2004 17 UCB connectivity to CalREN 1 Total of six GE links GE to CalREN HPR in CEV near Recreational Sports Facility GE to CalREN DC in CEV near Recreational Sports Facility GE to CalREN ISP in CEV near Recreational Sports Facility GE to CalREN HPR in Evans Hall GE to CalREN DC in Evans Hall GE to CalREN ISP in Evans Hall share a single 1 Gbps channel over SBC EGM not yet in service expect service June 2004 Redundancy two campus locations connected to different CalREN POPs via diverse fiber paths OAK POP Emeryville via dark fiber leased from MFN SFO POP via SBC Extended GigaMan EGM service EE122 7 May 2004 18 new UCB border routers existing border routers were not up to the task inr 666 a cisco 7507 can handle GE interfaces but with only 300 400 Mbps onto the backplane inr 000 a cisco
View Full Document
Unlocking...