1UCB External ConnectivityUCB External Connectivityken lindahlcommunication and network servicesuc [email protected], 7 May 2004EE122, 7 May 2004 2CalRENCalREN--2: 1998 to 20032: 1998 to 2003• conceived as a state-wide high-performance, advanced services network serving California’s research and higher education communities.• charter members: UC, CSU, Stanford, CalTech, USC• CalREN-2 network was supposed to OC-48 (2.5Gbps), but what got delivered was two OC-12 (622 Mbps) SONET rings– should have asked for “OC-12c.” oops.• implemented as one POS (Packet Over SONET) ring and one ATM ringEE122, 7 May 2004 3CalRENCalREN--2 northern 2 northern gigaPOPgigaPOP, ca. 11/98, ca. 11/98• “distributed gigaPOP,” with backbone equipment (SONET MUXes, ATM switches, IP routers) located in campus facilities.• commercial SONET service for connections between campuses.EE122, 7 May 2004 4CalRENCalREN--2 southern 2 southern gigaPOPgigaPOP, ca. 11/98, ca. 11/98EE122, 7 May 2004 5CalRENCalREN--2 backbone, circa 11/982 backbone, circa 11/98EE122, 7 May 2004 6CalRENCalREN--2: 1998 to 20032: 1998 to 2003• UCB initially connected to CalREN-2 with a singleOC-12 POS link• subsequently added one, then two, OC-3 (155 Mbps) ATM links for ISP traffic– separate path required in order to control campus ISP costs– diverse routing provided a measure of redundancy, for protectionagainst failures (but Evans Hall Machine Room was still single point of failure)2EE122, 7 May 2004 7UCB external connectivity, ca. 5/2003 (1)UCB external connectivity, ca. 5/2003 (1)inr-2051000100010001000inr-220inr-230 inr-240 inr-260 inr-270inr-281inr-280inr-120 inr-130 inr-140 inr-160 inr-170 inr-100100inr-150100100 100 100 100 100inr-101inr-102inr-299ResHallEECS Network1001551000 1000Internet2622CalREN-2inr-000100010001000100155CENIC ISP(Level3, Qwest)100100inr-667inr-666100100inr-668Cogent(SETI@home ISP)PAIX100AffiliatesAffiliatesAffiliatesinr-201 inr-202100rate limitingdevicesinr-203100Tier 1BackboneTier 2BackboneSeti@HomeBorderRoutersEE122, 7 May 2004 8shortcomings of the original CalRENshortcomings of the original CalREN--2 network2 network• campus facilities often not hardened against power outages, flooding, rats, … (read: Evans Hall)• very little redundancy in network; failures tended to have wide-spread, significant effects• campus support staff often not funded for 24x7 support• commercial SONET circuits very expensive, especially after initial 3-year contract expired• mixing research and ISP traffic on a single backbone had an unexpected result: reliability and availability (for ISP) became more important than flexibility (for research), it became very difficult to deploy new features on the backbone.EE122, 7 May 2004 9CENIC next generation backbone considerations (1)CENIC next generation backbone considerations (1)• running out of bandwidth on OC-12 backbone ciruits• eliminate dependence on flakey campus facilities• provide redundant connections to campuses• need to separate production and research networks• need for true separation of ISP traffic for rate-limitingcapacity planning based on large amounts of headroom needed for occasional usecapacity planning based on expected long-term utilizationoutages are undesirable, but acceptablevery low tolerance for outagesrun new code as soon as it becomes availablerun well-tested code, let somebody else find the bugsconstantly changingavoid change whenever possibleAdvanced Services NetworksProduction NetworksEE122, 7 May 2004 10CENIC next generation backbone considerations (2)CENIC next generation backbone considerations (2)• over-investment in fiber plants causing many carriers to sell or lease fiber at low cost– vs. recurring high costs for commercial circuits• development of wave division multiplexing technology– providing multiple high-speed (2.5 Gbps, 10 Gbps, 40 Gbps, …) channels over a single fiber pair (single fiber strand in some cases)• researchers requesting “private pipes” between campuses– experiment with non-IP (unroutable) packets– experiment with routing protocols– experiment with “dangerous” packets• small number of researchers requesting dark fiber between campusesEE122, 7 May 2004 11the pyramid!the pyramid!• plus, ISP “sub-layer” within CalREN-DCEE122, 7 May 2004 12CalREN Optical Backbone, ca. 3/2004CalREN Optical Backbone, ca. 3/20043EE122, 7 May 2004 13Wave Division Multiplexing (sort of) demystifiedWave Division Multiplexing (sort of) demystified• multiple channels of data carried over single fiber pair between WDM terminals• each channel has its own wavelength (“lambda” or “wave”)• optical multiplexers combine multiple wavelengths into single composite signal; optical demultiplexers split composite signal into individual wavelengths• amplification and regeneration needed for longer distancesEE122, 7 May 2004 14CalREN backbone wavesCalREN backbone wavesSan DiegoTustinLos AngelesSanta BarbaraSunnyvaleOaklandSan LuisObispoSoledadHPRTeragridHPR & DCCalREN DCFergusFresnoBakersfieldSacramentoUC DavisTriangle CourtEE122, 7 May 2004 15Bay Area Metro RingBay Area Metro RingEE122, 7 May 2004 16CalRENCalREN--DC: 10/2003 to presentDC: 10/2003 to present• OC-48c (2.5 Gbps) backbone over private DWDM fiber• Six GE links to the commodity Internet (Qwest, Level3, Cogent)• “no-cost” private peerings at PAIX, LAAP, SDNAP• Serving:– California State University (all campuses)– California Community Colleges (all campuses)– University of California (all campuses)– ~80% of K-12 schools in California• Estimated user base is 10 million faculty, staff and students!AOL: 26 million usersCalREN-DC: 10 million usersMSN: 9 million usersEarthlink: 5 million usersNetzero/Juno: 5 million usersComcast/ATT: 3.6 million usersSBC Internet Services: 2.2 million usersEE122, 7 May 2004 17CalRENCalREN--HPR: 2/2004 to presentHPR: 2/2004 to present• 10Gigabit Ethernet backbone over private DWDM fiber• Cisco 12400 routers• 10GE connection to Abilene• connection to CUDI (Mexico’s version of Internet2)• Jumbo frames (9180) and IPV6 enabled• Serving:– University of California (all campuses)– Stanford– University of Southern California – CalTech & Jet Propulsion Laboratory (JPL)• (future) 10GE to campuses, 12-18 months from now• (future) PacificWave: distributed exchange point with POPs between LA and Seattle, ideal for landing trans-Pacific
View Full Document
Unlocking...