First and second level packaging for the IBM eServer z900 by H Harrer H Pross T M Winkel W D Becker H I Stoller M Yamamoto S Abe B J Chamberlin G A Katopis This paper describes the system packaging of the processor cage for the IBM eServer z900 This server contains the world s most complex multichip module MCM with a wiring length of 1 km and a maximum power of 1300 W on a glass ceramic substrate The z900 MCM contains 35 chips comprising the heart of the central electronic complex CEC of this server This MCM was implemented using two different glass ceramic technologies one an MCM D technology using thin film and glass ceramic and the other a pure MCM C technology using glass ceramic with more aggressive wiring ground rules In this paper we compare these two technologies and describe their impact on the MCM electrical design Similarly two different board technologies for the housing of the CEC are discussed and the impact of their electrical properties on the system design is described The high frequency requirements of this design due to operating frequencies of 918 MHz for on chip and 459 MHz for off chip interconnects make a comprehensive design methodology and post routing electrical verification necessary The design methodology including the wiring strategy needed for its success is described in detail in the paper 1 Introduction The IBM S 390 platform has seen a new revitalization with the movement to complementary metal oxide semiconductor CMOS servers which began in 1993 because of the reduced hardware costs high integration density excellent reliability and lower power of CMOS compared to bipolar technology Table 1 shows the development of the S 390 CMOS servers for the last four machine generations From 1998 to 2000 the symmetric multiprocessor SMP MIPS number tripled in two years This improvement in MIPS performance was achieved by using a faster processor cycle time due to chip technology scaling improved cycles per instructions CPI and an increase from 12 to 20 in the number of central processor units CPUs per system The 20 CPUs allow the implementation of a 16 way node with four service processors for the z900 server Table 1 also shows the continued increase of CPU electrical power during the last four years which has led to the significant challenge of cooling a 1300 W multichip module In order to achieve the extremely high system performance for the z900 server an elaborate hierarchical system design had to be followed The basic strategy was Copyright 2002 by International Business Machines Corporation Copying in printed form for private use is permitted without payment of royalty provided that 1 each reproduction is done without alteration and 2 the Journal reference and IBM copyright notice are included on the first page The title and abstract but no other portions of this paper may be copied or distributed royalty free without further permission by computer based and other information service systems Permission to republish any other portion of this paper must be obtained from the Editor 0018 8646 02 5 00 2002 IBM IBM J RES DEV VOL 46 NO 4 5 JULY SEPTEMBER 2002 H HARRER ET AL 397 Table 1 Development of the zServer from 1998 to 2002 Year Machine 1998 1999 2000 2002 398 G5 G6 z900 z900 Uni MIPS SMP MIPS Processors per MCM Processor power W Chip technology m Processor cycle time ns Package cycle time ns 127 152 178 205 250 250 901 1069 1441 1644 2694 2694 12 14 20 20 31 36 25 31 32 38 0 25 0 22 0 18 0 18 2 4 2 0 1 8 1 57 1 3 1 09 4 8 4 0 3 6 3 14 2 6 2 18 to package the zServer core chips consisting of the processor second level L2 cache system control memory bus adapter and memory storage control chips on a single MCM first level package Here the short interconnect lengths with well defined electrical behavior allowed a 1 2 cycle time ratio between the processor and the shared L2 cache This approach has been used in previous S 390 server designs 1 The 20 processors required about 16 000 interconnections For this number of interconnections the MCM technology was the only costeffective packaging solution for supporting the required bus widths which are essential for the performance of the zSeries SMP node For MCM cost performance issues the reader is referred to 2 which is also valid for this design This MCM technology also enabled the use of an easily implementable and reliable refrigeration system for the CEC chips by using a redundant cooler scheme This scheme achieves a low temperature operating point for the MCM chips at which the chip junction temperature is 0 C The multichip module was interconnected to the rest of the system elements e g memory and I O using an advanced printed wiring board based technology Section 2 gives a detailed overview of the logic system structure of the z900 server In the first system released in 2000 the chip set had a cycle time of 1 3 ns using the IBM 0 18 m technology However the MCM was designed to support a CEC chip set that operates at 1 09 ns allowing the processor chips to be upgraded to a faster technology in 2002 This aggressive package design approach enables an easy upgrade at the customer s site by simply replacing the MCM with another MCM containing new processor chips It also minimized the package development costs by using a single MCM design for two product offerings A Hitachi IBM partnership enabled us to have two suppliers for the MCM used in the same z900 server product Specifically there are two types of multichip modules used One manufactured by the IBM Microelectronics Division MD uses glass ceramic technology with a thin film wiring plane pair The other MCM which is functionally equivalent and is manufactured by Hitachi uses a glass ceramic technology with tighter ceramic ground rules and no thin film H HARRER ET AL wiring However since these two designs have the same mechanical dimensions and are functionally equivalent while employing the same bottom side connector to the processor planar board they can be used interchangeably This is the first time that such a complex design total wiring length of 1000 m connecting 16 000 nets in which 80 of all connections must operate at 459 MHz has been implemented in two different technologies in a completely transparent manner and in record design time Although many factors contributed to this achievement the primary ones were excellent team cooperation and an efficient and effective design verification system The two MCM technologies mentioned earlier are
View Full Document
Unlocking...