Tony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 1Efficient Implementation of High-Energy Physics Processing Using Modified Jet Reconstruction and Active Size Partitioning Tony GregersonUniversity of WisconsinTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 2100m below groundThe Large Hadron ColliderTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 3Compact Muon Solenoid (CMS) ExperimentTrackerCollisionFigure: D. BarneyTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 4Regional Calorimeter Trigger• Energy-based Particle Identification• Designed at UW-Madison• 4,000 1.2 Gigabit links• ~300 boards• 1000s of custom ASICs• Reducing implementation cost is importantTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 5Super-LHC Calorimeter Trigger• Designed at UW-Madison• Specifications• New data set every 25 ns• 4 Tb/s throughput• Computation latency < 1 us• 10x increased luminosityTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 62D Cartesian Tower Mapping3D Cylindrical Tower MappingClustering GridPhysical Detector4096 Towers 17-bit towers processed every 25 nsGrid is divided into small (~ 12 x 12) blocks that are processed in parallel on different chipsMapping Calorimeter DataTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 7Parallel Processing………Cluster FinderE/P ID andWeighting Cluster FinderE/P ID and WeightingCluster FinderE/P ID and WeightingOverlapFilterOverlapFilterOverlapFilter……………………………..…ClusterFinderE/P ID andWeightingOverlapFilterJet ReconstructionIsolationET sumsIDJet ReconstructionIsolationET sumsIDJet ReconstructionIsolationET sumsIDJet ReconstructionIsolationET sumsIDJetSortereGamma,IsoEgammaTau sorterMET,HT,SumEtcalculationTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 8Jet Reconstruction Challenges• Jet Reconstruction operates on a large area (10 x 10 towers)• This requires a large amount of padding elements• Bandwidth limits the size of the grid in each chip (~ 12 x 12 to 15 x 15)• Padding elements are a major hardware overheadTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 9Modified Jet Reconstruction• Different Jet sizes (6x6 – 8x8)• Operate on raw towers instead of filtered clusters (Old Method+)• Build jets using reduced input granularity (Region-based Jet Finding)• Reduces the positional resolution based on region size• Evaluated region sizes from 1x1 (no reduction) to 4x4Several ideas to reduce Jet Reconstruction cost by altering the specificationTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 10Region-based Jets (2x2)2x2 Region-based Jet finder produces half as many jets, but requires 1/3 the input bandwidth to Reconstruction chipTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 11Modified Jet Reconstruction01020304050607080901000 100 200 300 400 500Number of FPGAs to fit algorithmBandwidth per FPGA (Gbps)Jet Finders Compared6x6 Jet - 1x1 Region6x6 Jet - 2x2 Region6x6 Jet - 3x3 Region7x7 Jet - 1x1 Region8x8 Jet - 1x1 Region8x8 Jet - 2x2 Region8x8 Jet - 4x4 Region8x8 Jet - Old MethodTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 1201020304050607080901000 100 200 300 400 500Number of FPGAsBandwidth per FPGA (Gbps)8x8 Jet Finders8x8 Jet - 1x1 Region8x8 Jet - 2x2 Region8x8 Jet - 4x4 Region8x8 Jet - Old Method8x8 Jet - Old Method+8x8 Modified Jet ReconstructionTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 138x8 Jet Reconstruction on Commercially-Available FPGAsFPGA (Gbps) Original Original+ 1x1 Region2x2 Region4x4 RegionV5-LX110T (48) N/A N/A 320 90 70V5-TX240T (240) 51 34 25 16 13V6-HX565T (450) 16 13 13 9 8• FPGA used in system likely to be similar to 240T• Region-based Jet Finder requires 2-4 times fewer FPGAs at this bandwidth• 2 x 2 Region is almost as efficient as 4 x 4 Region but has 4 times the positional resolution• Final choice also depends on physics needsTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 14Active Size Partitioning• Algorithm & System partitioning algorithm designed to minimize padding overhead for very-high throughput grid-processing applications• See Paper for detailsTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 15Thanks for your attention.Tony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 16Backup SlidesBackup SlidesTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 17Clustering Algorithm - ParticlesECALe/γECALHCALτECALHCALjetφηφηφ• Taus• Confined in 2x3 Clusters• Small energy leak in surrounding towers• Jets• Most of the energy confined in a central core• For jets over 20 GeV, the energy is included in a 8x8 region• Electrons/Photons– Spatially confined in a cluster of 2x2 trigger towers– Significantly higher ECAL contribution– Isolated e/γ should have low energy deposits in the surrounding areaTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 18Clustering Algorithm – I/O• Electromagnetic energy (ECAL) [8]• Hadron energy (HCAL) [8]• Finegrain Veto Bit (FG) [1]Input [17 bits x 4096 Towers] (3 Tb/s)Output [748 bits] (30 Gb/s)• 4 e/p, 4 tau, 12 jets (energy & position)• Total energy (ET)• Missing energy (MET)• Total jet energy (HT)• Missing jet energy (MHT)Tony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 19• Particle Cluster Finder– Filter out low-energy towers– Forms clusters– Initial energy calculations• Electron/Photon ID– Analyzes the energy contribution of ECAL & HCAL in a cluster• Cluster Weighting– Calculates ‘Center-of-Energy’ position for each cluster• Cluster Overlap Filter– Removes overlapping towers between clusters, creating local maxima– Prunes low energy clustersClustering Algorithm SubsystemsTony Gregerson, U. Wisconsin, 13 April 2010Computing at the Large Hadron Collider- 20• Particle Isolation– Calculates isolation deposits around 2x2, 2x3 clusters• Particle ID– Characterizes clusters as
View Full Document
Unlocking...