6.189 IAP 2007 MIT6.189 IAP 2007Student Project PresentationMolecular DynamicsPintilieMolecular Dynamics on the Playstation 3Greg PintilieOverview• Molecular Dynamics• Algorithm• Parallelization ApproachesMolecular Dynamics• Potential Energy:()bondednonbondedpEExE−+=vMolecular Dynamics• Potential Energy:()bondednonbondedpEExE−+=vimpdihanglesbondsbondedEEEE,++=Molecular Dynamics• Potential Energy:()∑−=bondsbbondsllkE20()bondednonbondedpEExE−+=vimpdihanglesbondsbondedEEEE,++=Molecular Dynamics• Potential Energy:()20∑−=anglesangleskEθθθ()bondednonbondedpEExE−+=vimpdihanglesbondsbondedEEEE,++=Molecular Dynamics• Potential Energy:()()()()∑∑−+−+−=impropersdihedralsimpdihkknkF00,cos1ωωθθφωθφ()bondednonbondedpEExE−+=vimpdihanglesbondsbondedEEEE,++=Molecular Dynamics• Potential Energy:()∑−=bondsbbondsbbkE20()20∑−=anglesangleskEθθθ()bondednonbondedpEExE−+=v()()()()∑∑−+−+−=impropersdihedralsimpdihkknkF00,cos1ωωθθφωθφimpdihanglesbondsbondedEEEE,++=Molecular Dynamics• Potential Energy:ticelectrostaWaalsdervanbondednonEEE+=−−−()bondednonbondedpEExE−+=vMolecular Dynamics• Potential Energy:ticelectrostaWaalsdervanbondednonEEE+=−−−∑⎟⎟⎠⎞⎜⎜⎝⎛−=−−kiatomsikikikikWaalsdervanrCrAE,_612()bondednonbondedpEExE−+=vMolecular Dynamics• Potential Energy:ticelectrostaWaalsdervanbondednonEEE+=−−−∑⎟⎟⎠⎞⎜⎜⎝⎛−=−−kiatomsikikikikWaalsdervanrCrAE,_612∑=kiatomsikkiticelectrostaDrqqE,_()bondednonbondedpEExE−+=v• Compute forces :• Integrate to obtain velocity, position:Molecular Dynamics()mtftttvttvvvΔ+⎟⎠⎞⎜⎝⎛Δ−=⎟⎠⎞⎜⎝⎛Δ+2121()()⎟⎠⎞⎜⎝⎛Δ+⋅Δ+=Δ+ ttvttxttx21vvv() () ()xExtaMtfpvvvv∂∂−==• Kinetic Energy/Temperature:– from classical equipartition theory, each degree of freedom has, at thermal equilibrium, this much energy:Kinetic EnergyTkB21TkNvmEBFNiiik2121312==∑=Langevin Dynamics())(tRMvxEaMFp+−−∇==γvv0)( =tR)'(2)'(),( ttTMktRtRBT−=δγ• Account for collisions with imaginary molecules (heat bath)• e.g. in solvent such as waterSolvation in Dielectric Material• Molecules that are polar/ionic ‘shield’ electrostatic forces• Water:– distance-dependent dielectric:∑=kiatomsikkiticelectrostaDrqqE,_rD=∑=kiatomsikkiticelectrostarqqE,_2Non-bonded Cut-offs• Cut-off ~12A∑⎪⎭⎪⎬⎫⎪⎩⎪⎨⎧+⎟⎟⎠⎞⎜⎜⎝⎛−=−kiatomsikkiikikikikbondednonDrqqrCrAE,_612Non-bonded Cut-offs• Cut-off ~12A∑⎪⎭⎪⎬⎫⎪⎩⎪⎨⎧+⎟⎟⎠⎞⎜⎜⎝⎛−=−kiatomsikkiikikikikbondednonDrqqrCrAE,_612Basic MD AlgorithmIntegrateCompute ‘non-bonded’ forcesatom pairsCompute ‘bonded’ forcesbonds, angles, dihedrals, impropersif i % ap_freq == 0find atom pairsFor i=0 to numstepsData StructuresAtomVector pos, vel, forcedouble Massdouble Chargedouble Rmindouble EpsVectordouble x, y, zBondAtom *a1, *a2double k, b0AngleAtom *a1, *a2, *a3double k, t0ImproperAtom *a1, *a2, *a3, *a4double k, t0DihedralAtom *a1, *a2, *a3, *a4list DihedralValuevalsDihedralValuedouble k, phaseint nAtom PairAtom *a1, *a2double eijMoleculelist Atom atomslist Bondbondslist Angleangleslist Improperimproperslist Dihedralsdihedralslist AtomPairatompairsBonded Forces Non-Bonded Forces•A8m–‘Bonded’-total 41,652• 146 atoms x 104 bytes = 15,184• 147 bonds x 24 bytes= 3,528• 275 angles x 28 bytes = 7,700• 393 dihedrals x 16 bytes +414 dihedral values x 20 bytes = 14,568• 21 impropers 32 bytes = 672– ‘Non-bonded’ –total 176,000• 11,000 Atom Pairs x 16 bytes– (1-3 bonded atoms excluded)• 10 x A8m–‘Bonded’–total 416,520– ‘Non-bonded’ –total 16,976,000• 20 x A8m–‘Bonded’–total 833,040– ‘Non-bonded’ –total 68,064,000Sequential Algorithm10ms10msintegrate150ms116,434 pairs1,480ms1,060,850 pairs‘non-bonded’ forcesatom pairs50ms50ms‘bonded’ forcesbonds, angles, dihedrals, impropersBf / Kd6,090 / 880 ms24,440 / 1,750 ms0 msif i%ap_freq==0find atom pairsWith cutoffNo cutoffFor i=0 to numstepsParallelization Approaches• Force Decomposition…………A3A1A2A1• force operation includes both atom positions, returns the force on both atoms• scales well with system size and #processorsAn…A2A1Parallelization Approaches• Force Decomposition…………A3A1A2A1for j=0 to #SPUs•send control block to SPU-jfor step i=0 to num steps•compute bonded forces•compute non-bonded forces• while non-bonded operations remaining• for j=0 to #SPUs• create block with force-operations (200)• send control block with #ops to SPU-j• tell SPU-j to start processing• for j=0 to #SPUs• if SPU-j finished, add forces to atoms• break if all SPUs finished• integrate forcesPPUiter 0iter 1iter 2Parallelization Approaches• Force Decomposition - performance2602702803103906303106SPUs5SPUs4SPUs3SPUs2SPUs1SPUPPUNon-bonded Forces Computation Time0100200300400500600700PPU 1SPU 2SPUs 3SPUs 4SPUs 5SPUs 6SPUsmsParallelization Approaches• Atomic Decomposition…………A3A1A2A1An…A2A1• atoms and forces stored independently• doesn’t scale as easily with system sizeParallelization Approaches• Spatial Decomposition• not load-balanced• atom positions must be communicated between processors• periodically re-assign atomsState of the Art - NAMD• force-spatial