Lecture Notes CMSC 251 Lecture 26 Chain Matrix Multiplication Thursday April 30 1998 Read Section 16 1 of CLR Chain Matrix Multiplication This problem involves the question of determining the optimal sequence for performing a series of operations This general class of problem is important in compiler design for code optimization and in databases for query optimization We will study the problem in a very restricted instance where the dynamic programming issues are easiest to see Suppose that we wish to multiply a series of matrices A1 A2 An Matrix multiplication is an associative but not a commutative operation This means that we are free to parenthesize the above multiplication however we like but we are not free to rearrange the order of the matrices Also recall that when two nonsquare matrices are being multiplied there are restrictions on the dimensions A p q matrix has p rows and q columns You can multiply a p q matrix A times a q r matrix B and the result will be a p r matrix C The number of columns of A must equal the number of rows of B In particular for 1 i p and 1 j r C i j q X A i k B k j k 1 Observe that there are pr total entries in C and each takes O q time to compute thus the total time e g number of multiplications to multiply these two matrices is p q r A B q C Multiplication time pqr p p r q r Figure 33 Matrix Multiplication Note that although any legal parenthesization will lead to a valid result not all involve the same number of operations Consider the case of 3 matrices A1 be 5 4 A2 be 4 6 and A3 be 6 2 mult A1 A2 A3 mult A1 A2 A3 5 4 6 5 6 2 180 4 6 2 5 4 2 88 Even for this small example considerable savings can be achieved by reordering the evaluation sequence The Chain Matrix Multiplication problem is Given a sequence of matrices A1 A2 An and dimensions p0 p1 pn where Ai is of dimension pi 1 pi determine the multiplication sequence that minimizes the number of operations Important Note This algorithm does not perform the multiplications it just figures out the best order in which to perform the multiplications 79 Lecture Notes CMSC 251 Naive Algorithm We could write a procedure which tries all possible parenthesizations Unfortunately the number of ways of parenthesizing an expression is very large If you have just one item then there is only one way to parenthesize If you have n items then there are n 1 places where you could break the list with the outermost pair of parentheses namely just after the 1st item just after the 2nd item etc and just after the n 1 st item When we split just after the kth item we create two sublists to be parenthesized one with k items and the other with n k items Then we could consider all the ways of parenthesizing these Since these are independent choices if there are L ways to parenthesize the left sublist and R ways to parenthesize the right sublist then the total is L R This suggests the following recurrence for P n the number of different ways of parenthesizing n items 1 if n 1 Pn 1 P n if n 2 k 1 P k P n k This is related to a famous function in combinatorics called the Catalan numbers which in turn is related to the number of different binary trees on n nodes In particular P n C n 1 and 1 2n C n n 1 n Applying Stirling s formula we find that C n 4n n3 2 Since 4n is exponential and n3 2 is just polynomial the exponential will dominate and this grows very fast Thus this will not be practical except for very small n Dynamic Programming Solution This problem like other dynamic programming problems involves determining a structure in this case a parenthesization We want to break the problem into subproblems whose solutions can be combined to solve the global problem For convenience we can write Ai j to be the product of matrices i through j It is easy to see that Ai j is a pi 1 pj matrix In parenthesizing the expression we can consider the highest level of parenthesization At this level we are simply multiplying two matrices together That is for any k 1 k n 1 A1 n A1 k Ak 1 n Thus the problem of determining the optimal sequence of multiplications is broken up into 2 questions how do we decide where to split the chain what is k and how do we parenthesize the subchains A1 k and Ak 1 n The subchain problems can be solved by recursively applying the same scheme The former problem can be solved by just considering all possible values of k Notice that this problem satisfies the principle of optimality because if we want to find the optimal sequence for multiplying A1 n we must use the optimal sequences for A1 k and Ak 1 n In other words the subproblems must be solved optimally for the global problem to be solved optimally We will store the solutions to the subproblems in a table and build the table in a bottom up manner For 1 i j n let m i j denote the minimum number of multiplications needed to compute Ai j The optimum cost can be described by the following recursive definition As a basis observe that if i j then the sequence contains only one matrix and so the cost is 0 There is nothing to multiply Thus m i i 0 If i j then we are asking about the product Ai j This can be split by considering each k i k j as Ai k times Ak 1 j The optimum time to compute Ai k is m i k and the optimum time to compute Ak 1 j is m k 1 j We may assume that these values have been computed previously and stored in our array Since Ai k is a pi 1 pk matrix and Ak 1 j is a pk pj matrix the time to multiply them is pi 1 pk pj This suggests the following recursive rule for computing m i j m i i m i j 0 min m i k m k 1 j pi 1 pk pj i k j 80 for i j Lecture Notes CMSC 251 It is not hard to convert this rule into a procedure which is given below The only tricky part is arranging the order in which to compute the values In the process of computing m i j we will need to access values m i k and m k 1 j for k lying between i and j This suggests that we should organize things our computation according to the number of matrices in the subchain Let L j i 1 denote the length of the …