Lecture Notes CMSC 251Lecture 5: Asymptotics(Tuesday, Feb 10, 1998)Read: Chapt. 3 in CLR. The Limit Rule is not really covered in the text. Read Chapt. 4 for next time.Asymptotics: We have introduced the notion of Θ() notation, and last time we gave a formal definition.Today, we will explore this and other asymptotic notations in greater depth, and hopefully give a betterunderstanding of what they mean.Θ-Notation: Recall the following definition from last time.Definition: Given any function g(n), we define Θ(g(n)) to be a set of functions:Θ(g(n)) = {f(n) | there exist strictly positive constants c1, c2, and n0such that0 ≤ c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ n0}.Let’s dissect this definition. Intuitively, what we want to say with “f(n) ∈ Θ(g(n))” is that f(n) andg(n) are asymptotically equivalent. This means that they have essentially the same growth rates forlarge n. For example, functions like 4n2, (8n2+2n−3), (n2/5+√n−10 log n), and n(n −3) are allintuitively asymptotically equivalent, since as n becomes large, the dominant (fastest growing) term issome constant times n2. In other words, they all grow quadratically in n. The portion of the definitionthat allows us to select c1and c2is essentially saying “the constants do not matter because you maypick c1and c2however you like to satisfy these conditions.” The portion of the definition that allowsus to select n0is essentially saying “we are only interested in large n, since you only have to satisfythe condition for all n bigger than n0, and you may make n0as big a constant as you like.”An example: Consider the function f (n)=8n2+2n−3. Our informal rule of keeping the largest termand throwing away the constants suggests that f (n) ∈ Θ(n2) (since f grows quadratically). Let’s seewhy the formal definition bears out this informal observation.We need to show two things: first, that f(n) does grows asymptotically at least as fast as n2, andsecond, that f(n) grows no faster asymptotically than n2. We’ll do both very carefully.Lower bound: f(n) grows asymptotically at least as fast as n2: This is established by the portionof the definition that reads: (paraphrasing): “there exist positive constants c1and n0, such thatf(n) ≥ c1n2for all n ≥ n0.” Consider the following (almost correct) reasoning:f(n)=8n2+2n−3≥8n2−3=7n2+(n2−3) ≥ 7n2=7n2.Thus, if we set c1=7, then we are done. But in the above reasoning we have implicitly madethe assumptions that 2n ≥ 0 and n2−3 ≥ 0. These are not true for all n, but they are true for allsufficiently large n. In particular, if n ≥√3, then both are true. So let us select n0≥√3, andnow we have f (n) ≥ c1n2, for all n ≥ n0, which is what we need.Upper bound: f(n) grows asymptotically no faster than n2: This is established by the portion ofthe definition that reads “there exist positive constants c2and n0, such that f(n) ≤ c2n2for alln ≥ n0.” Consider the following reasoning (which is almost correct):f(n)=8n2+2n−3≤8n2+2n≤8n2+2n2=10n2.This means that if we let c2=10, then we are done. We have implicitly made the assumptionthat 2n ≤ 2n2. This is not true for all n, but it is true for all n ≥ 1. So, let us select n0≥ 1, andnow we have f (n) ≤ c2n2for all n ≥ n0, which is what we need.16Lecture Notes CMSC 251From the lower bound, we have n0≥√3 and from the upper bound we have n0≥ 1, and so combiningthese we let n0be the larger of the two: n0=√3. Thus, in conclusion, if we let c1=7,c2=10, andn0=√3, then we have0 ≤ c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ n0,and this is exactly what the definition requires. Since we have shown (by construction) the existence ofconstants c1, c2, and n0, we have established that f(n) ∈ n2. (Whew! That was a lot more work thanjust the informal notion of throwing away constants and keeping the largest term, but it shows how thisinformal notion is implemented formally in the definition.)Now let’s show why f (n) is not in some other asymptotic class. First, let’s show that f(n) /∈ Θ(n).If this were true, then we would have to satisfy both the upper and lower bounds. It turns out thatthe lower bound is satisfied (because f(n) grows at least as fast asymptotically as n). But the upperbound is false. In particular, the upper bound requires us to show “there exist positive constants c2and n0, such that f(n) ≤ c2n for all n ≥ n0.” Informally, we know that as n becomes large enoughf(n)=8n2+2n−3will eventually exceed c2n no matter how large we make c2(since f(n) isgrowing quadratically and c2n is only growing linearly). To show this formally, suppose towards acontradiction that constants c2and n0did exist, such that 8n2+2n−3≤c2nfor all n ≥ n0. Sincethis is true for all sufficiently large n then it must be true in the limit as n tends to infinity. If we divideboth side by n we have:limn→∞8n +2−3n≤c2.It is easy to see that in the limit the left side tends to ∞, and so no matter how large c2is, this statementis violated. This means that f(n) /∈ Θ(n).Let’s show that f (n) /∈ Θ(n3). Here the idea will be to violate the lower bound: “there exist positiveconstants c1and n0, such that f(n) ≥ c1n3for all n ≥ n0.” Informally this is true because f(n) isgrowing quadratically, and eventually any cubic function will exceed it. To show this formally, supposetowards a contradiction that constants c1and n0did exist, such that 8n2+2n−3 ≥ c1n3for all n ≥ n0.Since this is true for all sufficiently large n then it must be true in the limit as n tends to infinity. If wedivide both side by n3we have:limn→∞8n+2n2−3n3≥ c1.It is easy to see that in the limit the left side tends to 0, and so the only way to satisfy this requirementis to set c1=0, but by hypothesis c1is positive. This means that f(n) /∈ Θ(n3).O-notation and Ω-notation: We have seen that the definition of Θ-notation relies on proving both a lowerand upper asymptotic bound. Sometimes we are only interested in proving one bound or the other. TheO-notation allows us to state asymptotic upper bounds and the Ω-notation allows us to state asymptoticlower bounds.Definition: Given any function g(n),O(g(n)) = {f (n) | there exist positive constants c and n0such that0 ≤ f(n) ≤ cg(n) for all n ≥ n0}.Definition: Given any function g(n),Ω(g(n)) = {f(n) | there exist positive constants c and n0such that0 ≤ cg(n) ≤ f(n) for all n ≥ n0}.17Lecture Notes CMSC