Unformatted text preview:

IE172 – Algorithms in Systems EngineeringLecture 3Pietro BelottiDepartment of Industrial & Systems EngineeringLehigh UniversityJanuary 16, 2009Our problem◮We want to understand how many steps f (n) it takes toexecute a certain algorithm on a problem of size n◮We care about large values of n, i.e., in the asymptoticalbehavior of f (n) (i.e. how f (n) behaves for n → ∞)◮That allows us to compare tw o algorithms: if A and B takef (n) and g(n), resp., and f(n) grows faster than g(n), then Bis a better algorithm than AComparing Algorithms◮Consider algorithm A with running time given by f andalgorithm B with running time given by g.◮We are interested in knowingL = limn→∞f (n)g(n)◮What are the four possibilities?◮L = 0: g grows faster than f◮L = ∞: f grows faster than g◮L = c: f and g grow at the same rate.◮The limit d oesn’t exist.Θ Notation◮We now define th e setΘ(g) = {f : ∃ c1, c2, n0> 0 such thatc1g(n) ≤ f (n) ≤ c2g(n) ∀n ≥ n0}◮If f ∈ Θ(g), then we say that f and g grow at the same rateor that they are of the same order.◮Note t h atf ∈ Θ(g) ⇔ g ∈ Θ(f)◮We also know that iflimn→∞f (n)g(n)= cfor some constant c, then f ∈ Θ(g).Big-O NotationO(g) = {f | ∃ constants c, n0> 0 s.t. f (n) ≤ cg(n) ∀n ≥ n0}◮If f ∈ O(g), then we say that “f is big-O of” g or that ggrows at least as fast as f◮If we say 2n2+ 3n + 1 = 2n2+ O(n) this means that2n2+ 3n + 1 = 2n2+ f (n) for some f ∈ O(n) (e. g.f (n) = 3n + 1).Big-Ω NotationΩ(g) = {f | ∃ constants c, n0> 0 s.t. 0 ≤ cg(n) ≤ f(n) ∀n ≥ n0}◮f ∈ Ω(g) means that g is an asymptotic lower bound on f◮f “grows faster than” gNote◮f ∈ Θ(g) ⇔ f ∈ O(g) and f ∈ Ω(g).◮f ∈ Ω(g) ⇔ g ∈ O(f ).Strict Asymptotic Bounds. “Little oh”f ∈ o(g) ⇔ limn→∞f (n)g(n)= 0f ∈ ω(g) ⇔ g ∈ o(f) ⇔ limn→∞f (n)g(n)= ∞Note◮f ∈ o(g) ⇒ f ∈ O(g) \ Θ(g).◮f ∈ ω(g) ⇒ f ∈ O(g) \ Θ(g).Comparing Functions◮The notation we have just defined gives us a way ofordering functions.◮This gives us a metho d for comparing algorithms based ontheir running times!The Upshot!◮f ∈ O(g) is like “f ≤ g,”◮f ∈ Ω(g) is like “f ≥ g,”◮f ∈ o(g) is like “f < g,”◮f ∈ ω(g) is like “f > g,” and◮f ∈ Θ(g) is like “f = g.”Commonly Occurring FunctionsPolynomials◮f (n) =Pki=0ainiis a polynomial of degree k◮Polynomials f of degree k are in Θ(nk).Exponentials◮A function in which n appears as an exponent on aconstant is anexponential function, i.e., 2n.◮For all constants a > 0 and b > 1, limn→∞nabn= 0.⇒ Exponential functions grow faste r than polynomialsMore FunctionsLogarithms◮x = logb(a) ⇔ bx= a◮Logarithms of different bases differ only by a constantmultiple, so they all grow at the same rate.◮A po lylogarithmic function is a function inO(lgk) = O((lg n)k).◮Polylogarithmic functions always grow more slowlythan polynomials.Factorials◮n! = n(n − 1)(n − 2) · · · 3 · 2 · 1◮n! ∈ o(nn), n! ∈ ω(2n)◮lg(n!) ∈ Θ(n lg n)Logs◮anam= an+m◮We use the notation◮lg n = log2n◮ln n = logen◮lgkn = (lg n)k◮Changing the base of alogarithm change s itsvalue by a constant factorLog Rules◮a = blogba◮lgQnk=1ak=Pnk=1lg ak◮logban= n logba◮logba = (logca)/(logcb)◮logba = 1/(logab)◮alogbn= nlogbaProblem difficulty◮The difficulty of a problem can be judged by the(worst-case) running time of the best-known algorithm.◮Problems for which there is an algorithm with polynomialrunning time (or be tter) are calledpolynomially s olvable.◮Generally, these problems are conside red to be easy.◮Formally, they are in the complexity class P◮There are many interesting problems for w h ich it is notknown if there is a polynomial-time algorithm.◮These problems are generally considered difficult.◮This is known as the complexity class N P.One of the great open qu estions in Mathematics1:is P = NP?That is, are those in N P difficult problems, or have we not yetdiscovered the right algorithm?1If you prove it, you can win amillion dollars:http://www.claymath.org/millennium/P_vs_NP/Measuring the running time of algorithmsFibonacci, direct methodint Fib (int n) {f [0] = 0; f [1] = 1;for (int i=2; i<=n; i++)f [i] = f [i-1] + f [i-2];return f[n];}Easy... it takes n + 1 steps.Fibonacci, the recursive wayint Fib (int n) {if (n==0) return 0;if (n==1) return 1;return (Fib (n-1) + Fib (n-2);}Not so easy with recursive algorithms. We need other tools.Analyzing recurrences◮General me thods for analyzing recurrences◮Substitution◮Master Theorem◮(Generating Functions)◮Note t h at when we analyze a recurrence, we may not getor ne ed an exact answer, only an asymptotic one◮We may prove the running time is in O(f) or Θ(f)We are looking at the asymptotic behavior of recurrences⇒ Ignore floors and ceilings: (Asymptotic behavior doesn’tcare if y ou round down or up)⇒ Assume th at all algorithms run in Θ(1) (constant time) fora s mall enough fix ed input size n0. This makes the basecase of induction easy.A Few Examples of Recurrences◮This recurrence arises in algorithms that loop through theinput to eliminate one item.T(n) =(1 n = 1T(n − 1) + n n > 1◮This recurrence arises in algorithms that halve the input inone step.T(n) =(1 n = 1T(n/2) + 1 n > 1Some More Recurrences◮This recurrence arises in algorithms that halve the input inone step, but have to scan through the data at each step.T(n) =(1 n = 1T(n/2) + n n > 1◮This recurrence arises in algorithms that quarter the inputin o n e step, but have to scan through the data 4 times ateach step.T(n) =(1 n = 1T(n/4) + 4n n > 1Solving Recurrences by SubstitutionA Simple Two P art Plan◮Guess an answer◮Use induction to prove o r disprove your guess◮Here let’s show that ifT(n) = T(⌈n/2⌉) + 1 ⇒ T ∈ O(lg n)The Master Theorem◮Most recurrences that we care about are of the formT(n) =(1 n = 1aT(n/b) + f (n) n > 1◮Use the Master Theorem to analyze such recurrences:a) If f ∈ O(nlogba−ε), for some ε > 0, then T ∈ Θ(nlogba).b) If f ∈ Θ(nlogba), then T ∈ Θ(nlogbalg n).c) If f ∈ Ω(nlogba+ε), for some ε > 0, and if af (n/b) ≤ cf(n) forsome c < 1 and n > n0, then T ∈ Θ(f ).◮How do we interpret this?A Few More Exam ples◮This recurrence arises in algorithms t h at partition the inputin o n e step, but then make recursive calls on both pieces.T(n) =(1 n = 12T(n/2) + 1 n >


View Full Document

Clemson IE 172 - lecture 3

Download lecture 3
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view lecture 3 and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view lecture 3 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?