61A Lecture 34November 21st, 2011Last weekDistributed computing!Client-server!Peer-to-peer!Message passing!Modularity!InterfacesParallel computing!Threads!Shared memory!Problems: Synchronization and stale data!Solutions: Locks, semaphores (and conditions)!Deadlock2Sequential dataSome of the most interesting real-world problems in computer science center around sequential data.DNA sequencesWeb and cell-phone traffic streamsThe social data streamSeries of measurements from instruments on a robotStock prices, weather patterns3So far: the sequence abstractionSequences have!Length!Element selection!In python• Membership testing• SlicingData structures that support the sequence abstraction!Nested tuples!Tuples!Strings!Lists (mutable)4Problems with sequencesMemory!Each item must be explicitly represented!Even if all can be generated by a common formula or functionUp-front computation!Have to compute all items up-front!Even if using them one by oneCan’t be infinite!Why care about “infinite” sequences?•They’re everywhere!•Internet and cell phone traffic•Instrument measurement feeds, real-time data•Mathematical sequences5Finding prime numbersSieve of Erastothenes!Find prime numbers by walking down integers!For each integer, eliminate all multiples of that integer!Left with indivisible numbers!2 3 4 5 6 7 8 9 10 11 12 13!2 3 4 5 6 7 8 9 10 11 12 13!2 3 4 5 6 7 8 9 10 11 12 13!2 3 4 5 6 7 8 9 10 11 12 13!2 3 4 5 6 7 8 9 10 11 12 13!2 3 4 5 6 7 8 9 10 11 12 13!2 3 4 5 6 7 8 9 10 11 12 13!2 3 4 5 6 7 8 9 10 11 12 13!2 3 4 5 6 7 8 9 10 11 12 13!2 3 4 5 6 7 8 9 10 11 12 13!2 3 4 5 6 7 8 9 10 11 12 13!2 3 4 5 6 7 8 9 10 11 12 136Working example: finding prime numbers7def primes_sieve(limit):# mark all numbers as prime at first prime = [True] * (limit+1) primes = [] # eliminate multiples of previous numbers for i in range(2, limit+1): if prime[i]: primes.append(i) multiple = i*i while multiple <= limit : prime[multiple] = False multiple += i return primesprimes_sieve(1000000000) anyone?1 billioneach number = 64 bits = 8 bytes8 bytes * 1 billion * 2 = 16 billion bytes = ~14.9 GB of memorysequencesequenceIterators: another abstraction for sequential dataIterators!Store how to compute items instead of items themselves!Give out one item at a time!Save the next until asked (lazy evaluation)Compared with sequences!Length not explicitly defined!Element selection not supported•Element selection -- random access•Iterators -- sequential access!No up-front computation of all items!Only one item stored at a time!CAN be infinite8Implementation: nested delayed evaluation9Nested pairsStreamfirst restnow, explicitstore how to compute itcompute when askedStreams10class Stream(object): def __init__(self, first, compute_rest, empty= False): self.first = first self._compute_rest = compute_rest self.empty = empty self._rest = None self._computed = False @property def rest(self): assert not self.empty, 'Empty streams have no rest.' if not self._computed: self._rest = self._compute_rest() self._computed = True return self._restempty_stream = Stream(None, None, True)first restSequential data: nested streams11Nest streams inside each otherOnly compute one element of a sequence at a timedef make_integer_stream(first=1): def compute_rest(): return make_integer_stream(first+1) return Stream(first, compute_rest)live examplePrime numbers with nested streams12def primes(positive_ints): def not_divible(x): return (x % positive_integers.first) != 0 def sieve(): return primes(filter_stream(not_divible, positive_ints.rest)) return Stream(pos_stream.first, sieve)def filter_stream(filter_func, stream): def make_filtered_rest(): return filter_stream(filter_func, stream.rest) if stream.empty: return stream if filter_func(stream.first): return Stream(s.first, make_filtered_rest)else: return filter_stream(filter_funct, stream.rest)Prime numbers with nested streams13def primes(positive_ints): def not_divible(x): return (x % positive_integers.first) != 0 def sieve(): return primes(filter_stream(not_divible, positive_ints.rest)) return Stream(pos_stream.first, sieve)>>> p = primes(make_integer_stream(5))>>> p.first5>>> p.rest>>> p.rest.first7<Stream instance at ... >Native python iteratorsPython natively supports iteratorsThe Iterator interface in python:!__iter__ •should return an iterator object!__next__•should return a value OR!raise StopIteration•when end of sequence is reached•on all subsequent calls14Native python iterators: example15class Letters(object): def __init__(self, start, finish): self.current = start self.finish = finish def __next__(self): if self.current > self.finish: raise StopIteration result = self.current self.current = chr(ord(result)+1) return result def __iter__(self): return self>>> letters = Letters('a', 'd')>>> letters.__next__()'a'>>> letters.__next__()'b'>>> letters.__next__()'c'>>> letters.__next__()'d'>>> letters.__next__()Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 12, in nextStopIterationFrom a native python iterator to a nested stream16empty_stream = Stream(None, None, True)def iterator_to_stream(iterator): def streamify(): try: first = iterator.__next__() return Stream(first, streamify) except: return empty_stream stream = streamify() return streamMore support: for loops!“for” loops use iterators!Step 1: get an iterator!iterator = obj.__iter__()!Step 2:! try iterator.__next__()! assign value to “item”! do body of loop! until StopIteration is raised17def for_each(sequence, function): iterator = sequence.__iter__() try: while True : element = iterator.__next__() function(element) except StopIteration as e: passfor item in obj:do stufflive exampleEven more support: generator functions18class Letters(object): def __init__(self, start, finish): self.current = start self.finish = finish def __next__(self): if self.current > self.finish: raise StopIteration result = self.current self.current = chr(ord(result)+1) return result def __iter__(self): return selfdef letters(start, finish): current = start while current <= finish: yield current current = chr(ord(current)+1)Generator versiondef letters(start, finish): current = start while current <= finish: yield current current = chr(ord(current)+1)Yield: a built-in flow-control statementDoes nothing at firstwhen __next__() is called,
View Full Document