Darwin Phones: The Evolution of Sensing and Inference on Mobile Phones Presented By: Brandon OchsWhat does Darwin do?What about battery life?Common Urban Sensing ChallengesTypes of LearningDarwin StepsClassifier EvolutionModel PoolingCollaborative InferenceDarwin Design: ComputationDarwin Design: ContextDarwin Design: Co-locationSpeaker RecognitionSpeaker ModelingClassifier Evolution: Training StepClassifier Evolution: Evolution StepSlide 17Slide 18Slide 19Privacy and TrustExperimental ResultsExperiment 1 ParametersExperiment 1 Results: Without EvolutionExperiment 2 ParametersExperiment 2 ResultsSlide 26Experiment 3 ParametersExperiment 3 ResultsSlide 29Slide 30Slide 31Experiment 4 ParametersExperiment 4 ResultsSlide 34Time and Energy MeasurementsSlide 36Possible ApplicationsFuture WorkImprovements On The PaperConclusionReferencesDARWIN PHONES: THE EVOLUTION OF SENSING AND INFERENCE ON MOBILE PHONESPRESENTED BY: BRANDON OCHSEmiliano Miluzzo, Cory T. Cornelius, Ashwin Ramaswamy, Tanzeem Choudhury, Zhigang Liu, Andrew T. Campbell, "Darwin phones: the evolution of sensing and inference on mobile phones," In Proc. of 8th ACM Conference on Mobile Systems, Applications, and Services (MobiSys), 2010, pp. 5-20.What does Darwin do?A Smartphone platform for urban sensingProof of concept model uses microphoneCommunicates with other local devices to improve inference accuracy (collaborative inference)Framework can be expanded to gatherinformation using a range of sensor dataWhat about battery life?Communicates with backend server to do the CPU-intensive machine learning algorithmsLocal devices share models rather than re-computing themSensing is enabled/disabled as the system sees fitCommon Urban Sensing ChallengesHuman burden of training classifiersAbility to perform reliably in different environments (indoor vs outdoor)The ability to scale to a large number of phones without hurting usability and battery life.Darwin overcomes all of these through classifier/model evolution, model pooling, and collaborative inferenceTypes of LearningSupervised: Given a fully-labeled training setSemi-Supervised: Given a small training set that is evolvedUnsupervised: No training set is givenDarwin StepsEvolution, Pooling, and Collaborative InferenceThese represent Darwin’s novel evolve-pool-collaborate model implemented on mobile phonesClassifier EvolutionAutomated approach to updating models over timeNeeds to account for variability in sensing conditions and settingsVariability in background noise and phone location require separate modelsModel PoolingReuses models that have already been built and evolved on other phonesExchange classification models whenever the model is available from another phoneClassifiers do not need to be retrained, which increases scalabilityCan pool models from backend serversCollaborative InferenceCombines results from multiple phonesRun inference algorithms in parallel on the same classifiersSystem is more robust to degradation in sensing qualityIncreases accuracyDarwin Design: ComputationReduces the on-the-phone computation by offloading some of the work to backend serversBackend server uses a machine learning algorithm to compute a Gaussian Mixture Model (2 hours for 15 seconds of audio)Feature vectors are computedlocallyDarwin Design: ContextContext (in/out of pocket, in/out of bag) will impact the sensing and inference capabilityClassifier evolution makes sure the classifier of an event is robust across different environmentsDarwin Design: Co-locationAccounts for a group of co-located phones running the same classification algorithm and sensing the same event but computing different inference resultsPhones pool classification models when collocated or from backend serversCompares against its own model and the co-located modelDrastically reduces classification latencyExploits diversity of different phone sensing context viewpointsSpeaker RecognitionAttempts to identify a speaker by analyzing the microphone’s audio streamSuppresses silence, low amplitude audio, and chunks that do not contain human voiceReduce false positives by pre-processing in 32ms blocksSpeaker ModelingFeature vector consisting ofMel Frequency Cepstral CoefficientsEach speaker is modeled with 20 GaussiansAn initial speaker model is built by collecting a short training sampleClassifier Evolution: Training StepShort training phase (30 seconds) used to build a model which is later evolvedFirst 15 seconds used as the training setLast 15 seconds used as baseline for evolutionClassifier Evolution: Evolution StepSemi-supervised learning strategyIf the likelihood of the incoming audio stream is much lower than any of the baselines then a new model is evolvedCollaborative InferenceLocal inference phase can be broken into three steps:Local inference operated by each individual phonePropagation of the result of the local inference to the neighboring phonesFinal inference based on the neighboring mobile phones local inference resultsEach node individually operates inference on the sensed eventResults and confidence broadcastedPrivacy and TrustRaw sensor data is not stored on or leaves the mobile phoneThe content of a conversation or raw audio data is never disclosedUsers can choose to opt out of DarwinExperimental ResultsTested using a mixture of five N97 and iPhones used by eight people over a period of two weeksAudio recorded in different locationsClassifier trained indoorsExperiment 1 ParametersThree people walk along a sidewalk of a busy road and engage in conversationThe speaker recognition application without the Darwin components runs on each of the phones carried by the peopleExperiment 1 Results: Without EvolutionExperiment 2 ParametersMeeting setting in an office environment where 8 people are involved in conversationThe phones are located at different distances from people in the meeting, some on the table and some in people’s pocketsExperiment 2 ResultsExperiment 2 ResultsExperiment 3 ParametersFive phones in a noisy restaurantThree of the five people are engaged in conversationTwo of the five phones are placed on the tablePhone 4 Is the closest phone to speaker 4 and also the closest phone to another group of people having a loud conversationExperiment 3
View Full Document