DSP.Ear: Leveraging co-processor support for continuous audio sensing on smartphones

SenSys 2014 - Proceedings of the 12th ACM Conference on Embedded Networked Sensor Systems

Volumn , Issue , 2014, Pages 295-309

  Georgie, Petko ^a   Lane, Nicholas D ^b   Rachuri, Kiran K ^a,c   Mascolo, Cecilia ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b MICROSOFT RESEARCH   (United States)

^c SAMSUNG RESEARCH AMERICA   (United States)

Author keywords

Audio; Co processor; DSP; Energy; Mobile sensing

Indexed keywords

AUDIO ACOUSTICS; BEHAVIORAL RESEARCH; COPROCESSOR; ELECTRIC BATTERIES; EMBEDDED SYSTEMS; LARGE DATASET; SMARTPHONES; SPEECH RECOGNITION;

AUDIO; ENERGY; LARGE-SCALE DATASET; MOBILE SENSING; PIPELINE OPTIMIZATION; PROCESSOR TECHNOLOGY; PROTOTYPE IMPLEMENTATIONS; SIMULTANEOUS OPERATION;

AUDIO EQUIPMENT;

EID: 84914146168     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2668332.2668349     Document Type: Conference Paper

References (47)

1. British Library of Sounds. http://sounds.bl.uk/.
2. Monsoon Power Monitor. http://www.msoon.com/LabEquipment/PowerMonitor/.
3. Free Sound Effects. http://www.freesfx.co.uk/.
4. Google Nexus 5. https://www.google.com/nexus/5/.
5. HTK Speech Recognition Toolkit. http://htk.eng.cam.ac.uk/.
6. iPhone 5s M7 Motion Coprocessor. https://www.apple.com/iphone-5s/specs/.
7. Motorola Moto X. http://www.motorola.com/us/FLEXR1-1/moto-x-specifications.html.
8. Qualcomm Hexagon SDK. https://developer.qualcomm.com/mobile-development/maximize-hardware/multimedia-optimization-hexagon-sdk.
9. Qualcomm Snapdragon 800 MDP. https://developer.qualcomm.com/mobile-development/development-devices/snapdragon-mobile-development-platform-mdp.
10. Qualcomm Snapdragon 800 Processors. http://www.qualcomm.com/snapdragon/processors/800.
11. Scikit-Learn Python Library. http://scikit-learn.org/stable/.
12. Trepn Profiler. https://developer.qualcomm.com/mobile-development/increase-app-performance/trepn-profiler.
13. Hexagon DSP processor. https://developer.qualcomm.com/mobile-development/maximize-hardware/multimedia-optimization-hexagon-sdk/hexagon-dsp-processor.
14. R. J. Baken. Clinical Measurement of Speech and Voice. Taylor & Francis Ltd, London, 1987.
15. N. Lane, et al. Piggyback CrowdSensing (PCS): Energy Efficient Crowdsourcing of Mobile Sensor Data by Exploiting Smartphone App Opportunities. In SenSys '13.
16. A. Carroll and G. Heiser. An Analysis of Power Consumption in a Smartphone. In USENIXATC '10.
17. G. Chechik, E. Ie, M. Rehn, S. Bengio, and D. Lyon. Large-scale content-based audio retrieval from text queries. In MIR '08.
18. B.-G. Chun, S. Ihm, P. Maniatis, M. Naik, and A. Patti. Clone cloud: Elastic execution between mobile device and cloud. In EuroSys '11.
19. A. de Cheveigné and H. Kawahara. YIN, a fundamental frequency estimator for speech and music. The Journal of the Acoustical Society of America, 111(4):1917-1930, 2002.
20. S. Dixon. Onset Detection Revisited. In Proc. of the Int. Conf. on Digital Audio Effects (DAFx-06), pages 133-137, Montreal, Quebec, Canada, Sept. 2006.
21. Z. Fang, et al. Comparison of different implementations of MFCC. J. Comput. Sci. Technol., 16(6):582-589, Nov. 2001.
22. H. Hermansky. Perceptual linear predictive (PLP) analysis of speech. J. Acoust. Soc. Am., 57(4):1738-52, Apr. 1990.
23. Y. Lee, et al. Sociophone: Everyday face-to-face interaction monitoring platform using multi-phone sensor fusion. In MobiSys '13.
24. D. Li, et al. Classification of general audio data for content-based retrieval. Pattern Recognition Letters, 22(5):533-544, 2001.
25. T. Li. Musical genre classification of audio signals. In IEEE Transactions on Speech and Audio Processing, pages 293-302, 2002.
26. M. Liberman, K. Davis, M. Grossman, N. Martey, and J. Bell. Emotional prosody speech and transcripts. 2002.
27. H. Lu, A. J. B. Brush, B. Priyantha, A. K. Karlson, and J. Liu. Speakersense: Energy efficient unobtrusive speaker identification on mobile phones. In Pervasive'11.
28. H. Lu, et al. Stresssense: Detecting stress in unconstrained acoustic environments using smartphones. In UbiComp '12.
29. H. Lu, et al. Sound sense: Scalable sound sensing for people-centric applications on mobile phones. In MobiSys '09.
30. H. Lu, et al. The JigSaw continuous sensing engine for mobile phone applications. In SenSys '10.
31. P. Mohan, V. N. Padmanabhan, and R. Ramjee. Nericell: Rich monitoring of road and traffic conditions using mobile smartphones. In SenSys '08.
32. S. Nath. ACE: Exploiting correlation for energy-efficient and continuous context sensing. In MobiSys '12.
33. S. Ntalampiras, I. Potamitis, and N. Fakotakis. Acoustic detection of human activities in natural environments. Journal of Audio Engineering Society, 2012 2012.
34. B. Priyantha, D. Lymberopoulos, and J. Liu. Enabling energy efficient continuous sensing on mobile phones with littlerock. In IPSN '10.
35. K. K. Rachuri, et al. Sociablesense: Exploring the trade-offs of adaptive sampling and computation offloading for social sensing. In Mobi- Com '11.
36. K. K. Rachuri, et al. Emotionsense: A mobile phones based adaptive platform for experimental social psychology research. Ubicomp '10.
37. J. Saunders. Real-time discrimination of broadcast speech/music. In ICASSP '96.
38. E. Scheirer and M. Slaney. Construction and evaluation of a robust multifeature speech/music discriminator. In ICASSP '97.
39. I. Constandache et al. EnLoc: Energy-Efficient Localization for Mobile Phones. In InfoCom '09.
40. C. Shen et al. Exploiting Processor Heterogeneity for Energy Efficient Context Inference on Mobile Phones. In HotPower '13.
41. D. Zhang et al. ACC: Generic On-demand Accelerations for Neighbor Discovery in Mobile Applications. In SenSys '12.
42. M. Ra et al. Improving Energy Efciency of Personal Sensing Applications with Heterogeneous Multi-Processors. In Ubicomp '12.
43. G. Schwarz. Estimating the Dimension of a Model. The Annals of Statistics, 6(2):461-464, 1978.
44. S. Verma, A. Robinson, and P. Dutta. Audiodaq: Turning the mobile phone's ubiquitous headset port into a universal data acquisition interface. In SenSys '12.
45. Y. Wang, et al. A framework of energy efficient mobile sensing for automatic user state recognition. In MobiSys '09.
46. C. Xu, et al. Crowd++: Unsupervised speaker count with smart phones. In UbiComp '13.
47. B. Yan, and G. Chen. App Joy: Personalized Mobile Application Discovery In MobiSys '11.