Controlling free flight of a robotic fly using an onboard vision sensor inspired by insect ocelli

Journal of the Royal Society Interface

Volumn 11, Issue 97, 2014, Pages

  Fuller, Sawyer B ^a   Karpelson, Michael ^a   Censi, Andrea ^b   Ma, Kevin Y ^a   Wood, Robert J ^a  

^a HARVARD UNIVERSITY   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Biology inspired robotics; Phasic response; Stability; Velocity feedback

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; ROBOTICS; ROBOTS;

BIOLOGY-INSPIRED; FEEDBACK CONTROLLER; ON-BOARD SENSORS; PHASIC RESPONSE; SOURCE OF LIGHT; STABILIZATION MECHANISMS; UPRIGHT ORIENTATIONS; VELOCITY FEEDBACK;

SENSORS;

ARTICLE; DIPTERA; FEEDBACK SYSTEM; FLIGHT; LIGHT; ROBOTIC FLY; ROBOTICS; SENSOR; TORQUE; VELOCITY; AIRCRAFT; ANIMAL; BIOMIMETICS; COMPOUND EYE; DEVICE FAILURE ANALYSIS; DEVICES; ELECTRONICS; EQUIPMENT DESIGN; FLYING; FORELIMB; ORIENTATION; PHYSIOLOGY; SENSORY FEEDBACK; TRANSDUCER;

AIRCRAFT; ANIMALS; BIOMIMETICS; COMPOUND EYE, ARTHROPOD; DIPTERA; EQUIPMENT DESIGN; EQUIPMENT FAILURE ANALYSIS; FEEDBACK; FEEDBACK, SENSORY; FLIGHT, ANIMAL; MINIATURIZATION; ORIENTATION; ROBOTICS; TORQUE; TRANSDUCERS; WING;

EID: 84903649934     PISSN: 17425689     EISSN: 17425662     Source Type: Journal    
DOI: 10.1098/rsif.2014.0281     Document Type: Article

References (72)

1. Land MF, Collett TS. 1974 Chasing behaviour of houseflies. J. Comp. Physiol. A 89, 331-357. (doi:10.1007/BF00695351)
2. Dalton S. 1975 Borne on the wind. London, UK: Chatto & Windus.
3. Graham KT, Holger GK. 2007 Sensory systems and flight stability: what do insects measure and why? In Advances in insect physiology. Vol. 34. Insect mechanics and control (eds J Casas, SJ Simpson), pp. 231-316. New York, NY: Academic Press.
4. Taylor GK. 2001 Mechanics and aerodynamics of insect flight control. Biol. Rev. 76, 449-471. (doi:10.1017/S1464793101005759)
5. Trimmer WSN. 1989 Microbots and micromechanical systems. Sensors Actuators 19, 267-287. (doi:10. 1016/0250-6874(89)87079-9)
6. Wood R, Finio B, Karpelson M, Ma K, Perez-Arancibia N, Sreetharan P, Tanaka H, Whitney J. 2012 Progress on 'pico' air vehicles. Int. J. Robot. Res. 31, 1292-1302. (doi:10.1177/02783649 12455073)
7. Pesavento U, Wang ZJ. 2009 Flapping wing flight can save aerodynamic power compared to steady flight. Phys. Rev. Lett. 103, 118102. (doi:10.1103/ PhysRevLett.103.118102)
8. Kumar V, Michael N. 2012 Opportunities and challenges with autonomous micro aerial vehicles. Int. J. Robot. Res. 31, 1279-1291. (doi:10.1177/ 0278364912455954)
9. Astrom KJ, Murray RM. 2008 Feedback systems: an introduction for scientists and engineers. Princeton, NJ: Princeton University Press.
10. Keennon M, Klingebiel K, Won H, Andriukov A. 2012 Development of the nano hummingbird: a tailless flapping wing micro air vehicle. In Proc. 50th AIAA Aerospace Sciences Meeting, Nashville, TN, 9-12 January 2012, pp. 1-24. Reston, VA: American Institute of Aeronautics and Astronautics.
11. Beyeler A, Zufferey J-C, Floreano D. 2009 Vision-based control of near-obstacle flight. Auton. Robot. 27, 201-219. (doi:10.1007/s10514-009-9139-6)
12. Duhamel P-EJ, Perez-Arancibia NO, Barrows GL, Wood RJ. 2013 Biologically inspired optical-flow sensing for altitude control of flapping-wing microrobots. IEEE/ASME Trans. Mechatron. 18, 556-568. (doi:10.1109/TMECH.2012.2225635)
13. Serres J, Dray D, Ruffier F, Franceschini N. 2008 A vision-based autopilot for a miniature air vehicle: joint speed control and lateral obstacle avoidance. Auton. Robot. 25, 103-122. (doi:10.1007/s10514-007-9069-0)
14. Conroy J, Gremillion G, Ranganathan B, Humbert J. 2009 Implementation of wide-field integration of optic flow for autonomous quadrotor navigation. Auton. Robot. 27, 189-198. (doi:10.1007/s10514-009-9140-0)
15. Fuller SB, Murray RM. 2011 A hovercraft robot that uses insect-inspired visual autocorrelation for motion control in a corridor. In Proc. IEEE Int. Conf. On Robotics and Biomimetics (ROBIO), Phuket Island, Thailand, 7-11 December 2011, pp. 1474-1481. Piscataway, NJ: IEEE.
16. Floreano D et al. 2013 Miniature curved artificial compound eyes. Proc. Natl Acad. Sci. USA 110, 9332-9337. (doi:10.1073/pnas.1219068110)
17. Zufferey J-C, Floreano D. 2006 Fly-inspired visual steering of an ultralight indoor aircraft. IEEE Trans. Robot. 22, 137-146. (doi:10.1109/ TRO.2005.858857)
18. De Wagter C, Tijmons S, Remes BDW, de Croon GCHE. 2014 Autonomous flight of a 20-gram flapping wing mav with a 4-gram onboard stereo vision system. In Proc. 2014 IEEE/RSJ Int. Conf. On Robotics and Autonomous Systems (ICRA), Hong Kong, China, 2-5 June 2014. Piscataway, NJ: IEEE.
19. Franceschini N, Ruffier F, Serres J. 2007 A bioinspired flying robot sheds light on insect piloting abilities. Curr. Biol. 17, 1-7. (doi:10.1016/j.cub. 2006.12.032)
20. Srinivasan MV, Moore RJD, Thurrowgood S, Soccol D, Bland D. 2012 From biology to engineering: insect vision and applications to robotics. In Frontiers in sensing (eds FG Barth, JAC Humphrey, MV Srinivasan), pp. 19-39. Vienna, Austria: Springer.
21. Han S, Censi A, Straw AD, Murray RM. 2010 A bioplausible design for visual pose stabilization. In Proc. 2010 IEEE/RSJ Int. Conf. On Intelligent Robots and Systems (IROS), Taipei, Taiwan, 18-22 October 2010, pp. 5679-5686. Piscataway, NJ: IEEE.
22. Wood RJ. 2008 The first takeoff of a biologically inspired at-scale robotic insect. IEEE Trans. Robot. 24, 341-347. (doi:10.1109/TRO.2008.916997)
23. Wood RJ, Fearing RS, Steltz E. 2005 Optimal energy density piezoelectric bending actuators. Sensors Actuators A, Phys. 119, 476-488. (doi:10.1016/j. sna.2004.10.024)
24. Whitney JP, Wood RJ. 2010 Aeromechanics of passive rotation in flapping flight. J. Fluid Mech. 660, 197-220. (doi:10.1017/S002211201000265X)
25. Desbiens AL, Chen Y, Wood RJ. 2013 A wing characterization method for flapping-wing robotic insects. In Proc. 2013 IEEE/RSJ Int. Conf. On Intelligent Robots and Systems (IROS), Tokyo, Japan, 3-8 November 2013, pp. 1367-1373. Piscataway, NJ: IEEE.
26. Pérez-Arancibia NO, Ma KY, Galloway KC, Greenberg JD, Wood RJ. 2011 First controlled vertical flight of a biologically inspired microrobot. Bioinspiration Biomimetics 6, 036009. (doi:10.1088/1748-3182/6/ 3/036009)
27. Abzug MJ, Larrabee EE. 2002 Airplane stability and control: a history of technologies that made aviation possible, 2nd edn. Cambridge, MA: Cambridge University Press.
28. Taylor GK, Thomas ALR. 2003 Dynamic flight stability in the desert locust Schistocerca gregaria. J. Exp. Biol. 206, 2803-2829. (doi:10.1242/ jeb.00115)
29. Sun M, Xiong Y. 2005 Dynamic flight stability of a hovering bumblebee. J. Exp. Biol. 208, 447-459. (doi:10.1242/jeb.01407)
30. Faruque I, Humbert JS. 2010 Dipteran insect flight dynamics. Part 1: longitudinal motion about hover. J. Theor. Biol. 264, 538-552. (doi:10.1016/j.jtbi. 2010.02.018)
31. Faruque I, Humbert JS. 2010 Dipteran insect flight dynamics. Part 2: lateral-directional motion about hover. J. Theor. Biol. 265, 306-313. (doi:10.1016/j. jtbi.2010.05.003)
32. Ristroph L, Ristroph G, Morozova S, Bergou AJ, Chang S, Guckenheimer J, Wang ZJ, Cohen I. 2013 Active and passive stabilization of body pitch in insect flight. J. R. Soc. Interface 10, 20130237. (doi:10.1098/rsif.2013.0237)
33. Ma KY, Chirarattananon P, Fuller SB, Wood R. 2013 Controlled flight of a biologically inspired, insectscale robot. Science 340, 603-607. (doi:10.1126/ science.1231806)
34. Teoh ZE, Fuller SB, Chirarattananon PC, Perez-Arancibia NO, Greenberg JD, Wood RJ. 2012 A hovering flapping-wing microrobot with altitude control and passive upright stability. In Proc. 2012 IEEE/RSJ Int. Conf. On Intelligent Robots and Systems (IROS), Vilamoura, Portugal, 7-12 October 2012, pp. 3209-3216. Piscataway, NJ: IEEE.
35. Kalmus H. 1945 Correlations between flight and vision, and particularly between wings and ocelli, in insects. Proc. R. Entomol. Soc. Lond. A, Gen. Entomol. 20, 84-96. (doi:10.1111/j.1365-3032. 1945.tb01072.x)
36. Wilson M. 1978 The functional organisation of locust ocelli. J. Comp. Physiol. 124, 297-316. (doi:10.1007/BF00661380)
37. Hengstenerg R. 1993 Visual motion and its role in the stabilization of gaze. In Multisensory control in insect oculomotor systems (eds FA Miles, J Wallman), pp. 285-298. Amsterdam, The Netherlands: Elsevier Science Publishers.
38. Goodman LJ. 1965 The role of certain optomotor reactions in regulating stability in the rolling plane during flight in the desert locust, Schistocerca gregaria. J. Exp. Biol. 42, 385-407.
39. Parsons MM, Krapp HG, Laughlin SB. 2010 Sensor fusion in identified visual interneurons. Curr. Biol. 20, 624-628. (doi:10.1016/j.cub.2010. 01.064)
40. Pringle JWS. 1948 The gyroscopic mechanism of the halteres of diptera. Phil. Trans. R. Soc. Lond. B 233, 347-384. (doi:10.1098/rstb.1948.0007)
41. Dickinson MH. 1999 Haltere-mediated equilibrium reflexes of the fruit fly, Drosophila melanogaster. Phil. Trans. R. Soc. Lond. B 354, 903-916. (doi:10. 1098/rstb.1999.0442)
42. Sane SP, Dieudonné A, Willis MA, Daniel TL. 2007 Antennal mechanosensors mediate flight control in moths. Science 315, 863-866. (doi:10.1126/ science.1133598)
43. Hinterwirth AJ, Daniel TL. 2010 Antennae in the hawkmoth Manduca sexta (Lepidoptera, Sphingidae) mediate abdominal flexion in response to mechanical stimuli. J. Comp. Physiol. A 196, 947-956. (doi:10.1007/s00359-010-0578-5)
44. Wu W-C, Schenato L, Wood RJ, Fearing RS. 2003 Biomimetic sensor suite for flight control of a micromechanical flying insect: design and experimental results. In Proc. IEEE Int. Conf. Robotics and Automation ICRA '03, Taipei, Taiwan, 14-19 September 2003, vol. 1, pp. 1146-1151. Piscataway, NJ: IEEE.
45. Schenato L, Wu WC, Sastry S. 2004 Attitude control for a micromechanical flying insect via sensor output feedback. IEEE Trans. Robot. Autom. 20, 93-106. (doi:10.1109/TRA.2003.820863)
46. Deng X, Schenato L, Sastry SS. 2006 Flapping flight for biomimetic robotic insects. Part II. Flight control design. IEEE Trans. Robot. 22, 789-803. (doi:10. 1109/TRO.2006.875483)
47. Fuller SB, Sands A, Haggerty A, Karpelson M, Wood RJ. 2013 Estimating attitude and wind velocity using biomimetic sensors on a microrobotic bee. In Proc. IEEE Int. Conf. On Robotics and Automation (ICRA), Karlsruhe, Germany, 6-10 May 2013, pp. 1374-1380. Piscataway, NJ: IEEE.
48. Chahl J, Thakoor S, Bouffant NL, Stange G, Srinivasan MV, Hine B, Zornetzer S. 2003 Bioinspired engineering of exploration systems: a horizon sensor/attitude reference system based on the dragonfly ocelli for Mars exploration applications. J. Robot. Syst. 20, 35-42. (doi:10.1002/rob.10068)
49. Gremillion G, Galfond M, Krapp HG, Humbert JS. 2012 Biomimetic sensing and modeling of the ocelli visual system of flying insects. In Proc. 2012 IEEE/ RSJ Int. Conf. On Intelligent Robots and Systems (IROS), Vilamoura, Portugal, 7-12 October 2012, pp. 1454-1459. Piscataway, NJ: IEEE.
50. Moore R, Thurrowgood S, Bland D, Soccol D, Srinivasan MV. 2011 A fast and adaptive method for estimating UAV attitude from the visual horizon. In Proc. 2011 IEEE/RSJ Int. Conf. On Intelligent Robots and Systems (IROS), San Francisco, CA, 25-30 September 2011, pp. 4935-4940. Piscataway, NJ: IEEE.
51. Ellington CP. 1984 The aerodynamics of hovering insect flight. II. Morphological parameters. Phil. Trans. R. Soc. Lond. B 305, 17-40. (doi:10.1098/ rstb.1984.0050)
52. Taylor CP. 1981 Contribution of compound eyes and ocelli to steering of locusts in flight. I. Behavioural analysis. J. Exp. Biol. 93, 1-18.
53. Wu WC, Wood RJ, Fearing RS. 2002 Halteres for the micromechanical flying insect. In Proc. the IEEE Int. Conf. On Robotics and Automation, Washington, DC, 11-15 May 2002, pp. 60-65. Piscataway, NJ: IEEE.
54. Xinyan Deng LS, Wu WC, Sastry SS. 2006 Flapping flight for biomimetic robotic insects: part I. System modeling. IEEE Trans. Robot. 22, 776-788. (doi:10. 1109/TRO.2006.875480)
55. Schuppe H, Hengstenberg R. 1993 Optical properties of the ocelli of Calliphora erythrocephala ANMD their role in the dorsal light response. J. Comp. Physiol. A 173, 143-149. (doi:10.1007/ BF00192973)
56. Ma KY, Felton SM, Wood RJ. 2012 Design, fabrication, and modeling of the split actuator microrobotic bee. In Proc. IEEE Int. Robotics and Automation Conference, St. Paul, MN, 14-18 May 2012, pp. 1133-1140. Piscataway, NJ: IEEE.
57. Dickson WB, Polidoro P, Tanner MM, Dickinson MH. 2010 A linear systems analysis of the yaw dynamics of a dynamically scaled insect model. J. Exp. Biol. 213, 3047-3061. (doi:10.1242/jeb.042978)
58. Murray RM, Li Z, Sastry SS. 1994 A mathematical introduction to robotic manipulation. Boca Raton, FL: CRC Press.
59. Karpelson M, Whitney JP, Wei G-Y, Wood RJ. 2010 Energetics of flapping-wing robotic insects: towards autonomous hovering flight. In 2010 IEEE/RSJ Int. Conf. On Intelligent Robots and Systems (IROS), Taipei, Taiwan, 18-22 October 2010, pp. 1630-1637. Piscataway, NJ: IEEE.
60. Karpelson M, Wei G-Y, Wood RJ. 2012 Driving high voltage piezoelectric actuators in microrobotic applications. Sensors Actuators A, Phys. 176, 78-89. (doi:10.1016/j.sna.2011.11.035)
61. Censi A, Han S, Fuller SB, Murray RM. 2009 A bioplausible design for visual attitude stabilization. In Proc. IEEE Conf. On Decision and Control (CDC), Shanghai, China, 16-19 December 2009, pp. 3513-3520. Piscataway, NJ: IEEE.
62. Borst A, Egelhaaf M. 1989 Principles of visual motion detection. Trends Neurosci. 12, 297-306. (doi:10.1016/0166-2236(89)90010-6)
63. Wehrhahn C. 1984 Ocellar vision and orientation in flies. Proc. R. Soc. Lond. B 222, 409-411. (doi:10. 1098/rspb.1984.0073)
64. Cheng B, Deng X. 2011 Translational and rotational damping of flapping flight and its dynamics and stability at hovering. IEEE Trans. Robot. 27, 849-864. (doi:10.1109/TRO.2011.2156170)
65. Nabawy MRA, Crowther WJ. 2014 On the quasi-steady aerodynamics of normal hovering flight part ii: model implementation and evaluation. J. R. Soc. Interface 11, 20131197. (doi:10.1098/rsif. 2013.1197)
66. Ellington CP, van den Berg C, Willmott AP, Thomas ALR. 1996 Leading-edge vortices in insect flight. Nature 384, 626-630. (doi:10.1038/384626a0)
67. Sane SP, Dickinson MH. 2002 The aerodynamic effects of wing rotation and a revised quasi-steady model of flapping flight. J. Exp. Biol. 205, 1087-1096.
68. Wang ZJ, Birch JM, Dickinson MH. 2004 Unsteady forces and flows in low Reynolds number hovering flight: two-dimensional computations vs robotic wing experiments. J. Exp. Biol. 207, 449-460. (doi:10.1242/jeb.00739)
69. Tu J, Yeoh GH, Liu C. 2012 Computational fluid dynamics: a practical approach, 2nd edn. Waltham, MA: Butterworth-Heinemann.
70. Straw AD, Lee S, Dickinson MH. 2010 Visual control of altitude in flying Drosophila. Curr. Biol. 20, 1550-1556. (doi:10.1016/j.cub.2010. 07.025)
71. Borst A, Haag J, Reiff DF. 2010 Fly motion vision. Annu. Rev. Neurosci. 33, 49-70. (doi:10.1146/ annurev-neuro-060909-153155)
72. Kastberger G. 1990 The ocelli control the flight course in honeybee. Physiol. Entomol. 15, 337-346. (doi:10.1111/j.1365-3032.1990.tb00521.x)