Sensor and data fusion: Taxonomy, challenges, and applications

Handbook on Soft Computing for Video Surveillance

Volumn , Issue , 2012, Pages 139-183

  Klein, Lawrence A ^a   Mihaylova, Lyudmila ^b   El Faouzi, Nour Eddin ^c  

^a Consultant ^*  (United States)

^b LANCASTER UNIVERSITY   (United Kingdom)

^c UNIVERSITÉ DE LYON   (France)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85124057986     PISSN: None     EISSN: None     Source Type: Book    
DOI: None     Document Type: Chapter

References (88)

1. A. Doucet and N. Freitas and N. Gordon, Eds. Sequential Monte Carlo Meth ods in Practice. New York: Springer-Verlag, 2001.
2. H. Aghajan and A. Cavallaro. Multi-Camera Networks: Principles and Ap plications. New York: Academic Press, 2009.
3. M. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. on Signal Processing, 50(2):174-188, 2002.
4. Y. Bar-Shalom and W. Blair. Multitarget-Multisensor Tracking: Applications and Advances, Ch. 6 by S.S. Blackman, Association and fusion of multiple sensor data, volume III. Boston: Artech House, 2000.
5. J.A. Barnett. Computational methods for a mathematical theory of evidence. In Seventh International Joint Conference on Artificial Intelligence II, IJCAI'81, pages 868-875, 1981.
6. A. Bhattacharayya. On a measure of divergence between two statistical popula tions defined by their probability distributions. Bulletin Calcutta Math. Society, 35:99-110, 1943.
7. S. S. Blackman. Multiple Target Tracking with Radar Applications. Boston: Artech House, 1986.
8. E.P. Blasch and S. Plano. JDL level 5 fusion model 'user refinement' issues and application in group tracking. In Proceedings of SPIE 4729, Aerosense, pages 270-279, 2002.
9. H.A.P. Bloom and Y. Bar-Shalom. The interacting multiple-model algorithm for systems with switching coefficients. IEEE Transactions on Automatic Control, 33:780-783, 1988.
10. P.L. Bogler. Shafer-Dempster reasoning with applications to multisensor tar get identification systems. IEEE Transactions on Systems, Man, and Cybernetics, 17(6):968-977, 1987.
11. P. Brasnett, L. Mihaylova, N. Canagarajah, and D. Bull. Improved proposal distribution with gradient measures for tracking. In IEEE International Workshop on Machine Learning for Signal Processing, pages 105-110. Mystic, CT, USA, Sept. 28-30 2005.
12. P. Brasnett, L. Mihaylova, N. Canagarajah, and D. Bull. Particle filtering with multiple cues for object tracking in video sequences. In Proceedings of SPIE's 17th Annual Symposium on Electronic Imaging, Science and Technology, 5685:430-441, 2005.
13. P. Brasnett, L. Mihaylova, N. Canagarajah, and D. Bull. Sequential Monte Carlo tracking by fusing multiple cues in video sequences. Image and Vision Computing, 25(8):1217-1227, Aug. 2007.
14. B.R. Cobb and P.P. Shenoy. A comparison of methods for transforming belief function models to probability models. In LNCS in Artificial Intelligence. Symbolic and Quantitative Approaches to Reasoning with Uncertainty, Eds. T.D. Nielsen and N.L. Zhang, pages 255-266. Berlin: Springer-Verlag, 2003.
15. D. Comaniciu, V. Ramesh, and P. Meer. Kernel-based object tracking. IEEE Trans. Pattern Analysis Machine Intelligence, 25(5):564-575, 2003.
16. V.G. Comparato. Fusion - The key to tactical mission success. Sensor fusion. In Proc. SPIE 931, pages 2-7, 1988.
17. Data Fusion Development Strategy Panel. Functional description of the data fusion process. Technical report, Office of Naval Technology, 1991.
18. Data Fusion Subpanel. Data fusion lexicon. Technical report, Joint Directors of Laboratories Technical Panel for C3, 1991.
19. J. Dezert. Foundations for a new theory of plausible and paradoxical reasoning. Information and Security, 9:1-45, 2002.
20. R.A. Dillard. Computing confidences in tactical rule-based systems by using Dempster-Shafer theory. Technical Report NOSC TD 649 (AD A 137274), San Diego: Naval Ocean Systems Center, 1983.
21. T.D. Dixon, J. Li, J.M. Noyes, T. Troscianko, S.G. Nikolov, J. Lewis, E.-F. Canga, D.R. Bull, and C.N. Canagarajah. Scanpath analysis of fused multi-sensor images with luminance change: A pilot study. In Proceedings ofthe 9th International Conference on Information Fusion, Italy, July 2006.
22. O.E. Drummond and S.S. Blackman. Challenges of developing algorithms for multiple sensor, multiple target tracking. In Proc. SPIE 1096, pages 244-255, 1989.
23. D. Dubois, H. Prade, and H. Hguyen. Possibility theory, probability and fuzzy sets: Misunderstandings, bridges and gaps. In D. Dubois and H. Prade, Editors, Fundamentals of Fuzzy Sets. The Handbook of Fuzzy Sets. Kluwer Academic Pulications, Dordrecht, 1999.
24. N.-E. El Faouzi. Data fusion: Concepts and methods. Technical Report No. 01.01, Licit, INRETS-ENTPE, 2001.
25. N.-E. El Faouzi, H. Leung, and A. Kurian. Data fusion in intelligent transporta tion systems: Progress and challenges information fusion. Information Fusion, 12(1):4-10, 2011.
26. M. Fauvel, J. Chanussot, and J.A. Benediktsson. Decision fusion for the clas sification of urban remote sensing images. IEEE Transactions on Geo-science and Remote Sensing, 44(10):2828-2838, 2006.
27. F.E. White, Jr. Joint directors of laboratories data fusion subpanel report: Sigint session. In Tech. Proc. Joint Service Data Fusion Symposium I, DFS'90, pages 469-484, 1990.
28. D. Fixsen and R.P.S. Mahler. The modified Dempster-Shafer approach to classification. IEEE Transactions on Systems, Man, and Cybernetics. Part A: Systems and Humans, 27(1):96-104, 1997.
29. W. Freeman and E.H. Adelson. The design and use of steerable filters. IEEE Trans. Pattern Analysis & Machine Intelligence, 13:891-906, 1991.
30. T. Gandhi and M. Trivedi. Pedestrian protection systems: Issues, survey and chllenges. IEEE Transactions on Intelligent Transportation Systems, 8(3):413-430, 2007.
31. T.D. Garvey, J.D. Lowrance, and M.A. Fischler. An inference technique for integrating knowledge from disparate sources. In Proc. Seventh International Joint Conference on Artificial Intelligence I, IJCAI'81, pages 319-325, 1981.
32. D. Geronimo, A. M. Lopez, Angel D. Sappa, and T. Graf. Survey of pedestrian detection for advanced driver assistance systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32:1239-1258, 2010.
33. M. Goldstein and W.R. Dillon. Discrete Discriminant Analysis. Wiley, New York, 1978.
34. N. Gordon, D. Salmond, and A. Smith. A novel approach to nonlinear / non-Gaussian Bayesian state estimation. In IEE Proceedings-F, 140:107-113, April 1993.
35. D. L. Hall and J. Llinas, Editors. Handbook of Multisensor Data Fusion. Second Edition. Boca Raton, FL, USA, CRC Press, 2008.
36. D.L. Hall. Mathematical Techniques in Multisensor Data Fusion. Norwood: Artech House, 1992.
37. D.L. Hall and R.J. Linn. Algorithm selection for data fusion systems. In Tri-Service Data Fusion Symposium Technical Proceedings I, DFS' 87, Warminster: Naval Air Development Center, 1987.
38. D.L. Hall and R.J. Linn. A taxonomy of algorithms for multi-sensor data fusion. In Tech. Proc. Joint Service Data Fusion Symposium I, DFS'90, pages 594-610, Warminster: Naval Air Development Center, 1990.
39. J. Heather and M. Smith. Multimodel image registration with applications to image fusion. In Proceedings of the 8th International Conference on Information Fusion. USA, July 2005.
40. S.A. Hovanessian. Introduction to Sensor Systems. Chapter 7 in S.S. Black man, Multiple sensor tracking and data fusion. Boston: Artech House, 1988.
41. M. Isard and A. Blake. Condensation - Conditional density propagation for visual tracking. International Journal of Computer Vision, 28(1):5-28, 1998.
42. J. Jazwinski. Stochastic Processes and Filtering Theory. New York: Academic Press, 1970.
43. A. Josang. The consensus operator for combining beliefs. Artificial Intelligence Journal, 14(1-2):157-170, 2002.
44. R. Kalman. A new approach to linear filtering and prediction problems. Trans ASME, Serie D: J. Basic Eng., 82:35-45, 1960.
45. R. Kalman and R. Bucy. New results in linear filtering and prediction theory. Trans ASME, Serie D: J. Basic Eng., 83:95-108, 1961.
46. L.A. Klein. Sensor and Data Fusion: A Tool for Information Assessment and Decision Making. Monograph 138 (3rd printing). Bellingham: SPIE Press, 2010.
47. J. Lewis, S. Nikolov, N. Canagarajah, and D. Bull. Region-based image fusion. Technical Report UOB-DIF-DTC-PROJ201-TR02, Data and Information Fusion, Defence Technology Centre, University of Bristol, 2004.
48. J. Lewis, S. Nikolov, A. Loza, E.-F. Canga, N. Cvejic, J. Li, A. Cardinali, N. Canagarajah, and D. Bull. Multi-sensor data gathering. Technical Report UOB-DIF-DTC-PROJ201-TR11, Univ. of Bristol, Bristol, UK, Oct. 2005.
49. J. J. Lewis, R. J. O'Callaghan, S. G. Nikolov, D. R. Bull, and C. N. Cana-garajah. Pixel- and region-based image fusion using complex wavelets. Information Fusion, Special Issue on Image Fusion: Advances in the State of Art, 8(2):119-130, 2007.
50. J.J. Lewis, S.G. Nikolov, A. Loza, E. Fernandez-Canga, N. Cvejic, L. Li, A. Cardinali, C.N. Canagarajah, D.R. Bull, T. Riley, D. Hickman, and M.I. Smith. The Eden project multi-sensor data set. Technical report, University of Bristol, TR-UoB-WS-Eden-Project-Data-Set (ImageFusion.org), 2006.
51. J. Li, C. Benton, S. Nikolov, and N. Scott-Emanuel. Multi-sensor motion computation, analysis and fusion. Technical Report UOB-DIF-DTC-PROJ201-TR14, Department of Electrical and Electronic Engineering, University of Bristol, Bristol, UK, February 2006.
52. J. Liu. Monte Carlo Strategies in Scientific Computing. Springer Verlag, 2001.
53. A. Loza, L. Mihaylova, D. Bull, and N. Canagarajah. Structural similarity-based object tracking in multimodality surveillance videos. Machine Vision and Applications, 20(2):71-83, February 2009.
54. R.C. Luo and M.G. Kay. Multisensor integration and fusion: Issues and ap proaches. sensor fusion. In Proc. SPIE 931, pages 42-49, 1988.
55. R.P.S. Mahler. Combining ambiguous evidence with respect to ambiguous a priori knowledge, I: Boolean logic. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 26(1):27-41, 1996.
56. R. Manduchi. Bayesian fusion of colour and texture segmentations. In IEEE International Conference on Computer Vision, ICCV'99, September 1999.
57. M.E. Liggins II, C.-Y. Chong, I. Kadar, M.G. Alford, V. Vannicola, and S. Tho-mopoulos. Distributed fusion architectures and algorithms for target tracking. Proceedings of IEEE, 85(1):95-107, 1997.
58. L. Mihaylova, A. Loza, S. G. Nikolov, J. J. Lewis, E.-F. Canga, J. Li, C. N. Canagarajah, and D. R. Bull. Object tracking in multi-sensor video. Technical Report UOB-DIF-DTC-PROJ202-TR10, University of Bristol, UK, 2006.
59. P. Del Moral, A. Doucet, and A. Jasra. Sequential Monte Carlo for Bayesian computation. In J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Hecherman, A. F. M. Smith, and M. West, Editors, Bayesian Statistics 8, pages 115-148. Oxford University Press, 2007.
60. C.K. Murphy. Combining belief functions when evidence conflicts. Decision Support Systems, pages 1-9, 2000.
61. S. G. Nikolov, P. Hill, D. Bull, and N. Canagarajah. Wavelets for image fusion. In A. Petrosian and F. Meyer, Editors, Wavelets in Signal and Image Analysis, pages 213-244. Kluwer Academic Publishers, 2001.
62. S. G. Nikolov, J. J. Lewis, E. F. Canga, A. Loza, C. N. Canagarajah, and D. R. Bull. Fusion of Visible and Infrared Image Sequences Using Wavelets. Progress report UOB-DIF-DTC-PROJ201-TR06, DTC-UoB, 2004.
63. K. Nummiaro, E. B. Koller-Meier, and L. Van Gool. An adaptive color-based particle filter. Image and Vision Computing, 21(1):99-110, 2003.
64. E. Ozyildiz, N. Krahnstöver, and R. Sharma. Adaptive texture and color segmentation for tracking moving objects. Pattern Recognition, 35:20132029, 2002.
65. P. Smets (Ed.). Quantified Representation of Uncertainty and Imprecisions. Handbook of Defeasible Reasoning of Uncertainty Management Systems, volume I. Kluwer Academic Publ., Dordrecht, The Netherlands, 1998.
66. C.R. Parikh, M.J. Pont, N.B. Jones, and F.S. Schlindwein. Improving the performance of CMFD applications using multiple classifiers and a fusion framework. Transactions of Institute of Measurement and Control, 25(2), 2003.
67. J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA. Morgan Kaufmann, 1988.
68. P. Perez, J. Vermaak, and A. Blake. Data fusion for tracking with particles. Proceedings of the IEEE, 92(3):495-513, 2004.
69. Proceedings of the IEEE. Special issue. sequential state estimation: From Kalman filters to particle filters. 92(4), 2004.
70. G.S. Robinson and A.O. Aboutalib. Trade-off analysis of multisensor fusion levels. In Proc. of the 2nd National Symp. on Sensors and Sensor Fusion II, GACIAC PR 89-01, pages 21-34. Chicago: IIT Research Institute, 1990.
71. E. Saber and A. M. Tekalp. Integration of color, edge, shape and texture fea tures for automatic region-based image annotation and retrieval. Journal of Electronic Imaging (Special Issue), 7(3):684-700, 1998.
72. D. W. Scott. Multivariate Density Estimation: Theory, Practice and Visu alization. Wiley Series in Probability and Mathematical Statistics. New York: John Wiley & Sons, 1992.
73. G. Shafer. A Mathematical Theory of Evidence. Princeton University Press, 1976.
74. X. Shi and R. Manduchi. A study on Bayes feature fusion for image classifi cation. IEEE Workshop on Statistical Analysis in Computer Vision, 8:95, 2003.
75. P. Smets. Constructing the pignistic probability function in a context of un certainty. Uncertainty Artificial Inteligence, 5:29-39, 1990.
76. P. Smets. The transferable belief model and random sets. International Jour nal of Intelligent Systems, 7:37-46, 1992.
77. P. Smets. Imperfect information: Imprecision - uncertainty. In A. Motro and P. Smets, Editors, Uncertainty Management in Information Systems: From Needs to Solutions, pages 225-254. Kluwer Academic Publishers, 1997.
78. P. Smets and R. Kennes. The transferable belief model. Artificial Intelligence, 66:191-234, 1994.
79. M. Smithson. Ignorance and Uncertainty: Emerging Pradigms. Spinger Verlag, New York, 1989.
80. A.N. Steinberg, C.L. Bowman, and F.E. White. Revisions to the JDL data fusion model. In Procceedings of SPIE 3719, pages 430-441, 1999.
81. J. Triesch and C. von der Malsburg. Democratic integration: Self-organized in tegration of adaptive cues. Neural Computation, 13(9):2049-2074, 2001.
82. L. Valet, G. Mauris, and Ph. Bolon. A statistical overview of recent literature in information fusion. IEEE Aerospace and Electronic Systems Magazine, pages 7-14, 2001.
83. R. Viswanathan and P.K. Varshney. Distributed detection with multiple sen sors: Part I - fundamentals. Proc. IEEE, 85(1):54-63, 1997.
84. E. Waltz and J. Llinas. Multisensor Data Fusion. Boston, MA: Artech House, 1990.
85. Z. Wang, A. C. Bovik, and E. P. Simoncelli. Structural approaches to image quality assessment. In A. Bovik, Editor, Handbook of Image and Video Processing, 2nd Edition, chapter 8.3. Academic Press, 2005.
86. Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assess ment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600-612, 2004.
87. L.A. Zadeh. A theory of approximate reasoning. In J. Hayes, D. Michie, and L. Mikulich, Editors, Machine Intelligence 9, pages 149-194. 1979.
88. L.A. Zadeh. Review of Shafer's mathematical theory of evidence. Artificial Intelligence Magazine, 5(3):81-83, 1984.