Optimizing warehouse forklift dispatching using a sensor network and stochastic learning

IEEE Transactions on Industrial Informatics

Volumn 7, Issue 3, 2011, Pages 476-485

  Estanjini, Reza Moazzez ^a   Lin, Yingwei ^a   Li, Keyong ^a   Guo, Dong ^a   Paschalidis, Ioannis Ch ^a  

^a Boston University   (United States)

Author keywords

Decision support systems; dispatching; event driven sensing; industrial mornitoring; intelligent systems; statistical learning; wireless sensor networks

Indexed keywords

DISPATCHING; EVENT DRIVEN SENSING; INDUSTRIAL MORNITORING; STATISTICAL LEARNING; WIRELESS SENSOR;

AUTOMOBILES; DECISION SUPPORT SYSTEMS; DECISION THEORY; INTELLIGENT SYSTEMS; MATERIALS HANDLING EQUIPMENT; SENSOR NODES; SENSORS; STOCHASTIC SYSTEMS; TELECOMMUNICATION EQUIPMENT; TESTING; WAREHOUSES;

WIRELESS NETWORKS;

EID: 80051780798     PISSN: 15513203     EISSN: None     Source Type: Journal    
DOI: 10.1109/TII.2011.2158834     Document Type: Article

References (48)

1. I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, "Wireless sensor networks: A survey," Comput. Networks, vol. 38, no. 4, pp. 393-422, 2002.
2. C. Chong and S. Kumar, "Sensor networks: Evolution, opportunities, and challenges," Proc. IEEE, vol. 91, no. 8, pp. 1247-1256, Aug. 2003.
3. J. Gehrke and L. Liu, "Sensor-network applications," IEEE Internet Comput., vol. 10, no. 2, pp. 16-17, 2006.
4. G. J. Pottie and W. J. Kaiser, "Wireless integrated network sensors," Commun. ACM, vol. 43, no. 5, pp. 51-58, 2000.
5. D. S. Alberts, J. J. Garska, and F. P. Stein, "Network centric warfare: Developing and leveraging information superiority," 1999. [Online]. Available: Http://www.dodccrp.org/NCW/ncw.html
6. R. Hills, "Sensing for Danger," Sci. Technol. Rep., 2001. [Online]. Available: Http://www.llnl.gov/str/JulAug01/Hills.html
7. J. Xu, M. P. Johnson, P. S. Fischbeck, M. J. Small, and J. M. Van-Briesen, "Robust placement of sensors in dynamic water distribution systems," Eur. J. Oper. Res., vol. 202, no. 3, pp. 707-716, May 2010.
8. W. B. Heinzelman, A. L. Murphy, H. S. Carvalho, and M. A. Perillo, "Middleware to support sensor network applications," IEEE Network, vol. 18, no. 1, pp. 6-14, Jan.-Feb. 2004.
9. T. Liu, C. Sadler, P. Zhang, and M. Martonosi, "Implementing software on resource-constrained mobile sensors: Experiences with impala and zebranet," in Proc.2nd Int. Conf. Mobile Syst., Appl. Services, MOBISYS' 04, Jun. 2004.
10. A. Mainwaring, J. Polastre, R. Szewczyk, D. Culler, and J. Anderson, "Wireless sensor networks for habitat monitoring," in Proc. ACM Int. Workshop on Wireless Sensor Networks and Applications (WSNA'02), Atlanta, GA, Sep. 2002, pp. 88-97.
11. C. E. Nishimura and D. M. Conlon, "Iuss dual use: Monitoring whales and earthquakes using sosus," Mar. Technol. Soc. J., vol. 27, no. 4, pp. 13-21, 1994.
12. H. Wang, J. Elson, L. Girod, D. Estrin, and K. Yao, "Target classification and localization in habitat monitoring," in Proc. IEEE ICASSP 2003, Hong Kong, Apr. 2003, pp. IV-844-847.
13. P. Zhang, C. M. Sadler, S. A. Lyon, and M. Martonosi, "Hardware design experiences in zebranet," in Proc. 2nd ACM Conf. Embedded Networked Sensor Syst., SenSys'04, Nov. 2004, pp. 227-238.
14. B. Halweil, "Study finds modern farming is costly," World Watch, vol. 14, no. 1, pp. 9-10, 2001.
15. J. Hart and K. Martinez, "Environmental sensor networks: A revolution in the earth system science?," Earth-Sci. Rev., vol. 78, no. 3-4, pp. 177-191, Oct. 2006.
16. D. C. Steere, A. Baptista, D. McNamee, C. Pu, and J. Walpole, "Research challenges in environmental observation and forecasting systems," in Proc. 6th Annu. Int. Conf. Mobile Comput. Networking, 2000, pp. 292-299.
17. L. Yu, N. Wang, and X. Meng, "Real-time forest fire detection with wireless sensor networks," in Proc. Int. Conf. Wireless Commun., Networking, Mobile Comput., Sep. 2005, vol. 2, pp. 1214-1217.
18. J. Burrell, T. Brooke, and R. Beckwith, "Vineyard computing: Sensor networks in agricultural production," IEEE Pervasive Comput., vol. 3, no. 1, pp. 38-45, 2004.
19. A. Matese, S. D. Gennaro, A. Zaldei, L. Genesio, and F. Vaccari, "A wireless sensor network for precision viticulture: The NAV system," Comput. Electron. Agriculture, vol. 69, no. 1, pp. 51-58, Nov. 2009.
20. Y. Kan and C. Chen, "A wearable inertial sensor node for body motion analysis," IEEE Sensors J., vol. PP, 2011, to be published.
21. H. Ghasemzadeh and R. Jafari, "Physical movement monitoring using body sensor networks: A phonological approach to construct spatial decision trees," IEEE Trans. Ind. Informat., vol. 7, no. 1, pp. 66-77, Feb. 2011.
22. P. Bauer, M. Sichitiu, R. Istepanian, and K. Premaratne, "The mobile patient: Wireless distributed sensor networks for patient monitoring and care," in Proc. IEEE EMBS Int. Conf. Inform. Technol. Appl. Biomed., 2000, pp. 17-21.
23. D. Malan, T. Fulford-jones, M.Welsh, and S. Moulton, "Codeblue: An ad hoc sensor network infrastructure for emergency medical care," in Proc. Int. Workshop on Wearable and Implantable Body Sensor Networks, 2004, pp. 12-14.
24. L. Schwiebert, S. K. S. Gupta, and J. Weinmann, "Research challenges in wireless networks of biomedical sensors," Mobile Comput. Networking, pp. 151-165, 2001.
25. N. Noury, T. Herve, V. Rialle, G. Virone, E. Mercier, G. Morey, A. Moro, and T. Porcheron, "Monitoring behavior in home using a smart fall sensor," in Proc. IEEE-EMBS Special Topic Conf. Microtechnol. Med. Biol., Oct. 2000, pp. 607-610.
26. M. Ogawa, T. Tamura, and T. Togawa, "Fully automated biosignal acquisition in daily routine through 1 month," in Proc. IEEE Int. Conf., EMBS, 1998, pp. 1947-1950.
27. P. Johnson and D. Andrews, "Remote continuous physiological monitoring in the home," J. Telemed Telecare, vol. 2, no. 2, pp. 107-113, 1996.
28. Y.-D. Lee, W.-Y. Chung, and B. Chemical, "Wireless sensor network based wearable smart shirt for ubiquitous health and activity monitoring," Sens. Actuators, vol. 140, no. 2, pp. 390-395, Jul. 2009.
29. M. Whelan, M. Gangone, K. Janoyan, and R. Jha, "Real-time wireless vibration monitoring for operational modal analysis of an integral abutment highway bridge," Eng. Structures, vol. 31, no. 10, pp. 2224-2235, Oct. 2009.
30. A. Willig, "Recent and emerging topics in wireless industrial communications: A selection," IEEE Trans. Ind. Informat., vol. 4, no. 2, pp. 102-124, May 2008.
31. I. C. Paschalidis and D. Guo, "Robust and distributed stochastic localization in sensor networks: Theory and experimental results," ACM Trans. Sensor Networks, vol. 5, no. 4, pp. 341-3422, 2008.
32. I. C. Paschalids and D. Guo, "Robust and distributed localization in sensor networks," in Proc. 46th IEEE Conf. Decision and Control, New Orleans, LA, Dec. 2007, pp. 933-938.
33. I. C. Paschalidis, K. Li, and D. Guo, Model-Free Probabilistic Localization of Wireless Sensor Network Nodes in Indoor Environments. New York: Springer, 2009, vol. 5801, Lecture Notes in Computer Science, pp. 66-78.
34. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
35. D. P. Bertsekas and J. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
36. A. Barto, R. Sutton, and C. Anderson, "Neuron-like elements that can solve difficult learning control problems," IEEE Trans. Syst., Man, Cybern., no. 13, pp. 835-846, 1983.
37. J. Si, A. Barto, W. Powell, Handbook of Learning and Approximate Dynamic Programming, IEEE Series on Computational Intelligence. Piscataway, NJ: IEEE Press, 2004.
38. S. Bhatnagar, R. Sutton, M. Ghavamzadeh, and M. Lee, "Incremental natural actor-critic algorithms," Adv. Neural Inform. Process. Syst., vol. 20, pp. 105-112, 2007.
39. V. Borkar, "An actor-critic algorithm for constrained Markov decision processes," Syst. Control Lett., vol. 54, pp. 207-213, 2005.
40. R. H. Crites and A. G. Barto, "An actor/critic algorithm that is equivalent to Q-learning," Adv. Neural Inform. Process. Syst., vol. 7, pp. 401-408, 1994.
41. M. Ghavamzadeh and Y. Engel, "Bayesian actor-critic algorithms," in Proc. ACM Int. Conf. Series, 2007, vol. 227, pp. 297-304.
42. E. Mizutani and S. Dreyfus, "Two stochastic dynamic programming problems by model-free actor-critic recurrent-network learning in non-Markovian settings," in Proc. IEEE Int. Joint Conf. Neural Networks, 2004, vol. 2, pp. 1079-1084.
43. C. Niedzwiedz, I. Elhanany, Z. Liu, and S. Livingston, "A consolidated actor-critic model with function approximation for high-dimensional POMDPS," in Proc. Workshop for Advancement in POMDP Solvers (Part of the AAAI'08 Conf.), Chicago, IL, 2008.
44. M. Rosenstein and A. Barto, "Supervised actor-critic reinforcement learning," in Learning and Approximate Dynamic Programming: Scaling Up to the Real World, W. P. J. Si, A. Barto, and D. Wunsch, Eds. New York: Wiley, 2004, pp. 359-380.
45. I. Paschalidis, K. Li, and R. E. Moazzez, "An actor-critic method using least squares temporal difference learning," in Proc. 48th IEEE Conf. Decision and Control, Shanghai, China, Dec. 2009, pp. 2564-2569.
46. V. R. Konda, "Actor-critic algorithms," Ph.D. dissertation, MIT, Cambridge, MA, 2002.
47. H.Yu and D. Bertsekas, "Convergence results for some temporal difference methods based on least squares," IEEE Trans. Automat. Control, vol. 54, no. 7, pp. 1515-1531, Jul. 2009.
48. V. R. Konda and J. N. Tsitsiklis, "On actor-critic algorithms," SIAM J. Control and Optimization, vol. 42, no. 4, pp. 1143-1166, 2003