A multiresolution analysis-assisted reinforcement learning approach to run-by-run control

IEEE Transactions on Automation Science and Engineering

Volumn 4, Issue 2, 2007, Pages 182-193

  Ganesan, Rajesh ^a   Das, Tapas K ^b   Ramachandran, Kandethody M ^c  

^a Krasnow Institute for Advanced Study   (United States)

^b University of South Florida   (United States)

^c UNIVERSITY OF SOUTH FLORIDA   (United States)

Author keywords

Chemical mechanical planarization (CMP); Exponentially weighted moving average (EWMA); Multiresolution; Reinforcement learning; Run by run control; Wavelet; Wavelet modulated reinforcement learning run by run (WRL RbR)

Indexed keywords

CHEMICAL MECHANICAL POLISHING; MATHEMATICAL MODELS; MULTIRESOLUTION ANALYSIS; PRODUCTION CONTROL; SEMICONDUCTOR DEVICE MANUFACTURE;

EXPONENTIALLY WEIGHTED MOVING AVERAGE (EWMA); PROCESS MODELS; PRODUCTION RUNS; RUN-BY RUN (RBR) CONTROL;

REINFORCEMENT LEARNING;

EID: 34147169178     PISSN: 15455955     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASE.2006.879915     Document Type: Article

References (41)

1. A. Ingolfsson and E. Sachs, "Stability and sensitivity of an EWMA controller," J. Qual. Technol., vol. 25, no. 4, pp. 271-287, 1993.
2. E. Del Castillo and J. Yeh, "An adaptive optimizing quality controller for linear and nonlinear semiconductor processes," IEEE Trans. Semicond. Manuf., vol. 11, no. 2, pp. 285-295, May 1998.
3. W. J. Campbell, "Model predictive run-to-run control of chemical mechanical planarization," Ph.D. dissertation, Univ. Texas at Austin, Austin, 1999.
4. Z. Ning, J. R. Moyne, T. Smith, D. Boning, E. D. Castillo, J. Y. Yeh, and A. Hurwitz, "A comparative analysis of run-to-run control algorithms in the semiconductor manufacturing industry," in Proc. Advanced Semiconductor Manufacturing, 1996, pp. 375-381, IEEE/SEMI.
5. K. Chamness, G. Cherry, R. Good, and S. J. Qin, "Comparison of r2r control algorithms for the CMP with measurement delays," in Proc. AEC/APC XIII Symp., Banff, AB, Canada, 2001.
6. E. Sachs, A. Hu, and A. Ingolfsson, "Run by run process control: Combining SPC and feedback control," IEEE Trans. Semicond. Manuf., vol. 8, no. 1, pp. 26-43, Feb. 1995.
7. S. W. Butler and J. A. Stefani, "Supervisory run-to-run control of a polysilicon gate etch using in situ ellipsometry," IEEE Trans. Semicond. Manuf., vol. 7, no. 2, pp. 193-201, May 1994.
8. E. Del Castillo and A. M. Hurwitz, "Run-to-run process control: Literature review and extensions," J. Qual. Technol., vol. 29, no. 2, pp. 184-196, 1997.
9. T. H. Smith and D. S. Boning, "Artificial neural network exponentially weighted moving average control for semiconductor processes," J. Vac. Sci. Technol. A, vol. 15, no. 3, pp. 1377-1384, 1997.
10. E. Del Castillo and R. Rajagopal, "A multivariate double EWMA process adjustment scheme for drifting processes," IIE Trans., vol. 34, no. 12, pp. 1055-1068, 2002.
11. R. Rajagopal and E. Del Castillo, "An analysis and MIMO extension of a double EWMA run-to-run controller for non-squared systems," Int. J. Rel., Qual. Safety Eng., vol. 10, no. 4, pp. 417-428, 2003.
12. S. K. S. Fan, B. C. Jiang, C. H. Jen, and C. C. Wang, "SISO run-to-run feedback controller using triple EWMA smoothing foe semiconductor manufacturing processes," Int. J. Prod. Res., vol. 40, no. 13, pp. 3093-3120, 2002.
13. S. T. Tseng, A. B. Yeh, F. Tsung, and Y. Y. Chan, "A study of variable EWMA controller," IEEE Trans. Semicond. Manuf., vol. 16, no. 4, pp. 633-643, Nov. 2003.
14. N. S. Patel and S. T. Jenkins, "Adaptive optimization of run-by-run controllers," IEEE Trans. Semicond. Eng., vol. 13, no. 1, pp. 97-107, Feb. 2000.
15. C. T. Su and C. C. Hsu, "On-line tuning of a single EWMA controller based on the neural technique," Int. J. Prod. Res., vol. 42, no. 11, pp. 2163-2178, 2004.
16. D. Shi and F. Tsung, "Modeling and diagnosis of feedback-controlled process using dynamic PCA and neural networks," Int. J. Prod. Res., vol. 41, no. 2, pp. 365-379, 2003.
17. E. Del Castillo, "Long ran transient analysis of a double EWMA feedback controller," IIE Trans., vol. 31, pp. 1157-1169, 1999.
18. R. Ganesan, T. K. Das, and V. Venkataraman, "Wavelet based multi-scale statistical process monitoring - A literature review," IIE Trans. Qual. Rel. Eng., vol. 36, no. 9, pp. 787-806, 2004.
19. A. Terchi and Y. H. J. Au, "Acoustic emission signal processing," Meas. Control, vol. 34, pp. 240-244, 2001.
20. I. N. Tansel, C. Mekdesi, O. Rodriguez, and B. Uragun, "Monitoring microdrilling operations with wavelets," Quality Assurance Through Integration of Manufacturing Processes and Systems, pp. 151-163, 1992.
21. X. Li, "A brief review: Acoustic emission method for tool wear monitoring during turning," Int. J. Mach. Tools Manuf., vol. 42, pp. 157-165, 2002.
22. G. Strang and T. Nguyen, Wavelets and Filter Banks. Wellesley, MA: Wellesley Cambridge Press, 1996.
23. I. Daubechies, Ten Lectures in Wavelets. Philadelphia, PA: SIAM, 1992.
24. D. L. Donoho, I. M. Johnstone, G. Kerkyacharian, and D. Picard, "Wavelet shrinkage: Asymptopia? (with discussion)," J. Roy. Statist. Soc., vol. 57, no. 2, pp. 301-369, 1995.
25. F. Abramovich and Y. Benjamini, "Thresholding of wavelet coefficients as multiple hypothesis testing procedure," in Wavelets and Statistics, ser. Lecture Notes in Statistics, A. Antoniadis and G. Oppenheim, Eds. New York: Springer-Verlag, 1995, vol. 103, pp. 5-14.
26. M. Neumann and R. V. Sachs, "Wavelet thresholding: Beyond the Gaussian iid situation," in Wavelets and Statistics, ser. Lecture Notes in Statistics, A. Antoniadis and G. Oppenheim, Eds. New York: Springer-Verlag, 1995, vol. 103, pp. 301-329.
27. A. J. Toprac, H. Luna, B. Withers, M. Bedrin, and S. Toy, "Developing and implementing an advanced CMP runto-run controller," Micromagazine, 2003 [Online]. Available: available URL: http://www.micromagazine.com/archive/03/08/toprac.html.
28. R. Bellman, "The theory of dynamic programming," Bull. Amer. Math. Soc., vol. 60, pp. 503-516, 1954.
29. H. Robbins and S. Monro, "A stochastic approximation method," Ann. Math. Statist., vol. 22, pp. 400-407, 1951.
30. M. L. Puterman, Markov Decision Processes. New York: Wiley, 1994.
31. J. Abounadi, "Stochastic approximation for non-expansive maps: Application to q-learning algorithms," Ph.D. dissertation, Mass. Inst. Technol., Cambridge, MA, Feb. 1998.
32. A. Gosavi, "An algorithm for solving semi-markov decision problem using reinforcement learning: convergence analysis and numerical results," Ph.D. dissertation, IMSE Dept., Univ. South Florida, Tampa, FL, 1998.
33. J. Bremer, R. Coifman, M. Maggioni, and A. Szlam, Diffusion Wavelet Packets Yale Univ., New Haven, CT, Tech. Rep. YALE/DCS/TR-1304, 2004, to appear in Appl. Camp. Harm. Anal.
34. S. Mahadevan and M. Maggioni, "Value function approximation using diffusion wavelets and Laplacian eigenfunctions," Dept. Comput. Sci., Univ. Mass., Amherst, MA, 2005, Tech. Rep. TR-2005-38.
35. R. Coifman and M. Maggioni, "Diffusion wavelets," Yale University, New Haven, CT, Tech. Rep. YALE/DCS/TR-1303, 2004.
36. R. Ganesan, T. K. Das, A. K. Sikder, and A. Kumar, "Wavelet based identification of delamination of low-k dielectric layers in a copper damascene CMP process," IEEE Trans. Semicond. Manuf., vol. 16, no. 4, pp. 677-685, Nov. 2003.
37. B. R. Bakshi, "Multiscale statistical process control and model-based denoising," in Wavelets in Chemistry, ser. Data Handling in Science and Technology. B. Walczak, Ed. Amsterdam, The Netherlands: Elsevier, 2000, vol. 22, ch. 17, pp. 411-436.
38. V. S. Borkar, "Stochastic approximation with two-time scales," Syst. Control Lett., vol. 29, pp. 291-294, 1997.
39. A. Gosavi, "Reinforcement learning for long-run average cost," Eur. J. Oper. Res., vol. 155, pp. 654-674, 2004.
40. V. S. Borkar and K. Soumyanath, "An analog scheme for fixed point computation, part 1: Theory," IEEE Trans. Circuits Syst. I. Fundam. Theory Appl., vol. 44, pp. 351-354, 1997.
41. D. Bertsekas and J. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1995.