메뉴 건너뛰기




Volumn 130, Issue 5, 2006, Pages 626-629

Comparing methods to measure error in gynecologic cytology and surgical pathology

Author keywords

[No Author keywords available]

Indexed keywords

ANALYTIC METHOD; CLINICAL PRACTICE; COMPARATIVE STUDY; CONFERENCE PAPER; CYTOLOGY; DIAGNOSTIC ERROR; GOLD STANDARD; GYNECOLOGY; INTERMETHOD COMPARISON; OUTCOMES RESEARCH; PATHOLOGY; PATIENT SAFETY; REPRODUCIBILITY; SAMPLING; SCREENING; SURGERY;

EID: 33646703051     PISSN: 00039985     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Conference Paper
Times cited : (18)

References (49)
  • 1
    • 27644513255 scopus 로고    scopus 로고
    • Clinical impact and frequency of anatomic pathology errors in cancer diagnoses
    • Raab SS, Grzybicki DM, Janosky JE, et al. Clinical impact and frequency of anatomic pathology errors in cancer diagnoses. Cancer. 2005;104:2205-2213.
    • (2005) Cancer , vol.104 , pp. 2205-2213
    • Raab, S.S.1    Grzybicki, D.M.2    Janosky, J.E.3
  • 2
    • 16344384736 scopus 로고    scopus 로고
    • Patient safety in anatomic pathology: Measuring discrepancy frequencies and causes
    • Raab SS, Nakhleh RE, Ruby SG. Patient safety in anatomic pathology: measuring discrepancy frequencies and causes. Arch Pathol Lab Med. 2005;129:459-466.
    • (2005) Arch Pathol Lab Med , vol.129 , pp. 459-466
    • Raab, S.S.1    Nakhleh, R.E.2    Ruby, S.G.3
  • 3
    • 9144259753 scopus 로고    scopus 로고
    • Improving patient safety by examining pathology errors
    • Raab SS. Improving patient safety by examining pathology errors. Clin Lab Med. 2004;24:849-863.
    • (2004) Clin Lab Med , vol.24 , pp. 849-863
    • Raab, S.S.1
  • 4
    • 33646715394 scopus 로고    scopus 로고
    • Internal quality assurance activities of a surgical pathology department in an Australian teaching hospital
    • Zadawi IM, Bennett G, Jain S, Brown M. Internal quality assurance activities of a surgical pathology department in an Australian teaching hospital. J Clin Pathol. 1998;1998:51.
    • (1998) J Clin Pathol , vol.1998 , pp. 51
    • Zadawi, I.M.1    Bennett, G.2    Jain, S.3    Brown, M.4
  • 6
    • 0025298158 scopus 로고
    • Interpretation of diagnostic cytology with likelihood ratios
    • Giard RWM, Hermans D. Interpretation of diagnostic cytology with likelihood ratios. Arch Pathol Lab Med. 1990;114:852-854.
    • (1990) Arch Pathol Lab Med , vol.114 , pp. 852-854
    • Giard, R.W.M.1    Hermans, D.2
  • 7
    • 0031891916 scopus 로고    scopus 로고
    • Qualifying the cytologic diagnosis of "atypical squamous cells of undetermined significance" affects the predictive value of a squamous intraepithelial lesion on subsequent biopsy
    • Genest DR, Dean B, Lee KR, Sheets E, Crum CP, Cibas ES. Qualifying the cytologic diagnosis of "atypical squamous cells of undetermined significance" affects the predictive value of a squamous intraepithelial lesion on subsequent biopsy. Arch Pathol Lab Med. 1998;122:338-341.
    • (1998) Arch Pathol Lab Med , vol.122 , pp. 338-341
    • Genest, D.R.1    Dean, B.2    Lee, K.R.3    Sheets, E.4    Crum, C.P.5    Cibas, E.S.6
  • 8
    • 2342582767 scopus 로고    scopus 로고
    • Treatment for cervical intraepithelial neoplasia and risk of preterm delivery
    • Sadler L, Saftlas A, Wang W, Exeter M, Whittaker J, McCowan L. Treatment for cervical intraepithelial neoplasia and risk of preterm delivery. JAMA. 2004; 291:2100-2106.
    • (2004) JAMA , vol.291 , pp. 2100-2106
    • Sadler, L.1    Saftlas, A.2    Wang, W.3    Exeter, M.4    Whittaker, J.5    McCowan, L.6
  • 9
    • 3242805971 scopus 로고    scopus 로고
    • Vanishing prostate cancer in radical prostatectomy specimens: Incidence and long-term follow-up in 38 cases
    • Bostwick DG, Bostwick KC. Vanishing prostate cancer in radical prostatectomy specimens: incidence and long-term follow-up in 38 cases. BJU Int. 2004; 94:57-58.
    • (2004) BJU Int , vol.94 , pp. 57-58
    • Bostwick, D.G.1    Bostwick, K.C.2
  • 10
    • 28444433765 scopus 로고    scopus 로고
    • Comparison of disagreement and error rates for three types of interdepartmental consultations
    • Renshaw AA, Gould EW. Comparison of disagreement and error rates for three types of interdepartmental consultations. Am J Clin Pathol. 2005;124:878-882.
    • (2005) Am J Clin Pathol , vol.124 , pp. 878-882
    • Renshaw, A.A.1    Gould, E.W.2
  • 11
    • 0033429881 scopus 로고    scopus 로고
    • Mandatory second opinion surgical pathology at a large referral hospital
    • Kronz JD, Westra WH, Epstein JI. Mandatory second opinion surgical pathology at a large referral hospital. Cancer. 1999;86:2426-2438.
    • (1999) Cancer , vol.86 , pp. 2426-2438
    • Kronz, J.D.1    Westra, W.H.2    Epstein, J.I.3
  • 12
    • 26844489722 scopus 로고    scopus 로고
    • Results of multiple blinded review of Pap slides in the context of litigation: Determining what can be regularly and reliably detected
    • Renshaw AA, Holladay EB, Geils KB. Results of multiple blinded review of Pap slides in the context of litigation: determining what can be regularly and reliably detected. Cancer Cytopathol. 2005;105:263-269.
    • (2005) Cancer Cytopathol , vol.105 , pp. 263-269
    • Renshaw, A.A.1    Holladay, E.B.2    Geils, K.B.3
  • 13
    • 0036724693 scopus 로고    scopus 로고
    • Review of 'outside' pathology before treatment should be mandatory
    • Rosen PP. Review of 'outside' pathology before treatment should be mandatory. Am J Surg Pathol. 2002;26:1235-1236.
    • (2002) Am J Surg Pathol , vol.26 , pp. 1235-1236
    • Rosen, P.P.1
  • 14
    • 0035925786 scopus 로고    scopus 로고
    • Interobserver reproducibility of cervical cytologic and histologic interpretations: Realistic estimates from the ASCUS-LSIL triage study
    • Stoler MH, Schiffman M. Interobserver reproducibility of cervical cytologic and histologic interpretations: realistic estimates from the ASCUS-LSIL triage study. JAMA. 2001;285:1500-1505.
    • (2001) JAMA , vol.285 , pp. 1500-1505
    • Stoler, M.H.1    Schiffman, M.2
  • 15
    • 33646671275 scopus 로고    scopus 로고
    • Interlaboratory reproducibility of atypical squamous cells of undetermined significance
    • Confortini M, Carozzi F, Dalla Palma P, et al. Interlaboratory reproducibility of atypical squamous cells of undetermined significance. Cytopathology. 2003;14:231-234.
    • (2003) Cytopathology , vol.14 , pp. 231-234
    • Confortini, M.1    Carozzi, F.2    Dalla Palma, P.3
  • 16
    • 18144373458 scopus 로고    scopus 로고
    • Measuring the significance of field validation in the College of American Pathologists gynecologic cytology program: How good are the experts?
    • Renshaw AA, Wang E, Mody DR, Wilbur DC, Davey DD, Colgan TJ. Measuring the significance of field validation in the College of American Pathologists gynecologic cytology program: how good are the experts? Arch Pathol Lab Med. 2005;129:609-613.
    • (2005) Arch Pathol Lab Med , vol.129 , pp. 609-613
    • Renshaw, A.A.1    Wang, E.2    Mody, D.R.3    Wilbur, D.C.4    Davey, D.D.5    Colgan, T.J.6
  • 17
    • 0031038971 scopus 로고    scopus 로고
    • Diagnostic discrepancies and their clinical impact in a neuropathology referral practice
    • Bruner JM, Inouye L, Fuller GN, Langford LA. Diagnostic discrepancies and their clinical impact in a neuropathology referral practice. Cancer. 1997;79: 796-803.
    • (1997) Cancer , vol.79 , pp. 796-803
    • Bruner, J.M.1    Inouye, L.2    Fuller, G.N.3    Langford, L.A.4
  • 18
    • 0032915178 scopus 로고    scopus 로고
    • The false negative rate of Pap smear screening as determined by rapid rescreening
    • Renshaw AA, Bellerose B, DiNisco SA, Minier LJ, Lee KR. The false negative rate of Pap smear screening as determined by rapid rescreening. Acta Cytol. 1999;43:344-350.
    • (1999) Acta Cytol , vol.43 , pp. 344-350
    • Renshaw, A.A.1    Bellerose, B.2    Dinisco, S.A.3    Minier, L.J.4    Lee, K.R.5
  • 19
    • 0033038338 scopus 로고    scopus 로고
    • Performance characteristics of rapid (30 second) prescreening: Implications for calculating the false-negative rate and comparison with other quality assurance techniques
    • Renshaw AA, Cronin JA, Minier LJ, Whitman T, Jiroutek M, Cibas ES. Performance characteristics of rapid (30 second) prescreening: implications for calculating the false-negative rate and comparison with other quality assurance techniques. Am J Clin Pathol. 1999;111:517-522.
    • (1999) Am J Clin Pathol , vol.111 , pp. 517-522
    • Renshaw, A.A.1    Cronin, J.A.2    Minier, L.J.3    Whitman, T.4    Jiroutek, M.5    Cibas, E.S.6
  • 20
    • 0036235534 scopus 로고    scopus 로고
    • Quantifying the value of in-house consultation in surgical pathology
    • Renshaw AA, Pinnar NE, Jiroutek MR, Young ML. Quantifying the value of in-house consultation in surgical pathology. Am J Clin Pathol. 2002;117:751-754.
    • (2002) Am J Clin Pathol , vol.117 , pp. 751-754
    • Renshaw, A.A.1    Pinnar, N.E.2    Jiroutek, M.R.3    Young, M.L.4
  • 21
  • 22
    • 85047688752 scopus 로고    scopus 로고
    • Agreement and error rates using blinded review to evaluate surgical pathology material
    • Renshaw AA, Cartagena N, Granter SR, Gould EW. Agreement and error rates using blinded review to evaluate surgical pathology material. Am J Clin Pathol. 2003;119:797-800.
    • (2003) Am J Clin Pathol , vol.119 , pp. 797-800
    • Renshaw, A.A.1    Cartagena, N.2    Granter, S.R.3    Gould, E.W.4
  • 23
    • 0242304209 scopus 로고    scopus 로고
    • Interpretation of skin biopsies by general pathologists: Diagnostic discrepancy rates measured by blinded review
    • Trotter MJ, Bruecks AK. Interpretation of skin biopsies by general pathologists: diagnostic discrepancy rates measured by blinded review. Arch Pathol Lab Med 2003;127:1489-1492.
    • (2003) Arch Pathol Lab Med , vol.127 , pp. 1489-1492
    • Trotter, M.J.1    Bruecks, A.K.2
  • 24
    • 0023493339 scopus 로고
    • Retrospective evaluation of gynecologic cytodiagnosis, II: Interlaboratory reproducibility as shown in rescreening large consecutive samples of reported cases
    • Yobs AR, Plott AE, Hicklin MD, et al. Retrospective evaluation of gynecologic cytodiagnosis, II: interlaboratory reproducibility as shown in rescreening large consecutive samples of reported cases. Acta Cytol. 1987;31:900-910.
    • (1987) Acta Cytol , vol.31 , pp. 900-910
    • Yobs, A.R.1    Plott, A.E.2    Hicklin, M.D.3
  • 25
    • 17744402794 scopus 로고    scopus 로고
    • Do proficiency test results correlate with the work performance of screeners who screen Papanicolaou smears?
    • Keenlyside RA, Collins CL, Hancock JS, et al. Do proficiency test results correlate with the work performance of screeners who screen Papanicolaou smears? Am J Clin Pathol. 1999;112:769-776.
    • (1999) Am J Clin Pathol , vol.112 , pp. 769-776
    • Keenlyside, R.A.1    Collins, C.L.2    Hancock, J.S.3
  • 26
    • 0035045625 scopus 로고    scopus 로고
    • The human false negative rate of rescreening in a two arm prospective clinical trial
    • Renshaw AA, Lezon KM, Wilbur DC. The human false negative rate of rescreening in a two arm prospective clinical trial. Cancer Cytopathol. 2001;93: 106-110.
    • (2001) Cancer Cytopathol , vol.93 , pp. 106-110
    • Renshaw, A.A.1    Lezon, K.M.2    Wilbur, D.C.3
  • 27
    • 0026161655 scopus 로고
    • Diagnosis of diffuse malignant mesothelioma: Experience of a US/Canadian mesothelioma panel
    • McCaughey WT, Colby TV, Battifora H, et al. Diagnosis of diffuse malignant mesothelioma: experience of a US/Canadian mesothelioma panel. Mod Pathol. 1991;4:342-353.
    • (1991) Mod Pathol , vol.4 , pp. 342-353
    • McCaughey, W.T.1    Colby, T.V.2    Battifora, H.3
  • 28
    • 0028271150 scopus 로고
    • Random rescreening of cytologic smears: A practical and effective component of quality assurance programs in both large and small cytology laboratories
    • Krieger P, Naryshkin S. Random rescreening of cytologic smears: a practical and effective component of quality assurance programs in both large and small cytology laboratories. Acta Cytol. 1994;38:291-298.
    • (1994) Acta Cytol , vol.38 , pp. 291-298
    • Krieger, P.1    Naryshkin, S.2
  • 29
    • 0022641141 scopus 로고
    • Diagnostic vigilance
    • Fowkes FGR. Diagnostic vigilance. Lancet. 1986;8479:493-494.
    • (1986) Lancet , vol.8479 , pp. 493-494
    • Fowkes, F.G.R.1
  • 30
    • 0033739971 scopus 로고    scopus 로고
    • Expert review of histologic slides and Papanicolaou tests in the context of litigation or potential litigation
    • Surgical Pathology Committee and Cytopathology Committee of the College of American Pathologists
    • Fitzgibbons PL, Austin RM. Expert review of histologic slides and Papanicolaou tests in the context of litigation or potential litigation. Surgical Pathology Committee and Cytopathology Committee of the College of American Pathologists. Arch Pathol Lab Med. 2000;124:1717-1719.
    • (2000) Arch Pathol Lab Med , vol.124 , pp. 1717-1719
    • Fitzgibbons, P.L.1    Austin, R.M.2
  • 31
    • 0032709411 scopus 로고    scopus 로고
    • Distraction index, part I: The elusive trich
    • Neal MH, Kline TS. Distraction index, part I: the elusive trich. Diagn Cytopathol. 1999;6:367-369.
    • (1999) Diagn Cytopathol , vol.6 , pp. 367-369
    • Neal, M.H.1    Kline, T.S.2
  • 32
    • 0033840220 scopus 로고    scopus 로고
    • Distraction index, part II: Inflammation in nongynecologic cytology
    • Schulte MA. Distraction index, part II: inflammation in nongynecologic cytology. Diagn Cytopathol. 2000;23:149-150.
    • (2000) Diagn Cytopathol , vol.23 , pp. 149-150
    • Schulte, M.A.1
  • 33
    • 0035048666 scopus 로고    scopus 로고
    • An accurate and precise methodology for routine determination of the false negative rate of Pap smear screening
    • Renshaw AA. An accurate and precise methodology for routine determination of the false negative rate of Pap smear screening. Cancer Cytopathol. 2001; 93:86-92.
    • (2001) Cancer Cytopathol , vol.93 , pp. 86-92
    • Renshaw, A.A.1
  • 34
    • 0037173945 scopus 로고    scopus 로고
    • Measuring sensitivity in gynecologic cytology: A review
    • Renshaw AA. Measuring sensitivity in gynecologic cytology: a review. Cancer Cytopathol. 2002;96:210-217.
    • (2002) Cancer Cytopathol , vol.96 , pp. 210-217
    • Renshaw, A.A.1
  • 35
    • 2942664573 scopus 로고    scopus 로고
    • Multiple case blinded rescreening for review of litigation cases
    • Allen KA, Holladay EB, eds. Raleigh, NC: American Society of Cytotechnologists
    • Holladay EB, Austin RM. Multiple case blinded rescreening for review of litigation cases. In: Allen KA, Holladay EB, eds. Risk Management for the Cytology Laboratory. Raleigh, NC: American Society of Cytotechnologists; 2001:149-162.
    • (2001) Risk Management for the Cytology Laboratory , pp. 149-162
    • Holladay, E.B.1    Austin, R.M.2
  • 36
    • 2942631428 scopus 로고    scopus 로고
    • Blinded review of Pap smears in the context of litigation: Using statistical analysis to define appropriate thresholds
    • Renshaw AA, Young ML, Holladay EB. Blinded review of Pap smears in the context of litigation: using statistical analysis to define appropriate thresholds. Cancer Cytopathol. 2004;102:136-141.
    • (2004) Cancer Cytopathol , vol.102 , pp. 136-141
    • Renshaw, A.A.1    Young, M.L.2    Holladay, E.B.3
  • 37
    • 0035949173 scopus 로고    scopus 로고
    • Estimating the percentage of Papanicolaou smears that can be reproducibly identified: Modeling Papanicolaou smear interpretation based on multiple blinded rescreenings
    • Renshaw AA. Estimating the percentage of Papanicolaou smears that can be reproducibly identified: modeling Papanicolaou smear interpretation based on multiple blinded rescreenings. Cancer Cytopathol. 2001;93:241-245.
    • (2001) Cancer Cytopathol , vol.93 , pp. 241-245
    • Renshaw, A.A.1
  • 38
    • 0000912177 scopus 로고    scopus 로고
    • A case-control study of true positive versus false negative smears in women with CIN 3
    • O'Sullivan JP, A'Hern RP, Chapman PA, et al. A case-control study of true positive versus false negative smears in women with CIN 3 [abstract]. Acta Cytol. 1997;41:1543.
    • (1997) Acta Cytol , vol.41 , pp. 1543
    • O'Sullivan, J.P.1    A'Hern, R.P.2    Chapman, P.A.3
  • 39
    • 0033979732 scopus 로고    scopus 로고
    • Characteristics of high grade dyskaryotic cervical smears likely to be missed on rapid rescreening
    • O'Sullivan JP, Chapman PA, Jenkins L, Smith R. Characteristics of high grade dyskaryotic cervical smears likely to be missed on rapid rescreening. Acta Cytol. 2000;44:37-40.
    • (2000) Acta Cytol , vol.44 , pp. 37-40
    • O'Sullivan, J.P.1    Chapman, P.A.2    Jenkins, L.3    Smith, R.4
  • 40
    • 1842531412 scopus 로고    scopus 로고
    • Cytologic features of squamous cell carcinoma in ThinPrep slides
    • Renshaw AA, Dubray-Benstein B, Cobb CJ, et al. Cytologic features of squamous cell carcinoma in ThinPrep slides. Arch Pathol Lab Med. 2004;128: 403-405.
    • (2004) Arch Pathol Lab Med , vol.128 , pp. 403-405
    • Renshaw, A.A.1    Dubray-Benstein, B.2    Cobb, C.J.3
  • 41
    • 0742283968 scopus 로고    scopus 로고
    • Adenocarcinoma in situ of the cervix is significantly more difficult to identify and categorize than adenocarcinoma, high grade squamous intraepithelial lesion, or squamous cell carcinoma
    • Renshaw AA, Mody DR, Lozano RL, Volk EE, Davey DD, Birdsong GG. Adenocarcinoma in situ of the cervix is significantly more difficult to identify and categorize than adenocarcinoma, high grade squamous intraepithelial lesion, or squamous cell carcinoma. Arch Pathol Lab Med. 2004;128:153-157.
    • (2004) Arch Pathol Lab Med , vol.128 , pp. 153-157
    • Renshaw, A.A.1    Mody, D.R.2    Lozano, R.L.3    Volk, E.E.4    Davey, D.D.5    Birdsong, G.G.6
  • 42
    • 3042841598 scopus 로고    scopus 로고
    • Cytologic features of high grade squamous intraepithelial lesion in ThinPrep slides: Comparison of cases which performed poorly and well in the College of American Pathologists Interlaboratory Comparison Program in cervicovaginal cytology
    • Renshaw AA, Plott E, Dubray-Benstein B, et al. Cytologic features of high grade squamous intraepithelial lesion in ThinPrep slides: comparison of cases which performed poorly and well in the College of American Pathologists Interlaboratory Comparison Program in cervicovaginal cytology. Arch Pathol Lab Med. 2004;128:746-748.
    • (2004) Arch Pathol Lab Med , vol.128 , pp. 746-748
    • Renshaw, A.A.1    Plott, E.2    Dubray-Benstein, B.3
  • 43
    • 19544378848 scopus 로고    scopus 로고
    • Cytologic features of high grade squamous intraepithelial lesion in conventional slides: Comparison of cases which performed poorly and well in the College of American Pathologists gynecologic cytology program
    • Renshaw AA, Prey MU, Hodes L, Weisson M, Haja J, Moriarty AT. Cytologic features of high grade squamous intraepithelial lesion in conventional slides: comparison of cases which performed poorly and well in the College of American Pathologists gynecologic cytology program. Arch Pathol Lab Med. 2005;129:733-735.
    • (2005) Arch Pathol Lab Med , vol.129 , pp. 733-735
    • Renshaw, A.A.1    Prey, M.U.2    Hodes, L.3    Weisson, M.4    Haja, J.5    Moriarty, A.T.6
  • 44
    • 11844283238 scopus 로고    scopus 로고
    • Cytologic features of low grade squamous intraepithelial lesion in ThinPrep and conventional smear specimens
    • Renshaw AA, Dubray-Benstein BL, Haja J, Hughes JL. Cytologic features of low grade squamous intraepithelial lesion in ThinPrep and conventional smear specimens. Arch Pathol Lab Med. 2005;129:23-25.
    • (2005) Arch Pathol Lab Med , vol.129 , pp. 23-25
    • Renshaw, A.A.1    Dubray-Benstein, B.L.2    Haja, J.3    Hughes, J.L.4
  • 45
    • 30444457922 scopus 로고    scopus 로고
    • Cytologic features of adenocarcinoma, not otherwise specified, in conventional smears: Comparison of cases that performed poorly with those that performed well in the College of American Pathologists Interlaboratory Comparison Program in cervicovaginal cytology
    • Renshaw AA, Schwartz MR, Wang E, Haja J, Hughes JH. Cytologic features of adenocarcinoma, not otherwise specified, in conventional smears: comparison of cases that performed poorly with those that performed well in the College of American Pathologists Interlaboratory Comparison Program in cervicovaginal cytology. Arch Pathol Lab Med. 2006;130:23-26.
    • (2006) Arch Pathol Lab Med , vol.130 , pp. 23-26
    • Renshaw, A.A.1    Schwartz, M.R.2    Wang, E.3    Haja, J.4    Hughes, J.H.5
  • 47
    • 0141706739 scopus 로고    scopus 로고
    • Rescreening in cervical cytology for quality control: When bad data is worse than no data, or what works, what doesn't, and why
    • Renshaw AA. Rescreening in cervical cytology for quality control: when bad data is worse than no data, or what works, what doesn't, and why. Clin Lab Med. 2003;23:695-708.
    • (2003) Clin Lab Med , vol.23 , pp. 695-708
    • Renshaw, A.A.1
  • 48
    • 19544380246 scopus 로고    scopus 로고
    • A preliminary diagnosis service provides prospective blinded dual-review of all general surgical pathology cases in an academic institution
    • Weydert JA, De Young BR, Cohen MB. A preliminary diagnosis service provides prospective blinded dual-review of all general surgical pathology cases in an academic institution. Am J Surg Pathol. 2005;29:801-805.
    • (2005) Am J Surg Pathol , vol.29 , pp. 801-805
    • Weydert, J.A.1    De Young, B.R.2    Cohen, M.B.3
  • 49
    • 85047690599 scopus 로고    scopus 로고
    • Measuring the value of review of pathology material by a second pathologist
    • In press
    • Renshaw AA, Gould EW. Measuring the value of review of pathology material by a second pathologist. Am J Clin Pathol. In press.
    • Am J Clin Pathol
    • Renshaw, A.A.1    Gould, E.W.2


* 이 정보는 Elsevier사의 SCOPUS DB에서 KISTI가 분석하여 추출한 것입니다.