Physical activity and suicidal ideation: A systematic review and meta-analysis

Journal of Affective Disorders

Volumn 225, Issue , 2018, Pages 438-448

  Vancampfort, Davy ^a   Hallgren, Mats ^b   Firth, Joseph ^c,d   Rosenbaum, Simon ^e,f   Schuch, Felipe B ^g,h,i   Mugisha, James ^j,k   Probst, Michel ^a   Van Damme, Tine ^a   Carvalho, André F ^l   Stubbs, Brendon ^m,n  

^a UNIVERSITY OF LEUVEN   (Belgium)

^b KAROLINSKA INSTITUTE   (Sweden)

^c UNIVERSITY OF MANCHESTER   (United Kingdom)

^d UNIVERSITY OF WESTERN SYDNEY   (Australia)

^e UNIVERSITY OF NEW SOUTH WALES   (Australia)

^f PRINCE OF WALES HOSPITAL   (Australia)

^g Mestrado em Avaliacao de Impactos Ambientais em Mineracao Victor Barreto   (Brazil)

^h Laboratorio de Pesquisa do Exercicio   (Brazil)

ⁱ HOSPITAL DE CLÍNICAS DE PORTO ALEGRE   (Brazil)

^j KYAMBOGO UNIVERSITY   (Uganda)

^k Butabika National Referral and Teaching Mental Hospital   (Uganda)

^l FEDERAL UNIVERSITY OF CEARÁ   (Brazil)

^m SOUTH LONDON AND MAUDSLEY NHS FOUNDATION TRUST   (United Kingdom)

ⁿ KING'S COLLEGE LONDON   (United Kingdom)

more..

Author keywords

Mortality; Physical exercise; Suicide

Indexed keywords

DEPRESSION; HUMAN; META ANALYSIS; MORTALITY; PHYSICAL ACTIVITY; PHYSICAL INACTIVITY; PRACTICE GUIDELINE; PRIORITY JOURNAL; REVIEW; SUICIDAL IDEATION; SYSTEMATIC REVIEW; ADOLESCENT; ADULT; AGE; AGED; ATTITUDE TO HEALTH; CROSS-SECTIONAL STUDY; EXERCISE; HEALTH BEHAVIOR; MIDDLE AGED; MOTOR ACTIVITY; PROSPECTIVE STUDY; PSYCHOLOGY; SUICIDE; YOUNG ADULT;

ADOLESCENT; ADULT; AGE FACTORS; AGED; ATTITUDE TO HEALTH; CROSS-SECTIONAL STUDIES; EXERCISE; HEALTH BEHAVIOR; HUMANS; MIDDLE AGED; MOTOR ACTIVITY; PROSPECTIVE STUDIES; SUICIDAL IDEATION; SUICIDE; YOUNG ADULT;

EID: 85028387828     PISSN: 01650327     EISSN: 15732517     Source Type: Journal    
DOI: 10.1016/j.jad.2017.08.070     Document Type: Review

References (91)

1. Abdel-Baki, A., Brazzini-Poisson, V., Marois, F., Letendre, E., Karelis, A.D., Effects of aerobic interval training on metabolic complications and cardiorespiratory fitness in young adults with psychotic disorders: a pilot study. Schizophr. Res. 149 (2013), 112–115.
2. Abdollahi, A., LeBouthillier, D.M., Najafi, M., Asmundson, G.J.G., Hosseinian, S., Shahidi, S., Carlbring, P., Kalhori, A., Sadeghi, H., Jalili, M., Effect of exercise augmentation of cognitive behavioural therapy for the treatment of suicidal ideation and depression. J. Affect. Disord. 219 (2017), 58–63.
3. Adams, T.B., Moore, M.T., Dye, J., The relationship between physical activity and mental health in a national sample of college females. Women Health 45 (2007), 69–85.
4. An, K.O., Jang, J.Y., Kim, J., Sedentary behavior and sleep duration are associated with both stress symptoms and suicidal thoughts in Korean adults. Tohoku J. Exp. Med. 237 (2015), 279–286.
5. Arango, V., Underwood, M.D., Mann, J.J., Serotonin brain circuits involved in major depression and suicide. Prog. Brain Res. 136 (2002), 443–453.
6. Arat, G., Wong, P.W.-C., The relationship between physical activity and mental health among adolescents in six middle-income countries: a cross-sectional study. Child Youth Serv., 2017, 1–16.
7. Babiss, L.A., Gangwisch, J.E., Sports participation as a protective factor against depression and suicidal ideation in adolescents as mediated by self-esteem and social support. J. Dev. Behav. Pediatr. 30 (2009), 376–384.
8. Bailey, M., McLaren, S., Physical activity alone and with others as predictors of sense of belonging and mental health in retirees. Aging Ment. Health 9 (2005), 82–90.
9. Begg, C.B., Mazumdar, M., Operating characteristics of a rank correlation test for publication bias. Biometrics, 1994, 1088–1101.
10. Borges, G., Nock, M.K., Abad, J.M.H., Hwang, I., Sampson, N.A., Alonso, J., Andrade, L.H., Angermeyer, M.C., Beautrais, A., Bromet, E., Twelve month prevalence of and risk factors for suicide attempts in the WHO World Mental Health Surveys. J. Clin. Psychiatry, 71, 2010, 1617.
11. Brezo, J., Paris, J., Barker, E.D., Tremblay, R., Vitaro, F., Zoccolillo, M., Hebert, M., Turecki, G., Natural history of suicidal behaviors in a population-based sample of young adults. Psychol. Med. 37 (2007), 1563–1574.
12. Brown, D.R., Blanton, C.J., Physical activity, sports participation, and suicidal behavior among college students. Med. Sci. Sports Exerc. 34:7 (2002), 1087–1096.
13. Brown, D.R., Galuska, D.A., Zhang, J., Eaton, D.K., Fulton, J.E., Lowry, R., Maynard, L.M., Psychobiology and behavioral strategies. Physical activity, sport participation, and suicidal behavior: US high school students. Med. Sci. Sports Exerc. 39 (2007), 2248–2257.
14. Carroll, R., Metcalfe, C., Gunnell, D., Hospital presenting self-harm and risk of fatal and non-fatal repetition: systematic review and meta-analysis. PLoS One, 9, 2014, e89944.
15. Caspersen, C.J., Powell, K.E., Christenson, G.M., Physical activity, exercise, and physical fitness: definitions and distinctions for health-related research. Public Health Rep., 100, 1985, 126.
16. Cho, K.-O., Physical activity and suicide attempt of South Korean adolescents-evidence from the Eight Korea Youth Risk Behaviors Web-based Survey. J. Sports Sci. Med., 13, 2014, 888.
17. Conwell, Y., Van Orden, K., Caine, E.D., Suicide in older adults. Psychiatr. Clin. N. Am. 34 (2011), 451–468.
18. Davidson, C.L., Babson, K.A., Bonn‐Miller, M.O., Souter, T., Vannoy, S., The impact of exercise on suicide risk: examining pathways through depression, PTSD, and sleep in an inpatient sample of veterans. Suicide Life-Threat. Behav. 43 (2013), 279–289.
19. DeVon, H.A., Block, M.E., Moyle‐Wright, P., Ernst, D.M., Hayden, S.J., Lazzara, D.J., Savoy, S.M., Kostas‐Polston, E., A psychometric toolbox for testing validity and reliability. J. Nurs. Scholarsh. 39 (2007), 155–164.
20. Dinger, M.K., Vesely, S.K., Relationships between physical activity and other health-related behaviors in a representative sample of US college students. J. Health Educ. 32 (2001), 83–88.
21. Duval, S., Tweedie, R., Trim and fill: a simple funnel‐plot–based method of testing and adjusting for publication bias in meta‐analysis. Biometrics 56 (2000), 455–463.
22. Egger, M., Smith, G.D., Schneider, M., Minder, C., Bias in meta-analysis detected by a simple, graphical test. Br. Med. J. 315 (1997), 629–634.
23. Elliot, C.A., Kennedy, C., Morgan, G., Anderson, S.K., Morris, D., Undergraduate physical activity and depressive symptoms: a national study. Am. J. Health Behav. 36 (2012), 230–241.
24. Endrighi, R., Steptoe, A., Hamer, M., The effect of experimentally induced sedentariness on mood and psychobiological responses to mental stress. Br. J. Psychiatry 208 (2016), 245–251.
25. Firth, J., Cotter, J., Elliott, R., French, P., Yung, A., A systematic review and meta-analysis of exercise interventions in schizophrenia patients. Psychol. Med. 45 (2015), 1343–1361.
26. Galling, B., Roldán, A., Nielsen, R.E., Nielsen, J., Gerhard, T., Carbon, M., Stubbs, B., Vancampfort, D., De Hert, M., Olfson, M., Type 2 Diabetes mellitus in youth exposed to antipsychotics: a systematic review and meta-analysis. JAMA Psychiatry 73:3 (2016), 247–259.
27. Gutierrez, P.M., Davidson, C.L., Friese, A.H., Forster, J.E., Physical activity, suicide risk factors, and suicidal ideation in a veteran sample. Suicide Life-Threat. Behav. 46:3 (2015), 284–292.
28. Hallgren, M., Vancampfort, D., Giesen, E.S., Lundin, A., Stubbs, B., Exercise as treatment for alcohol use disorders: systematic review and meta-analysis. Br. J. Sports Med. 51:14 (2017), 1058–1064.
29. Hawton, K., Comabella, C.C., Haw, C., Saunders, K., Risk factors for suicide in individuals with depression: a systematic review. J. Affect. Disord. 147 (2013), 17–28.
30. Herzog, R., Álvarez-Pasquin, M.J., Díaz, C., Del Barrio, J.L., Estrada, J.M., Gil, Á., Are healthcare workers’ intentions to vaccinate related to their knowledge, beliefs and attitudes? A systematic review. BMC Public Health, 13, 2013, 154.
31. Higgins, J., 2011. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1. 0. The Cochrane Collaboration.
32. Hoying, J., Melnyk, B.M., COPE: a pilot study with urban-dwelling minority sixth-grade youth to improve physical activity and mental health outcomes. J. Sch. Nurs. 32 (2016), 347–356.
33. Jeong, K.C., Koo, K.-M., Kim, C.-J., An analysis on relationships among exercise participation, depression experience, and suicidal ideation of people with visual impairment. Indian J. Sci. Technol., 2016, 9.
34. Jia, C.-X., Li, S.-B., Han, M., Bo, Q.-G., Health-related factors and suicidal ideation in high school students in rural China. OMEGA J. Death Dying 73 (2016), 263–274.
35. Kang, H.-J., Stewart, R., Jeong, B.-O., Kim, S.-Y., Bae, K.-Y., Kim, S.-W., Kim, J.-M., Shin, I.-S., Yoon, J.-S., Suicidal ideation in elderly Korean population: a two-year longitudinal study. Int. Psychogeriatr. 26 (2014), 59–67.
36. Klonsky, E.D., May, A.M., Saffer, B.Y., Suicide, suicide attempts, and suicidal ideation. Annu. Rev. Clin. Psychol. 12 (2016), 307–330.
37. Kredlow, M.A., Capozzoli, M.C., Hearon, B.A., Calkins, A.W., Otto, M.W., The effects of physical activity on sleep: a meta-analytic review. J. Behav. Med. 38 (2015), 427–449.
38. Krysinska, K., Lester, D., Post-traumatic stress disorder and suicide risk: a systematic review. Arch. Suicide Res. 14 (2010), 1–23.
39. Lee, C.G., Cho, Y., Yoo, S., The relations of suicidal ideation and attempts with physical activity among Korean adolescents. J. Phys. Act. Health 10 (2013), 716–726.
40. Liu, M., Wu, L., Ming, Q., How does physical activity intervention improve self-esteem and self-concept in children and adolescents? Evidence from a meta-analysis. PLoS One, 10, 2015, e0134804.
41. MacKinnon, N., Colman, I., Factors associated with suicidal thought and help-seeking behaviour in transition-aged youth versus adults. Can. J. Psychiatry 61 (2016), 789–796.
42. Mayes, S.D., Calhoun, S.L., Baweja, R., Mahr, F., Suicide ideation and attempts in children with psychiatric disorders and typical development. Crisis 36:1 (2015), 55–60.
43. Melancon, M.O., Lorrain, D., Dionne, I.J., Changes in markers of brain serotonin activity in response to chronic exercise in senior men. Appl. Physiol. Nutr. Metab. 39 (2014), 1250–1256.
44. Min, K.-B., Kim, H.-J., Kim, H.-J., Min, J.-Y., Parks and green areas and the risk for depression and suicidal indicators. Int. J. Public Health, 2017, 1–10.
45. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med., 6, 2009, e1000097.
46. Nevalainen, O., Simola, M., Ansakorpi, H., Raitanen, J., Artama, M., Isojärvi, J., Auvinen, A., Epilepsy, excess deaths and years of life lost from external causes. Eur. J. Epidemiol. 31 (2016), 445–453.
47. Nock, M.K., Borges, G., Bromet, E.J., Alonso, J., Angermeyer, M., Beautrais, A., Bruffaerts, R., Chiu, W.T., De Girolamo, G., Gluzman, S., Cross-national prevalence and risk factors for suicidal ideation, plans and attempts. Br. J. Psychiatry 192 (2008), 98–105.
48. Nock, M.K., Green, J.G., Hwang, I., McLaughlin, K.A., Sampson, N.A., Zaslavsky, A.M., Kessler, R.C., Prevalence, correlates, and treatment of lifetime suicidal behavior among adolescents: results from the National Comorbidity Survey Replication Adolescent Supplement. JAMA Psychiatry 70 (2013), 300–310.
49. Pedersen, B., Saltin, B., Exercise as medicine–evidence for prescribing exercise as therapy in 26 different chronic diseases. Scand. J. Med. Sci. Sports 25 (2015), 1–72.
50. Petersen, I., Evans-Lacko, S., Semrau, M., Barry, M.M., Chisholm, D., Gronholm, P., Egbe, C.O., Thornicroft, G., Promotion, prevention and protection: interventions at the population-and community-levels for mental, neurological and substance use disorders in low-and middle-income countries. Int. J. Ment. Health Syst., 10, 2016, 30.
51. Radovic, S., Gordon, M.S., Melvin, G.A., Should we recommend exercise to adolescents with depressive symptoms? A meta‐analysis. J. Paediatr. Child Health 53:3 (2017), 214–220.
52. Ro, J., Park, J., Lee, J., Jung, H., Factors that affect suicidal attempt risk among Korean elderly adults: a path analysis. J. Prev. Med. Public Health, 48, 2015, 28.
53. Rosenbaum, S., Tiedemann, A., Sherrington, C., Curtis, J., Ward, P.B., Physical activity interventions for people with mental illness: a systematic review and meta-analysis. J. Clin. Psychiatry 75 (2014), 964–974.
54. Rosenbaum, S., Vancampfort, D., Steel, Z., Newby, J., Ward, P.B., Stubbs, B., Physical activity in the treatment of post-traumatic stress disorder: a systematic review and meta-analysis. Psychiatry Res. 230 (2015), 130–136.
55. Sallis, J.F., Prochaska, J.J., Taylor, W.C., A review of correlates of physical activity of children and adolescents. Med. Sci. Sports Exerc. 32 (2000), 963–975.
56. Schuch, F., Vancampfort, D., Firth, J., Rosenbaum, S., Ward, P., Reichert, T., Bagatini, N.C., Bgeginski, R., Stubbs, B., Physical activity and sedentary behavior in people with major depressive disorder: a systematic review and meta-analysis. J. Affect. Disord. 210 (2017), 139–150.
57. Schuch, F.B., Vancampfort, D., Richards, J., Rosenbaum, S., Ward, P.B., Stubbs, B., Exercise as a treatment for depression: a meta-analysis adjusting for publication bias. J. Psychiatr. Res. 77 (2016), 42–51.
58. Sibold, J., Edwards, E., Murray-Close, D., Hudziak, J.J., Physical activity, sadness, and suicidality in bullied US adolescents. J. Am. Acad. Child Adolesc. Psychiatry 54 (2015), 808–815.
59. Song, H.-B., Lee, S.-A., Socioeconomic and lifestyle factors as risks for suicidal behavior among Korean adults. J. Affect. Disord. 197 (2016), 21–28.
60. Southerland, J.L., Zheng, S., Dula, M., Cao, Y., Slawson, D.L., Relationship between physical activity and suicidal behaviors among 65,182 middle school students. J. Phys. Act. Health 13 (2016), 809–815.
61. Stang, A., Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur. J. Epidemiol. 25 (2010), 603–605.
62. Strid, C., Andersson, C., Forsell, Y., Öjehagen, A., Lundh, L.G., Internet‐based cognitive behaviour therapy and physical exercise–effects studied by automated telephone assessments in mental ill‐health patients; a randomized controlled trial. Br. J. Clin. Psychol. 55 (2016), 414–428.
63. Stroup, D.F., Berlin, J.A., Morton, S.C., Olkin, I., Williamson, G.D., Rennie, D., Moher, D., Becker, B.J., Sipe, T.A., Thacker, S.B., Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. J. Am. Med. Assoc. 283 (2000), 2008–2012.
64. Stubbs, B., The prevalence and odds of suicidal thoughts, behaviours and deaths among people with painful comorbidities: an updated meta-analysis accounting for publication bias. J. Psychiatr. Res., 72, 2016, 72.
65. Stubbs, B., Eggermont, L., Soundy, A., Probst, M., Vandenbulcke, M., Vancampfort, D., What are the factors associated with physical activity (PA) participation in community dwelling adults with dementia? A systematic review of PA correlates. Arch. Gerontol. Geriatr. 59 (2014), 195–203.
66. Stubbs, B., Hurley, M., Smith, T., What are the factors that influence physical activity participation in adults with knee and hip osteoarthritis? A systematic review of physical activity correlates. Clin. Rehabil. 29 (2015), 80–94.
67. Stubbs, B., Vancampfort, D., Rosenbaum, S., Ward, P.B., Richards, J., Soundy, A., Veronese, N., Solmi, M., Schuch, F.B., Dropout from exercise randomized controlled trials among people with depression: a meta-analysis and meta regression. J. Affect. Disord. 190 (2016), 457–466.
68. Stubbs, B., Wu, Y.-T., Prina, A.M., Leng, Y., Cosco, T.D., A population study of the association between sleep disturbance and suicidal behaviour in people with mental illness. J. Psychiatr. Res. 82 (2016), 149–154.
69. Stubbs, B., Vancampfort, D., Rosenbaum, S., Firth, J., Cosco, T., Veronese, N., Salum, G.A., Schuch, F.B., An examination of the anxiolytic effects of exercise for people with anxiety and stress-related disorders: a meta-analysis. Psychiatry Res. 249 (2017), 102–108.
70. Sturm, J., Plöderl, M., Fartacek, C., Kralovec, K., Neunhäuserer, D., Niederseer, D., Hitzl, W., Niebauer, J., Schiepek, G., Fartacek, R., Physical exercise through mountain hiking in high‐risk suicide patients. A randomized crossover trial. Acta Psychiatr. Scand. 126 (2012), 467–475.
71. Takada, M., Suzuki, A., Shima, S., Inoue, K., Kazukawa, S., Hojoh, M., Associations between lifestyle factors, working environment, depressive symptoms and suicidal ideation: a large-scale study in Japan. Ind. Health 47 (2009), 649–655.
72. Taliaferro, L.A., Rienzo, B.A., Pigg, R.M., Miller, M.D., Dodd, V.J., Associations between physical activity and reduced rates of hopelessness, depression, and suicidal behavior among college students. J. Am. Coll. Health 57 (2009), 427–436.
73. Tao, F.B., Xu, M.L., Kim, S.D., Sun, Y., Su, P.Y., Huang, K., Physical activity might not be the protective factor for health risk behaviours and psychopathological symptoms in adolescents. J. Paediatr. Child Health 43 (2007), 762–767.
74. Taylor, M.K., Beckerley, S.E., Henniger, N.E., Hernández, L.M., Larson, G.E., Granger, D.A., A genetic risk factor for major depression and suicidal ideation is mitigated by physical activity. Psychiatry Res. 249 (2017), 304–306.
75. Trost, S.G., Owen, N., Bauman, A.E., Sallis, J.F., Brown, W., Correlates of adults’ participation in physical activity: review and update. Med. Sci. Sports Exerc. 34:12 (2002), 1996–2001.
76. Turecki, G., Brent, D.A., Suicide and suicidal behaviour. Lancet 387 (2016), 1227–1239.
77. Unger, J.B., Physical activity, participation in team sports, and risk of suicidal behavior in adolescents. Am. J. Health Promot., 12, 1997 (90-90).
78. Valtonen, M., Laaksonen, D.E., Laukkanen, J., Tolmunen, T., Rauramaa, R., Viinamäki, H., Mursu, J., Savonen, K., Lakka, T.A., Niskanen, L., Sedentary lifestyle and emergence of hopelessness in middle-aged men. Eur. J. Cardiovasc. Prev. Rehabil. 17 (2010), 524–529.
79. Vancampfort, D., Knapen, J., Probst, M., Scheewe, T., Remans, R., De Hert, M., A systematic review of correlates of physical activity in patients with schizophrenia. Acta Psychiatr. Scand. 125 (2012), 352–362.
80. Vancampfort, D., Correll, C.U., Probst, M., Sienaert, P., Wyckaert, S., De Herdt, A., Knapen, J., De Wachter, D., De Hert, M., A review of physical activity correlates in patients with bipolar disorder. J. Affect. Disord. 145 (2013), 285–291.
81. Vancampfort, D., Vanderlinden, J., Stubbs, B., Soundy, A., Pieters, G., Hert, M.D., Probst, M., Physical activity correlates in persons with binge eating disorder: a systematic review. Eur. Eat. Disord. Rev. 22 (2014), 1–8.
82. Vancampfort, D., De Hert, M., Stubbs, B., Soundy, A., De Herdt, A., Detraux, J., Probst, M., A systematic review of physical activity correlates in alcohol use disorders. Arch. Psychiatr. Nurs. 29:4 (2015), 196–201.
83. Vancampfort, D., Stubbs, B., Mitchell, A.J., De Hert, M., Wampers, M., Ward, P.B., Rosenbaum, S., Correll, C.U., Risk of metabolic syndrome and its components in people with schizophrenia and related psychotic disorders, bipolar disorder and major depressive disorder: a systematic review and meta‐analysis. World Psychiatry 14 (2015), 339–347.
84. Vancampfort, D., Stubbs, B., Sienaert, P., Wyckaert, S., De Hert, M., Rosenbaum, S., Probst, M., What are the factors that influence physical activity participation in individuals with depression? A review of physical activity correlates from 59 studies. Psychiatr. Danub., 27, 2015, 210.
85. Vancampfort, D., Richards, J., Stubbs, B., Akello, G., Gbiri, C.A., Ward, P.B., Rosenbaum, S., Physical activity in people with PTSD: a systematic review of correlates. J. Phys. Act. Health 13:8 (2016), 910–918.
86. Vancampfort, D., Mugisha, J., Richards, J., Hert, M.D., Probst, M., Stubbs, B., Physical activity correlates in people living with HIV: a systematic review of 44 studies. Disabil. Rehabil., 2017, 10.1080/09638288.2017.1306587.
87. Vancampfort, D., Stubbs, B., De Hert, M., du Plessis, C., Gbiri, C.A.O., Kibet, J., Wanyonyi, N., Mugisha, J., A systematic review of physical activity policy recommendations and interventions for people with mental health problems in Sub-Saharan African countries. Pan Afr. Med. J., 2017, 26.
88. Wang, H., Dwyer-Lindgren, L., Lofgren, K.T., Rajaratnam, J.K., Marcus, J.R., Levin-Rector, A., Levitz, C.E., Lopez, A.D., Murray, C.J., Age-specific and sex-specific mortality in 187 countries, 1970–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 380 (2013), 2071–2094.
89. World Health Organization, The ICD-10 Classification of Mental and Behavioural Disorders – Diagnostic Criteria for Research. 1993, World Health Organization, Geneva.
90. World Health Organization, Suicide. 2017, World Health Organization, Geneva.
91. Zalsman, G., Hawton, K., Wasserman, D., van Heeringen, K., Arensman, E., Sarchiapone, M., Carli, V., Höschl, C., Barzilay, R., Balazs, J., Suicide prevention strategies revisited: 10-year systematic review. Lancet Psychiatry 3 (2016), 646–659.