Unboxing cluster heatmaps

BMC Bioinformatics

Volumn 18, Issue , 2017, Pages

  Engle, Sophie ^a   Whalen, Sean ^b   Joshi, Alark ^a   Pollard, Katherine S ^b,c  

^a UNIVERSITY OF SAN FRANCISCO   (United States)

^b GLADSTONE INSTITUTES   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Bioinformatics visualization; Data clustering; Hierarchy data; Qualitative evaluation; Quantitative evaluation; Systems biology omics data

Indexed keywords

CLUSTERING ALGORITHMS; HIERARCHICAL SYSTEMS; SURVEYS; VISUALIZATION;

BIOINFORMATICS VISUALIZATIONS; DATA CLUSTERING; HIERARCHY DATA; QUALITATIVE EVALUATIONS; QUANTITATIVE EVALUATION; SYSTEMS BIOLOGY;

DATA VISUALIZATION;

BIOINFORMATICS; HUMAN; INTERVIEW; MAJOR CLINICAL STUDY; PHYSICIAN; QUALITATIVE ANALYSIS; SYSTEMS BIOLOGY; BIOLOGY; CLUSTER ANALYSIS; K-562 CELL LINE; TUMOR CELL LINE;

CELL LINE, TUMOR; CLUSTER ANALYSIS; COMPUTATIONAL BIOLOGY; HUMANS; K562 CELLS;

EID: 85013813125     PISSN: None     EISSN: 14712105     Source Type: Journal    
DOI: 10.1186/s12859-016-1442-6     Document Type: Article

References (66)

1. Wilkinson L, Friendly M. The history of the cluster heat map. Am Stat. 2009; 63(2):179-84. doi: http://dx.doi.org/10.1198/tas.2009.0033.
2. Weinstein JN. A postgenomic visual icon. Science. 2008; 319(5871):1772-3. doi: http://dx.doi.org/10.1126/science.1151888.
3. Ram O, Goren A, Amit I, Shoresh N, Yosef N, Ernst J, Kellis M, Gymrek M, Issner R, Coyne M, Durham T, Zhang X, Donaghey J, Epstein CB, Regev A, Bernstein BE. Combinatorial patterning of chromatin regulators uncovered by genome-wide location analysis in human cells. Cell. 2011; 147(7):1628-39. doi: http://dx.doi.org/10.1016/j.cell.2011.09.057.
4. Hoffman MM, Buske OJ, Wang J, Weng Z, Bilmes JA, Noble WS. Unsupervised pattern discovery in human chromatin structure through genomic segmentation. Nat Methods. 2012; 9(5):473-6. doi: http://dx.doi.org/10.1038/nmeth.1937.
5. Kim N, Park H, He N, Lee HY, Yoon S. QCanvas: An advanced tool for data clustering and visualization of genomics data. Genomics Informat. 2012; 10(4):263-5. doi: http://dx.doi.org/10.5808/GI.2012.10.4.263.
6. Wang X, Eijkemans MJC, Wallinga J, Biesbroek G, Trzciński K, Sanders EAM, Bogaert D. Multivariate approach for studying interactions between environmental variables and microbial communities. PLOS ONE. 2012; 7(11):50267. doi: http://dx.doi.org/10.1371/journal.pone.0050267.
7. Heinrich J, Vehlow C, Battke F, Jäger G, Weiskopf D, Nieselt K. iHAT: Interactive hierarchical aggregation table for genetic association data. BMC Bioinforma. 2012; 13(Suppl 8):2. doi: http://dx.doi.org/10.1186/1471-2105-13-S8-S2.
8. Streit M, Gratzl S, Gillhofer M, Mayr A, Mitterecker A, Hochreiter S. Furby: Fuzzy force-directed bicluster visualization. BMC Bioinforma. 2014; 15(Suppl 6):4. doi: http://dx.doi.org/10.1186/1471-2105-15-S6-S4.
9. Hu Y, Yan C, Hsu CH, Chen QR, Niu K, Komatsoulis GA, Meerzaman D. OmicCircos: a simple-to-use R package for the circular visualization of multidimensional omics data. Cancer Informat. 2014; 13:13-20. doi: http://dx.doi.org/10.4137/CIN.S13495.
10. Zhao S, Guo Y, Sheng Q, Shyr Y. Advanced heat map and clustering analysis using Heatmap3. BioMed Res Int. 2014. doi: http://dx.doi.org/10.1155/2014/986048.
11. Gehlenborg N, Wong B. Points of view: heat maps. Nat Methods. 2012; 9(3):213-3. doi: http://dx.doi.org/10.1038/nmeth.1902.
12. Colet E, Aaronson D. Visualization of multivariate data: Human-factors considerations. Behav Res Methods Instrum Comput. 1995; 27(2):257-63. doi: http://dx.doi.org/10.3758/BF03204742.
13. van der Laan MJ, Pollard KS. Hybrid clustering of gene expression data with visualization and the bootstrap. J Stat Plan Infer. 2003; 117:275-303.
14. Bar-Joseph Z, Gifford DK, Jaakkola TS. Fast optimal leaf ordering for hierarchical clustering. Bioinformatics. 2001; 17(Suppl. 1):22-9. doi: http://dx.doi.org/10.1093/bioinformatics/17.suppl_1.S22.
15. Sakai R, Winand R, Verbeiren T, Moere A, Aerts J. dendsort: Modular leaf ordering methods for dendrogram representations in R. F1000Research. 2014; 3(177). doi: http://dx.doi.org/10.12688/f1000research.4784.1.
16. Heer J, Bostock M, Ogievetsky V. A tour through the visualization zoo. Commun ACM. 2010; 53(6):59-67. doi: http://dx.doi.org/10.1145/1743546.1743567.
17. Sakai R. gapmap: Functions for Drawing Gapped Cluster Heatmap with ggplot2. 2015. R package version 0.0.4. https://CRAN.Rproject.org/package=gapmap.
18. Vehlow C, Beck F, Weiskopf D. The state of the art in visualizing group structures in graphs. In: Proceedings of the 2015 Eurographics Conference on Visualization. Hoboken: Wiley: 2015.
19. Reingold EM, Tilford JS. Tidier drawings of trees. IEEE Trans Softw Eng. 1981; SE-7(2):223-8. doi: http://dx.doi.org/10.1109/TSE.1981.234519.
20. Dwyer T. Scalable, versatile and simple constrained graph layout. Comput Graph Forum. 2009; 28(3):991-8. doi: http://dx.doi.org/10.1111/j.1467-8659.2009.01449.x.
21. Stasko J, Zhang E. Focus+Context display and navigation techniques for enhancing radial, space-filling hierarchy visualizations. In: Proceedings of the 2000 IEEE Symposium on Information Visualization: 2000. p. 57-65. doi: http://dx.doi.org/10.1109/INFVIS.2000.885091.
22. Johnson B, Shneiderman B. Tree-maps: A space-filling approach to the visualization of hierarchical information structures. In: Proceedings of the 1991 IEEE Conference on Visualization: 1991. p. 284-91. doi: http://dx.doi.org/10.1109/VISUAL.1991.175815.
23. Bruls M, Huizing K, Van Wijk JJ. Squarified treemaps. In: Proceedings of the 2000 Eurographics Conference on Visualization. Hoboken: Wiley: 2000. p. 33-42.
24. Wang W, Wang H, Dai G, Wang H. Visualization of large hierarchical data by circle packing. In: Proceedings of the 2006 Conference on Human Factors in Computing Systems: 2006. p. 517-20. doi: http://dx.doi.org/10.1145/1124772.1124851.
25. Elmqvist N, Yi JS. Patterns for visualization evaluation. In: Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors - Novel Evaluation Methods for Visualization: 2012. doi: http://dx.doi.org/10.1145/2442576.2442588.
26. Hirsch O, Bösner S, Hüllermeier E, Senge R, Dembczynski K, Donner-Banzhoff N. Multivariate modeling to identify patterns in clinical data: The example of chest pain. BMC Med Res Methodol. 2011; 11(1):155. doi: http://dx.doi.org/10.1186/1471-2288-11-155.
27. Huber W, Carey VJ, Gentleman R, Anders S, Carlson M, Carvalho BS, Bravo HC, Davis S, Gatto L, Girke T, Others. Orchestrating high-throughput genomic analysis with Bioconductor. Nat Methods. 2015; 12(2):115-21. doi: http://dx.doi.org/10.1038/nmeth.3252.
28. Waskom M. Seaborn: Statistical Data Visualization. https://github.com/mwaskom/seaborn. Accessed 1 Jan 2016.
29. Shannon P, Markiel A, Ozier O, Baliga NS, Wang JT, Ramage D, Amin N, Schwikowski B, Ideker T. Cytoscape: A software environment for integrated models of biomolecular interaction networks. Genome Res. 2003; 13(11):2498-504. doi: http://dx.doi.org/10.1101/gr.1239303.
30. Gould J. GENE-E. http://www.broadinstitute.org/cancer/software/GENE-E. Accessed 1 Jan 2016.
31. Saldanha AJ. Java Treeview - extensible visualization of microarray data. Bioinformatics. 2004; 20(17):3246-248. doi: http://dx.doi.org/10.1093/bioinformatics/bth349.
32. Metsalu T, Vilo J. ClustVis: A web tool for visualizing clustering of multivariate data using principal component analysis and heatmap. Nucleic Acids Res. 2015; 43(W1):566-70. doi: http://dx.doi.org/10.1093/nar/gkv468.
33. Škuta C, Bartuněk P, Svozil D. InCHlib - interactive cluster heatmap for web applications. Journal of Cheminformatics. 2014; 6(44). doi: http://dx.doi.org/10.1186/s13321-014-0044-4.
34. Streit M, Lex A, Kalkusch M, Zatloukal K, Schmalstieg D. Caleydo: connecting pathways and gene expression. Bioinformatics. 2009; 25(20):2760-1. doi: http://dx.doi.org/10.1093/bioinformatics/btp432.
35. Burch M, Heinrich J, Konevtsova N, Höferlin M, Weiskopf D. Evaluation of traditional, orthogonal, and radial tree diagrams by an eye tracking study. IEEE Trans Vis Comput Graph. 2011; 17(12):2440-8. doi: http://dx.doi.org/10.1109/TVCG.2011.193.
36. Diehl S, Beck F, Burch M. Uncovering strengths and weaknesses of radial visualizations-an empirical approach. IEEE Trans Vis Comput Graph. 2010; 16(6):935-42. doi: http://dx.doi.org/10.1109/TVCG.2010.209.
37. Holten D, van Wijk JJ. A user study on visualizing directed edges in graphs. In: Proceedings of the 27th International Conference on Human Factors in Computing Systems: 2009. p. 2299-308. doi: http://dx.doi.org/10.1145/1518701.1519054. http://dl.acm.org/citation.cfm?id=1518701.1519054.
38. Kobsa A. User experiments with tree visualization systems. In: Proceedings of the 2004 IEEE Symposium on Information Visualization: 2004. p. 9-16. doi: http://dx.doi.org/10.1109/INFVIS.2004.70. http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=1382885.
39. Heer J, Bostock M. Crowdsourcing graphical perception: Using Mechanical Turk to assess visualization design. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems: 2010. p. 203-12. doi: http://dx.doi.org/10.1145/1753326.1753357. http://dl.acm.org/citation.cfm?id=1753326.1753357.
40. Lee B, Plaisant C, Parr CS, Fekete JD, Henry N. Task taxonomy for graph visualization. In: Proceedings of the 2006 Workshop on Beyond Time and Errors: Novel Evaluation Methods for Information Visualization: 2006. p. 1-5. doi: http://dx.doi.org/10.1145/1168149.1168168. http://dl.acm.org/citation.cfm?id=1168149.1168168.
41. Amar R, Eagan J, Stasko J. Low-level components of analytic activity in information visualization. In: Proceedings of the 2005 IEEE Symposium on Information Visualization: 2005. doi: http://dx.doi.org/10.1109/INFOVIS.2005.24. http://dl.acm.org/citation.cfm?id=1106328.1106582.
42. Saket B, Simonetto P, Kobourov SG. Group-level graph visualization taxonomy. In: Proceedings of the 2014 Eurographics Conference on Visualization. Hoboken: Wiley: 2014.
43. Shneiderman B. Tree visualization with tree-maps: 2-D space-filling approach. ACM Trans Graph. 1992; 11(1):92-9.
44. Stephenson K. Circle packing: a mathematical tale. Not AMS. 2003; 50(11):1376-88.
45. Ghoniem M, Fekete JD, Castagliola P. A comparison of the readability of graphs using node-link and matrix-based representations. In: Proceedings of the 2004 IEEE Symposium on Information Visualization: 2004. p. 17-24. doi: http://dx.doi.org/10.1109/INFVIS.2004.1. http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=1382886.
46. Jianu R, Rusu A, Taggart D. How to display group information on node-link diagrams: An evaluation. IEEE Trans Vis Comput Graph. 2014; 20(11):1530-41. doi: http://dx.doi.org/10.1109/TVCG.2014.2315995.
47. Collins C, Penn G, Carpendale S. Bubble Sets: Revealing set relations with isocontours over existing visualizations. IEEE Trans Vis Comput Graph. 2009; 15(6):1009-16. doi: http://dx.doi.org/10.1109/TVCG.2009.122.
48. Alper B, Riche NH, Ramos G, Czerwinski M. Design study of LineSets, a novel set visualization technique. IEEE Trans Vis Comput Graph. 2011; 17(12):2259-67. doi: http://dx.doi.org/10.1109/TVCG.2011.186.
49. Gansner ER, Hu Y, Kobourov S. GMap: Visualizing graphs and clusters as maps. In: Proceedings of the 2010 IEEE Pacific Visualization Symposium: 2010. p. 201-8. doi: http://dx.doi.org/10.1109/PACIFICVIS.2010.5429590. http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=5429590.
50. ENCODE Project Consortium. The ENCODE (ENCyclopedia Of DNA elements) project. Science. 2004; 306(5696):636-40. doi: http://dx.doi.org/10.1126/science.1105136.
51. Warnes GR, Bolker B, Bonebakker L, Gentleman R, Huber W, Liaw A, Lumley T, Maechler M, Magnusson A, Moeller S, Schwartz M, Venables B. gplots: Various R Programming Tools for Plotting Data. 2015. R package version 2.16.0. http://CRAN.R-project.org/package=gplots. Accessed 22 Dec 2016.
52. Wickham H, Francois R. dplyr: A Grammar of Data Manipulation. 2015. R Package Version 0.4.3. https://CRAN.R-project.org/package=dplyr. Accessed 22 Dec 2016.
53. Wickham H. Reshaping data with the reshape package. J Stat Softw. 2007; 21(12):1-20.
54. Lang DT. RJSONIO: Serialize R Objects to JSON, JavaScript Object Notation. 2014. R package version 1.3-0. https://CRAN.R-project.org/package=RJSONIO. Accessed 22 Dec 2016.
55. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E. Scikit-learn: machine learning in python. J Mach Learn Res. 2011; 12(Oct):2825-830.
56. Guyon I. Design of Experiments for the NIPS 2003 Variable Selection Benchmark. 2003. http://clopinet.com/isabelle/Projects/NIPS2003/Slides/NIPS2003-Datasets.pdf.
57. Jones E, Oliphant T, Peterson P, Others. SciPy: Open Source Scientific Tools for Python. 2001. http://www.scipy.org/. Accessed 22 Dec 2016.
58. Hagberg AA, Schult DA, Swart PJ. Exploring network structure, dynamics, and function using NetworkX. In: Proceedings of the 7th Python in Science Conference (SciPy2008). Pasadena: 2008. p. 11-15. http://conference.scipy.org/proceedings/scipy2008/paper_2/.
59. R Core Team. R: A Language and Environment for Statistical Computing. 2014. http://www.r-project.org. Accessed 22 Dec 2016.
60. Bostock M, Ogievetsky V, Heer J. D3: Data-driven documents. IEEE Trans Vis Comput Graph. 2011; 17(12):2301-309. doi: http://dx.doi.org/10.1109/TVCG.2011.185.
61. Mason W, Suri S. Conducting behavioral research on Amazon's Mechanical Turk. Behav Res Methods. 2012; 44(1):1-23. doi: http://dx.doi.org/10.3758/s13428-011-0124-6.
62. McKinney W. Data structures for statistical computing in Python In: van der Walt S, Millman J, editors. Proceedings of the 9th Python in Science Conference: 2010. p. 51-6. http://conference.scipy.org/proceedings/scipy2010/mckinney.html.
63. Holme P, Huss M, Lee SH. Atmospheric reaction systems as null-models to identify structural traces of evolution in metabolism. PLoS ONE. 2011; 6(5):1-8. doi: http://dx.doi.org/10.1371/journal.pone.0019759.
64. Knight CG, Pinney JW. Making the right connections: Biological networks in the light of evolution. Bioessays. 2009; 31(10):1080-90. doi: http://dx.doi.org/10.1002/bies.200900043.
65. Mackinlay J. Automating the design of graphical presentations of relational information. ACM Trans Graph. 1986; 5(2):110-41. doi: http://dx.doi.org/10.1145/22949.22950.
66. Harrower M, Brewer CA. ColorBrewer.org: An Online Tool for Selecting Colour Schemes for Maps. Chichester: Wiley; 2011, pp. 261-8. doi: http://dx.doi.org/10.1002/9780470979587.ch34. http://dx.doi.org/10.1002/9780470979587.ch34.