Don't leave the data in the dark: Issues in digitizing print statistical publications

D-Lib Magazine

Volumn 12, Issue 1, 2006, Pages 29-36

  Linden, Julie ^a,b   Green, Ann ^a,b  

^a YALE UNIVERSITY   (United States)

^b Digital Life Cycle Research and Consulting   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 31144470797     PISSN: 10829873     EISSN: 10829873     Source Type: Journal    
DOI: 10.1045/january2006-linden     Document Type: Article

References (17)

1. "Supporting Economic Development Research: A Collaborative Project to Create Access to Statistical Sources Not Born Digital" (June 2003-April 2005). Project web site, which includes the prototype statistical digital library and links to presentations given at the Digital Library Federation Fall Forum and the IASSIST Conference: 〈http://ssrs.yale.edu/egcdl〉.
2. U.S. Census Bureau, Statistical Abstracts, 〈http://www.census.gov/prod/www/abs/statab.html〉.
3. FRASER® Federal Reserve Archival System for Economic Research, 〈http://fraser.stlouisfed.org/〉.
4. Jostein Ryssevik, The Data Documentation Initiative (DDI) metadata specification (n.d.), 〈http://www.icpsr.umich.edu/DDI/papers/ryssevik.pdf〉.
5. Data users need to know a dataset's context (methodology, sampling, etc), source, funding and authoring body, purpose, validity, accuracy, version, orientation in space and time, provenance, relationships to related variables, and potential applications.
6. U.S. Government Printing Office, FDsys Specification for Converted Content (Version 3.0) (June 2005), 〈http://www.gpoaccess.gov/legacy/FDsys_ccspecs.pdf〉, 4.
7. U.S. Government Printing Office, Council Discussion, Questions and Answers from the Fall Meeting 2004: Digitization of the Legacy Collection, 〈http://www.access.gpo.gov/su_docs/fdlp/pubs/proceedings/04cbt/ digitization_legacy_collection.pdf〉, 11.
8. AHDS History, Online Historical Population Reports Project, 〈http://histpop.org/〉. accessed October 26, 2005.
9. Instituto Nacional de Estadística, Geografía e Informática (INEGI), Anuarios Estadísticos de los Estados, 1994-2000. INEGI provides Excel tables from this series, back to 2002, on its web site: 〈http://www.inegi.gob.mx/inegi/〉, accessed November 30, 2005.
10. We selected PDF/A format in anticipation of its suitability for long-term preservation. "Among federal archivists and records managers, PDF-A is viewed as one of two leading data format candidates for preserving future access to electronic records and documents....The proposed PDF-A standard specifies what should be stored in an archived file by prohibiting, for example, proprietary encryption schemes and embedded files such as executable scripts." Florence Olsen, "Archive-friendly PDF in the works," Federal Computer Week (March 15, 2004), 〈htttp://www.fcw.com/fcw/articles/2004/0315/news-pdf-03-15-04.asr〉. The Library of Congress provides details about PDF/A, including evaluation of sustainability, quality, and functionality factors: Sustainability of Digital Formats: Planning for Library of Congress Collections: PDF/A, PDF for Long-term Preservation, 〈http://www.digitalpreservation.gov/formats/fdd/fdd000125.shtml〉, accessed December 9, 2005.
11. We limited spreadsheet production to this subset for budgetary reasons. Document Solutions, Inc. (DSI) used custom zoned scanning software to process an image file for a particular page and determine the boundaries of the individual table or tables on the page. The text was processed with optical character recognition (OCR) software. DSI staff reviewed and corrected suspected OCR errors and manually corrected the layout. We also contracted for checksum macros, and the accuracy rate for alpha and numeric characters was certified at least 99%.
12. DDI Version 2.1, 〈http://www.icpsr.umich.edu/DDI/users/dtd/index.html#version2.0〉.
13. A sample DDI record is available at: 〈http://ssrs.yale.edu/egcdl/xml/Aguascalientes/2000/ Aguascalientes_2000_03_11.xml〉. The same record, rendered with an XSL stylesheet, is available at: 〈http://webapp.icpsr.umich.edu/cocoon/DDI/SAMPLES /Aguascalientes_2000_03_011.xml〉.
14. Ronald Jantz and Michael J. Giarlo, "Digital Preservation: Architecture and Technology for Trusted Digital Repositories," D-Lib Magazine 11 no. 6 (June 2005), 〈doi: 10.1045/june2005-jantz〉.
15. Digital Library Federation, "Benchmark for Faithful Digital Reproductions of Monographs and Serials," Version 1 (December 2002), 〈http://purl.oclc.org/DLF/benchrepro0212〉, provides a definition and discussion of "faithful digital reproduction."
16. Ann Green, Sandra K. Peterson, and Julie Linden, "Supporting Economic Development Research: A Collaborative Project to Create Access to Statistical Sources Not Born Digital" (April 27, 2005), 〈http://ssrs.yale.edu/egcdl/Yale_EGCDL_report_0505.pdf〉, 10-16.
17. John McCarthy, "Scientific Information = Data + Meta-data," photocopy of draft to be published in Database Management: Proceedings of a Workshop November 1-2, 1984, held at the U.S. Navy Postgraduate School, Monterey, California (Department of Statistics Technical Report, Stanford University, 1985), 16.