IsisCB Explore

Article ID: CBB221447639

Challenges of Large-Scale Data Processing in the 1990s: The IPUMS Experience (2022)

Diana L Magnuson (Author)
Steven Ruggles (Author)

IEEE Annals of the History of Computing
Volume: 44
Issue: 04
Pages: 71-83
Publication date: 2022
Language: English

URI: https://www.computer.org/csdl/magazine/an/2022/04/09972862/1ISVQV7hudq

DOI: 10.1109/MAHC.2022.3214736

Outside Links

WorldCat

Google Scholar

JSTOR

IPUMS Demography; population research Social sciences Technological innovation Computers and computing Data processing 20th century, late Minnesota (U.S.) University of Minnesota

When it was launched in 1991, the Integrated Public Use Microdata Series (IPUMS) project faced a challenging environment and limited resources. Few datasets were interoperable and much data collected at great public expense was inaccessible to most researchers. Documentation of datasets was nonstandardized, incomplete, and inadequate for automated processing. With insufficient attention to preservation, valuable scientific data were disappearing (see Bogue et al., 1976). IPUMS was established to address these critical issues. At the outset, IPUMS faced daunting barriers of inadequate data processing, storage, and network capacity. This anecdote describes the improvised computational infrastructure developed in the decade from 1989 to 1999 to process, manage, and disseminate the world's largest population datasets. We use a combination of archival sources, interviews, and our own memories to trace the development of the IPUMS computing environment during a period of explosive technical innovation. The development of IPUMS is part of a larger story of the development of social science infrastructure in the late 20th century and its contribution to democratizing data access.

...More

Included in

Article Emily Klancher Merchant; Myron P. Gutmann (2022) The IT of Demography. IEEE Annals of the History of Computing (pp. 6-15). unapi

Citation URI

data.isiscb.org/p/isis/citation/CBB221447639

This citation is part of the Isis database.

Citations Indexes

Similar Citations

Article Sean F. Johnston; (March 2018)
The Technological Fix as Social Cure-All: Origins and Implications (/p/isis/citation/CBB451903772/)

Book Barbara Coffin; Don Luce; Lansing Shepard; Gwen Schagrin; (2022)
A Natural Curiosity: The Story of the Bell Museum (/p/isis/citation/CBB594027695/)

Article George Borg; (2023)
Discoverer and Methodologist Alfred O. C. Nier and the Mid-Twentieth-Century Instrumental Revolution in Geochemistry (/p/isis/citation/CBB276066311/)

Book Misa, Thomas J.; Seidel, Robert W.; (2010)
College of Science and Engineering: The Institute of Technology Years (1935--2010) (/p/isis/citation/CBB001251430/)

Book Todd Tucker; (2007)
The Great Starvation Experiment: Ancel Keys and the Men Who Starved for Science (/p/isis/citation/CBB809511253/)

Thesis Holtan, Neal Ross; (2011)
From Eugenics to Public Health Genetics in Mid-Twentieth Century Minnesota (/p/isis/citation/CBB001562725/)

Article Philip L. Frana; (2023)
Demographics, Inc., Computerized Direct Mail, and the Rise of the Digital Attention Economy (/p/isis/citation/CBB797292983/)

Book Thomas J. Misa; (2013)
Digital State: The Story of Minnesota's Computing Industry (/p/isis/citation/CBB673942238/)

Article Cortada, James W.; (2001)
Using Textual Demographics to Understand Computer Use: 1950-1990 (/p/isis/citation/CBB000102444/)

Book Halfon, Saul; (2006)
The Cairo Consensus: Demographic Surveys, Women's Empowerment, and Regime Change in Population Policy (/p/isis/citation/CBB000772375/)

Article Michal Kravel-Tovi; (2020)
The Specter of Dwindling Numbers: Population Quantity and Jewish Biopolitics in the United States (/p/isis/citation/CBB978239418/)

Article Kruke, Anja; Woyke, Meik; (2010)
Editorial (/p/isis/citation/CBB001023652/)

Article Fenwick McKelvey; (2022)
When the New Magic was New: The Claritas Corporation and the Clustering of America (/p/isis/citation/CBB518108478/)

Book Fiona Farr; Murray, Liam; (2016)
The Routledge Handbook of Language Learning and Technology (/p/isis/citation/CBB304460544/)

Book Cortada, James W.; (2006)
The digital hand: Volume II, How computers changed the work of American financial, telecommunications, media, and entertainment industries (/p/isis/citation/CBB001180893/)

Book Samuel Greengard; (2021)
The Internet of Things (/p/isis/citation/CBB874725102/)

Article Fevolden, Arne Martin; (2013)
The Best of Both Worlds? A History of Time-Shared Microcomputers, 1977--1983 (/p/isis/citation/CBB001212988/)

Book Sito, Tom; (2013)
Moving Innovation: A History of Computer Animation (/p/isis/citation/CBB001214442/)

Book Carl Therrien; (2019)
The Media Snatcher: PC/CORE/TURBO/ENGINE/GRAFX/16/CDROM2/SUPER/DUO/ARCADE/RX (/p/isis/citation/CBB709874682/)

Book Gerard Alberts; Jan Friso Groote; (2023)
Tales of Electrologica: Computers, Software and People (/p/isis/citation/CBB540613020/)