CIS 512 Case Study 2 How Will Astronomy Archives Survive the Data Tsunami

In it, we describe the practices that can keep astronomy archives ahead of the tsunami, which will deposit an estimated 60-100 PB of data on-line by 2020. We describe how archive performance is already being degraded by the rapid growth and data volume and growing use of sophisticated queries - see the figure below. We describe how practices such as rigorous evaluation of new technologies, partnerships between astronomers and computer scientists, and training of scientists in high-end software engineering engineering skills will all be necessary. Astronomy is already awash with data: currently 1 PB (petabyte) of public data is electronically accessible, and this volume is growing at 0.5 PB per year. The...