Opening a Window of Discovery on the Dynamic Universe

Technology Innovation

Astronomy is undergoing a revolution in the way we probe the universe and the way we answer fundamental questions. New technology enables this: novel detectors are opening new windows on the universe, creating unprecedented volumes of high-quality data, and computing technology is keeping up with this explosion and driving a shift in the way science is produced in astronomy and astrophysics. Huge surveys of the sky over many wavelengths of light can be analyzed statistically for hidden correlations and explanations for puzzling observational data found using the technique of statistical inversion to aid in better understanding of the underlying physics. LSST is the lighthouse project in this revolution, and solutions to LSST’s challenges are already having spin-off effects in broader areas of technology and “big data” science.

Realizing the vision of LSST requires facing, and solving extraordinary engineering and technological challenges: the fabrication of large, high-precision non-spherical optics; construction of a huge, highly-integrated array of sensitive, wide-band imaging sensors; and the operation of a data management facility handling tens of terabytes of data each day. The design and development effort includes structural, thermal, and optical analyses of all key hardware subsystems, prototyping and development of data management systems, and extensive systems engineering studies. To validate system performance, full end-to-end simulations are being done, with more than 100 technical personnel at a range of institutions currently engaged in this program. 

LSST R&D has led to a new-generation imaging CCD which is highly segmented, low noise, and sensitive from the UV to the near IR. The speed with which LSST can cover half the sky will produce about 15 terabytes (TB) per night, leading to a total database over the ten years of operations of order 50 petabytes (PB) for the raw data, and 15 PB for the catalog database. The total data volume after processing will be over 100 PB, processed using 250 trillion floating-point operations per second of computing power. Processing such a large volume of data, converting the raw images into a faithful representation of the universe, automating data quality assessment, and archiving the results in useful form for a broad community of users is a major challenge.

The acquisition of scientific data in all disciplines is now accelerating, causing a nearly insurmountable data avalanche. It is no longer possible for humans to look at any representative fraction of the data. Instead, we may be looking over the shoulders of assisted learning machines at innovative visualizations of metadata. Discoveries will be made via searches for correlations. The role of the experimental scientist increasingly is as inventor of ambitious new searches and new algorithms. Novel theories of nature are tested through searching for predicted statistical relationships across big databases. 

In this era of big data, we will require novel, increasingly automated, and increasingly more effective ways to fish scientific knowledge from oceans of bytes.

The LSST scientific database will include:
  • A source catalog with 350 billion rows
  • An object catalog with 37 billion rows, each with 200+ attributes
  • A moving object catalog with 6 million rows
  • An alerts database, with alerts issued worldwide within 60 seconds, and
  • Calibration, configuration, processing, and provenance metadata.

The science archive will consist of 400,000 sixteen-megapixel images per night (for 10 years), comprising 60 PB of pixel data. This enormous LSST data archive and object database enables a diverse multidisciplinary research program: astronomy and astrophysics; machine learning (data mining); exploratory data analysis; extremely large databases; scientific visualization; computational science and distributed computing; and inquiry-based science education (using data in the classroom). The advances in these technology areas will be exported to other big data science applications (biology, remote sensing, etc.) and will drive innovations in industry. Already a collaboration is forming between industry and LSST on the design of extremely large databases.

 
Image Credit: 
iStock

Financial support for LSST comes from the National Science Foundation (NSF) through Cooperative Agreement No. 1258333, the Department of Energy (DOE) Office of Science under Contract No. DE-AC02-76SF00515, and private funding raised by the LSST Corporation. The NSF-funded LSST Project Office for construction was established as an operating center under management of the Association of Universities for Research in Astronomy (AURA).  The DOE-funded effort to build the LSST camera is managed by the SLAC National Accelerator Laboratory (SLAC). 


Contact   |   We are Hiring   |   Business with LSST

Admin Login

Back to Top