Data Management System Requirements

The Rubin Observatory Data Management System (DMS) must 

• Reliably process unprecedented data volumes
• Ensure consistent data quality without manual intervention
• Meet stringent near-real-time transient alerting deadlines
• Accommodate both scientific and computing technology evolution over at least a decade
• Serve Rubin Observatory data products to a diverse community of users located across several continents.

The Rubin Observatory DMS is composed of a set of products, each of which is made up of a series of pipelines, a large archive of images, and a number of catalogs containing the detected astronomical sources and resolved astronomical objects. Underneath these are the software middleware and technology infrastructure that permit the visible elements to work securely, reliably, and scalably. The processing and data are distributed across multiple computing centers on the observatory mountaintop, in a base facility near the observatory at La Serena, Chile, at an archive center at the University of Illinois National Center for Supercomputing Applications in Urbana-Champaign, Illinois, and multiple Data Access Centers.

The results of a processing run form a data release, which is a static, self-consistent data set for use in performing scientific analysis of Rubin Observatory data and publication of the results. Periodically, new calibration data products are created, such as bias frames and flat fields (needed to remove sensor "noise" from exposures) that will be used by the other processing functions. All Rubin Observatory data must be made available through an interface that uses, to the maximum possible extent, community-based standards such as those being developed by the Virtual Observatory.