Explore how Rubin Observatory data will be structured, how to model Rubin Observatory from sky to detector with the simulation framework, and how you can access and process Rubin Observatory/LSST data and simulations.
Rubin Observatory will deliver calibrated images and data products on a daily and annual basis, and will provide the science community with the Rubin Science Platform to enable data access and analysis. The annual data releases will include a reprocessing of all the LSST data acquired up until that year.
Short on time? Need a citable resource for your grant proposal? See this page for a short description or a short recorded presentation about the planned data products, their processing pipelines, and the resources for the science community.
For more information please see "The Rubin Data Products, Abridged" or the full Rubin "Data Products Definition Document".
The Data Management (DM) team periodically hosts DM Boot Camps. Tutorials and lectures are led by current DM team members and are geared towards the new scientists who have joined the Data Release Pipelines (DRP) team. More information can be found in this community post.
Highly sophisticated and open-source tools exist for you to simulate everything from how Rubin Observatory could make its observations over time, how light passes through the telescope system and spreads out on the detector, and what alerts and catalogs of astronomical sources will be created.
Catalogs | Images | Survey Strategy |
|---|---|---|
CatSim: the catalog simulator simulates the properties and distributions of stars, galaxies, and asteroids that Rubin Observatory expects to observe. |
ImSim: is an open source software package used to drive simulations of the LSST survey. imSim is used both for small scale studies in the Dark Energy Science Collaboration and Rubin Observatory, and also for large scale data challenges. |
OpSim: the operations simulator generates sequences of Rubin Observatory observations based on one or more science programs and the historical weather, accounting for the expected performance of the telescope and site. |
|
PhoSim: the photon simulator generates representative images of the sky as Rubin Observatory would observe it by raytracing photons through an atmosphere, telescope, and camera. |
MAF: the metrics analysis framework is an application used to analyze the outputs of the Operations Simulator to evaluate the science and technical performance of the Rubin Observatory LSST survey strategy. |
How can I access and process Rubin Observatory data and simulations?
All of the Rubin Observatory data for the LSST (raw and calibrated images, yearly data releases) will be archived in its entirety in two facilities on different continents: SLAC in the US and CC-IN2P3 in France. Users with data rights will access the survey releases and data products through tools provided by the Project, from a Data Access Center (DAC). Currently planned DACs are at SLAC, at CC-IN2P3, and in Chile; additional DACs are under consideration in other participating countries.
Many users will only desire the output catalogs provided in each data release.
DACs will provide access via QServ, which allows efficient querying of the petabyte-size catalogs.
http://slac.stanford.edu/exp/lsst/qserv/
https://ieeexplore.ieee.org/document/6114487/
The recommended way to access images is with Rubin Observatory's applications layer of the data processing software, which is entirely open-source. It has two components for users, together called the “DM stack”: a pipeline for data reduction, and tools for interacting with calibrated images and catalogues. More information on DM stack.
As part of the data releases, the reduction pipeline will be run by the Rubin Observatory's Project to create the data products, and users will only have to run it if they need more specific raw data processing. You can use the pipeline to process simulated images or observed images (e.g. from Subaru HSC) into catalogues: see the tutorial.
For interacting with calibrated data, the DM stack tools provide a Python-interface environment for further processing of data products and visualising data: see the tutorials.
Among the Science Collaborations, many working groups are developing software tailored to their specific needs.
The Rubin Observatory Project regularly simulates Rubin Observatory strategies through OpSim. The OpSim products are all available for download (as sql databases).
The DESC SC generates “mock universes” for the purpose of creating and testing Rubin Observatory LSST pipelines. The software used to generate these simulations is generally public on the DESC github repository.
