← Back to rubinobservatory.org

Understanding Data and Simulations

Explore how Rubin Observatory data will be structured, how to model Rubin Observatory from sky to detector with the simulation framework, and how you can access and process Rubin Observatory/LSST data and simulations.


What will Rubin Observatory data look like?

Rubin Observatory will deliver calibrated images and data products on a daily and annual basis, and will provide the science community with the Rubin Science Platform to enable data access and analysis. The annual data releases will include a reprocessing of all the LSST data acquired up until that year.

The Rubin Data Products, Abridged (with DOI)

Short on time? Need a citable resource for your grant proposal? See this page for a short description or a short recorded presentation about the planned data products, their processing pipelines, and the resources for the science community. 

Prompt data products (for transients, variables, and moving objects):
  • Alert packets for sources detected in difference images (difference = direct-template), delivered to brokers (60 seconds).
  • Catalogs of sources (associated by location into "objects") detected in difference images (24 hours).
  • Catalogs of moving objects with orbital parameters from the MPC (a result of daytime processing).
  • Direct and difference images (80 hours).
Data Releases data products (for static sky sources like stars and galaxies):
  • A global and uniform processing of all the data taken from the start of the survey until a given date (typically 6 months before the release).
  • Raw and calibrated images, the calibration data, and deeply stacked coadded images.
  • Source and object catalogs with measurements (centroids, fluxes, magnitudes, shape and size parameters, and more).
  • Includes re-processed Prompt data products and direct image light curves.

For more information please see "The Rubin Data Products, Abridged" or the full Rubin "Data Products Definition Document".

Data Management BootCamps

The Data Management (DM) team periodically hosts DM Boot Camps.  Tutorials and lectures are led by current DM team members and are geared towards the new scientists who have joined the Data Release Pipelines (DRP) team. More information can be found in this community post.

 

Modelling Rubin Observatory from sky to detector: the simulation framework

Highly sophisticated and open-source tools exist for you to simulate everything from how Rubin Observatory could make its observations over time, how light passes through the telescope system and spreads out on the detector, and what alerts and catalogs of astronomical sources will be created.

Catalogs
Images
Survey Strategy

CatSim: the catalog simulator simulates the properties and distributions of stars, galaxies, and asteroids that Rubin Observatory expects to observe.

ImSim: is an open source software package used to drive simulations of the LSST survey.  imSim is used both for small scale studies in the Dark Energy Science Collaboration and Rubin Observatory, and also for large scale data challenges.

OpSim: the operations simulator generates sequences of Rubin Observatory observations based on one or more science programs and the historical weather, accounting  for the expected performance of the telescope and site.

 

PhoSim: the photon simulator generates representative images of the sky as Rubin Observatory would observe it by raytracing photons through an atmosphere, telescope, and camera.

MAF: the metrics analysis framework is an application used to analyze the outputs of the Operations Simulator to evaluate the science and technical performance of the Rubin Observatory LSST survey strategy.

 

How can I access and process Rubin Observatory data and simulations?

Data

All of the Rubin Observatory data for the LSST (raw and calibrated images, yearly data releases) will be archived in its entirety in two facilities on different continents: SLAC in the US and CC-IN2P3 in France. Users with data rights will access the survey releases and data products through tools provided by the Project, from a Data Access Center (DAC). Currently planned DACs are at SLAC, at CC-IN2P3, and in Chile; additional DACs are under consideration in other participating countries.

Database queries for catalogs

Many users will only desire the output catalogs provided in each data release.

DACs will provide access via QServ, which allows efficient querying of the petabyte-size catalogs.
http://slac.stanford.edu/exp/lsst/qserv/ 
https://ieeexplore.ieee.org/document/6114487/

Accessing raw or calibrated images

The recommended way to access images is with Rubin Observatory's applications layer of the data processing software, which is entirely open-source. It has two components for users, together called the “DM stack”: a pipeline for data reduction, and tools for interacting with calibrated images and catalogues.  More information on DM stack.

As part of the data releases, the reduction pipeline will be run by the Rubin Observatory's Project to create the data products, and users will only have to run it if they need more specific raw data processing. You can use the pipeline to process simulated images or observed images (e.g. from Subaru HSC) into catalogues: see the tutorial.

For interacting with calibrated data, the DM stack tools provide a Python-interface environment for further processing of data products and visualising data: see the tutorials.

Working group specific tools

Among the Science Collaborations, many working groups are developing software tailored to their specific needs. 

Simulations

The Rubin Observatory Project regularly simulates Rubin Observatory strategies through OpSim. The OpSim products are all available for download (as sql databases).

The DESC SC generates “mock universes” for the purpose of creating and testing Rubin Observatory LSST pipelines. The software used to generate these simulations is generally public on the DESC github repository.

Financial support for Rubin Observatory comes from the National Science Foundation (NSF) through Cooperative Agreement No. 1258333, the Department of Energy (DOE) Office of Science under Contract No. DE-AC02-76SF00515, and private funding raised by the LSST Corporation. The NSF-funded Rubin Observatory Project Office for construction was established as an operating center under management of the Association of Universities for Research in Astronomy (AURA).  The DOE-funded effort to build the Rubin Observatory LSST Camera (LSSTCam) is managed by the SLAC National Accelerator Laboratory (SLAC).
The National Science Foundation (NSF) is an independent federal agency created by Congress in 1950 to promote the progress of science. NSF supports basic research and people to create knowledge that transforms the future.
NSF and DOE will continue to support Rubin Observatory in its Operations phase. They will also provide support for scientific research with LSST data.   




Contact   |   We are Hiring

Admin Login

Back to Top