A challenge for the LSST is to recognize important transient events—in real time—in a scene full of normal variations. On the one hand, there will be too much data for hands- or eyes-on analysis by scientists, no matter how skilled. On the other hand, there are so many different types of transient activity that writing specific programs to search for each one is far too likely to miss some very interesting and valuable phenomena. In addition, we know far too little about the variable Universe to predict, and thus be able to look for, important changes.
We are examining the potential of modern machine learning (ML) techniques for solving this problem. In particular, we are applying ML techniques for automated anomaly detection to the job of identifying transients without an existing description. Many anomalies will be instrumentation errors, also called artifacts. Automating their identification will allow prompt action to maintain LSST data quality. But some of the anomalies are likely to be things that we have not yet thought of.
Supervised Machine Learning
It's easier to show a machine what to find ...
-Machine Learning derives classification algorithms directly from examples of data
... than to tell a machine how to find it
-Requires domain expertise
-Involves software development
-Demands careful attention to statistical characterization
-Entails substantial amount of trial and error
The LSST will find normal source variations and instrumental artifacts in every image. The key to success will be real-time identification of important variations in a "forest" of normal ones. To efficiently identify the truly new and unique events the LSST will require multiple approaches to automated transient detection. For example, integrating machine learning techniques and context information provided by virtual observatories with the real-time analysis pipeline will be essential for identifying fast transients while they are still present.
An advantage of using ML techniques like anomaly detection algorithms is that they can find transients with properties we have not previously considered. ML techniques also allow the system to be trained, both by mining its own data and by interacting with human analysts. This will give the system an ability to bootstrap its capabilities by ignoring artifacts "like that" or by finding more "like this" without generating new hard-wired code. ML techniques can therefore enable the efficient construction of queries for the LSST to act as an autonomous discovery engine that searches the night sky.
STIS image and light curve of GRB990123. It peaked at brighter than 9th magnitude in the optical, easily visible with a small telescope—despite having a cosmological redshift, which means it is so far away we must include the expansion of space itself when calculating the speed at which it's receding from us. That makes it the most energetic event yet detected and poses a severe challenge to theorists who model these events. The technology now exists to monitor the sky in the optical for rare faint transients of all types—a million times fainter. (Image courtesy Bloom et al., Astrophysical Journal (Letters).)
The detection of transient emission provides a window on diverse astrophysical objects, from variable stars to stellar explosions to the mergers of compact stellar remnants. Perhaps even more exciting is the potential for discovering new, unanticipated phenomena. A possible example is a few short lived optical bursts seemingly without a detectable source that have already been seen in supernova surveys and by the Deep Lens Survey. LSST will obtain deeper and better data on tens of thousands of such events.
The rightmost image is an optical burst that is revealed by comparing two Deep Lens Survey images of the same place in the sky taken at different times (left and center images).
