In-Class Assignment 24

In-Class Assignment 24#

Transient Searches with ZTF data#

Learning Objectives#

  • Develop an understanding of ZTF datasets.

  • Methods for querying large ZTF datasets.

  • Plot lightcurve data of ZTF transients.

  • Identify qualitative differences in lightcurve properties from different sources.

Credit: Author: Igor Andreoni
Contact: andreoni@umd.edu
ZTF Summer School 2022

Introduction#

Wide-field surveys like the Zwicky Transient Facility (ZTF) image the entire observable sky almost every night multiple times. This makes ZTF as excellent instrument for time-domain astronomy. Image subtraction between a new “science” image and a deep “reference” (or “template”) image makes every new source, or every source that changes in luminosity, stand out. When such a source is identified in a new image, an alert packet is issued, which is a dictionary rich with information, including differential photometry between the science and the template images.

ZTF issues about 300,000 alerts every night. Searching for extragalactic fast transients constitutes a “needle in the haystack” big data challenge.

The data set#

Two data files were prepared for this school activity, both in JSON format. JSON files are, essentially, dictionaries.

The alerts were all issued on the night of 2021-02-05 UT. The total number of alerts issued on that night approximated 1 million. If all the alerts were to be used, complete with all their entries, they total disk space for the 2021-02-05 night is larger than 16GB. To facilitate the download and handling of the data, alerts were selected that:

  • have at least 2 detections for the source (ndethist >= 2)

  • are likely real according to two real/bogus classifiers (drb > 0.8; braai > 0.8)

  • left a positive residual after image subtraction, i.e. the flux in the science image is larger than in the template image (warning: the source might have been fainter than the template in a past science image, generating a “negative” subtraction!)

The data files are data/fast_transient_alerts.json for the alerts, data/fast_transient_lc.json for the light curves.

  • data/fast_transient_alerts.json Uniform JSON file (readable as a table using pandas) containing a selection of relevant information from the original alerts.

  • data/fast_transient_lc.json Light curves. The light curve of some transients have thousands of data points. To keep the data set manageable and our eyes on the scientific objective to discover fast transients, some the light curves were cut. In particular, empty light curves were assigned to those transients that:

    • have at least one “negative” subtraction in the past (see above)

    • are located at Galactic latitude -8 deg < b < +8 deg
      In addition, only those data points within the last 30 days before the alert was issued are present in the light curves. Long-duration transients, variables, and repeatingly bursting sources are outside the scope of this activity. However, data points acquired after the last alert included in the fast_transient_alerts.json file will be present.

Requirements#

  • python 3

  • numpy, pandas, matplotlib, astropy, collections

Import python packages#

import json

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from collections import OrderedDict

a- Read and understand the data#

Download the necessary data files here and here.

  1. load the alerts and lightcurve data and follow the questions in the cell below

Read the data file. Beware that this action could take several minutes. The alerts JSON file is uniform and can be read as a pandas data frame.

# Open the file with the alerts
#alerts_filename = "./fast_transient_alerts.json"
#df_alerts = pd.read_json(alerts_filename)
# To have one alert per row instead of column, use transpose()
#df_alerts = df_alerts.transpose()

# Open the file with the light curves
#lc_filename = "./fast_transient_lc.json"
  
# Opening JSON file
#f = open(lc_filename)
  
# returns JSON object as a dictionary
#lc_dict = json.load(f)

What do the data look like? It is a good idea to see what the data variables include, but we have to be careful about printing such a large amount of information all at once. Exploring the length of what we are going to print first is racommended.

# How many alerts do we have?

# What is the name of the alerts data frame columns?

STOP. What do these column names mean? In order to use them in a smart way, check them out in the ZTF Avro schema

#What do alerts look like, in this short version?
# take a look at the first 3 alerts

b - Refining search criteria#

  1. Create a search criterion using boolean logic (you can chain them like - (condition 1) & (condition 2) & (condition 3)) with the following criteria:

    • one of the alerts has exactly 20 detections in its history (ndethist)

    • is at least 5 arcsec away from the closest PS1 catalog source (distpsnr1)

    • and the closest PS1 cataloged source is likely a galaxy (sgscore1' > 0.5)

  2. apply this filter to the alert data

How many alerts fall under these criteria?

# b result here

Let’s display a light curve. For example, the dataset includes ZTF21aagnvvk, which is a Type Ia supernova that was assigned IAU name AT2021bnf. You should have found this transient using the selection criteria in the cell above.

To make things easier, the filter id (fid) column was replaced with the filter name (g, r, or i) column. Note that upper limits from non-detections are not included in this data set, so you will find only information about detections, even if knowing the last non-detection time of a transient can be crucial to constrain its onset.

# you can view info about the lightcurve data for a given target by calling `lc_dict` for a given target,
# do so below for `ZTF21aagnvvk`

c - Plot a light curve#

More information about the ZTF filters are shown in Figure 1 of Ngeow et al 2019.

Using the routine below:

  1. Plot a lightcurve for the object we looked at above: ZTF21aagnvvk.

This object is a Type Ia supernova. What about the lightcurve suggests this classification?

  1. Plot a lightcurve for the other object identified by our criterion above.

What kind of transient does the ligthcurve suggest this object is?

def plot_lc(name):
    """
    Plot a light curve given a ZTF transient name.
    Assume lc_dict to be a global variable already defined.

    Parameters
    ----------
    name str
        name (objectId) of the ZTF transient
    """
    # Get the lc of interest
    lc = lc_dict[name]
    # What are the filters in which it was observed
    filters = [x["filter"] for x in lc]
    
    # Initialize the figure
    fig, ax = plt.subplots()
    
    # Plot the light curve in each filter
    for f in set(filters):
        time = [x["jd"] for x in lc if x['filter'] == f]
        mag = [x["magpsf"] for x in lc if x['filter'] == f]
        magerr = [x["sigmapsf"] for x in lc if x['filter'] == f]
        ax.errorbar(time, mag, yerr=magerr, marker="o", label=f)
    # Legend without repeatitions
    handles, labels = plt.gca().get_legend_handles_labels()
    by_label = OrderedDict(zip(labels, handles))
    ax.legend(by_label.values(), by_label.keys())
    # Axes name
    ax.set_xlabel("JD")
    ax.set_ylabel("Apparent magnitude")
    # Note: smaller magnitude means larger luminosity!
    ax.set_ylim(ax.get_ylim()[::-1])

# Try out the new function ZTF21aagnvvk

Bonus quiz: Now try out the plotting function on ZTF21aagmryd. Can you guess which type of supernova this is ONLY based on the light curve? [solution: a SN Ia is well recognizable from the red bump]

# Try out the new function on the other object

d - Searching for Core-Collapse Supernovae#

  1. Visit the Bright Transient Survey webpage and identify the object ID of two massive star supernovae of different sub-classes.

  2. Plot the lightcurve of these additional objects.

Comment qualitatively on the shape of the lightcurves compared to the objects above.

# massive star explosion lightcurve here

# second massive star explosion lightcurve here
Contents