gwcelery.tasks.superevents module

Superevents are an abstraction to unify gravitational-wave candidates from multiple search pipelines. Each superevent is intended to represent a single astrophysical event. A superevent consists of one or more event candidates, possibly from different pipelines, that are neighbors in time. At any given time, one event belonging to the superevent is identified as the preferred event.

This module provides the Superevent Manager, an IGWN Alert handler that creates and updates superevents whenever new events are uploaded to GraceDB. It also checks whether the superevent qualifies to be sent out as a LIGO-Virgo public alert.

Event candidates are only considered for membership in a superevent if their false alarm rate is less than or equal to the value of the superevent_far_threshold configuration setting.

Each superevent has a time window described by a central time t_0, a start time t_start, and a end time t_end. The central time t_0 is just the time of the preferred event. The start and end time are extended to encompass all of the events that belong to the superevent (see get_ts()).

The first candidate reported from a search pipeline creates a superevent, with it being the preferred event. Subsequent candidate additions to the superevent may result in a change of the superevent time window. The preferred event may also be updated as more significant candidates are added. However, this will stop once a candidate passing the public false alarm rate threshold (mentioned in preliminary_alert_far_threshold) is added to the superevent. At this point, the preferred event is frozen and an automatically generated preliminary notice is sent with the data products of the preferred event. Triggers could however still be added to the superevent as the preliminary alert and the ensuing annotations are being processed. Once the preliminary alert is dispatched to the GCN broker, the preferred event would be revised after a wait time of superevent_clean_up_timeout, following which a second automatic preliminary alert would be issued.

Selection of the preferred event

When a new event is added to a superevent, it may or may not become the new preferred event. The preferred event is selected by considering the following factors in order to resolve any ties:

  1. Completeness: Would the event be complete, as determined by the function is_complete(), for sending an automated public alert?

  2. Less-significant FAR threshold: Does the false alarm rate pass the false alarm rate threshold mentioned in preliminary_alert_far_threshold?

  3. Significant FAR threshold: Does the false alarm rate pass the false alarm rate threshold mentioned in significant_alert_far_threshold?

  4. Search group: Is it a CBC event or a burst event? CBC events take precedence.

  5. Number of detectors: How many detectors contributed data to the event? For CBC events, events with triggers from more detectors take precedence.

  6. Significance: For CBC events, which has the highest SNR? For burst events, which has the lowest FAR?

The selection of the preferred event from a pair of events is illustrated by the decision tree below.

digraph preferred_event { compound = true nodesep = 0.1 ranksep = 0.5 node [ fillcolor = white shape = box style = filled target = "_top" ] graph [ labeljust = "left" style = filled target = "_top" ] significant_far_differs [ label = "Does only\none event pass \n significant FAR\nthreshold?" shape = diamond ] less_significant_far_differs [ label = "Does only\none event pass \n less-significant FAR\nthreshold?" shape = diamond ] completeness_differs [ label = "Is only\none event\ncomplete?" shape = diamond ] significant_far_decides [ label = "Select the\npublishable\nevent" ] less_significant_far_decides [ label = "Select the\npublishable\nevent" ] completeness_decides [ label = "Select the\ncomplete\nevent" ] group_differs [ label = "Are the events\nfrom different\nsearch groups?" shape = diamond ] group_decides [ label = "Select the\nCBC event" ] which_group [ label = "From which\nsearch group are\nthe events?" shape = diamond ] detectors_differ [ label = "Does one event\ninvolve more\ndetectors?" shape = diamond ] detectors_decide [ label = "Select the events\nwith the greatest\nnumber of detectors" ] cbc_significance [ label = "Select event\nwith the\nhighest SNR" ] burst_significance [ label = "Select event\nwith the\nlowest FAR" ] completeness_differs -> completeness_decides [label = Yes] completeness_differs -> significant_far_differs[label = No] significant_far_differs -> significant_far_decides [label = Yes] significant_far_differs -> less_significant_far_differs [label = No] less_significant_far_differs -> group_differs [label = No] less_significant_far_differs -> less_significant_far_decides [label = Yes] group_differs -> group_decides [label = Yes] group_differs -> which_group [label = No] which_group -> detectors_differ [label = CBC] detectors_differ -> detectors_decide [label = Yes] detectors_differ -> cbc_significance [label = No] which_group -> burst_significance [label = Burst] }

Note

When a preferred event is assigned to a superevent, it may not be complete i.e., its data products may not have been computed yet. Once all the data products of the preferred event is ready, the EM_READY label is applied to the superevent.

The preferred event is frozen once an event candidate passing the less-significant false alarm rate threshold is added to the superevent. This is denoted by the application of the EM_Selected label on the superevent. If the event instead passes the significant false alarm rate threshold, the EM_SelectedConfident label gets applied. All EM_SelectedConfident events are EM_Selected.

When the first significant preliminary alert has been dispatched to the GCN broker, the GCN_PRELIM_SENT label is applied to the superevent which is used to revise the preferred event and launch a second preliminary alert.

The second preliminary is sent even if the preferred event stays unchanged after the revision. In this case, it contains the same content as the first preliminary alert.

The application of ADVNO before the launching of the second preliminary alert stops the process. A retraction notice is sent instead.

Tasks

Module containing the functionality for creation and management of superevents.

  • There is serial processing of triggers from low latency pipelines.

  • Dedicated superevent queue for this purpose.

  • Primary logic to respond to low latency triggers contained in process() function.

gwcelery.tasks.superevents.REQUIRED_LABELS_BY_GROUP = {'burst': {'SKYMAP_READY'}, 'cbc': {'EMBRIGHT_READY', 'PASTRO_READY', 'SKYMAP_READY'}}

These labels should be present on an event to consider it to be complete.

gwcelery.tasks.superevents.FROZEN_LABEL = 'LOW_SIGNIF_LOCKED'

This label indicates that the superevent manager should make no further changes to the preferred event.

gwcelery.tasks.superevents.SIGNIFICANT_LABEL = 'SIGNIF_LOCKED'

This label indicates that the superevent is elevated to significant

gwcelery.tasks.superevents.EARLY_WARNING_LABEL = 'EARLY_WARNING'

This label indicates that the superevent contains a significant early warning event.

gwcelery.tasks.superevents.EARLY_WARNING_SEARCH_NAME = 'EarlyWarning'

Search name for Early Warning searches. Only significant events result in consideration by the superevent manager.

gwcelery.tasks.superevents.SUBSOLAR_SEARCH_NAME = 'SSM'

Search name for subsolar mass search. These are ignored by the superevent manager.

gwcelery.tasks.superevents.READY_LABEL = 'EM_READY'

This label indicates that a preferred event has been assigned and it has all data products required to make it ready for annotations.

gwcelery.tasks.superevents.VT_SEARCH_NAME = 'VTInjection'

Search name for events uploaded as a part of estimating the spacetime sensitive volume. Events from this search do not form superevents, and are not annotated.

(task)gwcelery.tasks.superevents.handle(payload)[source]

Respond to IGWN alert topics from low-latency search pipelines and delegate to process() for superevent management.

(task)gwcelery.tasks.superevents.process(payload)[source]

Respond to payload and serially processes them to create new superevents, add events to existing ones.

Parameters:

payload (dict) – IGWN alert payload

gwcelery.tasks.superevents.get_category(event)[source]

Get the superevent category for an event.

Parameters:

event (dict) – Event dictionary (e.g., the return value from gwcelery.tasks.gracedb.get_event()).

Return type:

{‘mdc’, ‘test’, ‘production’}

gwcelery.tasks.superevents.get_ts(event)[source]

Get time extent of an event, depending on pipeline-specific parameters.

  • For CWB and MLy, use the event’s duration field.

  • For oLIB, use the ratio of the event’s quality_mean and frequency_mean fields.

  • For all other pipelines, use the superevent_d_t_start and superevent_d_t_end configuration values.

Parameters:

event (dict) – Event dictionary (e.g., the return value from gwcelery.tasks.gracedb.get_event() or preferred_event_data in igwn-alert packet.)

Returns:

  • t_0 (float) – Segment center time in GPS seconds.

  • t_start (float) – Segment start time in GPS seconds.

  • t_end (float) – Segment end time in GPS seconds.

gwcelery.tasks.superevents.get_snr(event)[source]

Get the SNR from the LVAlert packet.

Different groups and pipelines store the SNR in different fields.

Parameters:

event (dict) – Event dictionary (e.g., the return value from gwcelery.tasks.gracedb.get_event(), or preferred_event_data in igwn-alert packet.)

Returns:

snr – The SNR.

Return type:

float

gwcelery.tasks.superevents.get_instruments(event)[source]

Get the instruments that contributed data to an event.

Parameters:

event (dict) – Event dictionary (e.g., the return value from gwcelery.tasks.gracedb.get_event(), or preferred_event_data in igwn-alert packet.)

Returns:

The set of instruments that contributed to the event.

Return type:

set

gwcelery.tasks.superevents.get_instruments_in_ranking_statistic(event)[source]

Get the instruments that contribute to the false alarm rate.

Parameters:

event (dict) – Event dictionary (e.g., the return value from gwcelery.tasks.gracedb.get_event(), or preferred_event_data in igwn-alert packet.)

Returns:

The set of instruments that contributed to the ranking statistic for the event.

Return type:

set

Notes

The number of instruments that contributed data to an event is given by the instruments key of the GraceDB event JSON structure. However, some pipelines (e.g. gstlal) have a distinction between which instruments contributed data and which were considered in the ranking of the candidate. For such pipelines, we infer which pipelines contributed to the ranking by counting only the SingleInspiral records for which the chi squared field is non-empty.

For PyCBC Live in the O3 configuration, an empty chi^2 field does not mean that the detector did not contribute to the ranking; in fact, all detectors listed in the SingleInspiral table contribute to the significance even if the chi^2 is not computed for some of them. Hence PyCBC Live is handled as a special case.

(task)gwcelery.tasks.superevents.select_pipeline_preferred_event(events)[source]

Group list of G events by pipeline, and apply keyfunc() to select the pipeline preferred events.

Parameters:

events (list) – list of event dictionaries

Returns:

pipeline, graceid pairs

Return type:

dict

(task)gwcelery.tasks.superevents.select_preferred_event(events)[source]

Select the preferred event out of a list of G events, typically contents of a superevent, based on keyfunc().

Parameters:

events (list) – list of event dictionaries

gwcelery.tasks.superevents.is_complete(event)[source]

Determine if a G event is complete in the sense of the event has its data products complete i.e. has PASTRO_READY, SKYMAP_READY, EMBRIGHT_READY for CBC events and the SKYMAP_READY label for the Burst events. Test events are not processed by low-latency infrastructure and are always labeled complete.

Parameters:

event (dict) – Event dictionary (e.g., the return value from gwcelery.tasks.gracedb.get_event(), or preferred_event_data in igwn-alert packet.)

gwcelery.tasks.superevents.should_publish(event, significant=True)[source]

Determine whether an event should be published as a public alert.

All of the following conditions must be true for a public alert:

Parameters:
Returns:

should_publish – True if the event meets the criteria for a public alert or False if it does not.

Return type:

bool

gwcelery.tasks.superevents.keyfunc(event)[source]

Key function for selection of the preferred event.

Return a value suitable for identifying the preferred event. Given events a and b, a is preferred over b if keyfunc(a) > keyfunc(b), else b is preferred.

Parameters:

event (dict) – Event dictionary (e.g., the return value from gwcelery.tasks.gracedb.get_event()).

Returns:

key – The comparison key.

Return type:

tuple

Notes

Tuples are compared lexicographically in Python: they are compared element-wise until an unequal pair of elements is found.