Skip to main content

Create your first alert

info

In this guide you'll learn how to create your first alert. To follow the instructions in this guide, you will need to have installed ThirdEye.

Alerts are the rules you build to detect anomalies. An alert configuration contains all the information necessary to:

  • Fetch data from a datasource.
  • Preprocess data
  • Compute baselines
  • Detect anomalies
  • Send alerts to your subscription groups

In this tutorial, we will create an alert that detects when a metric does not lie between a minimum and a maximum value. We will also see how alerts are created in the ThirdEye UI. Let’s jump right in!

caution

The following assumes you have a working ThirdEye environment. See Install ThirdEye.
If you just want to understand detection in alerts, see understanding detection configuration.

Load a sample dataset​

Start by loading some data into Apache Pinot.

  1. Go to your Startree Dataset Manager at [your_startree_url]/datasets.
  2. Click on Add New Dataset.
  3. We will use a sample dataset:
    1. Name the dataset complexWebsite
    2. Click on Use Sample Data
    3. Select Website With Anomalies
    4. Click Next
  4. On the next screen, click Finish

The test dataset has now been loaded into Pinot.

Setup the datasource​

Configure ThirdEye to get access to the Pinot database.

ThirdEye connects to the Pinot Database automatically.
Pinot datasets are onboarded automatically.
Ask StarTree support if you cannot see Pinot datasets in configuration → datasets.

Create the alert configuration​

  1. Go to your Thirdeye app.

  2. Click on Alerts

  3. On the alert page, set the dates between January 1, 2021 and December 31, 2021. We will simulate an alert rule based on this timeframe.


  4. On the Alerts page, click on Create → Create Alert

  5. Copy and paste this JSON in the configuration field:

    quickstart_complexWebsite_alert.json
    {
      "name": "my-first-detection-configuration",
      "description": "Detect anomalies in pageviews.",
      "cron": "0 0/1 * 1/1 * ? *",
      "template": {
        "nodes": [
          {
            "name": "root",
            "type": "AnomalyDetector",
            "params": {
              "type": "THRESHOLD",
              "component.monitoringGranularity": "P1D",
              "component.timestamp": "ts",
              "component.metric": "met",
              "component.max": "${max}",
              "component.min": "${min}",
              "anomaly.metric": "${metric}"
            },
            "inputs": [
              {
                "targetProperty": "current",
                "sourcePlanNode": "missingDataManager",
                "sourceProperty": "currentOutput"
              }
            ],
            "outputs": []
          },
          {
            "name": "missingDataManager",
            "type": "TimeIndexFiller",
            "params": {
              "component.timestamp": "ts"
            },
            "inputs": [
              {
                "sourcePlanNode": "currentDataFetcher",
                "sourceProperty": "currentOutput"
              }
            ],
            "outputs": [
              {
                "outputName": "currentOutput"
              }
            ]
          },
          {
            "name": "currentDataFetcher",
            "type": "DataFetcher",
            "params": {
              "component.dataSource": "${dataSource}",
              "component.query": "SELECT __timeGroup(hoursSinceEpoch, '${timeColumnFormat}', '${monitoringGranularity}') as ts, ${metric} as met FROM ${dataset} WHERE __timeFilter(hoursSinceEpoch, '${timeColumnFormat}') GROUP BY ts ORDER BY ts LIMIT 10000"
            },
            "inputs": [],
            "outputs": [
              {
                "outputKey": "pinot",
                "outputName": "currentOutput"
              }
            ]
          }
        ],
        "metadata": {
        "datasource": {
          "name": "${dataSource}"
        },
        "dataset": {
          "name": "${dataset}"
        },
        "metric": {
          "name": "views"
        }
      }
      },
      "templateProperties": {
        "dataSource": "pinotQuickStart",
        "dataset": "complexWebsite",
        "metric": "sum(views)",
        "monitoringGranularity": "P1D",
        "timeColumn": "date",
        "timeColumnFormat": "EPOCH_HOURS",
        "max": "15000",
        "min": "5000"
      }
    }
    This configuration includes the following:

    • Data Fetcher that runs a SQL query that returns the number of views grouped by time at an hour granularity.

    • Time Index Filler that fills in missing values.

    • Anomaly Detector that finds outliers in the dataset.

      tip

      For more on alert configuration, see understanding detection configuration.

  6. Below the JSON configuration, in the preview panel, press the refresh button: you will see the result of the detection configuration. Notice that there are a few anomalies.


    Try changing the dates on the timeframe selector in the top right corner and the min/max values in the JSON to see what happens.

  7. Click Next. Subscription Groups are used to send anomaly alerts to external systems like Slack and email. This can be added later, so let’s skip it for the moment. Click Next again.


  8. A final page summarizes everything. Review and click on Finish.

That’s it! Your first alert is created, and you will be redirected to the alert page. When an alert is created, the detection is rerun on past data.

Anomalies will appear in the anomaly list. Let's now have a look at those.

View anomalies​

  1. Click on Anomalies in the left panel.


  2. You should see anomalies! Click on one: you can now analyze the anomaly.


Next Steps​

You've now successfully configured your first alert and been able to identify anomalies in the dataset.