StarTree Cloud Coding Competition With Grand Prizes
⌂
StarTree ThirdEye
ThirdEye operators
EventFetcher

EventFetcher

Fetch events data from ThirdEye internal events database. These events can be fed to detectors downstream.

Inputs

None.

Outputs

One output.

No naming constraint for outputKey and outputName.

Parameters

namedescriptiondefault value
component.startTimeLookbackOffset to apply to the startTime of the event timeframe. In ISO-8601 (opens in a new tab) format. Eg P7D.No offset ie eventStartTime = detectionStartTime
component.endTimeLookbackOffset to apply to the endTime of the event timeframe. In ISO-8601 (opens in a new tab) format. Eg P7D.No offset ie eventEndTime = detectionEndTime
component.lookaroundOffset to apply on startTime and endTime to look around the timeframe. In ISO-8601 (opens in a new tab) format. Eg P1D.P1D ie events are fetched on [minTime - 1 day, maxTime + 1 day]
component.eventTypesList of event types to filter by. Eg ["HOLIDAY", "DEPLOYMENT"]. [] or null means no filtering.[] ie no filtering
component.sqlFilterSql filter to apply on the events. See documentation below. Eg type='HOLIDAY' and 'US' member of dimensionMap['countryCode']. "" empty string or null means no sql filtering."" ie no sql filtering

SQL Filter

SQL filters are applied to a table with the following schema:

name: Stringtype: StringdimensionMap: Map[String, List(String)]
ChristmasHOLIDAY{"countryCode": ["US", "CA"]}
ChristmasHOLIDAY{"countryCode": ["NL", "FR"]}
Product X deploymentCUSTOM{"env": ["prod", "dev"], "device": ["android"]}
Product Y deploymentCUSTOM{"env": ["prod", "dev"], "device": ["android", "ios"]}
Product X deploymentCUSTOM{"env": ["prod"], "language": ["android"]}

The same event can happen in different timezone, so on a slightly different timeframe.
For instance, Christmas in France happens around 7 hours before Christmas in the United-States, so it is good practice to filter the HOLIDAY events by a limited set of countries.

Example:

  • Take all HOLIDAY events happening in the US and all CUSTOM events created by the business team:
    'US' member of dimensionMap['countryCode'] OR (type = 'CUSTOM')

The dialect used is Apache Calcite SQL. It is very close to ANSI SQL. See SQL reference (opens in a new tab). Try Calcite SQL online here (opens in a new tab).

⚠️

Combining filtering The eventTypes filter is applied before the sqlFilter.

Example

{
  "name": "eventsDataFetcher",
  "type": "EventFetcher",
  "params": {
    "component.startTimeLookback": "P30D",
    "component.endTimeLookback": "P0D",
    "component.lookaround": "P1D",
    "component.eventTypes": ["HOLIDAY", "CUSTOM"],
    "component.sqlFilter": "'US' member of dimensionMap['countryCode'] OR (type = 'CUSTOM')"
  },
  "outputs": [
    {
      # no constraint
      "outputKey": "events",
      # no constraint
      "outputName": "events"
    }
  ]
},
DimensionExplorationSqlExecution