Skip to main content

Quick Start Guide

This guide will help you get up and running with the StarTree ThirdEye Community Edition on a local Kubernetes environment in a few minutes. For running it in AWS, visit Link

info

For differences between StarTree ThirdEye Community Edition vs Enterprise Edition, visit: Link

Watch this video to learn how to install StarTree Community Edition

Prerequisites for installing StarTree Community Edition

Getting the Helm Repo

Required Tools

info

You can skip this section if you have Docker, minikube, and helm installed on your machine.

Install Docker / Docker Desktop

Follow the instructions on the Docker Website to install Docker on your machine.

Please ensure that Docker Desktop (or Docker Daemon) is running after you install it.

caution

If you are using Docker Desktop on Mac or Windows, please allow at least 10G of memory and 4 CPU cores. For better performance and memory our recommendation is to install it in AWS EC2 instance (t2.xlarge)

Install Helm

Follow the instructions on the Helm Website to install the helm CLI tool.

Install Minikube

We recommend using Minikube to set up a local Kubernetes cluster. Follow the instructions on the Minikube Website to install it.

caution

If you have a minikube cluster already running. We recommend you delete it and create a new cluster with more resources allocated. You can delete an existing minikube cluster with minikube delete

create a minikube cluster by running

minikube start --driver=docker --vm=true --cpus=4 --memory=10g --disk-size=20g --kubernetes-version=v1.21.7

Preload Images (Apple M1/M2 Macbooks only)

Please skip this step if you are not on an M1 or M2 Macbook.

Run the following commands to prevent runtime issues on Apple Silicon and Docker

eval $(minikube docker-env)
docker pull --platform linux/x86_64 mysql:8.0.28
docker pull --platform linux/x86_64 adoptopenjdk/openjdk11:alpine
docker pull --platform linux/x86_64 apachepinot/pinot:latest-jdk11
docker pull --platform linux/x86_64 bitnami/zookeeper:3.7.0-debian-10-r56

Install StarTree Community Edition

If you followed the guide to Getting the Helm Repo, you should have received an email with the links to the Helm repo/chart.

Install the Helm Repo

helm repo add startree <REPO LINK FROM EMAIL>
helm install startree startree/startree-community-edition --set accesstoken=<access token shared in email>

Wait for all the pods to have a status of Running and have all their containers READY. The state transition can take about 3-5 minutes.

Before proceeding, ensure that the Status of all Pinot components are marked as Alive on the Pinot Controller UI.

You can access the Pinot Controller UI by port-forwarding port 9000 on the pod. Run the following command and visit http://localhost:9000/ on your browser.

kubectl port-forward svc/startree-pinot-controller 9000:9000

Running queries in Pinot Query Console for the Sample Data

image

Exploring Alerts and Anomalies with StarTree ThirdEye Community Edition

image

Sample Dataset

This Helm chart also includes a helper job to onboard a sample dataset into Pinot and explore anomalies and alerts using ThirdEye.

The helper job simulates a retail-order tracking event dataset from a global eCommerce company. With this sample, we can explore how the eCommerce company's business operations or data analytics team could monitor and track the order events with ThirdEye.

Apart from tracking, we can also:

  • Learn about patterns in the order events dataset. For example, if there are spikes, drops, seasonality, or fraudulent activity
  • Identify root causes for the patterns by drilling down into different dimensions of the dataset
  • Take necessary actions, such as resolving the fraudulent activity to deliver the best customer experience and prevent monetary losses in the order check-out process

The Community Edition Helm chart runs a Kubernetes Job called quickstart-helper that onboards a sample dataset for you to analyze. The job first checks to see if all Pinot and ThirdEye components are up. It then creates a Pinot Table with the sample retail-orders dataset and sets up anomaly detection in ThirdEye.

Wait for the quickstart-helper-XXXXX Job to go into the Completed state before proceeding. Please note that it should be in the Completed state and NOT in the Running state. You can view the Job state by running kubectl get pods.

Dataset Exploration

We can explore the data from the ThirdEye UI. Port-forward to the UI pod to access it from the browser. You can port-forward with the following command and visit the UI on your browser at http://localhost:8081/

kubectl port-forward svc/startree-startree-thirdeye-ui 8081:8081

You should see the following page. To view the alerts setup by the quickstart-helper job, click on the Alerts box.

image

View the alert by clicking on the alert name, in our case: threshold-alert-orders-sample

image

Our order-events sample dataset was created in the year 2021. Therefore please change the date range using the data-selector on the alert detail page. In the example beloew we are viewing the date range from Jan 1st, 2021 to March 31th, 2021.

image

Once you hit the Apply button on the data-range selector, you should be able to view the anomalies plotted on the chart.

image

You can view the alert configuration on the bottom of the alert page. The sample alert is pre-configured to use the startree-threshold Alert Template which detects threshold based anomalies for event_type in the order_events dataset.

Find anomalies

View the related anomalies for the alert by clicking on the link in the Anomalies in the Time Range section. Then from the list view select a specific anomaly to analyze.

image

We can perform root cause analysis for the drop in order_events by event_type by clicking on the Investigate Anomaly button. This opens up a page that allows us to perfor dimensional analysis.

imageimage

What went wrong?

We can see that the dimensional analysis indicates a drop in the number of orders which are in the IN_PROGRESS, PLACED states and increase in DELIVERED state.

What has contributed the most to the spike? Looking at the above heatmap it clearly indicates that the order bookings has increased. In order to what has contributed to the spike the business operations team can drill down on the "orders delivered" and look at other dimensions which has caused the spike.

As a call to action

  • The business operations team can reach out to order management team to understand why there are more order bookings on that day as compared to week-over-week or week-over-two-weeks.
  • If there is a public holiday or special occassion then it is an expected spike and probably no action is needed. Otherwise, the shipment department can proactively investigate and resolve the issue with the drop-in # of shipped orders for improved customer experience.
  • Maybe there is a specific campaign running or discount given which is resulting in an increase in order volume.
  • Sometimes the reason could be a pricing glitch and maybe selling at a cheaper price resulted in more orders getting booked and delivered however not good for revenue and business growth. Then the team can go and make the pricing adjustments accordingly to prevent business loss.

Using StarTree ThirdEye the custom events can be captured and correlated with the spikes or drops. This further helps users of StarTree ThirdEye to decide which specific action can be taken to resolve the outliers.

To learn more about StarTree ThirdEye please visit the documentation link which covers the concepts and how-to guide for using StarTree ThirdEye.