Using MinIO as deep store for an offline table

In this recipe we'll learn how to use MinIO (opens in a new tab) as a Pinot deep store.

To understand how this recipe processes data, examine the commands executed in the Makefile.

Prerequisites

To follow the code examples in this guide, do the following:

• Install Docker (opens in a new tab).
• Download recipes

Makefile

Clone this repository and navigate to this recipe:

make recipe

Validate

Check that minio has the segment in the deep store. You can also log into the minio console and check. http://localhost:9001/browser/deepstore (opens in a new tab).

docker exec minio mc ls myminio/deepstore/transcript

Clean up

make clean

Troubleshooting

To clean up old Docker installations that may be interfering with your testing of this recipe, run the following command:

docker system prune