Skip to main content

Upload file

In this guide we'll learn how to upload a file of up to 30MB in size in CSV, Avro, JSON, or Parquet format.

Select data source

Click on the Upload file button under the Which data do you want to use? heading:

Upload a file

Upload a file

Create a file SampleDataLoad.csv, based on the Pinot Batch import example, that contains the following:

SampleDataLoad.csv
order_date,order_type,branch_code,order_price,customer_loc,customerHasloyalty,distance_to_customer_KM,delivery_fee,distance_tier,fee_tier
2021-08-09,Lunch,SFO,328,San Francisco,FALSE,8.245,13.3796597,Medium,25-30
2021-02-08,Breakfast,SJC,219,San Jose,FALSE,8.752,13.8302326,Medium,25-30
2021-01-22,Breakfast,SJC,110.5,San Jose,FALSE,9.734,14.8748241,Medium,25-30
2021-01-02,Breakfast,SCC,597.5,Santa Clara,TRUE,13.202,7.3285306,Long,3-10

Drag the file onto the onto the Drop File Here or Browse area or click on that button to select the file using the file manager.

Enter transcripts as the Dataset Name. This value will also be used as the name for the underlying Pinot table.

Select CSV file

Select CSV file

If you want to delete the file, hover over the paper clip icon and click on the delete button. Once you're happy with the selected file, click the NEXT button.

Data Transformation

The Dataset Manager will make an educated guess at the field and data types for each of the columns in the CSV file.

Data Transformation

Data Transformation

You can update the selections on this screen, as well as adding any column transformation/mappings. We'll change the student data type to LONG, the score field type to METRIC, and the score data type to DOUBLE. The final version of the data transformation is shown below:

Data Transformation

Data Transformation

Once you're happy with the data transformations, scroll down, and click on the NEXT button.

Advanced settings

On this screen you'll be able to configure indexes, tennats, and data retention on this data source.

Configure indexes, tenants, and data retention

IndexesIndexes

For more information on the different types of indexes and when to use them, see the Apache Pinot Indexing Documentation.

Once you're happy with the configuration, scroll down, and click on the NEXT button.

Review

You'll now see the review and submit screen, where you can review everything that we've configured in the previous steps.

Review

Review

If anything doesn't look right, click on the PREV button to go back to the previous screen.

Once you're happy ready to create the data source, click on the FINISH button. You'll then see the following screen:

Data Source Created

DataSourceCreated

Query Data Source

To have a look at the data that we've imported, click on the Query Console link, which will open the Pinot Data Explorer. Click on the transcript table and then click RUN QUERY to run a basic query against the data source:

Query Transcript Data Source

Query Data Source