Generate Streaming Data
In this section, you will generate real-time data using the Amazon Kinesis Data Generator tool configured previously.
- Go Back to the console and use the username: kinesisgen and the password: kinesisgen1

- Select the following options:

- Go to the bottom part of the page and define the structure of our simulated data using the parametrized template. Copy and paste it to the Template1 tab. Template to use:
{
"productName" : "{{commerce.productName}}",
"color" : "{{commerce.color}}",
"department" : "{{commerce.department}}",
"product" : "{{commerce.product}}",
"imageUrl": "{{image.imageUrl}}",
"dateSoldSince": "{{date.past}}",
"dateSoldUntil": "{{date.future}}",
"price": {{random.number(
{
"min":10,
"max":150
}
)}},
"campaign": "{{random.arrayElement(
["BlackFriday","10Percent","NONE"]
)}}"
}
KDG makes use of an open-source toolkit called Faker.js (available in GitHub: ). You can customize the template as you wish. Check Faker.js documentation for details.
- Click on the Send Data button.
- You will see a message similar to this::
Hint: Don’t stop sending events from KDG. Let it run while you are working on the other sections.
- To verify that the Data Stream is working go back to the Kinesis Firehose console and select lf-kinesis-fh.

- Select the Monitoring tab and wait 3 minutes and you will see how the metrics start to change, if you can see it, you have successfully created a Kinesis firehose data stream, and are ready to add this data to Lake Formation. Please proceed to the next lab.

In the next section, you are going to create a table for the real time data.