Data Lake on AWS > Ingested and store > Generate sample data

Generate sample data

In this step we will configure Kinesis Data Generator to generate sample data and push them to Kinesis Firehose

Configure Amazon Cognito for Kinesis Data Generator

In this step, we will create a Cloud Formation Stack to configure Cognito. Scripts in CloudFormation will create in Region US East (N. Virginia) us-east-1

Go to AWS Management Console

Find CloudFormation
Select CloudFormation

Generate Sample Data

In the CloudFormation interface

Select Stacks
Select Create stack

Generate Sample Data

Download the file cognito-setup.json from First Cloud Journey

Use the keyboard shortcut Ctrl + S to Save

In the Create stack interface

Select Template is ready
Select Upload a template file
Select Choose file
Select cognito-setup.json
Select Next

Generate Sample Data

In the Specify stack details interface

Stack name, enter Kinesis-Data-Generator-Cognito-User
*User name, enter admin
Enter the password you want to use
Select Next

Generate Sample Data

Select Next

Generate Sample Data

In the Create stack interface

Select I acknowledge that AWS CloudFormation might create IAM resources

Generate Sample Data

Wait about 5 minutes, finish creating stack

Select Kinesis-Data-Generator-Cognito-User stack
Select Event to see the initialization process

Generate Sample Data

In the Stack interface

Select Kinesis-Data-Generator-Cognito-User stack
Select Outputs
Select KinesisDataGeneratorUrl

Generate Sample Data

In the Amazon Kinesis Data Generator interface, complete the information input (this information you just configured in the Stack creation step)

Enter Username
Enter Password
Select Sign In

Generate Sample Data

In the Amazon Kinesis Data Generator interface, after completing the login

Region, select us-east-1
Stream/delivery stream, select FCJ-Deliverystream
Records per second, enter 2000
Select Template 1
Enter the following code:

{
  "uuid": "{{random.uuid}}",
  "device_ts": "{{date.utc("YYYY-MM-DD HH:mm:ss.SSS")}}",
  "device_id": {{random.number(50)}},
  "device_temp": {{random.weightedArrayElement(
    {"weights":[0.30, 0.30, 0.20, 0.20],"data":[32, 34, 28, 40]}
  )}},
  "track_id": {{random.number(30)}},
  "activity_type": {{random.weightedArrayElement(
        {
            "weights": [0.1, 0.2, 0.2, 0.3, 0.2],
            "data": ["\"Running\"", "\"Working\"", "\"Walking\"", "\"Traveling\"", "\"Sitting\""]
        }
    )}}
}