continuously send data to your application. And application which is consuming this data is hosted on EC2 instances and ready to consume data in real-time and can apply the analytics on it and consumer can see this
analytics on their mobile phone. You have to implement a best solution for ingesting data with high throughput to all the application installed on EC2 hosts. How can you achieve this?
1. You will be using Kinesis API Library for data ingestion.
2. You will be using Kinesis KPL Library for data ingestion.
3. You will be using Kinesis KCL Library for data ingestion.
4. You will be using SNS Topic for data ingestion.
5. You will be using SQS Queue for data ingestion.
Correct Answer : 2 Exp : In this question you can say that IoT devices are sending continuous data from millions of consumer appliances. You need to consume this data. Now you want that application hosted on EC2
instances should be able to consume this data.
Kinesis Data stream producer is any application that puts user data records into a Kinesis data stream which is also known as Data Ingestion. The Kinesis Producer Library simplifies producer application development,
allowing developers to achieve high write throughput to a Kinesis Data Stream.
Using KPL you will be writing to Kinesis Data Stream, KPL library will work as an intermediary between your producer application code and the Kinesis Data Stream APIs and can perform the following tasks.
1. It can write data to one or more Kinesis Data Streams with an automatic and configurable retry mechanism.
2. Collects records and uses PutRecords to write multiple records to multiple shards per request.
3. Aggregate user records to increase payload size and improve throughput.
4. Integrate with the Kinesis Client Library (KCL) to de-aggregate batched records on the consumer.
5. Submit Amazon CloudWatch metrics on your behalf to provide visibility into producer performance.
There is a difference between KPL and Kinesis Data Stream API. Kinesis Data Stream API is available in the AWS SDK. Using Kinesis Data Stream you can create streams, re-sharding, and putting and getting records. While
KPL provides abstraction specifically for ingesting data.
Now in the question, EC2 instances has to receive all the events generated by the IOT devices and delivered for analytics. But it is very complicated stuff to receive the events and then aggregate all the events and
creating the bathes etc.
Hence, you will be using KPL which will help you in creating high-performance producers. Assume EC2 instances are getting all the events from the IoT devices and then writing those events to the Kinesis Data Stream.
Then application hosted on EC2 instance is responsible for writing thousands of events per second to your data stream. Hence, you need to write intermediate application which implement the complicated logic which does
the batching, multithreading etc. So rather than writing your own solution, you will be using the KPL for solving this problem. And this is what asked in the question and your answer would be option-2
2
