Amazon Kinesis, A New AWS Service To Process Real-Time Streams

Amazon is offering a new service called Kinesis that streams data in real time with the ability to process thousands of data streams on a per-second basis. The service, designed for real-time apps, allows developers to pull any amount of data, from any number of sources, scaling up and down as needed.

Kinesis can create any number of streams across multiple availability zones. The streams have no “intrinsic” capacity or rate limits. All incoming data is replicated across availability zones for high availability. Each stream can have multiple writers and multiple readers. The service shards the data into streams with each handling 1000 write transactions and up to 20 read transactions.

Kinesis is priced on a pay as you go basis by the amount of data that is processed and how it is packaged. According to the AWS blog, customers pay on a PUT basis and pay $0.028 per 1,000,000 PUT operations and you pay $0.015 per shard, per hour. For one hour of collecting game data you’d pay $0.30 for the shards and about $1.01 for the 36 million PUT calls, or $1.31.

CTO Werner Vogels introduced the discussion by providing context about the increasing use of sensors to record data. Construction sites will be covered in sensors so builders can look at the data, recording environmental conditions and determine, for example, when to pour the concrete in the foundation. That data can now be processed in real-time with Amazon Kinesis into applications of any variety.

His context illustrates how the data generated will continue to scale and with it there will be the need for services that can ingest, process and analyze on a per second basis. It illustrates how the world is now defined by the way we measure data and then act upon it.