Kafka Streams

http://kafka.apache.org/documentation/streams

http://www.confluent.io/blog/introducing-kafka-streams-stream-processing-made-simple/

http://docs.confluent.io/3.0.0/streams/

=Getting started=

wget http://packages.confluent.io/archive/3.0/confluent-3.0.0-2.11.zip unzip confluent-3.0.0-2.11.zip

cd confluent-3.0.0/
 * 1) *** IMPORTANT STEP ****
 * 2) The subsequent paths and commands used throughout this quickstart assume that
 * 3) your are in the following working directory:

cd confluent-3.0.0/ ./bin/zookeeper-server-start ./etc/kafka/zookeeper.properties
 * 1) Start ZooKeeper.  Run this command in its own terminal.

Next we launch the Kafka broker, which will listen on localhost:9092 and connect to the ZooKeeper instance we just started. Since this is a long-running service, too, you should run it in its own terminal. cd confluent-3.0.0/ ./bin/kafka-server-start ./etc/kafka/server.properties
 * 1) Start Kafka.  Run this command in its own terminal

First, we need to create the input topic, named streams-file-input:

./bin/kafka-topics --create \ --zookeeper localhost:2181 \ --replication-factor 1 \ --partitions 1 \ --topic streams-file-input

Next, we generate some input data and store it in a local file at /tmp/file-input.txt:

echo -e "all streams lead to kafka\nhello kafka streams\njoin kafka summit" > /tmp/file-input.txt