Apache Flink

https://flink.apache.org/

''Apache Flink® is an open source platform for distributed stream and batch data processing. Flink’s core is a streaming dataflow engine that provides data distribution, communication, and fault tolerance for distributed computations over data streams.''

=Getting started=

Installation
wget http://www.apache.org/dyn/closer.lua/flink/flink-1.1.2/flink-1.1.2-bin-hadoop27-scala_2.11.tgz tar xf flink-1.1.2-bin-hadoop27-scala_2.11.tgz FLINK_HOME=~/flink-1.1.2 cd $FLINK_HOME ls bin ls examples

cd $FLINK_HOME bin/start-local.sh

Open the UI http://localhost:8081/#/overview

cd $FLINK_HOME bin/start-scala-shell.sh

Run the SocketWindowWordCount example (source).

Terminal 1: Start netcat nc -l 9000

Submit the Flink program: bin/flink run examples/streaming/SocketWindowWordCount.jar --port 9000

Terminal 1: Add words in netcat input lorem ipsum ipsum ipsum ipsum bye

Terminal 3: tail -f log/flink-*-jobmanager-*.out

Terminal 4: stop Flink bin/stop-local.sh