InfluxDB



http://influxdb.com/

https://github.com/influxdb/influxdb

open-source distributed time series database with no external dependencies.

(metrics, events, and analytics)

developped in Go language

Voir https://speakerdeck.com/pauldix/introducing-influxdb-an-open-source-distributed-time-series-database

Demo sur http://play.influxdb.com/ (source code)

=InfluxDB @ AIR=
 * SmartCampus

=Installation=

http://influxdb.com/download/

On OS X brew update brew install influxdb

Démarrage influxdb -config=/usr/local/etc/influxdb.conf

Naviguez sur http://localhost:8083/ username : root & password : root

Pensez à changer ces valeurs quand vous mettez influxdb en production

=Premiers Pas=

Via l'interface Web
Depuis l'interface web, Créez une base : mydb

Depuis Database > Explore Data

Depuis write point, Ajoutez un point dans la série temporelle log_lines

{ "line":"here's some useful log info from paul@influx.com", "like":1, "star": 5 } puis un autre { "line":"here's another useful log info from paul@influx.com", "like":2, "star": 4 } puis un autre { "line":"here's some useful log info from didier@donsez.com", "like":1, "star": 5 }

Exécutez les requêtes suivantes depuis read point:

La liste des séries temporelles list series select * from /.*/ limit 1

Les points de la série log_lines select line from log_lines

select like, star from log_lines

Les deux points les plus récents select * from log_lines limit 2;

Les points de la série log_lines dont la colonne line contient paul@influx.com select line from log_lines where line =~ /paul@influx.com/

Les points de la série log_lines dont la colonne like est supérieure à 1 select line from log_lines where like > 1

Les points de la série log_lines des 24 dernières heures select * from log_lines where time > now - 24h

Agrégat temporel (les fonctions d'agrégat sont : count, min, max, mean, mode, median, distinct, percentile, histogram, derivative, sum, stddev, first, last). select sum(like) as number_of_likes, mean(star) as mean_of_star from log_lines group by time(1m) where time > now - 1d select star, sum(like) as number_of_likes from log_lines group by star, time(1m) where time > now - 1d

Remplissage des intervalles vides de points (avec fill) select count(like) from events group by time(1h) fill(0) where time > now - 24h

Fusion (merge) de séries select count(type) from user_events merge admin_events group by time(10m)

Attention, cette requête n'est pas un produit cartésien (SQL) select * from log_lines, log_cpu;

Jointure entre 2 séries select errors_per_minute.value / page_views_per_minute.value as error_rate_per_minute. from errors_per_minute inner join page_views_per_minute

Création d'un Requêtes continues select count(star) from log_lines group by time(10m), star into log_lines.count_per_star.10m

select * from log_lines.count_per_star.10m

drop continuous query 

Supprimez les points de toutes les séries temporelles dont la date est antérieure de 24 heures. delete from /.*/ where time < now - 24h select * from /.*/

Suppression d'une série drop series log_lines

Via l'interface REST
Execute a query (pretty=true --> the output is prettily formatted) curl -G 'http://localhost:8086/db/mydb/series?u=root&p=root&pretty=true' --data-urlencode "q=select * from log_lines"

Write points (with timestamp and sequence numbers : the time_precision can be s, us, ms) curl -X POST -d @points.json 'http://localhost:8086/db/mydb/series?u=root&p=root&time_precision=ms'

points.json [ {    "name": "log_lines", "columns": ["time", "sequence_number", "line"], "points": [ [1400425947368, 1, "this line is first"], [1400425947368, 2, "and this is second"], [1400425948368, 3, "and this is third"] ] } ]

Administration
create a database curl -X POST 'http://localhost:8086/db?u=root&p=root' -d '{"name": "site_development"}'

drop a database curl -X DELETE 'http://localhost:8086/db/site_development?u=root&p=root'

Autre : http://influxdb.com/docs/v0.8/api/administration.html

Dashboard avec Influga
Influga is a InfluxDB Dashboard and Graph Editor.

Installez influga sudo npm install influx sudo npm install -g influga

Editez la configuration dans influga-config.json { "dashboardDbPath": "./db/influga.db", "host": "localhost", "port": 8086, "database": "mydb", "username": "root", "password": "root" }

Lancez le service influga start -c influga-config.json

Naviguez sur http://localhost:8089/ et configurez le panel avec select star, like from log_lines

Dashboard avec Grafana
http://influxdb.com/docs/v0.7/ui/grafana.html

Téléchargez Grafana

Modifiez config.example.js avec datasources: { 'eu-metrics': { type: 'influxdb', url: 'http://localhost:8086/db/mydb', username: 'test', password: 'test', }, 'grafana': { type: 'influxdb', url: 'http://localhost:8086/db/grafana', username: 'test', password: 'test', grafanaDB: true }, },

et Sauvegardez dans config.js

Ouvrez index.html

=Premières Requêtes=

Node.js

 * https://github.com/bencevans/node-influx

Installez sudo npm install influx

Editez le programme testinflux.js var influx = require('influx');

console.log("Connect to database"); var username = 'root'; var password = 'root'; var database = 'mydb';

var dbInflux = influx({host : 'localhost', username : username, password : password, database : database});

/* console.log("Create database"); dbInflux.createDatabase('mydb', function(err) { if(err) throw err;  console.log('Database Created'); });

console.log("Write points");

var i=10; while(i--) { dbInflux.writePoint('log_lines', { line: 'Yo', like: Math.random * 10, star: Math.random * 5 }, function(err) {       if(err) throw err;        }); console.log("+");

}

console.log("Query series");

var query = 'SELECT * FROM log_lines WHERE time > now - 24h'; dbInflux.query(query, function(err, body) { if(err!=null) throw err;  console.log(JSON.stringify(body[0].columns, null, '\t'));  console.log(JSON.stringify(body[0].points, null, '\t')); });

Lancez le programme node testinflux.js

PubNub to InfluxDB
Créez un compte gratuit sur http://pubnub.com et renseignez les clés récues par mail (les clés demo & demo fonctionnent quand même)

Installez sudo npm install influx npm install pubnub

Lancez le programme node pubnub2influx.js dans un terminal

var influx = require('influx');

/* var pubnub = require("pubnub").init({   publish_key: 'your publish key',    subscribe_key: 'your subscribe key' });

var pubnub = require("pubnub").init({   publish_key: 'demo',    subscribe_key: 'demo' });

var channel = 'test/influxdb';

var dbhost = 'localhost'; var database = 'mydb'; var username = 'root'; var password = 'root'; var timeSerie = 'pubnub_test_influxdb';

var dbInflux = influx({host : dbhost, username : username, password : password, database : database});

pubnub.subscribe({   channel  : channel,    callback : function(message) {        console.log( " > ", message );        dbInflux.writePoint(timeSerie,message,function(err) { if(err) throw err; });   } });

console.log("PubNub to InfluxDB bridge is started");

Lancez le programme node pubnub-pub.js dans un autre terminal /* var pubnub = require("pubnub").init({   publish_key: 'your publish key',    subscribe_key: 'your subscribe key' });

var pubnub = require("pubnub").init({   publish_key: 'demo',    subscribe_key: 'demo' });

var channel = 'test/influxdb';

function publish { var message= { line: 'Yo', like: Math.random * 10, star: Math.random * 5 }; pubnub.publish({        channel   : channel,        message   : message,        callback  : function(e) { console.log( "SUCCESS!", e ); },        error     : function(e) { console.log( "FAILED! RETRY PUBLISH!", e ); }    }); setTimeout(publish, 500); }

publish;

console.log("PubNub publishing is started");

Naviguez dans la time serie pubnub_test_influxdb de la base mydb avec la requête SELECT star FROM pubnub_test_influxdb

Node.js et MQTT


Installez sudo npm install influx sudo npm install mqtt

Lancez le programme node mqtt2influx.js dans un terminal var mqtt = require('mqtt') var influx = require('influx');

var broker = 'test.mosquitto.org'; var port = 1883; var topic = 'test/influxdb';

var dbhost = 'localhost'; var database = 'mydb'; var username = 'root'; var password = 'root'; var timeSerie = 'mqtt_test_influxdb';

var dbInflux = influx({host : dbhost, username : username, password : password, database : database});

client = mqtt.createClient(port, broker);

client.subscribe(topic).on('message', function (topic, message) {   console.log("<");        dbInflux.writePoint(timeSerie,JSON.parse(message),function(err) { if(err) throw err; }); });

console.log("MQTT to InfluxDB bridge is started");

Lancez le programme node mqttpub.js dans un autre terminal var mqtt = require('mqtt')

var broker = 'test.mosquitto.org'; var port = 1883; var topic = 'test/influxdb';

client = mqtt.createClient(port, broker);

client.subscribe(topic);

function publish { var message= { line: 'Yo', like: Math.random * 10, star: Math.random * 5 }; client.publish(topic, JSON.stringify(message)); console.log(">"); setTimeout(publish, 500); }

publish;

console.log("MQTT publishing is started");

Naviguez dans la time serie mqtt_test_influxdb de la base mydb avec la requête SELECT mean(star) as Mean_Star FROM mqtt_test_influxdb GROUP BY time(10s)

Apache Kafka to InfluxDB
TODO

https://www.npmjs.org/package/kafka-node

STOMP to InfluxDB
TODO

https://github.com/benjaminws/stomp-js

Octoblu to InfluDB
TODO

npm install skynet npm install request

Node-RED

 * https://github.com/node-red/node-red/issues/462

Java
git clone https://github.com/influxdb/influxdb-java.git cd influxdb-java-master mvn clean install -DskipTests=true
 * 1) mvn clean install

mvn archetype:generate -DgroupId=org.influxdb -DartifactId=influxdb-example -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

See https://github.com/influxdb/influxdb-java/blob/master/src/test/java/org/influxdb/InfluxDBTest.java

String ip="localhost";

this.influxDB = InfluxDBFactory.connect("http://" + ip + ":8086", "root", "root"); boolean influxDBstarted = false; do { Pong response; try { response = this.influxDB.ping; if (response.getStatus.equalsIgnoreCase("ok")) { influxDBstarted = true; } } catch (Exception e) { // NOOP intentional } Thread.sleep(100L); } while (!influxDBstarted); this.influxDB.setLogLevel(LogLevel.FULL);

String logs = CharStreams.toString(new InputStreamReader(containerLogsStream, Charsets.UTF_8));

String dbName = "writeseriebuilder-unittest-" + System.currentTimeMillis; this.influxDB.createDatabase(dbName); int outer = 20; List series = Lists.newArrayList; for (int i = 0; i < outer; i++) { Serie serie = new Serie.Builder("serieFromBuilder") .columns("column1", "column2") .values(System.currentTimeMillis, 1) .values(System.currentTimeMillis, 2) .values(System.currentTimeMillis, 3) .values(System.currentTimeMillis, 4) .values(System.currentTimeMillis, 5) .values(System.currentTimeMillis, 6) .values(System.currentTimeMillis, 7) .values(System.currentTimeMillis, 8) .values(System.currentTimeMillis, 9) .values(System.currentTimeMillis, 10) .build; series.add(serie); } this.influxDB.write(dbName, TimeUnit.MILLISECONDS, series.toArray(new Serie[0]));

Serie serie = new Serie.Builder("testSeries") .columns("value1", "value2") .values(System.currentTimeMillis, 5) .build; this.influxDB.write(dbName, TimeUnit.MILLISECONDS, serie); List result = this.influxDB.query(dbName, "select value2 from testSeries", TimeUnit.MILLISECONDS);

this.influxDB.deleteDatabase(dbName);

=Benchmark=
 * http://influxdb.com/blog/2014/06/20/leveldb_vs_rocksdb_vs_hyperleveldb_vs_lmdb_performance.html
 * https://github.com/influxdb/influxdb/tree/master/tools