InfluxDB: Difference between revisions

From air
Jump to navigation Jump to search
(Replaced content with "InfluxDB 0.8 =Chronograf= ''Chronograf is a single binary web application that you can deploy behind your firewall to do ad hoc exploration of your time series data ...")
Line 1: Line 1:
[[Image:InfluxDB-WebUI.png|400px|thumb|right|InfluxDB WebUI]]
[[InfluxDB 0.8]]

'''REMARQUE : cette page concerne la version 0.8 d'InfluxDB. De nombreux changements sont été opérés dans la version 0.9.'''


http://influxdb.com/

https://github.com/influxdb/influxdb

open-source distributed time series database with no external dependencies.

(metrics, events, and analytics)

developped in [[Go]] language

Voir https://speakerdeck.com/pauldix/introducing-influxdb-an-open-source-distributed-time-series-database

Demo sur http://play.influxdb.com/ ([https://github.com/influxdb/play.influxdb.org source code])

=InfluxDB @ AIR=
* [[SmartCampus]]

=Installation=

http://influxdb.com/download/

==On Linux (Debian)==
<pre>
# for 64-bit systems
wget http://s3.amazonaws.com/influxdb/influxdb_latest_amd64.deb
sudo dpkg -i influxdb_latest_amd64.deb

# for 32-bit systems
wget http://s3.amazonaws.com/influxdb/influxdb_latest_i686.deb
sudo dpkg -i influxdb_latest_i686.deb
</pre>

==On OS X==
<pre>
brew update
brew install influxdb
</pre>

=Démarrage=
<pre>
sudo /etc/init.d/influxdb start
</pre>

Naviguez sur http://localhost:8083/
username : root & password : root

Pensez à changer ces valeurs dans /usr/local/etc/influxdb.conf quand vous mettez influxdb en production

=Premiers Pas=

==Via l'interface Web==
Depuis l'interface web http://localhost:8083/, Créez une base : mydb

Depuis ''Database > Explore Data''

Depuis ''write point'', Ajoutez un point dans la série temporelle ''log_lines''

<pre>
{ "line":"here's some useful log info from paul@influx.com", "like":1, "star": 5 }
</pre>
puis un autre
<pre>
{ "line":"here's another useful log info from paul@influx.com", "like":2, "star": 4 }
</pre>
puis un autre
<pre>
{ "line":"here's some useful log info from didier@donsez.com", "like":1, "star": 5 }
</pre>

Exécutez les requêtes suivantes depuis ''read point'':


La liste des séries temporelles
<pre>
list series
</pre>
<pre>
select * from /.*/ limit 1
</pre>


Les points de la série log_lines
<pre>
select line from log_lines
</pre>

<pre>
select like, star from log_lines
</pre>


Les deux points les plus récents
<pre>
select * from log_lines limit 2;
</pre>

Les points de la série log_lines dont la colonne line contient paul@influx.com
<pre>
select line from log_lines where line =~ /paul@influx.com/
</pre>

Les points de la série log_lines dont la colonne like est supérieure à 1
<pre>
select line from log_lines where like > 1
</pre>

Les points de la série log_lines des 24 dernières heures
<pre>
select * from log_lines
where time > now() - 24h
</pre>


Agrégat temporel (les fonctions d'agrégat sont : count(), min(), max(), mean(), mode(), median(), distinct(), percentile(), histogram(), derivative(), sum(), stddev(), first(), last()).
<pre>
select sum(like) as number_of_likes, mean(star) as mean_of_star from log_lines
group by time(1m)
where time > now() - 1d
</pre>
<pre>
select star, sum(like) as number_of_likes from log_lines
group by star, time(1m)
where time > now() - 1d
</pre>

Remplissage des intervalles vides de points (avec fill())
<pre>
select count(like) from events
group by time(1h) fill(0)
where time > now() - 24h
</pre>


Fusion (merge) de séries
<pre>
select count(type) from user_events merge admin_events group by time(10m)
</pre>


Attention, cette requête n'est pas un produit cartésien (SQL)
<pre>
select * from log_lines, log_cpu;
</pre>

Jointure entre 2 séries
<pre>
select errors_per_minute.value / page_views_per_minute.value as error_rate_per_minute.
from errors_per_minute
inner join page_views_per_minute
</pre>

Création d'un [http://influxdb.com/docs/v0.8/api/continuous_queries.html Requêtes continues]
<pre>
select count(star) from log_lines group by time(10m), star
into log_lines.count_per_star.10m
</pre>

<pre>
select * from log_lines.count_per_star.10m
</pre>

<pre>
drop continuous query <id>
</pre>



Supprimez les points de toutes les séries temporelles dont la date est antérieure de 24 heures.
<pre>
delete from /.*/ where time < now() - 24h
select * from /.*/
</pre>

Suppression d'une série
<pre>
drop series log_lines
</pre>

==Via l'interface REST==
Execute a query (pretty=true --> the output is prettily formatted)
<pre>
curl -G 'http://localhost:8086/db/mydb/series?u=root&p=root&pretty=true' --data-urlencode "q=select * from log_lines"
</pre>

Write points ([http://influxdb.com/docs/v0.8/api/reading_and_writing_data.html#specifying-time-and-sequence-number-on-writes with timestamp and sequence numbers : the time_precision can be s, us, ms])
<pre>
curl -X POST -d @points.json 'http://localhost:8086/db/mydb/series?u=root&p=root&time_precision=ms'
</pre>

'''points.json'''
<pre>
[
{
"name": "log_lines",
"columns": ["time", "sequence_number", "line"],
"points": [
[1400425947368, 1, "this line is first"],
[1400425947368, 2, "and this is second"],
[1400425948368, 3, "and this is third"]
]
}
]
</pre>


===Administration===
create a database
<pre>
curl -X POST 'http://localhost:8086/db?u=root&p=root' -d '{"name": "site_development"}'
</pre>

drop a database
<pre>
curl -X DELETE 'http://localhost:8086/db/site_development?u=root&p=root'
</pre>

Autre : http://influxdb.com/docs/v0.8/api/administration.html

==Dashboard avec Influga==
[[Image:Influga.png|400px|thumb|right|Influga]]
[https://github.com/hakobera/influga Influga] is a InfluxDB Dashboard and Graph Editor.

Installez influga
<pre>
sudo npm install influx
sudo npm install -g influga
</pre>

Editez la configuration dans influga-config.json
<pre>
{
"dashboardDbPath": "./db/influga.db",
"host": "localhost",
"port": 8086,
"database": "mydb",
"username": "root",
"password": "root"
}
</pre>

Lancez le service
<pre>
influga start -c influga-config.json
</pre>


Naviguez sur http://localhost:8089/ et configurez le panel avec
<pre>
select star, like from log_lines
</pre>

==Dashboard avec [[Grafana]]==
http://influxdb.com/docs/v0.7/ui/grafana.html

Téléchargez Grafana

Modifiez config.example.js avec
<pre>
datasources: {
'eu-metrics': {
type: 'influxdb',
url: 'http://localhost:8086/db/mydb',
username: 'test',
password: 'test',
},
'grafana': {
type: 'influxdb',
url: 'http://localhost:8086/db/grafana',
username: 'test',
password: 'test',
grafanaDB: true
},
},
</pre>

et Sauvegardez dans config.js

Ouvrez index.html

=Premières Requêtes=


==[[Node.js]]==
* https://github.com/bencevans/node-influx

Installez
<pre>
sudo npm install influx
</pre>

Editez le programme testinflux.js
<pre>
var influx = require('influx');

console.log("Connect to database");
var username = 'root';
var password = 'root';
var database = 'mydb';

var dbInflux = influx({host : 'localhost', username : username, password : password, database : database});

/*
console.log("Create database");
dbInflux.createDatabase('mydb', function(err) {
if(err) throw err;
console.log('Database Created');
});
*/

console.log("Write points");

var i=10;
while(i--) {
dbInflux.writePoint('log_lines', { line: 'Yo', like: Math.random() * 10, star: Math.random() * 5 }, function(err) {
if(err) throw err;
});
console.log("+");

}

console.log("Query series");

var query = 'SELECT * FROM log_lines WHERE time > now() - 24h';
dbInflux.query(query, function(err, body) {
if(err!=null) throw err;
console.log(JSON.stringify(body[0].columns, null, '\t'));
console.log(JSON.stringify(body[0].points, null, '\t'));
});

</pre>

Lancez le programme
<pre>
node testinflux.js
</pre>

==[[PubNub]] to InfluxDB==

Créez un compte gratuit sur http://pubnub.com et renseignez les clés récues par mail
(les clés demo & demo fonctionnent quand même)

Installez
<pre>
sudo npm install influx
npm install pubnub
</pre>


Lancez le programme ''node pubnub2influx.js'' dans un terminal
<pre>

var influx = require('influx');

/*
var pubnub = require("pubnub").init({
publish_key: 'your publish key',
subscribe_key: 'your subscribe key'
});
*/

var pubnub = require("pubnub").init({
publish_key: 'demo',
subscribe_key: 'demo'
});

var channel = 'test/influxdb';

var dbhost = 'localhost';
var database = 'mydb';
var username = 'root';
var password = 'root';
var timeSerie = 'pubnub_test_influxdb';

var dbInflux = influx({host : dbhost, username : username, password : password, database : database});

pubnub.subscribe({
channel : channel,
callback : function(message) {
console.log( " > ", message );
dbInflux.writePoint(timeSerie,message,function(err) {
if(err) throw err;
});
}
});

console.log("PubNub to InfluxDB bridge is started");
</pre>

Lancez le programme ''node pubnub-pub.js'' dans un autre terminal
<pre>
/*
var pubnub = require("pubnub").init({
publish_key: 'your publish key',
subscribe_key: 'your subscribe key'
});
*/

var pubnub = require("pubnub").init({
publish_key: 'demo',
subscribe_key: 'demo'
});

var channel = 'test/influxdb';

function publish() {
var message= { line: 'Yo', like: Math.random() * 10, star: Math.random() * 5 };
pubnub.publish({
channel : channel,
message : message,
callback : function(e) { console.log( "SUCCESS!", e ); },
error : function(e) { console.log( "FAILED! RETRY PUBLISH!", e ); }
});
setTimeout(publish, 500);
}

publish();

console.log("PubNub publishing is started");
</pre>

Naviguez dans la time serie ''pubnub_test_influxdb'' de la base ''mydb'' avec la requête
<pre>
SELECT star FROM pubnub_test_influxdb
</pre>

==[[Node.js]] et [[MQTT]]==
[[Image:InfluxDB-WebUI.png|400px|thumb|right|InfluxDB WebUI]]

Installez
<pre>
sudo npm install influx
sudo npm install mqtt
</pre>


Lancez le programme ''node mqtt2influx.js'' dans un terminal
<pre>
var mqtt = require('mqtt')
var influx = require('influx');

var broker = 'test.mosquitto.org';
var port = 1883;
var topic = 'test/influxdb';


var dbhost = 'localhost';
var database = 'mydb';
var username = 'root';
var password = 'root';
var timeSerie = 'mqtt_test_influxdb';

var dbInflux = influx({host : dbhost, username : username, password : password, database : database});

client = mqtt.createClient(port, broker);

client.subscribe(topic).on('message', function (topic, message) {
console.log("<");
dbInflux.writePoint(timeSerie,JSON.parse(message),function(err) {
if(err) throw err;
});
});

console.log("MQTT to InfluxDB bridge is started");
</pre>

Lancez le programme ''node mqttpub.js'' dans un autre terminal
<pre>
var mqtt = require('mqtt')

var broker = 'test.mosquitto.org';
var port = 1883;
var topic = 'test/influxdb';

client = mqtt.createClient(port, broker);

client.subscribe(topic);

function publish() {
var message= { line: 'Yo', like: Math.random() * 10, star: Math.random() * 5 };
client.publish(topic, JSON.stringify(message));
console.log(">");
setTimeout(publish, 500);
}

publish();

console.log("MQTT publishing is started");
</pre>

Naviguez dans la time serie ''mqtt_test_influxdb'' de la base ''mydb'' avec la requête
<pre>
SELECT mean(star) as Mean_Star FROM mqtt_test_influxdb GROUP BY time(10s)
</pre>


==[[Apache Kafka]] to InfluxDB==

TODO

https://www.npmjs.org/package/kafka-node

==[[STOMP]] to InfluxDB==
TODO

https://github.com/benjaminws/stomp-js

==[[Octoblu]] to InfluDB==
TODO

<pre>
npm install skynet
npm install request
</pre>

<pre>



</pre>

==[[Node-RED]]==
* https://github.com/node-red/node-red/issues/462

==Java==

<pre>
git clone https://github.com/influxdb/influxdb-java.git
cd influxdb-java-master
mvn clean install -DskipTests=true
# mvn clean install



mvn archetype:generate -DgroupId=org.influxdb -DartifactId=influxdb-example -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false
</pre>


See https://github.com/influxdb/influxdb-java/blob/master/src/test/java/org/influxdb/InfluxDBTest.java

<pre>
String ip="localhost";

this.influxDB = InfluxDBFactory.connect("http://" + ip + ":8086", "root", "root");
boolean influxDBstarted = false;
do {
Pong response;
try {
response = this.influxDB.ping();
if (response.getStatus().equalsIgnoreCase("ok")) {
influxDBstarted = true;
}
} catch (Exception e) {
// NOOP intentional
}
Thread.sleep(100L);
} while (!influxDBstarted);
this.influxDB.setLogLevel(LogLevel.FULL);

String logs = CharStreams.toString(new InputStreamReader(containerLogsStream, Charsets.UTF_8));


String dbName = "writeseriebuilder-unittest-" + System.currentTimeMillis();
this.influxDB.createDatabase(dbName);
int outer = 20;
List<Serie> series = Lists.newArrayList();
for (int i = 0; i < outer; i++) {
Serie serie = new Serie.Builder("serieFromBuilder")
.columns("column1", "column2")
.values(System.currentTimeMillis(), 1)
.values(System.currentTimeMillis(), 2)
.values(System.currentTimeMillis(), 3)
.values(System.currentTimeMillis(), 4)
.values(System.currentTimeMillis(), 5)
.values(System.currentTimeMillis(), 6)
.values(System.currentTimeMillis(), 7)
.values(System.currentTimeMillis(), 8)
.values(System.currentTimeMillis(), 9)
.values(System.currentTimeMillis(), 10)
.build();
series.add(serie);
}
this.influxDB.write(dbName, TimeUnit.MILLISECONDS, series.toArray(new Serie[0]));


Serie serie = new Serie.Builder("testSeries")
.columns("value1", "value2")
.values(System.currentTimeMillis(), 5)
.build();
this.influxDB.write(dbName, TimeUnit.MILLISECONDS, serie);
List<Serie> result = this.influxDB.query(dbName, "select value2 from testSeries", TimeUnit.MILLISECONDS);

this.influxDB.deleteDatabase(dbName);

</pre>


==[[OpenHAB]]==
The states of an item are persisted in time series with names equal to the name of the item. All values are stored in a field called "value" using integers or doubles, OnOffType and OpenClosedType values are stored using 0 or 1. The times for the entries are calculated by InfluxDB.

* https://github.com/openhab/openhab/wiki/InfluxDB-Persistence

=Benchmark=
* http://influxdb.com/blog/2014/06/20/leveldb_vs_rocksdb_vs_hyperleveldb_vs_lmdb_performance.html
* https://github.com/influxdb/influxdb/tree/master/tools




Revision as of 08:26, 13 September 2015

InfluxDB 0.8


Chronograf

Chronograf is a single binary web application that you can deploy behind your firewall to do ad hoc exploration of your time series data in InfluxDB.

Installation

wget https://s3.amazonaws.com/get.influxdb.org/chronograf/chronograf_0.1.0_amd64.deb
sudo dpkg -i chronograf_0.1.0_amd64.deb

sudo service chronograf start
Retrieved from "https://air.imag.fr/index.php?title=InfluxDB&oldid=23648"