Components of the Monitoring Pipeline

The complete performance monitoring infrastructure consist of several components forming a processing pipeline. The following graph visualizes the contained components and their data flow connections.

digraph component {
   compound = true;
   node [shape=record];

   "rsb-process-monitor*" -> "rsbperfmon-db-adapter" [label="rst.monitoring.ProcessCues"];
   "rsb-host-monitor*" -> "rsbperfmon-db-adapter" [label="rst.device.generic.HostInformation"];

   subgraph clusterbackend {
      label = "Timeseries Backend";

      "Graphite";
      "backenddummy" [shape=point style=invis];
      "InfluxDB";
   }

   "rsbperfmon-db-adapter" -> "backenddummy" [lhead=clusterbackend,minlen=1];
   "backenddummy" -> "Grafana Dashboard" [ltail=clusterbackend];

}

Measurement Layer

rsb-process-monitor and rsb-host-monitor perform the measurement of performance counters. Instances of the former program collect data for one or several processes identified via PID. rsb-host-monitor instances collect general performance data regarding a single computers. Both programs expose their data in a cyclic fashion via RSB using RST data types.

Backend Layer

In order to visualize the gathered performance data, a timeseries backend is filled with them so that visualizations can work on such a specific architecture. rsbperfmon-db-adapter accepts the incoming data via RSB and converts it in to a format suitable for timeseries databases like InfluxDB or Graphite (whisper) Graphite. Afterwards, the converted data is inserted into one of the available database backends.

Visualization Layer

In order to visualize the gathered data, the Grafana dashboard system is used. Grafana pulls the timeseries data from one of the database backends and visualizes it in a browser-based dashboard using client-side Javascript plotting.