Configuration

Measurement Cycle Time

Performance counters are measured periodically. Therefore the first aspect to decide is the measurement cycle time. This is the time between two consecutive measurements. It defines the resolution of available data but also the required storage space of the timeseries database backend as well as the computational load generated by the monitoring infrastructure. Additionally, too short cycle times will not increase the effective resolutions because several performance counters are only updated infrequently by the Linux kernel. Therefore recommended values could be 5 or 10 seconds.

Configuration of the Timeseries Backend

InfluxDB

Configure InfluxDB as desired using the example config files and the instructions found here in the installation instructions. An exemplary config file which uses just a local computer and /tmp for data can be downloaded. Please note that in this file the assets line in the admin section needs to be fixed so that it point to the installation of the required files for the admin web interface. After adapting the file, start the database server, e.g. using

$prefix/opt/influxdb/influxdb -config=path/to/influxdb-config.toml

Using the web interface (http://localhost:8083 per default), create a database for the monitoring data. The retention for the proposed default shard space should be set to the desired time you plan to keep data available in the database (e.g. 7d). Duration should probably be set to 1d.

Graphite

Unfortunately, Graphite configuration is a bit harder than for InfluxDB. Basically, you need to follow these instructions. The steps are:

1. Configure the carbon daemon, which accepts new data via HTTP and pushes them into the timeseries database.

   1. Create a carbon.conf file and adapt as required. The comments about all data directories descending from STORAGE_DIR are misleading. This is only true for the defaults. In case you want to move your data, you need to set all of these directories explicitly. An exemplary configuration file can be downloaded. The file needs to be placed in the conf directory of your Graphite installation root.
   2. Create a storage-schemas.conf file which defines the retention policies. The highest resolution must match the chosen measurement cycle time. Otherwise you will end up with null values in the database. An examplary file can be downloaded. The file needs to be placed in the conf directory of your Graphite installation root.

2. Configure the web application

   1. Create a local_settings.py according to the example file. The STORAGE_DIR` variable needs to match the value that has been used for carbon. Otherwise, the webapp won’t find the database. An example can be downloaded. This file needs to be placed in the webapp/graphite directory of your installation. In case you do not want to set up a custom webserver, it is important to set DEBUG = True.

   2. Collect static webapp files to be served by the wbeserver:

      export PYTHONPATH=$GRAPHITE_ROOT/webapp:$PYTHONPATH
      django-admin.py collectstatic --noinput --settings=graphite.settings
      

   3. Create a configuration database for django:

      PYTHONPATH=$GRAPHITE_ROOT/webapp django-admin.py syncdb --settings=graphite.settings
      

Configuration of the Grafana Dashboards

Create an ini file to configure the Grafana server. An exemplary file can be downloaded. You can place this file anywhere.

Initial Online Configuration

In addition to providing a config file, Grafana dashboards need to be configured once the Grafana server is running. For this purpose, first start the monitoring components as described at Component Startup. Afterwards, follow these steps once:

1. Clone the dashboard repository to your local computer:

   git clone https://code.cor-lab.org/git/rsb-performance-monitor.git.dashboards monitoring-dashboards
   

2. Open the Grafana webpage at http://localhost:8020.

3. On the left side, log in as admin (default: admin:admin).

4. In the left menu, select Data Sources ‣ Add new.

5. Add a data source matching your selected backend. For Graphite, the given port is the one used to start the devel server (as described in Component Startup). The InfluxDB data source must be called influxdb and the graphite one graphite.

6. Go to Dashboards in the left menu.

7. In the Home dropdown menu at the top, select Import and import the appropriate dashboards for the selected timeseries database backend from the folder that was cloned in step 1.