Eyer ingress API - add new data sources support

The Eyer ingress API is highly flexible, and can receive metrics and meta-data from any data source as long as the data transmitted is in JSON format.

For now, registering a new data source type with Eyer involves a manual process, the process is outlined below.

1. You need to be able to extract the metrics you want to monitor from the data source (application / system / sensor etc) and be able to store / send the data in JSON. This can be done by open source agents such as Telegraf or Prometheus, or by coding / scripting your own data collector.

2. The data output should be aggregated to a minimum of 1 minute. Lower resolutions are accepted. Example: you read a metric every 5 second, and aggregate it over 1 minute (select max, min, avg etc) before you output it.

3. The JSON output should have

   1. metrics metadata like name / id and aggregation (avg, sum, mean etc) and with Unix timestamps.

   2. If you want to group metrics by area / type, this information should also be included (for instance CPU and RAM used is grouped under “Operating System”).

4. Send the data format and an example output file to Eyer, with an explanation of the format, metrics and the grouping you want to be applied. For more complex data structures (nested structures) we can also do a quick meeting to ensure compatibility.

5. Eyer will add the new data source mapping and expose the API for the new data source.

Below you see an example JSON data structure, which is used to collect Boomi process performance metrics

This is a flat metric structure, with a single grouping extracted from the “tags” section (processName and atomId form a unique identifier per process). The timestamp for the metric collection is provided in the Unix format.