Metrics produced by the NuPIC Inference Server work well with the Prometheus monitoring tool, and the collected metrics can be visualized on a Grafana dashboard. Metrics displayed on the dashboard are useful for monitoring and verifying CPU resource utilization, and for a real-time visual reference of throughput and latency.

Quick Start

Before you start, make sure that your network security policy allows inbound and outbound traffic from Grafana over port 3000. Additionally, if you are accessing the Inference Server remotely, similarly ensure that your client machine's IP is allowed to access the Inference Server.

Additionally, we'll have to add a flag when launching the Inference Server so that it emits metrics that can be picked up by Prometheus:

./nupic_inference.sh --start --expose-metrics

Navigate to the directory containing the dashboard example:

cd nupic.examples/examples/dashboard

Now start the dashboard:

docker compose up -d

Grafana can now accessed from the following URL in your web browser. If you are accessing remotely, replace localhost with the IP address of the inference server.

http://localhost:3000

The visualization showed at the top of this page can be accessed by clicking on Dashboards in the sidebar of the web interface.

You can also view a more minimalist version using kiosk mode:

http://localhost:3000/d/nupic_inference_server/?theme=light&kiosk

When you're done, remember to shut down the dashboard:

docker compose down

Add the -v flag to the above if you want to also reset the metrics data.

In More Detail

Interactions between the NuPIC Inference Container, Prometheus and Grafana are orchestrated using Docker Compose. The key components of this setup are as follows:

nupic.examples/examples/dashboard/
├── compose.yml                                 # Main docker compose file
├── grafana                                     # Grafana visualization server
│   ├── dashboards                              # Preconfigured dashboards
│   │   ├── dashboard.yml                       # Dashboard configuration
│   │   └── nupic_inference_server.json         # Sample dashboard
│   └── datasources                             # Grafana metrics datasources
│       └── prometheus.yml                      # Prometheus datasource configuration
└── prometheus                                  # Prometheus metrics server
    └── prometheus.yml                          # Prometheus configuration

Overall orchestration is configured in the compose.yml file, while configurations for the constituent components are located within their respective directories.

Prometheus is configured to collect metrics from the Inference Server via port 8002, which is why we needed the --expose-metrics flag when launching the latter.

The Grafana web interface, on the other hand, uses port 3000. You can adjust this through compose.yml.