Benchmark Testing

Benchmarking refers to the process of measuring and comparing the performance of a system using a set of standardized tests. Tracking the benchmark scores of a system provides an objective way of identifying how hardware or software components can affect the system performance. For example, the benchmark score obtained from a computer vision benchmark test may indicate whether a GPU upgrade is required.

PSYGIG Mobility IoT SDK includes a comprehensive set of benchmark tests designed to measure the performance of the most basic components and functions of an embedded system.

The Benchmark Testing API includes many features suitable for integration into embedded systems:

  • Works out-of-the-box; just run psybench

  • Full benchmark test suite:

cpu

CPU

gpu

GPU

ram

RAM

disk

Storage devices

network

Network

cv

Computer vision 1
1

Stereo matching, Optical flow, Visual odometry, SLAM, Object detection, Object classification, Object tracking, 3D estimation, Velocity estimation

  • Customizable and extensible benchmark tests

  • View comparison charts of your device with built-in integration in the PSYGIG data platform

  • Generate an HTML report from the benchmark results

Note

The Benchmark Testing feature is a work-in-progress. Please contact us if you are interested in beta-testing this feature.

Running the benchmark tests

Simply run the included standalone binary psybench to run the full benchmark suite. For example:

psybench -f json -o benchmark-results.json

See psybench man page for a complete list of options.

Using PSYGIG SDK API

The Benchmark Testing API can be accessed by including a single header file and linking the libpsyiage shared library. See Advanced Setup - Building your application with libpsyiage for details.

Running the CPU Benchmark Test

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
#include <psyiage/psyiagesdk.h>

int main(int argc, char **argv)
{
    psyiage_handle     pah;  // handle to psyiage instance

    /* ... Initialize code omitted for simplicity ... */

    // Run CPU benchmark test
    float cpuscore;
    rc = psyiage_benchmark_run(pah, PSYIAGE_BENCH_CPU, cpuscore);

    /* ... Cleanup code omitted for simplicity ... */
}

Sample benchmark JSON results

{
    "BenchResults": {
        "Type: "CPU"
        "StartTime": 1547534065,
        "StartTimeOfDay": "2019-01-15 15:34:25",
        "TestDuration": 23,
        "Score": 1843.8
    }
}