MilSpec is a set of open-source benchmarks targeted at national defense applications – but also useful for the broader accelerator design space. These benchmarks focus on end-to-end applications that exercise a wide range of signal processing, communications, image processing/computer vision, knowledge formation, and other capabilities.
Benchmark | Functions | Getting Started | Code/Data | External Source/Docs | Updated | |
---|---|---|---|---|---|---|
The MUltiple SIgnal Classification (MUSIC) algorithm is a method for estimating the direction of arrival of signals received by an antenna array. This standalone test inputs a simulated set of signal samples, applies the MUSIC algorithm, and measures the angle estimation error and compute time. |
Linear algebra | Getting started with MUSIC | MUSIC Code/Data | 06/01/2020 | ||
Passive radar is a radar mechanism where the receiver relies on an external signal source, rather than transmitting radar pulses itself. This end-to-end test processes a time-series set of signal samples to determine target position and measures the compute time for each processing step. |
FFT, linear algebra, filtering, cross-correlation, beamforming, DoA estimation | Getting Started with Passive Radar | Passive Radar Code , Sample data (non-commercial use only) | 06/01/2020 | ||
Transcendental functions are a class of non-polynomial operations (sin, log, exp, etc.) that are fundamental to various algorithms, particularly in signal processing. This benchmark tests the performance of several methods for approximating transcendental function values at varying degrees of precision. |
CORDIC, Taylor series expansion | Getting Started with Transcendentals | Transcendentals Code/Data | 06/01/2020 | ||
Particle filtering is a method of efficiently estimating the true state of a complex environment given noisy data. This benchmark tests the performance and result quality of a Gaussian particle filter applied to estimate the true position of a moving object from noisy samples. |
Probability/statistics, weighted resampling | Getting started with Particle Filtering | Particle Filtering Code/Data | 07/20/2020 |
Benchmark | Functions | Getting Started | Code/Data | External Source/Docs | Updated | |
---|---|---|---|---|---|---|
Synthetic Aperture Radar constructs images based on data from a moving radar platform (e.g., mounted on an aircraft to scan a large ground region). This benchmark tests an end-to-end sensor processing pipeline, forming complete images from synthetically generated radar returns and then identifying objects in those returns. |
FFT, image processing, knowledge formation, data retrieval | Getting Started with SAR | SAR Code/Data | 02/04/2020 | ||
This benchmark uses a visual processing system to identify and track multiple human actors simultaneously in a variety of recorded video clips. |
Computer vision, multiple object tracking | Getting Started with RTTrack | RTTrack Code/Data | SORT Github MOT Challenge Sample Video |
01/21/2020 | |
Assignment problems use combinatorial optimization to find the most efficient mapping between two sets - for instance, when mapping a large number of tasks to agents that will perform them, as might be required with autonomous cooperative swarms of robots. This benchmark runs a synthetic test of the auction, Hungarian, and Jonker-Volgenant algorithms for finding such mappings. |
Combinatorial optimization, linear algebra, matrix math | Getting Started with LAP | Linear Assignment Code/Data | Auction algorithm overview Jonker-Volgenant algorithm overview Hungarian algorithm overview |
03/16/2020 | |
Pulse-Doppler radar transmits a series of radar pulses and then detects object motion per the Doppler effect. It is commonly used for tracking airborne targets or in weather applications. This benchmark processes simulated radar returns with a four-step pipeline to find Doppler peaks and detect object positions/velocities. |
Pulse compression, 3-Pulse cancellation, Doppler computation, FFT | Getting Started with Pulse-Doppler Radar | Pulse-Doppler Code/Data | Pulse-Doppler Radar Overview Baseline Implementation |
02/04/2020 | |
Epipolar geometry defines the relationships between points across multiple perspectives on a three-dimensional scene. This benchmark computes corresponding points in pairs of images using multiple user-selectable algorithms. |
Matrix math, feature detection, k-nearest-neighbors, fundamental matrix estimation, epipole determination/epipolar line calculation | Getting Started with Epipolar Geometry | Epipolar Geometry Code/Data | Epipolar Geometry Overview Original Python Source Additional Data - Generic Additional Data - Vehicular |
01/21/2020 | |
Optical flow is a means of detecting the relative motion of a scene and an observer. This benchmark identifies points of interest (whose motion can be easily tracked) in an image, then identifies the corresponding points in a second image and estimates relative motion. |
Computer vision/image processing, corner detection, Lucas-Kanade algorithm | Getting Started with Optical Flow | Optical Flow Code/Data | Optical Flow Overview Original Python Source Additional Data - Vehicular |
02/04/2020 | |
Matched filtering can improve an effective signal-to-noise ratio by "matching" a known signal template to an unknown input signal expected to follow that template. This test inputs a signal with configurable random noise, applies a matched filter, and then measures the output error rate and compute time. |
Matrix Math, convolution, correlation, 16QAM encode/decode | Getting started with Matched Filtering | Matched Filter Code/Data | Intro to matched filters | 03/16/2020 | |
Wavelets are signal pulses, with finite duration, based on assorted mathematical functions. They are useful for assorted signal and image processing applications e.g. noise reduction; this test evaluates the performance of a discrete wavelet transform algorithm on denoising a set of input images (or synthetic vectors). |
Discrete wavelet transform, matrix math, linear algebra, convolution, decomposition | Getting started with Wavelets | Wavelets Code/Data | Intro to wavelets | 03/16/2020 |
Benchmark | Functions | Getting Started | Code/Data | External Source/Docs | Updated | |
---|---|---|---|---|---|---|
GNU Radio is an open-source framework for software-defined radio (SDR). This benchmark evaluates the performance of an SDR receiver with DSP, decoding, and decryption steps, from the raw radio signal to the final audio output. |
SDR, FFT, DSP, ECC, cryptography | GNU Radio | 01/21/2020 | |||
A highly configurable benchmark for emulating end-to-end IoT analytics pipelines, including device-side kernels, server-side processing, and communication protocols. It includes several pluggable processing modules for different IoT applications, and reports aggregate and distributions of metrics (e.g. service latency and processing rate). |
device-server comms, CoAP, MQTT, Statistics, transformation, training, analytics | 01/21/2020 |