ADA-Contributed Benchmark Suites |
|||
|---|---|---|---|
| Benchmark Suite | Scope | URL | Projects that Adopted the Suite |
| BayesSuite | Bayesian Inference benchmark suite. | https://github.com/Emma926/BayesSuite | |
| BeXAI | A benchmark suite for explainable AI workloads. | https://adacenter.org/bexai/ | |
| DeepRecSys | A System for Optimizing End-To-End At-scale Neural Recommendation Inference | https://github.com/harvard-acc/DeepRecSys | |
| DyGraph | Dynamic Graph Datasets | https://adacenter.org/dygraph | OMEGA/ MessageFusion/ Centaur/ DREDGE |
| FPBench | Benchmarks, compilers, and standards for the floating-point research community. | https://github.com/FPBench | |
| Frontsuite | Suite of data center applications used for i-spy and ripple evaluation | https://github.com/efeslab/frontsuite | Intel, Arm, Qualcomm, University of Wisconsin, Texas A&M University, UPC Barcelona, EPFL, and the University of Edinburgh |
| GBBS | Graph-based algorithms: connectivity, covering, eigenvector, substructure, and shortest path | https://github.com/ldhulipala/gbbs | OMEGA/ MessageFusion/ Centaur/ DREDGE PI Shun's group is using the parallel primitives and graph representations in GBBS to implement new parallel static and dynamic graph algorithms. Google is adopting GBBS for their in-memory graph computations. |
| GenomicsBench | Benchmark suite of recent genomics applications | https://genomicsbench.eecs.umich.edu/ | PI Batten's group is using Genomics Bench for their Ephemeral Vector Engine (EVE) project and big.VLITTLE project PI Das's group is using GenomicBench for evaluating accelerator architectures in this domain. |
| ILLIXR | Illinois Extended Reality testbed: a fully open-source end-to-end Virtual and Augmented Reality system and research testbed (including standalone constituent components). | https://illixr.github.io/ | PIs Adve, Brooks and Wei are using ILLIXR for their work on accelerator design space exploration. PIs Adve and Gavrilovska are using ILLIXR for research on computation off-loading and multiparty AR. |
| Mobile Edge Computing (MEC) | A suite of application benchmarks and software stack components for a mobile edge computing testbed and different application use cases. | https://github.com/GTkernel/edge-benchmark-suite | |
| MilSpec | Suite of end-to-end open-source benchmarks relevant in national defense applications, employing key applications in the embedded systems space. | https://gitlab.eecs.umich.edu/wpehrett/milspec | |
| MLPerf | Machine-learning applications benchmark suite | https://github.com/mlcommons/ | PI Kasikci's group is using MLPerf for profile-driven optimizations for ML systems. |
| NTTSuite | A collection of NTT algorithms implemented as HLS-amenable code. | https://github.com/pbr- |
|
| ParaDNN | Parametric neural network benchmarks |
https://github.com/Emma926/paradnn | |
| Parapoly | A Massively Parallel Polymorphic Benchmark Suite | https://github.com/purdue-aalp/Parapoly | |
| RecPipe | HW/SW models for multi-stage recommendation systems’ performance | https://github.com/harvard-acc/RecPipe | harvard-acc/RecPipe |
| TinyMLPerf | ML benchmarks for embedded devices | https://github.com/mlcommons/tiny | ML Systems |
| VIP Bench | Privacy-enhanced computation benchmarks | https://bitbucket.org/vip-benchmarks/vip-bench/ | PI Austin’s group is using VIP Bench for Austin's startup, Agita Labs PI Reagan's group is using VIP Bench for accelerator evaluation research. PI Batten's group is to using VIP Bench and GenomicBench to explore hardware acceleration opportunities. |
JUMP-Contributed Benchmark Suites |
|||
| SAGA bench | Streaming graph benchmarks | https://github.com/abasak24/SAGA-Bench | |
Community-Contributed Benchmark Suites |
|||
| AutomataZoo | Benchmark repository for automata-based applications and automata engines for both von-Neumann and reconfigurable dataflow architectures. | https://github.com/tjt7a/Automatazoo | |
| CEP - Lincoln Lab | Publicly available SoC including several hardware blocks | https://www.ll.mit.edu/r-d/projects/common-evaluation-platform | |
| CloudSuite | Benchmark suite for cloud services. | https://github.com/parsa-epfl/cloudsuite | |
| CORAL-2 Benchmarks | Kernels inspired by natural science applications and core CS algorithms, including data science and ML | https://asc.llnl.gov/coral-2-benchmarks/ | |
| DIMACS implementation challenge - graph datasets | Road networks | http://users.diag.uniroma1.it/challenge9/download.shtml | |
| GAP Benchmark Suite | Static graph processing suite | https://github.com/sbeamer/gapbs | |
| HPC Challenge Benchmark | Suite for range memory access patterns. | http://icl.cs.utk.edu/hpcc/ | |
| MachSuite | Suite for accelerator design and customized architectures. | https://github.com/breagen/MachSuite | |
| MAVBench | Simulator and benchmark suite for micro aerial vehicle design | https://github.com/harvard-edge/MAVBench | |
| MEVBench | Mobile computer vision benchmarking suite. | http://vhosts.eecs.umich.edu/mevbench// | |
| MiBench and MiBench2 | Commercially representative embedded benchmark suite. MiBench2 is ported for IoT devices. | http://vhosts.eecs.umich.edu/mibench// | |
| MicroSuite | Benchmark suite for microservices. | https://github.com/wenischlab/MicroSuite | |
| NAS Parallel | Small set of programs designed to help evaluate the performance of parallel supercomputers. | https://www.nas.nasa.gov/publications/npb.html | |
| PARSEC | Benchmark suite composed of multithreaded programs, targeting emerging workloads for shared-memory programs for chip-multiprocessors. | https://parsec.cs.princeton.edu/index.htm | |
| Rodinia | Kernel suite of compute-intensive applications | https://rodinia.cs.virginia.edu/doku.php | |
| SNAP - Static Graph Datasets | Large graph datasets | http://snap.stanford.edu/data/index.html | |
| STINGER | Dynamic graph algorithms | http://www.stingergraph.com/ | |
| OpenNMT | Attention-based speech-to-text and machine translation | https://opennmt.net | |
| ERASER | Interpretable NLP models | http://www.eraserbenchmark.com | |
| Transformer | Transformer-based NLP models | https://github.com/huggingface/transformer | |
© 2023 Center for Applications Driving Architectures. All Rights Reserved.
