**This task is responsible for creating and maintaining the benchmarks for Theme #2, and for defining and measuring the key metrics of importance for all designs produced by the Theme #2’s PIs and the Theme #1 development tools.
The benchmarks domains of this theme will be the same as those defined in Task 1.5, specifically the visual computing and natural language processing benchmarks developed by Theme #1, but in the context of hardware solutions, and accelerators in particular. Moreover, benchmarks for this theme will have the added constraint that their use in this theme requires appropriate configuration of the applications for use in datacenter and edge devices. These varied targets will assess the ability of the design tools and implementation targets to serve two very different platform classes. The metrics in this theme will attempt to assess the effectiveness of the theme’s development tools to provide an low-cost and efficient target for the systems produced by Theme #1. In particular, the theme will measure, with respect to the benchmark development and redevelopment as additional algorithmic components become available. Specifically, for edge systems, we will target the following quality of results metrics: power (Watt), performance (sec/work), and area (mm2). For datacenter systems, we will measure power (Watt), performance (sec/work), and area (mm2).
