3 Industry standard performance and power benchmarking

While different SoC vendors use different metrics to benchmark deep learning performance and power across multiple devices, there is an initiative in the industry to standardize benchmarking for apples-to-apples comparison. Driven by more than 30 organizations, MLPerf Inference [9] prescribes a set of rules and best practices to ensure comparability across systems with wildly differing architectures. We will primarily use these guidelines for performance and power benchmarking.