The Right Metrics for Measuring Compute Efficiency
11/13/20251 min read
On a daily basis, I see multiple new and different metrics to indicate the progress of new neural networks or GPUs. However, these metrics tend to be super specific and are only representative for marketing purposes.
There is nothing worse than making product decisions based on the wrong metrics, defining wrong metrics as those that are not the most representative for the given product or service.
As a consequence, I think the most representative metrics for measuring compute are operations per watt and operations per dollar:
Operations per watt is an excellent metric due to the linkage of physical work to energy quantities. As a result, we can measure the raw performance of a system against others, without the differences of FP4, FP16, etc. So, I think operations per watt is the ultimate measure to quantify performance, because it measures the given output (operations) in relation to the given input (energy). However, this metric alone is incomplete to make conclusions because it completely ignores the cost of the solution. A system could be incredibly efficient but also prohibitively expensive.
Operations per dollar is representative because it links compute with its real cost, intuitively matching work with the true cost. This metric is complementary because it serves to justify the investment over a more efficient technology. If output increases at a higher rate than the cost of the more efficient technology, it would be financially viable. So, to sum up, operations per dollar truly represents how feasible the technology is from a financial point of view. Operations per dollar is, fundamentally, the ultimate metric factor, as it is the most useful to justify or not the investment in more powerful compute systems.
These two indicators are complementary because one expresses the given efficiency or performance of a given technology, and the other compares the relative output to the cost.
In the first place, we should increase the relative performance of a given system to scale faster with operations per watt.
In the second place, we should validate that performance grows faster than investment with operations per dollar.
