> Of course a local PC is used for other things and the idleness penalty must somehow account for that.
I'd argue the idleness penalty only applies if you wouldn't have otherwise had equivalent hardware. If you already have the exact same dGPU for gaming and it doubles up for inference then the only inefficiency is power consumption (both by the GPU and potentially by AC for your living space).
Conversely I think we should consider that privacy, distributed compute that you can access without an intermediary, and a more distributed power grid all provide net benefits to society at large.
I'd argue the idleness penalty only applies if you wouldn't have otherwise had equivalent hardware. If you already have the exact same dGPU for gaming and it doubles up for inference then the only inefficiency is power consumption (both by the GPU and potentially by AC for your living space).
Conversely I think we should consider that privacy, distributed compute that you can access without an intermediary, and a more distributed power grid all provide net benefits to society at large.