AI energy discussions often compress a system, a data center and an electricity grid into one dramatic number. Better decisions begin by declaring the boundary, workload and time period being measured.
Separate training from inference
Training is a bounded model-development workload; inference recurs with every production request. Report them separately, then include experimentation, failed runs and supporting services when they materially affect the decision. A model used at scale may consume more through inference over its lifetime than through one training run, but that balance depends on use.
For inference, define request type, input and output length, batch size, latency target and hardware. ‘One AI query’ is not a stable unit when generating an image and classifying a sentence require different work.
Measure IT energy and facility overhead
Accelerators are not the whole data center. Cooling, power conversion, networking and storage add overhead. Power usage effectiveness can help describe facility overhead, but it does not reveal the carbon intensity of electricity or the efficiency of a particular AI task.
Report measured energy where possible, with sampling method and uncertainty. If using estimates, state utilization assumptions and avoid presenting a single precise number as universal.
Add time and location
The same electricity use can carry different emissions depending on grid mix and hour. Location-based and market-based accounting answer different questions. Keep energy and emissions as separate reported quantities so readers can inspect the conversion.
DOE’s 2024 U.S. report estimated data centers used about 4.4% of national electricity in 2023 and projected 6.7% to 12% by 2028. That is a system-level scenario, not a per-model measurement, and should not be used to assign a footprint to one application.
Normalize by useful work
Track energy per successful task, not merely per token or request. A cheaper first attempt that fails and triggers three retries may be less efficient. Include quality thresholds so efficiency gains are not purchased by making the system unusable.
Compare architectures on the same evaluation set. Caching, smaller models, batching, shorter outputs and local processing can help, but each can alter quality, latency or hardware utilization.
Publish an auditable claim
A credible statement names the workload, model or system version, hardware, region, measurement window, facility boundary and quality level. It also states exclusions. Update the claim as deployment scale and grid conditions change.
This discipline does not produce one memorable universal number. It produces numbers a buyer, engineer or policymaker can actually use.
How to put the idea into practice
Begin with one bounded workflow related to ai energy claims need better measurement. Write a one-page baseline before changing the system: current completion time, quality checks, common failure categories, escalation path and the person responsible for the outcome. Select a representative sample rather than only the cleanest examples. Include ordinary cases, difficult edge cases and at least one case where the correct result is to stop or ask for more information.
Run the candidate beside the existing process before allowing it to replace that process. Review both successful and failed outputs, because a lower error rate can still conceal a new high-impact failure. Record the exact configuration used for each test and keep artifacts that let another reviewer reproduce the result. At the end of the pilot, decide whether to expand, revise or stop using thresholds agreed in advance—not a retrospective impression of the best demonstration.
Questions to ask a vendor or internal team
Ask what evidence supports the central claim, which system and data versions produced that evidence, and what conditions were excluded. Request results for the languages, input types and risk categories your deployment will encounter. Ask how changes are announced, how regressions are detected and how a customer can export logs needed for an incident review.
Also ask what the system does when confidence is low, a dependency fails or the request falls outside its supported scope. A dependable product should have a defined failure state, not merely a more polished answer. Ownership matters: identify who can pause the workflow, who approves exceptions and who informs affected users if a material error escapes into production.
A practical checklist
- Define the user task and the failure that matters before choosing a model or tool.
- Keep a small, versioned test set drawn from real work, including awkward and adversarial cases.
- Record model, prompt, tools, retrieval settings, data version, latency and cost for every run.
- Require human confirmation for irreversible, high-impact or externally visible actions.
- Review failures by category, not just by one average score, and add regressions to the test set.
Related reading from Meydo Journal
Primary sources
- 2024 Report on U.S. Data Center Energy Use announcement — U.S. Department of Energy
