The essential answer
Resolution describes a small change that can be distinguished by a measuring system or display. Accuracy concerns agreement with the quantity being measured and is not established by the number of displayed digits. A finely resolved instrument may have an incorrect gain or offset. Repeated measurements can average some noise but cannot automatically remove a stable scale error. Assess resolution as one contributor within the measurement model, alongside calibration, stability, environmental sensitivity, and the behavior of the complete acquisition chain.
How to reason about it
Digital steps create quantization, but the effective resolution of a system also depends on noise and processing. For an ideal rounded display with step q and a uniformly distributed rounding error, the corresponding standard uncertainty is q/sqrt(12). That model is useful only when the unknown position within a step is plausibly uniform. Repeatedly recording the same code does not supply independent rounding errors. Noise can dither the input across codes and permit sub-step mean estimates, but this requires a suitable model and sufficient linearity. Display interpolation and smoothing can add apparent digits without adding information. To judge usefulness, express the display step and other uncertainties in the final quantity's units. Then ask whether improving digitization would reduce the total uncertainty or merely make the readout appear more precise. Always keep corrected scale error separate from raw repeatability.
ILLUSTRATIVE · NOT A REPORTED RESULT
An example to work through
Assume a hypothetical force display rounds in 0.01 nN increments. Under uniform rounding, its standard quantization contribution is 0.01/sqrt(12) = 0.0029 nN. If independent calibration uncertainty is 0.20 nN and baseline uncertainty is 0.10 nN, total standard uncertainty is sqrt(0.0029²+0.20²+0.10²) = 0.224 nN. Changing to 0.001 nN increments leaves the total effectively unchanged. The extra digit may help observe tiny short-term changes, but it does not justify reporting the absolute force with a corresponding reduction in uncertainty. Calibration improvement would be more effective in this assumed case.
Where interpretation fails
Do not infer an uncertainty from the last displayed digit alone. Conversely, do not round every intermediate reading so aggressively that useful differential information disappears. Preserve raw data and calibration precision in processing, then round the reported estimate consistently with the evaluated uncertainty at the end.
The next decision
Upgrade resolution when quantization or readout granularity materially limits the intended estimator. If calibration, drift, or environmental terms dominate, improve those first. Require an independently applied small input to demonstrate usable change detection, and a reference comparison to demonstrate the absolute conversion at the relevant operating point.
Sources & context
These sources support the concepts noted below. Our synthetic examples and research framing are educational interpretation; they are not findings from these organizations or validation of our ventures.
- JCGM 100: Guide to uncertainty ↗
Further reading on uncertainty propagation; PDF text inspected, not a full document review.
Source check: Relevant excerpt or resource metadata checked; see support note. Original illustrative calculation, not source data.
Educational research context, not a validated device specification or clinical protocol. Read our editorial and correction policy.