Skip to content
Snippets Groups Projects
Commit abf941b2 authored by Simon Gene Gottlieb's avatar Simon Gene Gottlieb Committed by Jesse Beder
Browse files

Fix float precision (#649)

The issue is that numbers like
2.01 or 3.01 can not be precisely represented with binary floating point
numbers.

This replaces all occurrences of 'std::numeric_limits<T>::digits10 + 1' with
'std::numeric_limits<T>::max_digits10'.

Background:
Using 'std::numeric_limits<T>::digits10 + 1' is not precise enough.
Converting a 'float' into a 'string' and back to a 'float' will not always
produce the original 'float' value. To guarantee that the 'string'
representation has sufficient precision the value
'std::numeric_limits<T>::max_digits10' has to be used.
parent b659858b
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment