All Questions
Tagged with numeric-precision floating-point
14 questions
-4
votes
2
answers
127
views
Successor in a floating point system [closed]
I’m starting at numerical methods and I am introducing to the machine epsilon and succesor. I have this question, which is the succesor of .13579x10^2 at a floating point system of (10,5,-2,6). My ...
- 3
7
votes
6
answers
2k
views
When are equality comparisons between IEEE 754 floating point values acceptable?
My understanding is that it's rarely a good idea to compare two floating point numbers due to inherent inaccuracies. However, my understanding of those inaccuracies are that they are deterministic and ...
user361757
1
vote
4
answers
2k
views
Unit test a generic floating point equality function
I've written a function that tests two floating point numbers for approximate equality (see this Code Review question). I'd like to unit test my function, but I'm not positive of the best way to do ...
- 1,291
0
votes
2
answers
814
views
Floating point accuracy and conversions
I am trying to build a class where user can compute floating operations with a particular accuracy. For example, if the user wants to have up to 4 digit accuracy, everything less than 0.0001 is zero ...
- 103
-1
votes
1
answer
516
views
Floating point absorption phenomena and ULP
Numerical analysis text books talks about the absorption phenomena (Introduction to Numerical Analysis and Scientific Computing; Nabil Nassif; Dolly Khoueiri Fayyad; CRC Press; 2014) when adding ...
- 1
9
votes
3
answers
3k
views
Is order of arguments in an arithmetic expression important to achieve as most exact result as possible (speed is not necessary)?
Actually, in this question I do not ask about particular language or architecture, however I understand there might be some differences.
In physics / engineering it is usually better to handle ...
- 201
2
votes
3
answers
3k
views
How can a computer round the last digit in a floating point representation?
I'm confused by how a computer rounds off the last digit in the floating point representation. For example, I'm told that x=1.24327789 is stored in a computer with a 6-digit capacity then it;s ...
- 301
38
votes
9
answers
15k
views
Why are floating point numbers used often in Science/Engineering?
While investigating the accuracy of floating point numbers, I've seen in a few places a statement similar to
"float and double are (designed for/used often in) engineering and scientific calculation"...
- 633
9
votes
1
answer
515
views
Implied precision of floating-point functions
While reviewing another programmer's implementation of a function to calculate the normal distribution CDF, I made a suggestion to either replace the entire implementation with Python's built-in ...
- 1,578
35
votes
11
answers
16k
views
Why do we still use floats? [duplicate]
I understand why floats served a purpose in the past. And I think I can see why they're useful in some simulation examples today. But I think those example are more exceptional than common. So I ...
anon
2
votes
1
answer
10k
views
Addition of double's is NOT Equal to Sum of the double as a whole
I am aware of the floating point errors as I had gained some knowledge with my question asked here in SE Floating Point Errors.
What I am finding it difficult to understand is, the output of the ...
- 924
25
votes
5
answers
42k
views
Solutions for floating point rounding errors
In building an application that deals with a lot of mathematical calculations, I have encountered the problem that certain numbers cause rounding errors.
While I understand that floating point is ...
- 924
0
votes
1
answer
861
views
Conversion of a number from Single precision floating point representation to a Half precision floating point [closed]
I have a code where I have to work on Half precision floating point representation
numbers. To achieve that I have created my own C++ class fp16 with all operators(arithmetic logical, relational) ...
- 117
73
votes
9
answers
78k
views
What causes floating point rounding errors?
I am aware that floating point arithmetic has precision problems. I usually overcome them by switching to a fixed decimal representation of the number, or simply by neglecting the error.
However, I ...
- 863