Real numbers
Here we go again. This topic never gets old. It even seems that it gets worse due to software stacks with automatic type conversion.
Here is my attempt to help. I will let the image speak as this topic has been discussed too many times already.
Here are some links to back my statements and dive deeper:
General:
- Intel® 64 and IA-32 Architectures Developer’s Manual: Volume 1
- IEEE 754-2019 addendum
- IEEE 754 binary64
- IEEE 754 binary32
- What Every Computer Scientist Should Know About Floating-Point Arithmetic
- Why don’t my numbers add up
- Floating-Point Numbers
- All desktop calculators are wrong
- FFmpeg’s AVRational workaround
- JavaScript Number
- Java’s BigDecimal
MS Excel related:
- Numeric precision in Microsoft Excel
- Excel just stores 15 decimals
- Aspose.Cells has different numbers
- Apache POI’s Excel bug replication
NVIDIA related: