| 1 | <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> |
| 2 | <html> |
| 3 | <head> |
| 4 | <link rel="STYLESHEET" href="tut.css" type='text/css' /> |
| 5 | <link rel="SHORTCUT ICON" href="../icons/pyfav.png" type="image/png" /> |
| 6 | <link rel='start' href='../index.html' title='Python Documentation Index' /> |
| 7 | <link rel="first" href="tut.html" title='Python Tutorial' /> |
| 8 | <link rel='contents' href='node2.html' title="Contents" /> |
| 9 | <link rel='index' href='node19.html' title='Index' /> |
| 10 | <link rel='last' href='about.html' title='About this document...' /> |
| 11 | <link rel='help' href='about.html' title='About this document...' /> |
| 12 | <link rel="next" href="node17.html" /> |
| 13 | <link rel="prev" href="node15.html" /> |
| 14 | <link rel="parent" href="tut.html" /> |
| 15 | <link rel="next" href="node17.html" /> |
| 16 | <meta name='aesop' content='information' /> |
| 17 | <title>B. Floating Point Arithmetic: Issues and Limitations</title> |
| 18 | </head> |
| 19 | <body> |
| 20 | <DIV CLASS="navigation"> |
| 21 | <div id='top-navigation-panel' xml:id='top-navigation-panel'> |
| 22 | <table align="center" width="100%" cellpadding="0" cellspacing="2"> |
| 23 | <tr> |
| 24 | <td class='online-navigation'><a rel="prev" title="A. Interactive Input Editing" |
| 25 | href="node15.html"><img src='../icons/previous.png' |
| 26 | border='0' height='32' alt='Previous Page' width='32' /></A></td> |
| 27 | <td class='online-navigation'><a rel="parent" title="Python Tutorial" |
| 28 | href="tut.html"><img src='../icons/up.png' |
| 29 | border='0' height='32' alt='Up One Level' width='32' /></A></td> |
| 30 | <td class='online-navigation'><a rel="next" title="C. History and License" |
| 31 | href="node17.html"><img src='../icons/next.png' |
| 32 | border='0' height='32' alt='Next Page' width='32' /></A></td> |
| 33 | <td align="center" width="100%">Python Tutorial</td> |
| 34 | <td class='online-navigation'><a rel="contents" title="Table of Contents" |
| 35 | href="node2.html"><img src='../icons/contents.png' |
| 36 | border='0' height='32' alt='Contents' width='32' /></A></td> |
| 37 | <td class='online-navigation'><img src='../icons/blank.png' |
| 38 | border='0' height='32' alt='' width='32' /></td> |
| 39 | <td class='online-navigation'><a rel="index" title="Index" |
| 40 | href="node19.html"><img src='../icons/index.png' |
| 41 | border='0' height='32' alt='Index' width='32' /></A></td> |
| 42 | </tr></table> |
| 43 | <div class='online-navigation'> |
| 44 | <b class="navlabel">Previous:</b> |
| 45 | <a class="sectref" rel="prev" href="node15.html">A. Interactive Input Editing</A> |
| 46 | <b class="navlabel">Up:</b> |
| 47 | <a class="sectref" rel="parent" href="tut.html">Python Tutorial</A> |
| 48 | <b class="navlabel">Next:</b> |
| 49 | <a class="sectref" rel="next" href="node17.html">C. History and License</A> |
| 50 | </div> |
| 51 | <hr /></div> |
| 52 | </DIV> |
| 53 | <!--End of Navigation Panel--> |
| 54 | <div class='online-navigation'> |
| 55 | <!--Table of Child-Links--> |
| 56 | <A NAME="CHILD_LINKS"><STRONG>Subsections</STRONG></a> |
| 57 | |
| 58 | <UL CLASS="ChildLinks"> |
| 59 | <LI><A href="node16.html#SECTION0016100000000000000000">B.1 Representation Error</a> |
| 60 | </ul> |
| 61 | <!--End of Table of Child-Links--> |
| 62 | </div> |
| 63 | <HR> |
| 64 | |
| 65 | <H1><A NAME="SECTION0016000000000000000000"></A><A NAME="fp-issues"></A> |
| 66 | <BR> |
| 67 | B. Floating Point Arithmetic: Issues and Limitations |
| 68 | </H1> |
| 69 | |
| 70 | <P> |
| 71 | Floating-point numbers are represented in computer hardware as |
| 72 | base 2 (binary) fractions. For example, the decimal fraction |
| 73 | |
| 74 | <P> |
| 75 | <div class="verbatim"><pre> |
| 76 | 0.125 |
| 77 | </pre></div> |
| 78 | |
| 79 | <P> |
| 80 | has value 1/10 + 2/100 + 5/1000, and in the same way the binary fraction |
| 81 | |
| 82 | <P> |
| 83 | <div class="verbatim"><pre> |
| 84 | 0.001 |
| 85 | </pre></div> |
| 86 | |
| 87 | <P> |
| 88 | has value 0/2 + 0/4 + 1/8. These two fractions have identical values, |
| 89 | the only real difference being that the first is written in base 10 |
| 90 | fractional notation, and the second in base 2. |
| 91 | |
| 92 | <P> |
| 93 | Unfortunately, most decimal fractions cannot be represented exactly as |
| 94 | binary fractions. A consequence is that, in general, the decimal |
| 95 | floating-point numbers you enter are only approximated by the binary |
| 96 | floating-point numbers actually stored in the machine. |
| 97 | |
| 98 | <P> |
| 99 | The problem is easier to understand at first in base 10. Consider the |
| 100 | fraction 1/3. You can approximate that as a base 10 fraction: |
| 101 | |
| 102 | <P> |
| 103 | <div class="verbatim"><pre> |
| 104 | 0.3 |
| 105 | </pre></div> |
| 106 | |
| 107 | <P> |
| 108 | or, better, |
| 109 | |
| 110 | <P> |
| 111 | <div class="verbatim"><pre> |
| 112 | 0.33 |
| 113 | </pre></div> |
| 114 | |
| 115 | <P> |
| 116 | or, better, |
| 117 | |
| 118 | <P> |
| 119 | <div class="verbatim"><pre> |
| 120 | 0.333 |
| 121 | </pre></div> |
| 122 | |
| 123 | <P> |
| 124 | and so on. No matter how many digits you're willing to write down, the |
| 125 | result will never be exactly 1/3, but will be an increasingly better |
| 126 | approximation of 1/3. |
| 127 | |
| 128 | <P> |
| 129 | In the same way, no matter how many base 2 digits you're willing to |
| 130 | use, the decimal value 0.1 cannot be represented exactly as a base 2 |
| 131 | fraction. In base 2, 1/10 is the infinitely repeating fraction |
| 132 | |
| 133 | <P> |
| 134 | <div class="verbatim"><pre> |
| 135 | 0.0001100110011001100110011001100110011001100110011... |
| 136 | </pre></div> |
| 137 | |
| 138 | <P> |
| 139 | Stop at any finite number of bits, and you get an approximation. This |
| 140 | is why you see things like: |
| 141 | |
| 142 | <P> |
| 143 | <div class="verbatim"><pre> |
| 144 | >>> 0.1 |
| 145 | 0.10000000000000001 |
| 146 | </pre></div> |
| 147 | |
| 148 | <P> |
| 149 | On most machines today, that is what you'll see if you enter 0.1 at |
| 150 | a Python prompt. You may not, though, because the number of bits |
| 151 | used by the hardware to store floating-point values can vary across |
| 152 | machines, and Python only prints a decimal approximation to the true |
| 153 | decimal value of the binary approximation stored by the machine. On |
| 154 | most machines, if Python were to print the true decimal value of |
| 155 | the binary approximation stored for 0.1, it would have to display |
| 156 | |
| 157 | <P> |
| 158 | <div class="verbatim"><pre> |
| 159 | >>> 0.1 |
| 160 | 0.1000000000000000055511151231257827021181583404541015625 |
| 161 | </pre></div> |
| 162 | |
| 163 | <P> |
| 164 | instead! The Python prompt uses the builtin |
| 165 | <tt class="function">repr()</tt> function to obtain a string version of everything it |
| 166 | displays. For floats, <code>repr(<var>float</var>)</code> rounds the true |
| 167 | decimal value to 17 significant digits, giving |
| 168 | |
| 169 | <P> |
| 170 | <div class="verbatim"><pre> |
| 171 | 0.10000000000000001 |
| 172 | </pre></div> |
| 173 | |
| 174 | <P> |
| 175 | <code>repr(<var>float</var>)</code> produces 17 significant digits because it |
| 176 | turns out that's enough (on most machines) so that |
| 177 | <code>eval(repr(<var>x</var>)) == <var>x</var></code> exactly for all finite floats |
| 178 | <var>x</var>, but rounding to 16 digits is not enough to make that true. |
| 179 | |
| 180 | <P> |
| 181 | Note that this is in the very nature of binary floating-point: this is |
| 182 | not a bug in Python, and it is not a bug in your code either. You'll |
| 183 | see the same kind of thing in all languages that support your |
| 184 | hardware's floating-point arithmetic (although some languages may |
| 185 | not <em>display</em> the difference by default, or in all output modes). |
| 186 | |
| 187 | <P> |
| 188 | Python's builtin <tt class="function">str()</tt> function produces only 12 |
| 189 | significant digits, and you may wish to use that instead. It's |
| 190 | unusual for <code>eval(str(<var>x</var>))</code> to reproduce <var>x</var>, but the |
| 191 | output may be more pleasant to look at: |
| 192 | |
| 193 | <P> |
| 194 | <div class="verbatim"><pre> |
| 195 | >>> print str(0.1) |
| 196 | 0.1 |
| 197 | </pre></div> |
| 198 | |
| 199 | <P> |
| 200 | It's important to realize that this is, in a real sense, an illusion: |
| 201 | the value in the machine is not exactly 1/10, you're simply rounding |
| 202 | the <em>display</em> of the true machine value. |
| 203 | |
| 204 | <P> |
| 205 | Other surprises follow from this one. For example, after seeing |
| 206 | |
| 207 | <P> |
| 208 | <div class="verbatim"><pre> |
| 209 | >>> 0.1 |
| 210 | 0.10000000000000001 |
| 211 | </pre></div> |
| 212 | |
| 213 | <P> |
| 214 | you may be tempted to use the <tt class="function">round()</tt> function to chop it |
| 215 | back to the single digit you expect. But that makes no difference: |
| 216 | |
| 217 | <P> |
| 218 | <div class="verbatim"><pre> |
| 219 | >>> round(0.1, 1) |
| 220 | 0.10000000000000001 |
| 221 | </pre></div> |
| 222 | |
| 223 | <P> |
| 224 | The problem is that the binary floating-point value stored for "0.1" |
| 225 | was already the best possible binary approximation to 1/10, so trying |
| 226 | to round it again can't make it better: it was already as good as it |
| 227 | gets. |
| 228 | |
| 229 | <P> |
| 230 | Another consequence is that since 0.1 is not exactly 1/10, |
| 231 | summing ten values of 0.1 may not yield exactly 1.0, either: |
| 232 | |
| 233 | <P> |
| 234 | <div class="verbatim"><pre> |
| 235 | >>> sum = 0.0 |
| 236 | >>> for i in range(10): |
| 237 | ... sum += 0.1 |
| 238 | ... |
| 239 | >>> sum |
| 240 | 0.99999999999999989 |
| 241 | </pre></div> |
| 242 | |
| 243 | <P> |
| 244 | Binary floating-point arithmetic holds many surprises like this. The |
| 245 | problem with "0.1" is explained in precise detail below, in the |
| 246 | "Representation Error" section. See |
| 247 | <em class="citetitle"><a |
| 248 | href="http://www.lahey.com/float.htm" |
| 249 | title="The Perils of Floating |
| 250 | Point" |
| 251 | >The Perils of Floating |
| 252 | Point</a></em> for a more complete account of other common surprises. |
| 253 | |
| 254 | <P> |
| 255 | As that says near the end, ``there are no easy answers.'' Still, |
| 256 | don't be unduly wary of floating-point! The errors in Python float |
| 257 | operations are inherited from the floating-point hardware, and on most |
| 258 | machines are on the order of no more than 1 part in 2**53 per |
| 259 | operation. That's more than adequate for most tasks, but you do need |
| 260 | to keep in mind that it's not decimal arithmetic, and that every float |
| 261 | operation can suffer a new rounding error. |
| 262 | |
| 263 | <P> |
| 264 | While pathological cases do exist, for most casual use of |
| 265 | floating-point arithmetic you'll see the result you expect in the end |
| 266 | if you simply round the display of your final results to the number of |
| 267 | decimal digits you expect. <tt class="function">str()</tt> usually suffices, and for |
| 268 | finer control see the discussion of Python's <code>%</code> format |
| 269 | operator: the <code>%g</code>, <code>%f</code> and <code>%e</code> format codes |
| 270 | supply flexible and easy ways to round float results for display. |
| 271 | |
| 272 | <P> |
| 273 | |
| 274 | <H1><A NAME="SECTION0016100000000000000000"></A><A NAME="fp-error"></A> |
| 275 | <BR> |
| 276 | B.1 Representation Error |
| 277 | |
| 278 | </H1> |
| 279 | |
| 280 | <P> |
| 281 | This section explains the ``0.1'' example in detail, and shows how |
| 282 | you can perform an exact analysis of cases like this yourself. Basic |
| 283 | familiarity with binary floating-point representation is assumed. |
| 284 | |
| 285 | <P> |
| 286 | <i class="dfn">Representation error</i> refers to fact that some (most, actually) |
| 287 | decimal fractions cannot be represented exactly as binary (base 2) |
| 288 | fractions. This is the chief reason why Python (or Perl, C, C++, |
| 289 | Java, Fortran, and many others) often won't display the exact decimal |
| 290 | number you expect: |
| 291 | |
| 292 | <P> |
| 293 | <div class="verbatim"><pre> |
| 294 | >>> 0.1 |
| 295 | 0.10000000000000001 |
| 296 | </pre></div> |
| 297 | |
| 298 | <P> |
| 299 | Why is that? 1/10 is not exactly representable as a binary fraction. |
| 300 | Almost all machines today (November 2000) use IEEE-754 floating point |
| 301 | arithmetic, and almost all platforms map Python floats to IEEE-754 |
| 302 | "double precision". 754 doubles contain 53 bits of precision, so on |
| 303 | input the computer strives to convert 0.1 to the closest fraction it can |
| 304 | of the form <var>J</var>/2**<var>N</var> where <var>J</var> is an integer containing |
| 305 | exactly 53 bits. Rewriting |
| 306 | |
| 307 | <P> |
| 308 | <div class="verbatim"><pre> |
| 309 | 1 / 10 ~= J / (2**N) |
| 310 | </pre></div> |
| 311 | |
| 312 | <P> |
| 313 | as |
| 314 | |
| 315 | <P> |
| 316 | <div class="verbatim"><pre> |
| 317 | J ~= 2**N / 10 |
| 318 | </pre></div> |
| 319 | |
| 320 | <P> |
| 321 | and recalling that <var>J</var> has exactly 53 bits (is <code>>= 2**52</code> but |
| 322 | <code>< 2**53</code>), the best value for <var>N</var> is 56: |
| 323 | |
| 324 | <P> |
| 325 | <div class="verbatim"><pre> |
| 326 | >>> 2**52 |
| 327 | 4503599627370496L |
| 328 | >>> 2**53 |
| 329 | 9007199254740992L |
| 330 | >>> 2**56/10 |
| 331 | 7205759403792793L |
| 332 | </pre></div> |
| 333 | |
| 334 | <P> |
| 335 | That is, 56 is the only value for <var>N</var> that leaves <var>J</var> with |
| 336 | exactly 53 bits. The best possible value for <var>J</var> is then that |
| 337 | quotient rounded: |
| 338 | |
| 339 | <P> |
| 340 | <div class="verbatim"><pre> |
| 341 | >>> q, r = divmod(2**56, 10) |
| 342 | >>> r |
| 343 | 6L |
| 344 | </pre></div> |
| 345 | |
| 346 | <P> |
| 347 | Since the remainder is more than half of 10, the best approximation is |
| 348 | obtained by rounding up: |
| 349 | |
| 350 | <P> |
| 351 | <div class="verbatim"><pre> |
| 352 | >>> q+1 |
| 353 | 7205759403792794L |
| 354 | </pre></div> |
| 355 | |
| 356 | <P> |
| 357 | Therefore the best possible approximation to 1/10 in 754 double |
| 358 | precision is that over 2**56, or |
| 359 | |
| 360 | <P> |
| 361 | <div class="verbatim"><pre> |
| 362 | 7205759403792794 / 72057594037927936 |
| 363 | </pre></div> |
| 364 | |
| 365 | <P> |
| 366 | Note that since we rounded up, this is actually a little bit larger than |
| 367 | 1/10; if we had not rounded up, the quotient would have been a little |
| 368 | bit smaller than 1/10. But in no case can it be <em>exactly</em> 1/10! |
| 369 | |
| 370 | <P> |
| 371 | So the computer never ``sees'' 1/10: what it sees is the exact |
| 372 | fraction given above, the best 754 double approximation it can get: |
| 373 | |
| 374 | <P> |
| 375 | <div class="verbatim"><pre> |
| 376 | >>> .1 * 2**56 |
| 377 | 7205759403792794.0 |
| 378 | </pre></div> |
| 379 | |
| 380 | <P> |
| 381 | If we multiply that fraction by 10**30, we can see the (truncated) |
| 382 | value of its 30 most significant decimal digits: |
| 383 | |
| 384 | <P> |
| 385 | <div class="verbatim"><pre> |
| 386 | >>> 7205759403792794 * 10**30 / 2**56 |
| 387 | 100000000000000005551115123125L |
| 388 | </pre></div> |
| 389 | |
| 390 | <P> |
| 391 | meaning that the exact number stored in the computer is approximately |
| 392 | equal to the decimal value 0.100000000000000005551115123125. Rounding |
| 393 | that to 17 significant digits gives the 0.10000000000000001 that Python |
| 394 | displays (well, will display on any 754-conforming platform that does |
| 395 | best-possible input and output conversions in its C library -- yours may |
| 396 | not!). |
| 397 | |
| 398 | <P> |
| 399 | |
| 400 | <DIV CLASS="navigation"> |
| 401 | <div class='online-navigation'> |
| 402 | <p></p><hr /> |
| 403 | <table align="center" width="100%" cellpadding="0" cellspacing="2"> |
| 404 | <tr> |
| 405 | <td class='online-navigation'><a rel="prev" title="A. Interactive Input Editing" |
| 406 | href="node15.html"><img src='../icons/previous.png' |
| 407 | border='0' height='32' alt='Previous Page' width='32' /></A></td> |
| 408 | <td class='online-navigation'><a rel="parent" title="Python Tutorial" |
| 409 | href="tut.html"><img src='../icons/up.png' |
| 410 | border='0' height='32' alt='Up One Level' width='32' /></A></td> |
| 411 | <td class='online-navigation'><a rel="next" title="C. History and License" |
| 412 | href="node17.html"><img src='../icons/next.png' |
| 413 | border='0' height='32' alt='Next Page' width='32' /></A></td> |
| 414 | <td align="center" width="100%">Python Tutorial</td> |
| 415 | <td class='online-navigation'><a rel="contents" title="Table of Contents" |
| 416 | href="node2.html"><img src='../icons/contents.png' |
| 417 | border='0' height='32' alt='Contents' width='32' /></A></td> |
| 418 | <td class='online-navigation'><img src='../icons/blank.png' |
| 419 | border='0' height='32' alt='' width='32' /></td> |
| 420 | <td class='online-navigation'><a rel="index" title="Index" |
| 421 | href="node19.html"><img src='../icons/index.png' |
| 422 | border='0' height='32' alt='Index' width='32' /></A></td> |
| 423 | </tr></table> |
| 424 | <div class='online-navigation'> |
| 425 | <b class="navlabel">Previous:</b> |
| 426 | <a class="sectref" rel="prev" href="node15.html">A. Interactive Input Editing</A> |
| 427 | <b class="navlabel">Up:</b> |
| 428 | <a class="sectref" rel="parent" href="tut.html">Python Tutorial</A> |
| 429 | <b class="navlabel">Next:</b> |
| 430 | <a class="sectref" rel="next" href="node17.html">C. History and License</A> |
| 431 | </div> |
| 432 | </div> |
| 433 | <hr /> |
| 434 | <span class="release-info">Release 2.4.2, documentation updated on 28 September 2005.</span> |
| 435 | </DIV> |
| 436 | <!--End of Navigation Panel--> |
| 437 | <ADDRESS> |
| 438 | See <i><a href="about.html">About this document...</a></i> for information on suggesting changes. |
| 439 | </ADDRESS> |
| 440 | </BODY> |
| 441 | </HTML> |