Commit | Line | Data |
---|---|---|
920dae64 AT |
1 | <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> |
2 | <html> | |
3 | <head> | |
4 | <link rel="STYLESHEET" href="tut.css" type='text/css' /> | |
5 | <link rel="SHORTCUT ICON" href="../icons/pyfav.png" type="image/png" /> | |
6 | <link rel='start' href='../index.html' title='Python Documentation Index' /> | |
7 | <link rel="first" href="tut.html" title='Python Tutorial' /> | |
8 | <link rel='contents' href='node2.html' title="Contents" /> | |
9 | <link rel='index' href='node19.html' title='Index' /> | |
10 | <link rel='last' href='about.html' title='About this document...' /> | |
11 | <link rel='help' href='about.html' title='About this document...' /> | |
12 | <link rel="next" href="node17.html" /> | |
13 | <link rel="prev" href="node15.html" /> | |
14 | <link rel="parent" href="tut.html" /> | |
15 | <link rel="next" href="node17.html" /> | |
16 | <meta name='aesop' content='information' /> | |
17 | <title>B. Floating Point Arithmetic: Issues and Limitations</title> | |
18 | </head> | |
19 | <body> | |
20 | <DIV CLASS="navigation"> | |
21 | <div id='top-navigation-panel' xml:id='top-navigation-panel'> | |
22 | <table align="center" width="100%" cellpadding="0" cellspacing="2"> | |
23 | <tr> | |
24 | <td class='online-navigation'><a rel="prev" title="A. Interactive Input Editing" | |
25 | href="node15.html"><img src='../icons/previous.png' | |
26 | border='0' height='32' alt='Previous Page' width='32' /></A></td> | |
27 | <td class='online-navigation'><a rel="parent" title="Python Tutorial" | |
28 | href="tut.html"><img src='../icons/up.png' | |
29 | border='0' height='32' alt='Up One Level' width='32' /></A></td> | |
30 | <td class='online-navigation'><a rel="next" title="C. History and License" | |
31 | href="node17.html"><img src='../icons/next.png' | |
32 | border='0' height='32' alt='Next Page' width='32' /></A></td> | |
33 | <td align="center" width="100%">Python Tutorial</td> | |
34 | <td class='online-navigation'><a rel="contents" title="Table of Contents" | |
35 | href="node2.html"><img src='../icons/contents.png' | |
36 | border='0' height='32' alt='Contents' width='32' /></A></td> | |
37 | <td class='online-navigation'><img src='../icons/blank.png' | |
38 | border='0' height='32' alt='' width='32' /></td> | |
39 | <td class='online-navigation'><a rel="index" title="Index" | |
40 | href="node19.html"><img src='../icons/index.png' | |
41 | border='0' height='32' alt='Index' width='32' /></A></td> | |
42 | </tr></table> | |
43 | <div class='online-navigation'> | |
44 | <b class="navlabel">Previous:</b> | |
45 | <a class="sectref" rel="prev" href="node15.html">A. Interactive Input Editing</A> | |
46 | <b class="navlabel">Up:</b> | |
47 | <a class="sectref" rel="parent" href="tut.html">Python Tutorial</A> | |
48 | <b class="navlabel">Next:</b> | |
49 | <a class="sectref" rel="next" href="node17.html">C. History and License</A> | |
50 | </div> | |
51 | <hr /></div> | |
52 | </DIV> | |
53 | <!--End of Navigation Panel--> | |
54 | <div class='online-navigation'> | |
55 | <!--Table of Child-Links--> | |
56 | <A NAME="CHILD_LINKS"><STRONG>Subsections</STRONG></a> | |
57 | ||
58 | <UL CLASS="ChildLinks"> | |
59 | <LI><A href="node16.html#SECTION0016100000000000000000">B.1 Representation Error</a> | |
60 | </ul> | |
61 | <!--End of Table of Child-Links--> | |
62 | </div> | |
63 | <HR> | |
64 | ||
65 | <H1><A NAME="SECTION0016000000000000000000"></A><A NAME="fp-issues"></A> | |
66 | <BR> | |
67 | B. Floating Point Arithmetic: Issues and Limitations | |
68 | </H1> | |
69 | ||
70 | <P> | |
71 | Floating-point numbers are represented in computer hardware as | |
72 | base 2 (binary) fractions. For example, the decimal fraction | |
73 | ||
74 | <P> | |
75 | <div class="verbatim"><pre> | |
76 | 0.125 | |
77 | </pre></div> | |
78 | ||
79 | <P> | |
80 | has value 1/10 + 2/100 + 5/1000, and in the same way the binary fraction | |
81 | ||
82 | <P> | |
83 | <div class="verbatim"><pre> | |
84 | 0.001 | |
85 | </pre></div> | |
86 | ||
87 | <P> | |
88 | has value 0/2 + 0/4 + 1/8. These two fractions have identical values, | |
89 | the only real difference being that the first is written in base 10 | |
90 | fractional notation, and the second in base 2. | |
91 | ||
92 | <P> | |
93 | Unfortunately, most decimal fractions cannot be represented exactly as | |
94 | binary fractions. A consequence is that, in general, the decimal | |
95 | floating-point numbers you enter are only approximated by the binary | |
96 | floating-point numbers actually stored in the machine. | |
97 | ||
98 | <P> | |
99 | The problem is easier to understand at first in base 10. Consider the | |
100 | fraction 1/3. You can approximate that as a base 10 fraction: | |
101 | ||
102 | <P> | |
103 | <div class="verbatim"><pre> | |
104 | 0.3 | |
105 | </pre></div> | |
106 | ||
107 | <P> | |
108 | or, better, | |
109 | ||
110 | <P> | |
111 | <div class="verbatim"><pre> | |
112 | 0.33 | |
113 | </pre></div> | |
114 | ||
115 | <P> | |
116 | or, better, | |
117 | ||
118 | <P> | |
119 | <div class="verbatim"><pre> | |
120 | 0.333 | |
121 | </pre></div> | |
122 | ||
123 | <P> | |
124 | and so on. No matter how many digits you're willing to write down, the | |
125 | result will never be exactly 1/3, but will be an increasingly better | |
126 | approximation of 1/3. | |
127 | ||
128 | <P> | |
129 | In the same way, no matter how many base 2 digits you're willing to | |
130 | use, the decimal value 0.1 cannot be represented exactly as a base 2 | |
131 | fraction. In base 2, 1/10 is the infinitely repeating fraction | |
132 | ||
133 | <P> | |
134 | <div class="verbatim"><pre> | |
135 | 0.0001100110011001100110011001100110011001100110011... | |
136 | </pre></div> | |
137 | ||
138 | <P> | |
139 | Stop at any finite number of bits, and you get an approximation. This | |
140 | is why you see things like: | |
141 | ||
142 | <P> | |
143 | <div class="verbatim"><pre> | |
144 | >>> 0.1 | |
145 | 0.10000000000000001 | |
146 | </pre></div> | |
147 | ||
148 | <P> | |
149 | On most machines today, that is what you'll see if you enter 0.1 at | |
150 | a Python prompt. You may not, though, because the number of bits | |
151 | used by the hardware to store floating-point values can vary across | |
152 | machines, and Python only prints a decimal approximation to the true | |
153 | decimal value of the binary approximation stored by the machine. On | |
154 | most machines, if Python were to print the true decimal value of | |
155 | the binary approximation stored for 0.1, it would have to display | |
156 | ||
157 | <P> | |
158 | <div class="verbatim"><pre> | |
159 | >>> 0.1 | |
160 | 0.1000000000000000055511151231257827021181583404541015625 | |
161 | </pre></div> | |
162 | ||
163 | <P> | |
164 | instead! The Python prompt uses the builtin | |
165 | <tt class="function">repr()</tt> function to obtain a string version of everything it | |
166 | displays. For floats, <code>repr(<var>float</var>)</code> rounds the true | |
167 | decimal value to 17 significant digits, giving | |
168 | ||
169 | <P> | |
170 | <div class="verbatim"><pre> | |
171 | 0.10000000000000001 | |
172 | </pre></div> | |
173 | ||
174 | <P> | |
175 | <code>repr(<var>float</var>)</code> produces 17 significant digits because it | |
176 | turns out that's enough (on most machines) so that | |
177 | <code>eval(repr(<var>x</var>)) == <var>x</var></code> exactly for all finite floats | |
178 | <var>x</var>, but rounding to 16 digits is not enough to make that true. | |
179 | ||
180 | <P> | |
181 | Note that this is in the very nature of binary floating-point: this is | |
182 | not a bug in Python, and it is not a bug in your code either. You'll | |
183 | see the same kind of thing in all languages that support your | |
184 | hardware's floating-point arithmetic (although some languages may | |
185 | not <em>display</em> the difference by default, or in all output modes). | |
186 | ||
187 | <P> | |
188 | Python's builtin <tt class="function">str()</tt> function produces only 12 | |
189 | significant digits, and you may wish to use that instead. It's | |
190 | unusual for <code>eval(str(<var>x</var>))</code> to reproduce <var>x</var>, but the | |
191 | output may be more pleasant to look at: | |
192 | ||
193 | <P> | |
194 | <div class="verbatim"><pre> | |
195 | >>> print str(0.1) | |
196 | 0.1 | |
197 | </pre></div> | |
198 | ||
199 | <P> | |
200 | It's important to realize that this is, in a real sense, an illusion: | |
201 | the value in the machine is not exactly 1/10, you're simply rounding | |
202 | the <em>display</em> of the true machine value. | |
203 | ||
204 | <P> | |
205 | Other surprises follow from this one. For example, after seeing | |
206 | ||
207 | <P> | |
208 | <div class="verbatim"><pre> | |
209 | >>> 0.1 | |
210 | 0.10000000000000001 | |
211 | </pre></div> | |
212 | ||
213 | <P> | |
214 | you may be tempted to use the <tt class="function">round()</tt> function to chop it | |
215 | back to the single digit you expect. But that makes no difference: | |
216 | ||
217 | <P> | |
218 | <div class="verbatim"><pre> | |
219 | >>> round(0.1, 1) | |
220 | 0.10000000000000001 | |
221 | </pre></div> | |
222 | ||
223 | <P> | |
224 | The problem is that the binary floating-point value stored for "0.1" | |
225 | was already the best possible binary approximation to 1/10, so trying | |
226 | to round it again can't make it better: it was already as good as it | |
227 | gets. | |
228 | ||
229 | <P> | |
230 | Another consequence is that since 0.1 is not exactly 1/10, | |
231 | summing ten values of 0.1 may not yield exactly 1.0, either: | |
232 | ||
233 | <P> | |
234 | <div class="verbatim"><pre> | |
235 | >>> sum = 0.0 | |
236 | >>> for i in range(10): | |
237 | ... sum += 0.1 | |
238 | ... | |
239 | >>> sum | |
240 | 0.99999999999999989 | |
241 | </pre></div> | |
242 | ||
243 | <P> | |
244 | Binary floating-point arithmetic holds many surprises like this. The | |
245 | problem with "0.1" is explained in precise detail below, in the | |
246 | "Representation Error" section. See | |
247 | <em class="citetitle"><a | |
248 | href="http://www.lahey.com/float.htm" | |
249 | title="The Perils of Floating | |
250 | Point" | |
251 | >The Perils of Floating | |
252 | Point</a></em> for a more complete account of other common surprises. | |
253 | ||
254 | <P> | |
255 | As that says near the end, ``there are no easy answers.'' Still, | |
256 | don't be unduly wary of floating-point! The errors in Python float | |
257 | operations are inherited from the floating-point hardware, and on most | |
258 | machines are on the order of no more than 1 part in 2**53 per | |
259 | operation. That's more than adequate for most tasks, but you do need | |
260 | to keep in mind that it's not decimal arithmetic, and that every float | |
261 | operation can suffer a new rounding error. | |
262 | ||
263 | <P> | |
264 | While pathological cases do exist, for most casual use of | |
265 | floating-point arithmetic you'll see the result you expect in the end | |
266 | if you simply round the display of your final results to the number of | |
267 | decimal digits you expect. <tt class="function">str()</tt> usually suffices, and for | |
268 | finer control see the discussion of Python's <code>%</code> format | |
269 | operator: the <code>%g</code>, <code>%f</code> and <code>%e</code> format codes | |
270 | supply flexible and easy ways to round float results for display. | |
271 | ||
272 | <P> | |
273 | ||
274 | <H1><A NAME="SECTION0016100000000000000000"></A><A NAME="fp-error"></A> | |
275 | <BR> | |
276 | B.1 Representation Error | |
277 | ||
278 | </H1> | |
279 | ||
280 | <P> | |
281 | This section explains the ``0.1'' example in detail, and shows how | |
282 | you can perform an exact analysis of cases like this yourself. Basic | |
283 | familiarity with binary floating-point representation is assumed. | |
284 | ||
285 | <P> | |
286 | <i class="dfn">Representation error</i> refers to fact that some (most, actually) | |
287 | decimal fractions cannot be represented exactly as binary (base 2) | |
288 | fractions. This is the chief reason why Python (or Perl, C, C++, | |
289 | Java, Fortran, and many others) often won't display the exact decimal | |
290 | number you expect: | |
291 | ||
292 | <P> | |
293 | <div class="verbatim"><pre> | |
294 | >>> 0.1 | |
295 | 0.10000000000000001 | |
296 | </pre></div> | |
297 | ||
298 | <P> | |
299 | Why is that? 1/10 is not exactly representable as a binary fraction. | |
300 | Almost all machines today (November 2000) use IEEE-754 floating point | |
301 | arithmetic, and almost all platforms map Python floats to IEEE-754 | |
302 | "double precision". 754 doubles contain 53 bits of precision, so on | |
303 | input the computer strives to convert 0.1 to the closest fraction it can | |
304 | of the form <var>J</var>/2**<var>N</var> where <var>J</var> is an integer containing | |
305 | exactly 53 bits. Rewriting | |
306 | ||
307 | <P> | |
308 | <div class="verbatim"><pre> | |
309 | 1 / 10 ~= J / (2**N) | |
310 | </pre></div> | |
311 | ||
312 | <P> | |
313 | as | |
314 | ||
315 | <P> | |
316 | <div class="verbatim"><pre> | |
317 | J ~= 2**N / 10 | |
318 | </pre></div> | |
319 | ||
320 | <P> | |
321 | and recalling that <var>J</var> has exactly 53 bits (is <code>>= 2**52</code> but | |
322 | <code>< 2**53</code>), the best value for <var>N</var> is 56: | |
323 | ||
324 | <P> | |
325 | <div class="verbatim"><pre> | |
326 | >>> 2**52 | |
327 | 4503599627370496L | |
328 | >>> 2**53 | |
329 | 9007199254740992L | |
330 | >>> 2**56/10 | |
331 | 7205759403792793L | |
332 | </pre></div> | |
333 | ||
334 | <P> | |
335 | That is, 56 is the only value for <var>N</var> that leaves <var>J</var> with | |
336 | exactly 53 bits. The best possible value for <var>J</var> is then that | |
337 | quotient rounded: | |
338 | ||
339 | <P> | |
340 | <div class="verbatim"><pre> | |
341 | >>> q, r = divmod(2**56, 10) | |
342 | >>> r | |
343 | 6L | |
344 | </pre></div> | |
345 | ||
346 | <P> | |
347 | Since the remainder is more than half of 10, the best approximation is | |
348 | obtained by rounding up: | |
349 | ||
350 | <P> | |
351 | <div class="verbatim"><pre> | |
352 | >>> q+1 | |
353 | 7205759403792794L | |
354 | </pre></div> | |
355 | ||
356 | <P> | |
357 | Therefore the best possible approximation to 1/10 in 754 double | |
358 | precision is that over 2**56, or | |
359 | ||
360 | <P> | |
361 | <div class="verbatim"><pre> | |
362 | 7205759403792794 / 72057594037927936 | |
363 | </pre></div> | |
364 | ||
365 | <P> | |
366 | Note that since we rounded up, this is actually a little bit larger than | |
367 | 1/10; if we had not rounded up, the quotient would have been a little | |
368 | bit smaller than 1/10. But in no case can it be <em>exactly</em> 1/10! | |
369 | ||
370 | <P> | |
371 | So the computer never ``sees'' 1/10: what it sees is the exact | |
372 | fraction given above, the best 754 double approximation it can get: | |
373 | ||
374 | <P> | |
375 | <div class="verbatim"><pre> | |
376 | >>> .1 * 2**56 | |
377 | 7205759403792794.0 | |
378 | </pre></div> | |
379 | ||
380 | <P> | |
381 | If we multiply that fraction by 10**30, we can see the (truncated) | |
382 | value of its 30 most significant decimal digits: | |
383 | ||
384 | <P> | |
385 | <div class="verbatim"><pre> | |
386 | >>> 7205759403792794 * 10**30 / 2**56 | |
387 | 100000000000000005551115123125L | |
388 | </pre></div> | |
389 | ||
390 | <P> | |
391 | meaning that the exact number stored in the computer is approximately | |
392 | equal to the decimal value 0.100000000000000005551115123125. Rounding | |
393 | that to 17 significant digits gives the 0.10000000000000001 that Python | |
394 | displays (well, will display on any 754-conforming platform that does | |
395 | best-possible input and output conversions in its C library -- yours may | |
396 | not!). | |
397 | ||
398 | <P> | |
399 | ||
400 | <DIV CLASS="navigation"> | |
401 | <div class='online-navigation'> | |
402 | <p></p><hr /> | |
403 | <table align="center" width="100%" cellpadding="0" cellspacing="2"> | |
404 | <tr> | |
405 | <td class='online-navigation'><a rel="prev" title="A. Interactive Input Editing" | |
406 | href="node15.html"><img src='../icons/previous.png' | |
407 | border='0' height='32' alt='Previous Page' width='32' /></A></td> | |
408 | <td class='online-navigation'><a rel="parent" title="Python Tutorial" | |
409 | href="tut.html"><img src='../icons/up.png' | |
410 | border='0' height='32' alt='Up One Level' width='32' /></A></td> | |
411 | <td class='online-navigation'><a rel="next" title="C. History and License" | |
412 | href="node17.html"><img src='../icons/next.png' | |
413 | border='0' height='32' alt='Next Page' width='32' /></A></td> | |
414 | <td align="center" width="100%">Python Tutorial</td> | |
415 | <td class='online-navigation'><a rel="contents" title="Table of Contents" | |
416 | href="node2.html"><img src='../icons/contents.png' | |
417 | border='0' height='32' alt='Contents' width='32' /></A></td> | |
418 | <td class='online-navigation'><img src='../icons/blank.png' | |
419 | border='0' height='32' alt='' width='32' /></td> | |
420 | <td class='online-navigation'><a rel="index" title="Index" | |
421 | href="node19.html"><img src='../icons/index.png' | |
422 | border='0' height='32' alt='Index' width='32' /></A></td> | |
423 | </tr></table> | |
424 | <div class='online-navigation'> | |
425 | <b class="navlabel">Previous:</b> | |
426 | <a class="sectref" rel="prev" href="node15.html">A. Interactive Input Editing</A> | |
427 | <b class="navlabel">Up:</b> | |
428 | <a class="sectref" rel="parent" href="tut.html">Python Tutorial</A> | |
429 | <b class="navlabel">Next:</b> | |
430 | <a class="sectref" rel="next" href="node17.html">C. History and License</A> | |
431 | </div> | |
432 | </div> | |
433 | <hr /> | |
434 | <span class="release-info">Release 2.4.2, documentation updated on 28 September 2005.</span> | |
435 | </DIV> | |
436 | <!--End of Navigation Panel--> | |
437 | <ADDRESS> | |
438 | See <i><a href="about.html">About this document...</a></i> for information on suggesting changes. | |
439 | </ADDRESS> | |
440 | </BODY> | |
441 | </HTML> |