Commit | Line | Data |
---|---|---|
86530b38 AT |
1 | <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> |
2 | <html> | |
3 | <head> | |
4 | <link rel="STYLESHEET" href="lib.css" type='text/css' /> | |
5 | <link rel="SHORTCUT ICON" href="../icons/pyfav.png" type="image/png" /> | |
6 | <link rel='start' href='../index.html' title='Python Documentation Index' /> | |
7 | <link rel="first" href="lib.html" title='Python Library Reference' /> | |
8 | <link rel='contents' href='contents.html' title="Contents" /> | |
9 | <link rel='index' href='genindex.html' title='Index' /> | |
10 | <link rel='last' href='about.html' title='About this document...' /> | |
11 | <link rel='help' href='about.html' title='About this document...' /> | |
12 | <link rel="next" href="module-symbol.html" /> | |
13 | <link rel="prev" href="language.html" /> | |
14 | <link rel="parent" href="language.html" /> | |
15 | <link rel="next" href="node767.html" /> | |
16 | <meta name='aesop' content='information' /> | |
17 | <title>18.1 parser -- Access Python parse trees</title> | |
18 | </head> | |
19 | <body> | |
20 | <DIV CLASS="navigation"> | |
21 | <div id='top-navigation-panel' xml:id='top-navigation-panel'> | |
22 | <table align="center" width="100%" cellpadding="0" cellspacing="2"> | |
23 | <tr> | |
24 | <td class='online-navigation'><a rel="prev" title="18. Python Language Services" | |
25 | href="language.html"><img src='../icons/previous.png' | |
26 | border='0' height='32' alt='Previous Page' width='32' /></A></td> | |
27 | <td class='online-navigation'><a rel="parent" title="18. Python Language Services" | |
28 | href="language.html"><img src='../icons/up.png' | |
29 | border='0' height='32' alt='Up One Level' width='32' /></A></td> | |
30 | <td class='online-navigation'><a rel="next" title="18.1.1 Creating AST Objects" | |
31 | href="node767.html"><img src='../icons/next.png' | |
32 | border='0' height='32' alt='Next Page' width='32' /></A></td> | |
33 | <td align="center" width="100%">Python Library Reference</td> | |
34 | <td class='online-navigation'><a rel="contents" title="Table of Contents" | |
35 | href="contents.html"><img src='../icons/contents.png' | |
36 | border='0' height='32' alt='Contents' width='32' /></A></td> | |
37 | <td class='online-navigation'><a href="modindex.html" title="Module Index"><img src='../icons/modules.png' | |
38 | border='0' height='32' alt='Module Index' width='32' /></a></td> | |
39 | <td class='online-navigation'><a rel="index" title="Index" | |
40 | href="genindex.html"><img src='../icons/index.png' | |
41 | border='0' height='32' alt='Index' width='32' /></A></td> | |
42 | </tr></table> | |
43 | <div class='online-navigation'> | |
44 | <b class="navlabel">Previous:</b> | |
45 | <a class="sectref" rel="prev" href="language.html">18. Python Language Services</A> | |
46 | <b class="navlabel">Up:</b> | |
47 | <a class="sectref" rel="parent" href="language.html">18. Python Language Services</A> | |
48 | <b class="navlabel">Next:</b> | |
49 | <a class="sectref" rel="next" href="node767.html">18.1.1 Creating AST Objects</A> | |
50 | </div> | |
51 | <hr /></div> | |
52 | </DIV> | |
53 | <!--End of Navigation Panel--> | |
54 | ||
55 | <H1><A NAME="SECTION0020100000000000000000"> | |
56 | 18.1 <tt class="module">parser</tt> -- | |
57 | Access Python parse trees</A> | |
58 | </H1> | |
59 | ||
60 | <P> | |
61 | <A NAME="module-parser"></A> | |
62 | ||
63 | <P> | |
64 | <a id='l2h-4937' xml:id='l2h-4937'></a> | |
65 | ||
66 | <P> | |
67 | The <tt class="module">parser</tt> module provides an interface to Python's internal | |
68 | parser and byte-code compiler. The primary purpose for this interface | |
69 | is to allow Python code to edit the parse tree of a Python expression | |
70 | and create executable code from this. This is better than trying | |
71 | to parse and modify an arbitrary Python code fragment as a string | |
72 | because parsing is performed in a manner identical to the code | |
73 | forming the application. It is also faster. | |
74 | ||
75 | <P> | |
76 | There are a few things to note about this module which are important | |
77 | to making use of the data structures created. This is not a tutorial | |
78 | on editing the parse trees for Python code, but some examples of using | |
79 | the <tt class="module">parser</tt> module are presented. | |
80 | ||
81 | <P> | |
82 | Most importantly, a good understanding of the Python grammar processed | |
83 | by the internal parser is required. For full information on the | |
84 | language syntax, refer to the <em class="citetitle"><a | |
85 | href="../ref/ref.html" | |
86 | title="Python | |
87 | Language Reference" | |
88 | >Python | |
89 | Language Reference</a></em>. The parser itself is created from a grammar | |
90 | specification defined in the file <span class="file">Grammar/Grammar</span> in the | |
91 | standard Python distribution. The parse trees stored in the AST | |
92 | objects created by this module are the actual output from the internal | |
93 | parser when created by the <tt class="function">expr()</tt> or <tt class="function">suite()</tt> | |
94 | functions, described below. The AST objects created by | |
95 | <tt class="function">sequence2ast()</tt> faithfully simulate those structures. Be | |
96 | aware that the values of the sequences which are considered | |
97 | ``correct'' will vary from one version of Python to another as the | |
98 | formal grammar for the language is revised. However, transporting | |
99 | code from one Python version to another as source text will always | |
100 | allow correct parse trees to be created in the target version, with | |
101 | the only restriction being that migrating to an older version of the | |
102 | interpreter will not support more recent language constructs. The | |
103 | parse trees are not typically compatible from one version to another, | |
104 | whereas source code has always been forward-compatible. | |
105 | ||
106 | <P> | |
107 | Each element of the sequences returned by <tt class="function">ast2list()</tt> or | |
108 | <tt class="function">ast2tuple()</tt> has a simple form. Sequences representing | |
109 | non-terminal elements in the grammar always have a length greater than | |
110 | one. The first element is an integer which identifies a production in | |
111 | the grammar. These integers are given symbolic names in the C header | |
112 | file <span class="file">Include/graminit.h</span> and the Python module | |
113 | <tt class="module"><a href="module-symbol.html">symbol</a></tt>. Each additional element of the sequence represents | |
114 | a component of the production as recognized in the input string: these | |
115 | are always sequences which have the same form as the parent. An | |
116 | important aspect of this structure which should be noted is that | |
117 | keywords used to identify the parent node type, such as the keyword | |
118 | <tt class="keyword">if</tt> in an <tt class="constant">if_stmt</tt>, are included in the node tree without | |
119 | any special treatment. For example, the <tt class="keyword">if</tt> keyword is | |
120 | represented by the tuple <code>(1, 'if')</code>, where <code>1</code> is the | |
121 | numeric value associated with all <tt class="constant">NAME</tt> tokens, including | |
122 | variable and function names defined by the user. In an alternate form | |
123 | returned when line number information is requested, the same token | |
124 | might be represented as <code>(1, 'if', 12)</code>, where the <code>12</code> | |
125 | represents the line number at which the terminal symbol was found. | |
126 | ||
127 | <P> | |
128 | Terminal elements are represented in much the same way, but without | |
129 | any child elements and the addition of the source text which was | |
130 | identified. The example of the <tt class="keyword">if</tt> keyword above is | |
131 | representative. The various types of terminal symbols are defined in | |
132 | the C header file <span class="file">Include/token.h</span> and the Python module | |
133 | <tt class="module"><a href="module-token.html">token</a></tt>. | |
134 | ||
135 | <P> | |
136 | The AST objects are not required to support the functionality of this | |
137 | module, but are provided for three purposes: to allow an application | |
138 | to amortize the cost of processing complex parse trees, to provide a | |
139 | parse tree representation which conserves memory space when compared | |
140 | to the Python list or tuple representation, and to ease the creation | |
141 | of additional modules in C which manipulate parse trees. A simple | |
142 | ``wrapper'' class may be created in Python to hide the use of AST | |
143 | objects. | |
144 | ||
145 | <P> | |
146 | The <tt class="module">parser</tt> module defines functions for a few distinct | |
147 | purposes. The most important purposes are to create AST objects and | |
148 | to convert AST objects to other representations such as parse trees | |
149 | and compiled code objects, but there are also functions which serve to | |
150 | query the type of parse tree represented by an AST object. | |
151 | ||
152 | <P> | |
153 | <div class="seealso"> | |
154 | <p class="heading">See Also:</p> | |
155 | ||
156 | <dl compact="compact" class="seemodule"> | |
157 | <dt>Module <b><tt class="module"><a href="module-symbol.html">symbol</a></tt>:</b> | |
158 | <dd>Useful constants representing internal nodes of | |
159 | the parse tree. | |
160 | </dl> | |
161 | <dl compact="compact" class="seemodule"> | |
162 | <dt>Module <b><tt class="module"><a href="module-token.html">token</a></tt>:</b> | |
163 | <dd>Useful constants representing leaf nodes of the | |
164 | parse tree and functions for testing node values. | |
165 | </dl> | |
166 | </div> | |
167 | ||
168 | <P> | |
169 | ||
170 | <p><br /></p><hr class='online-navigation' /> | |
171 | <div class='online-navigation'> | |
172 | <!--Table of Child-Links--> | |
173 | <A NAME="CHILD_LINKS"><STRONG>Subsections</STRONG></a> | |
174 | ||
175 | <UL CLASS="ChildLinks"> | |
176 | <LI><A href="node767.html">18.1.1 Creating AST Objects</a> | |
177 | <LI><A href="node768.html">18.1.2 Converting AST Objects</a> | |
178 | <LI><A href="node769.html">18.1.3 Queries on AST Objects</a> | |
179 | <LI><A href="node770.html">18.1.4 Exceptions and Error Handling</a> | |
180 | <LI><A href="node771.html">18.1.5 AST Objects</a> | |
181 | <LI><A href="node772.html">18.1.6 Examples</a> | |
182 | <UL> | |
183 | <LI><A href="node773.html">18.1.6.1 Emulation of <tt class="function">compile()</tt></a> | |
184 | <LI><A href="node774.html">18.1.6.2 Information Discovery</a> | |
185 | </ul></ul> | |
186 | <!--End of Table of Child-Links--> | |
187 | </div> | |
188 | ||
189 | <DIV CLASS="navigation"> | |
190 | <div class='online-navigation'> | |
191 | <p></p><hr /> | |
192 | <table align="center" width="100%" cellpadding="0" cellspacing="2"> | |
193 | <tr> | |
194 | <td class='online-navigation'><a rel="prev" title="18. Python Language Services" | |
195 | href="language.html"><img src='../icons/previous.png' | |
196 | border='0' height='32' alt='Previous Page' width='32' /></A></td> | |
197 | <td class='online-navigation'><a rel="parent" title="18. Python Language Services" | |
198 | href="language.html"><img src='../icons/up.png' | |
199 | border='0' height='32' alt='Up One Level' width='32' /></A></td> | |
200 | <td class='online-navigation'><a rel="next" title="18.1.1 Creating AST Objects" | |
201 | href="node767.html"><img src='../icons/next.png' | |
202 | border='0' height='32' alt='Next Page' width='32' /></A></td> | |
203 | <td align="center" width="100%">Python Library Reference</td> | |
204 | <td class='online-navigation'><a rel="contents" title="Table of Contents" | |
205 | href="contents.html"><img src='../icons/contents.png' | |
206 | border='0' height='32' alt='Contents' width='32' /></A></td> | |
207 | <td class='online-navigation'><a href="modindex.html" title="Module Index"><img src='../icons/modules.png' | |
208 | border='0' height='32' alt='Module Index' width='32' /></a></td> | |
209 | <td class='online-navigation'><a rel="index" title="Index" | |
210 | href="genindex.html"><img src='../icons/index.png' | |
211 | border='0' height='32' alt='Index' width='32' /></A></td> | |
212 | </tr></table> | |
213 | <div class='online-navigation'> | |
214 | <b class="navlabel">Previous:</b> | |
215 | <a class="sectref" rel="prev" href="language.html">18. Python Language Services</A> | |
216 | <b class="navlabel">Up:</b> | |
217 | <a class="sectref" rel="parent" href="language.html">18. Python Language Services</A> | |
218 | <b class="navlabel">Next:</b> | |
219 | <a class="sectref" rel="next" href="node767.html">18.1.1 Creating AST Objects</A> | |
220 | </div> | |
221 | </div> | |
222 | <hr /> | |
223 | <span class="release-info">Release 2.4.2, documentation updated on 28 September 2005.</span> | |
224 | </DIV> | |
225 | <!--End of Navigation Panel--> | |
226 | <ADDRESS> | |
227 | See <i><a href="about.html">About this document...</a></i> for information on suggesting changes. | |
228 | </ADDRESS> | |
229 | </BODY> | |
230 | </HTML> |