<!DOCTYPE html PUBLIC
"-//W3C//DTD HTML 4.0 Transitional//EN">
<link rel=
"STYLESHEET" href=
"lib.css" type='text/css'
/>
<link rel=
"SHORTCUT ICON" href=
"../icons/pyfav.png" type=
"image/png" />
<link rel='start' href='../index.html' title='Python Documentation Index'
/>
<link rel=
"first" href=
"lib.html" title='Python Library Reference'
/>
<link rel='contents' href='contents.html'
title=
"Contents" />
<link rel='index' href='genindex.html' title='Index'
/>
<link rel='last' href='about.html' title='About this document...'
/>
<link rel='help' href='about.html' title='About this document...'
/>
<link rel=
"prev" href=
"http-error-processor-objects.html" />
<link rel=
"parent" href=
"module-urllib2.html" />
<link rel=
"next" href=
"module-httplib.html" />
<meta name='aesop' content='information'
/>
<title>11.5.22 Examples
</title>
<div id='top-navigation-panel' xml:id='top-navigation-panel'
>
<table align=
"center" width=
"100%" cellpadding=
"0" cellspacing=
"2">
<td class='online-navigation'
><a rel=
"prev" title=
"11.5.21 HTTPErrorProcessor Objects"
href=
"http-error-processor-objects.html"><img src='../icons/previous.png'
border='
0' height='
32' alt='Previous Page' width='
32'
/></A></td>
<td class='online-navigation'
><a rel=
"parent" title=
"11.5 urllib2 "
href=
"module-urllib2.html"><img src='../icons/up.png'
border='
0' height='
32' alt='Up One Level' width='
32'
/></A></td>
<td class='online-navigation'
><a rel=
"next" title=
"11.6 httplib "
href=
"module-httplib.html"><img src='../icons/next.png'
border='
0' height='
32' alt='Next Page' width='
32'
/></A></td>
<td align=
"center" width=
"100%">Python Library Reference
</td>
<td class='online-navigation'
><a rel=
"contents" title=
"Table of Contents"
href=
"contents.html"><img src='../icons/contents.png'
border='
0' height='
32' alt='Contents' width='
32'
/></A></td>
<td class='online-navigation'
><a href=
"modindex.html" title=
"Module Index"><img src='../icons/modules.png'
border='
0' height='
32' alt='Module Index' width='
32'
/></a></td>
<td class='online-navigation'
><a rel=
"index" title=
"Index"
href=
"genindex.html"><img src='../icons/index.png'
border='
0' height='
32' alt='Index' width='
32'
/></A></td>
<div class='online-navigation'
>
<b class=
"navlabel">Previous:
</b>
<a class=
"sectref" rel=
"prev" href=
"http-error-processor-objects.html">11.5.21 HTTPErrorProcessor Objects
</A>
<b class=
"navlabel">Up:
</b>
<a class=
"sectref" rel=
"parent" href=
"module-urllib2.html">11.5 urllib2
</A>
<b class=
"navlabel">Next:
</b>
<a class=
"sectref" rel=
"next" href=
"module-httplib.html">11.6 httplib
</A>
<!--End of Navigation Panel-->
<H2><A NAME=
"SECTION00135220000000000000000"></A><A NAME=
"urllib2-examples"></A>
This example gets the python.org main page and displays the first
100
<div class=
"verbatim"><pre>
>>> import urllib2
>>> f = urllib2.urlopen('http://www.python.org/')
>>> print f.read(
100)
<!DOCTYPE html PUBLIC
"-//W3C//DTD HTML 4.01 Transitional//EN">
<?xml-stylesheet
href=
"./css/ht2html
Here we are sending a data-stream to the stdin of a CGI and reading
the data it returns to us:
<div class="verbatim
"><pre>
>>> import urllib2
>>> req = urllib2.Request(url='https://localhost/cgi-bin/test.cgi',
... data='This data is passed to stdin of the CGI')
>>> f = urllib2.urlopen(req)
>>> print f.read()
Got Data: "This data is passed to stdin of the CGI
"
The code for the sample CGI used in the above example is:
<div class="verbatim
"><pre>
print 'Content-type: text-plain\n\nGot Data: "%s
"' % data
Use of Basic HTTP Authentication:
<div class="verbatim
"><pre>
# Create an OpenerDirector with support for Basic HTTP Authentication...
auth_handler = urllib2.HTTPBasicAuthHandler()
auth_handler.add_password('realm', 'host', 'username', 'password')
opener = urllib2.build_opener(auth_handler)
# ...and install it globally so it can be used with urlopen.
urllib2.install_opener(opener)
urllib2.urlopen('http://www.example.com/login.html')
<tt class="function
">build_opener()</tt> provides many handlers by default, including a
<tt class="class
">ProxyHandler</tt>. By default, <tt class="class
">ProxyHandler</tt> uses the
environment variables named <code><scheme>_proxy</code>, where <code><scheme></code>
is the URL scheme involved. For example, the <a class="envvar
" id='l2h-3294' xml:id='l2h-3294'>http_proxy</a>
environment variable is read to obtain the HTTP proxy's URL.
This example replaces the default <tt class="class
">ProxyHandler</tt> with one that uses
programatically-supplied proxy URLs, and adds proxy authorization support
with <tt class="class
">ProxyBasicAuthHandler</tt>.
<div class="verbatim
"><pre>
proxy_handler = urllib2.ProxyHandler({'http': 'http://www.example.com:3128/'})
proxy_auth_handler = urllib2.HTTPBasicAuthHandler()
proxy_auth_handler.add_password('realm', 'host', 'username', 'password')
opener = build_opener(proxy_handler, proxy_auth_handler)
# This time, rather than install the OpenerDirector, we use it directly:
opener.open('http://www.example.com/login.html')
Use the <var>headers</var> argument to the <tt class="class
">Request</tt> constructor, or:
<div class="verbatim
"><pre>
req = urllib2.Request('http://www.example.com/')
req.add_header('Referer', 'http://www.python.org/')
<tt class="class
">OpenerDirector</tt> automatically adds a <span class="mailheader
">User-Agent:</span>
header to every <tt class="class
">Request</tt>. To change this:
<div class="verbatim
"><pre>
opener = urllib2.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
opener.open('http://www.example.com/')
Also, remember that a few standard headers
(<span class="mailheader
">Content-Length:</span>, <span class="mailheader
">Content-Type:</span> and
<span class="mailheader
">Host:</span>) are added when the <tt class="class
">Request</tt> is passed to
<tt class="function
">urlopen()</tt> (or <tt class="method
">OpenerDirector.open()</tt>).
<div class='online-navigation'>
<table align="center
" width="100%
" cellpadding="0" cellspacing="2">
<td class='online-navigation'><a rel="prev
" title="11.5.21 HTTPErrorProcessor Objects
"
href="http-error-processor-objects.html
"><img src='../icons/previous.png'
border='0' height='32' alt='Previous Page' width='32' /></A></td>
<td class='online-navigation'><a rel="parent
" title="11.5 urllib2
"
href="module-urllib2.html
"><img src='../icons/up.png'
border='0' height='32' alt='Up One Level' width='32' /></A></td>
<td class='online-navigation'><a rel="next
" title="11.6 httplib
"
href="module-httplib.html
"><img src='../icons/next.png'
border='0' height='32' alt='Next Page' width='32' /></A></td>
<td align="center
" width="100%
">Python Library Reference</td>
<td class='online-navigation'><a rel="contents
" title="Table of Contents
"
href="contents.html
"><img src='../icons/contents.png'
border='0' height='32' alt='Contents' width='32' /></A></td>
<td class='online-navigation'><a href="modindex.html
" title="Module Index
"><img src='../icons/modules.png'
border='0' height='32' alt='Module Index' width='32' /></a></td>
<td class='online-navigation'><a rel="index
" title="Index
"
href="genindex.html
"><img src='../icons/index.png'
border='0' height='32' alt='Index' width='32' /></A></td>
<div class='online-navigation'>
<b class="navlabel
">Previous:</b>
<a class="sectref
" rel="prev
" href="http-error-processor-objects.html
">11.5.21 HTTPErrorProcessor Objects</A>
<b class="navlabel
">Up:</b>
<a class="sectref
" rel="parent
" href="module-urllib2.html
">11.5 urllib2 </A>
<b class="navlabel
">Next:</b>
<a class="sectref
" rel="next
" href="module-httplib.html
">11.6 httplib </A>
<span class="release-info
">Release 2.4.2, documentation updated on 28 September 2005.</span>
<!--End of Navigation Panel-->
See <i><a href="about.html
">About this document...</a></i> for information on suggesting changes.