Commit | Line | Data |
---|---|---|
920dae64 AT |
1 | """Generic socket server classes. |
2 | ||
3 | This module tries to capture the various aspects of defining a server: | |
4 | ||
5 | For socket-based servers: | |
6 | ||
7 | - address family: | |
8 | - AF_INET{,6}: IP (Internet Protocol) sockets (default) | |
9 | - AF_UNIX: Unix domain sockets | |
10 | - others, e.g. AF_DECNET are conceivable (see <socket.h> | |
11 | - socket type: | |
12 | - SOCK_STREAM (reliable stream, e.g. TCP) | |
13 | - SOCK_DGRAM (datagrams, e.g. UDP) | |
14 | ||
15 | For request-based servers (including socket-based): | |
16 | ||
17 | - client address verification before further looking at the request | |
18 | (This is actually a hook for any processing that needs to look | |
19 | at the request before anything else, e.g. logging) | |
20 | - how to handle multiple requests: | |
21 | - synchronous (one request is handled at a time) | |
22 | - forking (each request is handled by a new process) | |
23 | - threading (each request is handled by a new thread) | |
24 | ||
25 | The classes in this module favor the server type that is simplest to | |
26 | write: a synchronous TCP/IP server. This is bad class design, but | |
27 | save some typing. (There's also the issue that a deep class hierarchy | |
28 | slows down method lookups.) | |
29 | ||
30 | There are five classes in an inheritance diagram, four of which represent | |
31 | synchronous servers of four types: | |
32 | ||
33 | +------------+ | |
34 | | BaseServer | | |
35 | +------------+ | |
36 | | | |
37 | v | |
38 | +-----------+ +------------------+ | |
39 | | TCPServer |------->| UnixStreamServer | | |
40 | +-----------+ +------------------+ | |
41 | | | |
42 | v | |
43 | +-----------+ +--------------------+ | |
44 | | UDPServer |------->| UnixDatagramServer | | |
45 | +-----------+ +--------------------+ | |
46 | ||
47 | Note that UnixDatagramServer derives from UDPServer, not from | |
48 | UnixStreamServer -- the only difference between an IP and a Unix | |
49 | stream server is the address family, which is simply repeated in both | |
50 | unix server classes. | |
51 | ||
52 | Forking and threading versions of each type of server can be created | |
53 | using the ForkingServer and ThreadingServer mix-in classes. For | |
54 | instance, a threading UDP server class is created as follows: | |
55 | ||
56 | class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass | |
57 | ||
58 | The Mix-in class must come first, since it overrides a method defined | |
59 | in UDPServer! Setting the various member variables also changes | |
60 | the behavior of the underlying server mechanism. | |
61 | ||
62 | To implement a service, you must derive a class from | |
63 | BaseRequestHandler and redefine its handle() method. You can then run | |
64 | various versions of the service by combining one of the server classes | |
65 | with your request handler class. | |
66 | ||
67 | The request handler class must be different for datagram or stream | |
68 | services. This can be hidden by using the request handler | |
69 | subclasses StreamRequestHandler or DatagramRequestHandler. | |
70 | ||
71 | Of course, you still have to use your head! | |
72 | ||
73 | For instance, it makes no sense to use a forking server if the service | |
74 | contains state in memory that can be modified by requests (since the | |
75 | modifications in the child process would never reach the initial state | |
76 | kept in the parent process and passed to each child). In this case, | |
77 | you can use a threading server, but you will probably have to use | |
78 | locks to avoid two requests that come in nearly simultaneous to apply | |
79 | conflicting changes to the server state. | |
80 | ||
81 | On the other hand, if you are building e.g. an HTTP server, where all | |
82 | data is stored externally (e.g. in the file system), a synchronous | |
83 | class will essentially render the service "deaf" while one request is | |
84 | being handled -- which may be for a very long time if a client is slow | |
85 | to reqd all the data it has requested. Here a threading or forking | |
86 | server is appropriate. | |
87 | ||
88 | In some cases, it may be appropriate to process part of a request | |
89 | synchronously, but to finish processing in a forked child depending on | |
90 | the request data. This can be implemented by using a synchronous | |
91 | server and doing an explicit fork in the request handler class | |
92 | handle() method. | |
93 | ||
94 | Another approach to handling multiple simultaneous requests in an | |
95 | environment that supports neither threads nor fork (or where these are | |
96 | too expensive or inappropriate for the service) is to maintain an | |
97 | explicit table of partially finished requests and to use select() to | |
98 | decide which request to work on next (or whether to handle a new | |
99 | incoming request). This is particularly important for stream services | |
100 | where each client can potentially be connected for a long time (if | |
101 | threads or subprocesses cannot be used). | |
102 | ||
103 | Future work: | |
104 | - Standard classes for Sun RPC (which uses either UDP or TCP) | |
105 | - Standard mix-in classes to implement various authentication | |
106 | and encryption schemes | |
107 | - Standard framework for select-based multiplexing | |
108 | ||
109 | XXX Open problems: | |
110 | - What to do with out-of-band data? | |
111 | ||
112 | BaseServer: | |
113 | - split generic "request" functionality out into BaseServer class. | |
114 | Copyright (C) 2000 Luke Kenneth Casson Leighton <lkcl@samba.org> | |
115 | ||
116 | example: read entries from a SQL database (requires overriding | |
117 | get_request() to return a table entry from the database). | |
118 | entry is processed by a RequestHandlerClass. | |
119 | ||
120 | """ | |
121 | ||
122 | # Author of the BaseServer patch: Luke Kenneth Casson Leighton | |
123 | ||
124 | # XXX Warning! | |
125 | # There is a test suite for this module, but it cannot be run by the | |
126 | # standard regression test. | |
127 | # To run it manually, run Lib/test/test_socketserver.py. | |
128 | ||
129 | __version__ = "0.4" | |
130 | ||
131 | ||
132 | import socket | |
133 | import sys | |
134 | import os | |
135 | ||
136 | __all__ = ["TCPServer","UDPServer","ForkingUDPServer","ForkingTCPServer", | |
137 | "ThreadingUDPServer","ThreadingTCPServer","BaseRequestHandler", | |
138 | "StreamRequestHandler","DatagramRequestHandler", | |
139 | "ThreadingMixIn", "ForkingMixIn"] | |
140 | if hasattr(socket, "AF_UNIX"): | |
141 | __all__.extend(["UnixStreamServer","UnixDatagramServer", | |
142 | "ThreadingUnixStreamServer", | |
143 | "ThreadingUnixDatagramServer"]) | |
144 | ||
145 | class BaseServer: | |
146 | ||
147 | """Base class for server classes. | |
148 | ||
149 | Methods for the caller: | |
150 | ||
151 | - __init__(server_address, RequestHandlerClass) | |
152 | - serve_forever() | |
153 | - handle_request() # if you do not use serve_forever() | |
154 | - fileno() -> int # for select() | |
155 | ||
156 | Methods that may be overridden: | |
157 | ||
158 | - server_bind() | |
159 | - server_activate() | |
160 | - get_request() -> request, client_address | |
161 | - verify_request(request, client_address) | |
162 | - server_close() | |
163 | - process_request(request, client_address) | |
164 | - close_request(request) | |
165 | - handle_error() | |
166 | ||
167 | Methods for derived classes: | |
168 | ||
169 | - finish_request(request, client_address) | |
170 | ||
171 | Class variables that may be overridden by derived classes or | |
172 | instances: | |
173 | ||
174 | - address_family | |
175 | - socket_type | |
176 | - allow_reuse_address | |
177 | ||
178 | Instance variables: | |
179 | ||
180 | - RequestHandlerClass | |
181 | - socket | |
182 | ||
183 | """ | |
184 | ||
185 | def __init__(self, server_address, RequestHandlerClass): | |
186 | """Constructor. May be extended, do not override.""" | |
187 | self.server_address = server_address | |
188 | self.RequestHandlerClass = RequestHandlerClass | |
189 | ||
190 | def server_activate(self): | |
191 | """Called by constructor to activate the server. | |
192 | ||
193 | May be overridden. | |
194 | ||
195 | """ | |
196 | pass | |
197 | ||
198 | def serve_forever(self): | |
199 | """Handle one request at a time until doomsday.""" | |
200 | while 1: | |
201 | self.handle_request() | |
202 | ||
203 | # The distinction between handling, getting, processing and | |
204 | # finishing a request is fairly arbitrary. Remember: | |
205 | # | |
206 | # - handle_request() is the top-level call. It calls | |
207 | # get_request(), verify_request() and process_request() | |
208 | # - get_request() is different for stream or datagram sockets | |
209 | # - process_request() is the place that may fork a new process | |
210 | # or create a new thread to finish the request | |
211 | # - finish_request() instantiates the request handler class; | |
212 | # this constructor will handle the request all by itself | |
213 | ||
214 | def handle_request(self): | |
215 | """Handle one request, possibly blocking.""" | |
216 | try: | |
217 | request, client_address = self.get_request() | |
218 | except socket.error: | |
219 | return | |
220 | if self.verify_request(request, client_address): | |
221 | try: | |
222 | self.process_request(request, client_address) | |
223 | except: | |
224 | self.handle_error(request, client_address) | |
225 | self.close_request(request) | |
226 | ||
227 | def verify_request(self, request, client_address): | |
228 | """Verify the request. May be overridden. | |
229 | ||
230 | Return True if we should proceed with this request. | |
231 | ||
232 | """ | |
233 | return True | |
234 | ||
235 | def process_request(self, request, client_address): | |
236 | """Call finish_request. | |
237 | ||
238 | Overridden by ForkingMixIn and ThreadingMixIn. | |
239 | ||
240 | """ | |
241 | self.finish_request(request, client_address) | |
242 | self.close_request(request) | |
243 | ||
244 | def server_close(self): | |
245 | """Called to clean-up the server. | |
246 | ||
247 | May be overridden. | |
248 | ||
249 | """ | |
250 | pass | |
251 | ||
252 | def finish_request(self, request, client_address): | |
253 | """Finish one request by instantiating RequestHandlerClass.""" | |
254 | self.RequestHandlerClass(request, client_address, self) | |
255 | ||
256 | def close_request(self, request): | |
257 | """Called to clean up an individual request.""" | |
258 | pass | |
259 | ||
260 | def handle_error(self, request, client_address): | |
261 | """Handle an error gracefully. May be overridden. | |
262 | ||
263 | The default is to print a traceback and continue. | |
264 | ||
265 | """ | |
266 | print '-'*40 | |
267 | print 'Exception happened during processing of request from', | |
268 | print client_address | |
269 | import traceback | |
270 | traceback.print_exc() # XXX But this goes to stderr! | |
271 | print '-'*40 | |
272 | ||
273 | ||
274 | class TCPServer(BaseServer): | |
275 | ||
276 | """Base class for various socket-based server classes. | |
277 | ||
278 | Defaults to synchronous IP stream (i.e., TCP). | |
279 | ||
280 | Methods for the caller: | |
281 | ||
282 | - __init__(server_address, RequestHandlerClass) | |
283 | - serve_forever() | |
284 | - handle_request() # if you don't use serve_forever() | |
285 | - fileno() -> int # for select() | |
286 | ||
287 | Methods that may be overridden: | |
288 | ||
289 | - server_bind() | |
290 | - server_activate() | |
291 | - get_request() -> request, client_address | |
292 | - verify_request(request, client_address) | |
293 | - process_request(request, client_address) | |
294 | - close_request(request) | |
295 | - handle_error() | |
296 | ||
297 | Methods for derived classes: | |
298 | ||
299 | - finish_request(request, client_address) | |
300 | ||
301 | Class variables that may be overridden by derived classes or | |
302 | instances: | |
303 | ||
304 | - address_family | |
305 | - socket_type | |
306 | - request_queue_size (only for stream sockets) | |
307 | - allow_reuse_address | |
308 | ||
309 | Instance variables: | |
310 | ||
311 | - server_address | |
312 | - RequestHandlerClass | |
313 | - socket | |
314 | ||
315 | """ | |
316 | ||
317 | address_family = socket.AF_INET | |
318 | ||
319 | socket_type = socket.SOCK_STREAM | |
320 | ||
321 | request_queue_size = 5 | |
322 | ||
323 | allow_reuse_address = False | |
324 | ||
325 | def __init__(self, server_address, RequestHandlerClass): | |
326 | """Constructor. May be extended, do not override.""" | |
327 | BaseServer.__init__(self, server_address, RequestHandlerClass) | |
328 | self.socket = socket.socket(self.address_family, | |
329 | self.socket_type) | |
330 | self.server_bind() | |
331 | self.server_activate() | |
332 | ||
333 | def server_bind(self): | |
334 | """Called by constructor to bind the socket. | |
335 | ||
336 | May be overridden. | |
337 | ||
338 | """ | |
339 | if self.allow_reuse_address: | |
340 | self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) | |
341 | self.socket.bind(self.server_address) | |
342 | ||
343 | def server_activate(self): | |
344 | """Called by constructor to activate the server. | |
345 | ||
346 | May be overridden. | |
347 | ||
348 | """ | |
349 | self.socket.listen(self.request_queue_size) | |
350 | ||
351 | def server_close(self): | |
352 | """Called to clean-up the server. | |
353 | ||
354 | May be overridden. | |
355 | ||
356 | """ | |
357 | self.socket.close() | |
358 | ||
359 | def fileno(self): | |
360 | """Return socket file number. | |
361 | ||
362 | Interface required by select(). | |
363 | ||
364 | """ | |
365 | return self.socket.fileno() | |
366 | ||
367 | def get_request(self): | |
368 | """Get the request and client address from the socket. | |
369 | ||
370 | May be overridden. | |
371 | ||
372 | """ | |
373 | return self.socket.accept() | |
374 | ||
375 | def close_request(self, request): | |
376 | """Called to clean up an individual request.""" | |
377 | request.close() | |
378 | ||
379 | ||
380 | class UDPServer(TCPServer): | |
381 | ||
382 | """UDP server class.""" | |
383 | ||
384 | allow_reuse_address = False | |
385 | ||
386 | socket_type = socket.SOCK_DGRAM | |
387 | ||
388 | max_packet_size = 8192 | |
389 | ||
390 | def get_request(self): | |
391 | data, client_addr = self.socket.recvfrom(self.max_packet_size) | |
392 | return (data, self.socket), client_addr | |
393 | ||
394 | def server_activate(self): | |
395 | # No need to call listen() for UDP. | |
396 | pass | |
397 | ||
398 | def close_request(self, request): | |
399 | # No need to close anything. | |
400 | pass | |
401 | ||
402 | class ForkingMixIn: | |
403 | ||
404 | """Mix-in class to handle each request in a new process.""" | |
405 | ||
406 | active_children = None | |
407 | max_children = 40 | |
408 | ||
409 | def collect_children(self): | |
410 | """Internal routine to wait for died children.""" | |
411 | while self.active_children: | |
412 | if len(self.active_children) < self.max_children: | |
413 | options = os.WNOHANG | |
414 | else: | |
415 | # If the maximum number of children are already | |
416 | # running, block while waiting for a child to exit | |
417 | options = 0 | |
418 | try: | |
419 | pid, status = os.waitpid(0, options) | |
420 | except os.error: | |
421 | pid = None | |
422 | if not pid: break | |
423 | self.active_children.remove(pid) | |
424 | ||
425 | def process_request(self, request, client_address): | |
426 | """Fork a new subprocess to process the request.""" | |
427 | self.collect_children() | |
428 | pid = os.fork() | |
429 | if pid: | |
430 | # Parent process | |
431 | if self.active_children is None: | |
432 | self.active_children = [] | |
433 | self.active_children.append(pid) | |
434 | self.close_request(request) | |
435 | return | |
436 | else: | |
437 | # Child process. | |
438 | # This must never return, hence os._exit()! | |
439 | try: | |
440 | self.finish_request(request, client_address) | |
441 | os._exit(0) | |
442 | except: | |
443 | try: | |
444 | self.handle_error(request, client_address) | |
445 | finally: | |
446 | os._exit(1) | |
447 | ||
448 | ||
449 | class ThreadingMixIn: | |
450 | """Mix-in class to handle each request in a new thread.""" | |
451 | ||
452 | # Decides how threads will act upon termination of the | |
453 | # main process | |
454 | daemon_threads = False | |
455 | ||
456 | def process_request_thread(self, request, client_address): | |
457 | """Same as in BaseServer but as a thread. | |
458 | ||
459 | In addition, exception handling is done here. | |
460 | ||
461 | """ | |
462 | try: | |
463 | self.finish_request(request, client_address) | |
464 | self.close_request(request) | |
465 | except: | |
466 | self.handle_error(request, client_address) | |
467 | self.close_request(request) | |
468 | ||
469 | def process_request(self, request, client_address): | |
470 | """Start a new thread to process the request.""" | |
471 | import threading | |
472 | t = threading.Thread(target = self.process_request_thread, | |
473 | args = (request, client_address)) | |
474 | if self.daemon_threads: | |
475 | t.setDaemon (1) | |
476 | t.start() | |
477 | ||
478 | ||
479 | class ForkingUDPServer(ForkingMixIn, UDPServer): pass | |
480 | class ForkingTCPServer(ForkingMixIn, TCPServer): pass | |
481 | ||
482 | class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass | |
483 | class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass | |
484 | ||
485 | if hasattr(socket, 'AF_UNIX'): | |
486 | ||
487 | class UnixStreamServer(TCPServer): | |
488 | address_family = socket.AF_UNIX | |
489 | ||
490 | class UnixDatagramServer(UDPServer): | |
491 | address_family = socket.AF_UNIX | |
492 | ||
493 | class ThreadingUnixStreamServer(ThreadingMixIn, UnixStreamServer): pass | |
494 | ||
495 | class ThreadingUnixDatagramServer(ThreadingMixIn, UnixDatagramServer): pass | |
496 | ||
497 | class BaseRequestHandler: | |
498 | ||
499 | """Base class for request handler classes. | |
500 | ||
501 | This class is instantiated for each request to be handled. The | |
502 | constructor sets the instance variables request, client_address | |
503 | and server, and then calls the handle() method. To implement a | |
504 | specific service, all you need to do is to derive a class which | |
505 | defines a handle() method. | |
506 | ||
507 | The handle() method can find the request as self.request, the | |
508 | client address as self.client_address, and the server (in case it | |
509 | needs access to per-server information) as self.server. Since a | |
510 | separate instance is created for each request, the handle() method | |
511 | can define arbitrary other instance variariables. | |
512 | ||
513 | """ | |
514 | ||
515 | def __init__(self, request, client_address, server): | |
516 | self.request = request | |
517 | self.client_address = client_address | |
518 | self.server = server | |
519 | try: | |
520 | self.setup() | |
521 | self.handle() | |
522 | self.finish() | |
523 | finally: | |
524 | sys.exc_traceback = None # Help garbage collection | |
525 | ||
526 | def setup(self): | |
527 | pass | |
528 | ||
529 | def handle(self): | |
530 | pass | |
531 | ||
532 | def finish(self): | |
533 | pass | |
534 | ||
535 | ||
536 | # The following two classes make it possible to use the same service | |
537 | # class for stream or datagram servers. | |
538 | # Each class sets up these instance variables: | |
539 | # - rfile: a file object from which receives the request is read | |
540 | # - wfile: a file object to which the reply is written | |
541 | # When the handle() method returns, wfile is flushed properly | |
542 | ||
543 | ||
544 | class StreamRequestHandler(BaseRequestHandler): | |
545 | ||
546 | """Define self.rfile and self.wfile for stream sockets.""" | |
547 | ||
548 | # Default buffer sizes for rfile, wfile. | |
549 | # We default rfile to buffered because otherwise it could be | |
550 | # really slow for large data (a getc() call per byte); we make | |
551 | # wfile unbuffered because (a) often after a write() we want to | |
552 | # read and we need to flush the line; (b) big writes to unbuffered | |
553 | # files are typically optimized by stdio even when big reads | |
554 | # aren't. | |
555 | rbufsize = -1 | |
556 | wbufsize = 0 | |
557 | ||
558 | def setup(self): | |
559 | self.connection = self.request | |
560 | self.rfile = self.connection.makefile('rb', self.rbufsize) | |
561 | self.wfile = self.connection.makefile('wb', self.wbufsize) | |
562 | ||
563 | def finish(self): | |
564 | if not self.wfile.closed: | |
565 | self.wfile.flush() | |
566 | self.wfile.close() | |
567 | self.rfile.close() | |
568 | ||
569 | ||
570 | class DatagramRequestHandler(BaseRequestHandler): | |
571 | ||
572 | # XXX Regrettably, I cannot get this working on Linux; | |
573 | # s.recvfrom() doesn't return a meaningful client address. | |
574 | ||
575 | """Define self.rfile and self.wfile for datagram sockets.""" | |
576 | ||
577 | def setup(self): | |
578 | import StringIO | |
579 | self.packet, self.socket = self.request | |
580 | self.rfile = StringIO.StringIO(self.packet) | |
581 | self.wfile = StringIO.StringIO() | |
582 | ||
583 | def finish(self): | |
584 | self.socket.sendto(self.wfile.getvalue(), self.client_address) |