Commit | Line | Data |
---|---|---|
920dae64 AT |
1 | =head1 NAME |
2 | ||
3 | perliol - C API for Perl's implementation of IO in Layers. | |
4 | ||
5 | =head1 SYNOPSIS | |
6 | ||
7 | /* Defining a layer ... */ | |
8 | #include <perliol.h> | |
9 | ||
10 | =head1 DESCRIPTION | |
11 | ||
12 | This document describes the behavior and implementation of the PerlIO | |
13 | abstraction described in L<perlapio> when C<USE_PERLIO> is defined (and | |
14 | C<USE_SFIO> is not). | |
15 | ||
16 | =head2 History and Background | |
17 | ||
18 | The PerlIO abstraction was introduced in perl5.003_02 but languished as | |
19 | just an abstraction until perl5.7.0. However during that time a number | |
20 | of perl extensions switched to using it, so the API is mostly fixed to | |
21 | maintain (source) compatibility. | |
22 | ||
23 | The aim of the implementation is to provide the PerlIO API in a flexible | |
24 | and platform neutral manner. It is also a trial of an "Object Oriented | |
25 | C, with vtables" approach which may be applied to perl6. | |
26 | ||
27 | =head2 Basic Structure | |
28 | ||
29 | PerlIO is a stack of layers. | |
30 | ||
31 | The low levels of the stack work with the low-level operating system | |
32 | calls (file descriptors in C) getting bytes in and out, the higher | |
33 | layers of the stack buffer, filter, and otherwise manipulate the I/O, | |
34 | and return characters (or bytes) to Perl. Terms I<above> and I<below> | |
35 | are used to refer to the relative positioning of the stack layers. | |
36 | ||
37 | A layer contains a "vtable", the table of I/O operations (at C level | |
38 | a table of function pointers), and status flags. The functions in the | |
39 | vtable implement operations like "open", "read", and "write". | |
40 | ||
41 | When I/O, for example "read", is requested, the request goes from Perl | |
42 | first down the stack using "read" functions of each layer, then at the | |
43 | bottom the input is requested from the operating system services, then | |
44 | the result is returned up the stack, finally being interpreted as Perl | |
45 | data. | |
46 | ||
47 | The requests do not necessarily go always all the way down to the | |
48 | operating system: that's where PerlIO buffering comes into play. | |
49 | ||
50 | When you do an open() and specify extra PerlIO layers to be deployed, | |
51 | the layers you specify are "pushed" on top of the already existing | |
52 | default stack. One way to see it is that "operating system is | |
53 | on the left" and "Perl is on the right". | |
54 | ||
55 | What exact layers are in this default stack depends on a lot of | |
56 | things: your operating system, Perl version, Perl compile time | |
57 | configuration, and Perl runtime configuration. See L<PerlIO>, | |
58 | L<perlrun/PERLIO>, and L<open> for more information. | |
59 | ||
60 | binmode() operates similarly to open(): by default the specified | |
61 | layers are pushed on top of the existing stack. | |
62 | ||
63 | However, note that even as the specified layers are "pushed on top" | |
64 | for open() and binmode(), this doesn't mean that the effects are | |
65 | limited to the "top": PerlIO layers can be very 'active' and inspect | |
66 | and affect layers also deeper in the stack. As an example there | |
67 | is a layer called "raw" which repeatedly "pops" layers until | |
68 | it reaches the first layer that has declared itself capable of | |
69 | handling binary data. The "pushed" layers are processed in left-to-right | |
70 | order. | |
71 | ||
72 | sysopen() operates (unsurprisingly) at a lower level in the stack than | |
73 | open(). For example in UNIX or UNIX-like systems sysopen() operates | |
74 | directly at the level of file descriptors: in the terms of PerlIO | |
75 | layers, it uses only the "unix" layer, which is a rather thin wrapper | |
76 | on top of the UNIX file descriptors. | |
77 | ||
78 | =head2 Layers vs Disciplines | |
79 | ||
80 | Initial discussion of the ability to modify IO streams behaviour used | |
81 | the term "discipline" for the entities which were added. This came (I | |
82 | believe) from the use of the term in "sfio", which in turn borrowed it | |
83 | from "line disciplines" on Unix terminals. However, this document (and | |
84 | the C code) uses the term "layer". | |
85 | ||
86 | This is, I hope, a natural term given the implementation, and should | |
87 | avoid connotations that are inherent in earlier uses of "discipline" | |
88 | for things which are rather different. | |
89 | ||
90 | =head2 Data Structures | |
91 | ||
92 | The basic data structure is a PerlIOl: | |
93 | ||
94 | typedef struct _PerlIO PerlIOl; | |
95 | typedef struct _PerlIO_funcs PerlIO_funcs; | |
96 | typedef PerlIOl *PerlIO; | |
97 | ||
98 | struct _PerlIO | |
99 | { | |
100 | PerlIOl * next; /* Lower layer */ | |
101 | PerlIO_funcs * tab; /* Functions for this layer */ | |
102 | IV flags; /* Various flags for state */ | |
103 | }; | |
104 | ||
105 | A C<PerlIOl *> is a pointer to the struct, and the I<application> | |
106 | level C<PerlIO *> is a pointer to a C<PerlIOl *> - i.e. a pointer | |
107 | to a pointer to the struct. This allows the application level C<PerlIO *> | |
108 | to remain constant while the actual C<PerlIOl *> underneath | |
109 | changes. (Compare perl's C<SV *> which remains constant while its | |
110 | C<sv_any> field changes as the scalar's type changes.) An IO stream is | |
111 | then in general represented as a pointer to this linked-list of | |
112 | "layers". | |
113 | ||
114 | It should be noted that because of the double indirection in a C<PerlIO *>, | |
115 | a C<< &(perlio->next) >> "is" a C<PerlIO *>, and so to some degree | |
116 | at least one layer can use the "standard" API on the next layer down. | |
117 | ||
118 | A "layer" is composed of two parts: | |
119 | ||
120 | =over 4 | |
121 | ||
122 | =item 1. | |
123 | ||
124 | The functions and attributes of the "layer class". | |
125 | ||
126 | =item 2. | |
127 | ||
128 | The per-instance data for a particular handle. | |
129 | ||
130 | =back | |
131 | ||
132 | =head2 Functions and Attributes | |
133 | ||
134 | The functions and attributes are accessed via the "tab" (for table) | |
135 | member of C<PerlIOl>. The functions (methods of the layer "class") are | |
136 | fixed, and are defined by the C<PerlIO_funcs> type. They are broadly the | |
137 | same as the public C<PerlIO_xxxxx> functions: | |
138 | ||
139 | struct _PerlIO_funcs | |
140 | { | |
141 | Size_t fsize; | |
142 | char * name; | |
143 | Size_t size; | |
144 | IV kind; | |
145 | IV (*Pushed)(pTHX_ PerlIO *f,const char *mode,SV *arg, PerlIO_funcs *tab); | |
146 | IV (*Popped)(pTHX_ PerlIO *f); | |
147 | PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab, | |
148 | AV *layers, IV n, | |
149 | const char *mode, | |
150 | int fd, int imode, int perm, | |
151 | PerlIO *old, | |
152 | int narg, SV **args); | |
153 | IV (*Binmode)(pTHX_ PerlIO *f); | |
154 | SV * (*Getarg)(pTHX_ PerlIO *f, CLONE_PARAMS *param, int flags) | |
155 | IV (*Fileno)(pTHX_ PerlIO *f); | |
156 | PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o, CLONE_PARAMS *param, int flags) | |
157 | /* Unix-like functions - cf sfio line disciplines */ | |
158 | SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count); | |
159 | SSize_t (*Unread)(pTHX_ PerlIO *f, const void *vbuf, Size_t count); | |
160 | SSize_t (*Write)(pTHX_ PerlIO *f, const void *vbuf, Size_t count); | |
161 | IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence); | |
162 | Off_t (*Tell)(pTHX_ PerlIO *f); | |
163 | IV (*Close)(pTHX_ PerlIO *f); | |
164 | /* Stdio-like buffered IO functions */ | |
165 | IV (*Flush)(pTHX_ PerlIO *f); | |
166 | IV (*Fill)(pTHX_ PerlIO *f); | |
167 | IV (*Eof)(pTHX_ PerlIO *f); | |
168 | IV (*Error)(pTHX_ PerlIO *f); | |
169 | void (*Clearerr)(pTHX_ PerlIO *f); | |
170 | void (*Setlinebuf)(pTHX_ PerlIO *f); | |
171 | /* Perl's snooping functions */ | |
172 | STDCHAR * (*Get_base)(pTHX_ PerlIO *f); | |
173 | Size_t (*Get_bufsiz)(pTHX_ PerlIO *f); | |
174 | STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f); | |
175 | SSize_t (*Get_cnt)(pTHX_ PerlIO *f); | |
176 | void (*Set_ptrcnt)(pTHX_ PerlIO *f,STDCHAR *ptr,SSize_t cnt); | |
177 | }; | |
178 | ||
179 | The first few members of the struct give a function table size for | |
180 | compatibility check "name" for the layer, the size to C<malloc> for the per-instance data, | |
181 | and some flags which are attributes of the class as whole (such as whether it is a buffering | |
182 | layer), then follow the functions which fall into four basic groups: | |
183 | ||
184 | =over 4 | |
185 | ||
186 | =item 1. | |
187 | ||
188 | Opening and setup functions | |
189 | ||
190 | =item 2. | |
191 | ||
192 | Basic IO operations | |
193 | ||
194 | =item 3. | |
195 | ||
196 | Stdio class buffering options. | |
197 | ||
198 | =item 4. | |
199 | ||
200 | Functions to support Perl's traditional "fast" access to the buffer. | |
201 | ||
202 | =back | |
203 | ||
204 | A layer does not have to implement all the functions, but the whole | |
205 | table has to be present. Unimplemented slots can be NULL (which will | |
206 | result in an error when called) or can be filled in with stubs to | |
207 | "inherit" behaviour from a "base class". This "inheritance" is fixed | |
208 | for all instances of the layer, but as the layer chooses which stubs | |
209 | to populate the table, limited "multiple inheritance" is possible. | |
210 | ||
211 | =head2 Per-instance Data | |
212 | ||
213 | The per-instance data are held in memory beyond the basic PerlIOl | |
214 | struct, by making a PerlIOl the first member of the layer's struct | |
215 | thus: | |
216 | ||
217 | typedef struct | |
218 | { | |
219 | struct _PerlIO base; /* Base "class" info */ | |
220 | STDCHAR * buf; /* Start of buffer */ | |
221 | STDCHAR * end; /* End of valid part of buffer */ | |
222 | STDCHAR * ptr; /* Current position in buffer */ | |
223 | Off_t posn; /* Offset of buf into the file */ | |
224 | Size_t bufsiz; /* Real size of buffer */ | |
225 | IV oneword; /* Emergency buffer */ | |
226 | } PerlIOBuf; | |
227 | ||
228 | In this way (as for perl's scalars) a pointer to a PerlIOBuf can be | |
229 | treated as a pointer to a PerlIOl. | |
230 | ||
231 | =head2 Layers in action. | |
232 | ||
233 | table perlio unix | |
234 | | | | |
235 | +-----------+ +----------+ +--------+ | |
236 | PerlIO ->| |--->| next |--->| NULL | | |
237 | +-----------+ +----------+ +--------+ | |
238 | | | | buffer | | fd | | |
239 | +-----------+ | | +--------+ | |
240 | | | +----------+ | |
241 | ||
242 | ||
243 | The above attempts to show how the layer scheme works in a simple case. | |
244 | The application's C<PerlIO *> points to an entry in the table(s) | |
245 | representing open (allocated) handles. For example the first three slots | |
246 | in the table correspond to C<stdin>,C<stdout> and C<stderr>. The table | |
247 | in turn points to the current "top" layer for the handle - in this case | |
248 | an instance of the generic buffering layer "perlio". That layer in turn | |
249 | points to the next layer down - in this case the lowlevel "unix" layer. | |
250 | ||
251 | The above is roughly equivalent to a "stdio" buffered stream, but with | |
252 | much more flexibility: | |
253 | ||
254 | =over 4 | |
255 | ||
256 | =item * | |
257 | ||
258 | If Unix level C<read>/C<write>/C<lseek> is not appropriate for (say) | |
259 | sockets then the "unix" layer can be replaced (at open time or even | |
260 | dynamically) with a "socket" layer. | |
261 | ||
262 | =item * | |
263 | ||
264 | Different handles can have different buffering schemes. The "top" | |
265 | layer could be the "mmap" layer if reading disk files was quicker | |
266 | using C<mmap> than C<read>. An "unbuffered" stream can be implemented | |
267 | simply by not having a buffer layer. | |
268 | ||
269 | =item * | |
270 | ||
271 | Extra layers can be inserted to process the data as it flows through. | |
272 | This was the driving need for including the scheme in perl 5.7.0+ - we | |
273 | needed a mechanism to allow data to be translated between perl's | |
274 | internal encoding (conceptually at least Unicode as UTF-8), and the | |
275 | "native" format used by the system. This is provided by the | |
276 | ":encoding(xxxx)" layer which typically sits above the buffering layer. | |
277 | ||
278 | =item * | |
279 | ||
280 | A layer can be added that does "\n" to CRLF translation. This layer | |
281 | can be used on any platform, not just those that normally do such | |
282 | things. | |
283 | ||
284 | =back | |
285 | ||
286 | =head2 Per-instance flag bits | |
287 | ||
288 | The generic flag bits are a hybrid of C<O_XXXXX> style flags deduced | |
289 | from the mode string passed to C<PerlIO_open()>, and state bits for | |
290 | typical buffer layers. | |
291 | ||
292 | =over 4 | |
293 | ||
294 | =item PERLIO_F_EOF | |
295 | ||
296 | End of file. | |
297 | ||
298 | =item PERLIO_F_CANWRITE | |
299 | ||
300 | Writes are permitted, i.e. opened as "w" or "r+" or "a", etc. | |
301 | ||
302 | =item PERLIO_F_CANREAD | |
303 | ||
304 | Reads are permitted i.e. opened "r" or "w+" (or even "a+" - ick). | |
305 | ||
306 | =item PERLIO_F_ERROR | |
307 | ||
308 | An error has occurred (for C<PerlIO_error()>). | |
309 | ||
310 | =item PERLIO_F_TRUNCATE | |
311 | ||
312 | Truncate file suggested by open mode. | |
313 | ||
314 | =item PERLIO_F_APPEND | |
315 | ||
316 | All writes should be appends. | |
317 | ||
318 | =item PERLIO_F_CRLF | |
319 | ||
320 | Layer is performing Win32-like "\n" mapped to CR,LF for output and CR,LF | |
321 | mapped to "\n" for input. Normally the provided "crlf" layer is the only | |
322 | layer that need bother about this. C<PerlIO_binmode()> will mess with this | |
323 | flag rather than add/remove layers if the C<PERLIO_K_CANCRLF> bit is set | |
324 | for the layers class. | |
325 | ||
326 | =item PERLIO_F_UTF8 | |
327 | ||
328 | Data written to this layer should be UTF-8 encoded; data provided | |
329 | by this layer should be considered UTF-8 encoded. Can be set on any layer | |
330 | by ":utf8" dummy layer. Also set on ":encoding" layer. | |
331 | ||
332 | =item PERLIO_F_UNBUF | |
333 | ||
334 | Layer is unbuffered - i.e. write to next layer down should occur for | |
335 | each write to this layer. | |
336 | ||
337 | =item PERLIO_F_WRBUF | |
338 | ||
339 | The buffer for this layer currently holds data written to it but not sent | |
340 | to next layer. | |
341 | ||
342 | =item PERLIO_F_RDBUF | |
343 | ||
344 | The buffer for this layer currently holds unconsumed data read from | |
345 | layer below. | |
346 | ||
347 | =item PERLIO_F_LINEBUF | |
348 | ||
349 | Layer is line buffered. Write data should be passed to next layer down | |
350 | whenever a "\n" is seen. Any data beyond the "\n" should then be | |
351 | processed. | |
352 | ||
353 | =item PERLIO_F_TEMP | |
354 | ||
355 | File has been C<unlink()>ed, or should be deleted on C<close()>. | |
356 | ||
357 | =item PERLIO_F_OPEN | |
358 | ||
359 | Handle is open. | |
360 | ||
361 | =item PERLIO_F_FASTGETS | |
362 | ||
363 | This instance of this layer supports the "fast C<gets>" interface. | |
364 | Normally set based on C<PERLIO_K_FASTGETS> for the class and by the | |
365 | existence of the function(s) in the table. However a class that | |
366 | normally provides that interface may need to avoid it on a | |
367 | particular instance. The "pending" layer needs to do this when | |
368 | it is pushed above a layer which does not support the interface. | |
369 | (Perl's C<sv_gets()> does not expect the streams fast C<gets> behaviour | |
370 | to change during one "get".) | |
371 | ||
372 | =back | |
373 | ||
374 | =head2 Methods in Detail | |
375 | ||
376 | =over 4 | |
377 | ||
378 | =item fsize | |
379 | ||
380 | Size_t fsize; | |
381 | ||
382 | Size of the function table. This is compared against the value PerlIO | |
383 | code "knows" as a compatibility check. Future versions I<may> be able | |
384 | to tolerate layers compiled against an old version of the headers. | |
385 | ||
386 | =item name | |
387 | ||
388 | char * name; | |
389 | ||
390 | The name of the layer whose open() method Perl should invoke on | |
391 | open(). For example if the layer is called APR, you will call: | |
392 | ||
393 | open $fh, ">:APR", ... | |
394 | ||
395 | and Perl knows that it has to invoke the PerlIOAPR_open() method | |
396 | implemented by the APR layer. | |
397 | ||
398 | =item size | |
399 | ||
400 | Size_t size; | |
401 | ||
402 | The size of the per-instance data structure, e.g.: | |
403 | ||
404 | sizeof(PerlIOAPR) | |
405 | ||
406 | If this field is zero then C<PerlIO_pushed> does not malloc anything | |
407 | and assumes layer's Pushed function will do any required layer stack | |
408 | manipulation - used to avoid malloc/free overhead for dummy layers. | |
409 | If the field is non-zero it must be at least the size of C<PerlIOl>, | |
410 | C<PerlIO_pushed> will allocate memory for the layer's data structures | |
411 | and link new layer onto the stream's stack. (If the layer's Pushed | |
412 | method returns an error indication the layer is popped again.) | |
413 | ||
414 | =item kind | |
415 | ||
416 | IV kind; | |
417 | ||
418 | =over 4 | |
419 | ||
420 | =item * PERLIO_K_BUFFERED | |
421 | ||
422 | The layer is buffered. | |
423 | ||
424 | =item * PERLIO_K_RAW | |
425 | ||
426 | The layer is acceptable to have in a binmode(FH) stack - i.e. it does not | |
427 | (or will configure itself not to) transform bytes passing through it. | |
428 | ||
429 | =item * PERLIO_K_CANCRLF | |
430 | ||
431 | Layer can translate between "\n" and CRLF line ends. | |
432 | ||
433 | =item * PERLIO_K_FASTGETS | |
434 | ||
435 | Layer allows buffer snooping. | |
436 | ||
437 | =item * PERLIO_K_MULTIARG | |
438 | ||
439 | Used when the layer's open() accepts more arguments than usual. The | |
440 | extra arguments should come not before the C<MODE> argument. When this | |
441 | flag is used it's up to the layer to validate the args. | |
442 | ||
443 | =back | |
444 | ||
445 | =item Pushed | |
446 | ||
447 | IV (*Pushed)(pTHX_ PerlIO *f,const char *mode, SV *arg); | |
448 | ||
449 | The only absolutely mandatory method. Called when the layer is pushed | |
450 | onto the stack. The C<mode> argument may be NULL if this occurs | |
451 | post-open. The C<arg> will be non-C<NULL> if an argument string was | |
452 | passed. In most cases this should call C<PerlIOBase_pushed()> to | |
453 | convert C<mode> into the appropriate C<PERLIO_F_XXXXX> flags in | |
454 | addition to any actions the layer itself takes. If a layer is not | |
455 | expecting an argument it need neither save the one passed to it, nor | |
456 | provide C<Getarg()> (it could perhaps C<Perl_warn> that the argument | |
457 | was un-expected). | |
458 | ||
459 | Returns 0 on success. On failure returns -1 and should set errno. | |
460 | ||
461 | =item Popped | |
462 | ||
463 | IV (*Popped)(pTHX_ PerlIO *f); | |
464 | ||
465 | Called when the layer is popped from the stack. A layer will normally | |
466 | be popped after C<Close()> is called. But a layer can be popped | |
467 | without being closed if the program is dynamically managing layers on | |
468 | the stream. In such cases C<Popped()> should free any resources | |
469 | (buffers, translation tables, ...) not held directly in the layer's | |
470 | struct. It should also C<Unread()> any unconsumed data that has been | |
471 | read and buffered from the layer below back to that layer, so that it | |
472 | can be re-provided to what ever is now above. | |
473 | ||
474 | Returns 0 on success and failure. If C<Popped()> returns I<true> then | |
475 | I<perlio.c> assumes that either the layer has popped itself, or the | |
476 | layer is super special and needs to be retained for other reasons. | |
477 | In most cases it should return I<false>. | |
478 | ||
479 | =item Open | |
480 | ||
481 | PerlIO * (*Open)(...); | |
482 | ||
483 | The C<Open()> method has lots of arguments because it combines the | |
484 | functions of perl's C<open>, C<PerlIO_open>, perl's C<sysopen>, | |
485 | C<PerlIO_fdopen> and C<PerlIO_reopen>. The full prototype is as | |
486 | follows: | |
487 | ||
488 | PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab, | |
489 | AV *layers, IV n, | |
490 | const char *mode, | |
491 | int fd, int imode, int perm, | |
492 | PerlIO *old, | |
493 | int narg, SV **args); | |
494 | ||
495 | Open should (perhaps indirectly) call C<PerlIO_allocate()> to allocate | |
496 | a slot in the table and associate it with the layers information for | |
497 | the opened file, by calling C<PerlIO_push>. The I<layers> AV is an | |
498 | array of all the layers destined for the C<PerlIO *>, and any | |
499 | arguments passed to them, I<n> is the index into that array of the | |
500 | layer being called. The macro C<PerlIOArg> will return a (possibly | |
501 | C<NULL>) SV * for the argument passed to the layer. | |
502 | ||
503 | The I<mode> string is an "C<fopen()>-like" string which would match | |
504 | the regular expression C</^[I#]?[rwa]\+?[bt]?$/>. | |
505 | ||
506 | The C<'I'> prefix is used during creation of C<stdin>..C<stderr> via | |
507 | special C<PerlIO_fdopen> calls; the C<'#'> prefix means that this is | |
508 | C<sysopen> and that I<imode> and I<perm> should be passed to | |
509 | C<PerlLIO_open3>; C<'r'> means B<r>ead, C<'w'> means B<w>rite and | |
510 | C<'a'> means B<a>ppend. The C<'+'> suffix means that both reading and | |
511 | writing/appending are permitted. The C<'b'> suffix means file should | |
512 | be binary, and C<'t'> means it is text. (Almost all layers should do | |
513 | the IO in binary mode, and ignore the b/t bits. The C<:crlf> layer | |
514 | should be pushed to handle the distinction.) | |
515 | ||
516 | If I<old> is not C<NULL> then this is a C<PerlIO_reopen>. Perl itself | |
517 | does not use this (yet?) and semantics are a little vague. | |
518 | ||
519 | If I<fd> not negative then it is the numeric file descriptor I<fd>, | |
520 | which will be open in a manner compatible with the supplied mode | |
521 | string, the call is thus equivalent to C<PerlIO_fdopen>. In this case | |
522 | I<nargs> will be zero. | |
523 | ||
524 | If I<nargs> is greater than zero then it gives the number of arguments | |
525 | passed to C<open>, otherwise it will be 1 if for example | |
526 | C<PerlIO_open> was called. In simple cases SvPV_nolen(*args) is the | |
527 | pathname to open. | |
528 | ||
529 | Having said all that translation-only layers do not need to provide | |
530 | C<Open()> at all, but rather leave the opening to a lower level layer | |
531 | and wait to be "pushed". If a layer does provide C<Open()> it should | |
532 | normally call the C<Open()> method of next layer down (if any) and | |
533 | then push itself on top if that succeeds. | |
534 | ||
535 | If C<PerlIO_push> was performed and open has failed, it must | |
536 | C<PerlIO_pop> itself, since if it's not, the layer won't be removed | |
537 | and may cause bad problems. | |
538 | ||
539 | Returns C<NULL> on failure. | |
540 | ||
541 | =item Binmode | |
542 | ||
543 | IV (*Binmode)(pTHX_ PerlIO *f); | |
544 | ||
545 | Optional. Used when C<:raw> layer is pushed (explicitly or as a result | |
546 | of binmode(FH)). If not present layer will be popped. If present | |
547 | should configure layer as binary (or pop itself) and return 0. | |
548 | If it returns -1 for error C<binmode> will fail with layer | |
549 | still on the stack. | |
550 | ||
551 | =item Getarg | |
552 | ||
553 | SV * (*Getarg)(pTHX_ PerlIO *f, | |
554 | CLONE_PARAMS *param, int flags); | |
555 | ||
556 | Optional. If present should return an SV * representing the string | |
557 | argument passed to the layer when it was | |
558 | pushed. e.g. ":encoding(ascii)" would return an SvPV with value | |
559 | "ascii". (I<param> and I<flags> arguments can be ignored in most | |
560 | cases) | |
561 | ||
562 | C<Dup> uses C<Getarg> to retrieve the argument originally passed to | |
563 | C<Pushed>, so you must implement this function if your layer has an | |
564 | extra argument to C<Pushed> and will ever be C<Dup>ed. | |
565 | ||
566 | =item Fileno | |
567 | ||
568 | IV (*Fileno)(pTHX_ PerlIO *f); | |
569 | ||
570 | Returns the Unix/Posix numeric file descriptor for the handle. Normally | |
571 | C<PerlIOBase_fileno()> (which just asks next layer down) will suffice | |
572 | for this. | |
573 | ||
574 | Returns -1 on error, which is considered to include the case where the | |
575 | layer cannot provide such a file descriptor. | |
576 | ||
577 | =item Dup | |
578 | ||
579 | PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o, | |
580 | CLONE_PARAMS *param, int flags); | |
581 | ||
582 | XXX: Needs more docs. | |
583 | ||
584 | Used as part of the "clone" process when a thread is spawned (in which | |
585 | case param will be non-NULL) and when a stream is being duplicated via | |
586 | '&' in the C<open>. | |
587 | ||
588 | Similar to C<Open>, returns PerlIO* on success, C<NULL> on failure. | |
589 | ||
590 | =item Read | |
591 | ||
592 | SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count); | |
593 | ||
594 | Basic read operation. | |
595 | ||
596 | Typically will call C<Fill> and manipulate pointers (possibly via the | |
597 | API). C<PerlIOBuf_read()> may be suitable for derived classes which | |
598 | provide "fast gets" methods. | |
599 | ||
600 | Returns actual bytes read, or -1 on an error. | |
601 | ||
602 | =item Unread | |
603 | ||
604 | SSize_t (*Unread)(pTHX_ PerlIO *f, | |
605 | const void *vbuf, Size_t count); | |
606 | ||
607 | A superset of stdio's C<ungetc()>. Should arrange for future reads to | |
608 | see the bytes in C<vbuf>. If there is no obviously better implementation | |
609 | then C<PerlIOBase_unread()> provides the function by pushing a "fake" | |
610 | "pending" layer above the calling layer. | |
611 | ||
612 | Returns the number of unread chars. | |
613 | ||
614 | =item Write | |
615 | ||
616 | SSize_t (*Write)(PerlIO *f, const void *vbuf, Size_t count); | |
617 | ||
618 | Basic write operation. | |
619 | ||
620 | Returns bytes written or -1 on an error. | |
621 | ||
622 | =item Seek | |
623 | ||
624 | IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence); | |
625 | ||
626 | Position the file pointer. Should normally call its own C<Flush> | |
627 | method and then the C<Seek> method of next layer down. | |
628 | ||
629 | Returns 0 on success, -1 on failure. | |
630 | ||
631 | =item Tell | |
632 | ||
633 | Off_t (*Tell)(pTHX_ PerlIO *f); | |
634 | ||
635 | Return the file pointer. May be based on layers cached concept of | |
636 | position to avoid overhead. | |
637 | ||
638 | Returns -1 on failure to get the file pointer. | |
639 | ||
640 | =item Close | |
641 | ||
642 | IV (*Close)(pTHX_ PerlIO *f); | |
643 | ||
644 | Close the stream. Should normally call C<PerlIOBase_close()> to flush | |
645 | itself and close layers below, and then deallocate any data structures | |
646 | (buffers, translation tables, ...) not held directly in the data | |
647 | structure. | |
648 | ||
649 | Returns 0 on success, -1 on failure. | |
650 | ||
651 | =item Flush | |
652 | ||
653 | IV (*Flush)(pTHX_ PerlIO *f); | |
654 | ||
655 | Should make stream's state consistent with layers below. That is, any | |
656 | buffered write data should be written, and file position of lower layers | |
657 | adjusted for data read from below but not actually consumed. | |
658 | (Should perhaps C<Unread()> such data to the lower layer.) | |
659 | ||
660 | Returns 0 on success, -1 on failure. | |
661 | ||
662 | =item Fill | |
663 | ||
664 | IV (*Fill)(pTHX_ PerlIO *f); | |
665 | ||
666 | The buffer for this layer should be filled (for read) from layer | |
667 | below. When you "subclass" PerlIOBuf layer, you want to use its | |
668 | I<_read> method and to supply your own fill method, which fills the | |
669 | PerlIOBuf's buffer. | |
670 | ||
671 | Returns 0 on success, -1 on failure. | |
672 | ||
673 | =item Eof | |
674 | ||
675 | IV (*Eof)(pTHX_ PerlIO *f); | |
676 | ||
677 | Return end-of-file indicator. C<PerlIOBase_eof()> is normally sufficient. | |
678 | ||
679 | Returns 0 on end-of-file, 1 if not end-of-file, -1 on error. | |
680 | ||
681 | =item Error | |
682 | ||
683 | IV (*Error)(pTHX_ PerlIO *f); | |
684 | ||
685 | Return error indicator. C<PerlIOBase_error()> is normally sufficient. | |
686 | ||
687 | Returns 1 if there is an error (usually when C<PERLIO_F_ERROR> is set, | |
688 | 0 otherwise. | |
689 | ||
690 | =item Clearerr | |
691 | ||
692 | void (*Clearerr)(pTHX_ PerlIO *f); | |
693 | ||
694 | Clear end-of-file and error indicators. Should call C<PerlIOBase_clearerr()> | |
695 | to set the C<PERLIO_F_XXXXX> flags, which may suffice. | |
696 | ||
697 | =item Setlinebuf | |
698 | ||
699 | void (*Setlinebuf)(pTHX_ PerlIO *f); | |
700 | ||
701 | Mark the stream as line buffered. C<PerlIOBase_setlinebuf()> sets the | |
702 | PERLIO_F_LINEBUF flag and is normally sufficient. | |
703 | ||
704 | =item Get_base | |
705 | ||
706 | STDCHAR * (*Get_base)(pTHX_ PerlIO *f); | |
707 | ||
708 | Allocate (if not already done so) the read buffer for this layer and | |
709 | return pointer to it. Return NULL on failure. | |
710 | ||
711 | =item Get_bufsiz | |
712 | ||
713 | Size_t (*Get_bufsiz)(pTHX_ PerlIO *f); | |
714 | ||
715 | Return the number of bytes that last C<Fill()> put in the buffer. | |
716 | ||
717 | =item Get_ptr | |
718 | ||
719 | STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f); | |
720 | ||
721 | Return the current read pointer relative to this layer's buffer. | |
722 | ||
723 | =item Get_cnt | |
724 | ||
725 | SSize_t (*Get_cnt)(pTHX_ PerlIO *f); | |
726 | ||
727 | Return the number of bytes left to be read in the current buffer. | |
728 | ||
729 | =item Set_ptrcnt | |
730 | ||
731 | void (*Set_ptrcnt)(pTHX_ PerlIO *f, | |
732 | STDCHAR *ptr, SSize_t cnt); | |
733 | ||
734 | Adjust the read pointer and count of bytes to match C<ptr> and/or C<cnt>. | |
735 | The application (or layer above) must ensure they are consistent. | |
736 | (Checking is allowed by the paranoid.) | |
737 | ||
738 | =back | |
739 | ||
740 | =head2 Utilities | |
741 | ||
742 | To ask for the next layer down use PerlIONext(PerlIO *f). | |
743 | ||
744 | To check that a PerlIO* is valid use PerlIOValid(PerlIO *f). (All | |
745 | this does is really just to check that the pointer is non-NULL and | |
746 | that the pointer behind that is non-NULL.) | |
747 | ||
748 | PerlIOBase(PerlIO *f) returns the "Base" pointer, or in other words, | |
749 | the C<PerlIOl*> pointer. | |
750 | ||
751 | PerlIOSelf(PerlIO* f, type) return the PerlIOBase cast to a type. | |
752 | ||
753 | Perl_PerlIO_or_Base(PerlIO* f, callback, base, failure, args) either | |
754 | calls the I<callback> from the functions of the layer I<f> (just by | |
755 | the name of the IO function, like "Read") with the I<args>, or if | |
756 | there is no such callback, calls the I<base> version of the callback | |
757 | with the same args, or if the f is invalid, set errno to EBADF and | |
758 | return I<failure>. | |
759 | ||
760 | Perl_PerlIO_or_fail(PerlIO* f, callback, failure, args) either calls | |
761 | the I<callback> of the functions of the layer I<f> with the I<args>, | |
762 | or if there is no such callback, set errno to EINVAL. Or if the f is | |
763 | invalid, set errno to EBADF and return I<failure>. | |
764 | ||
765 | Perl_PerlIO_or_Base_void(PerlIO* f, callback, base, args) either calls | |
766 | the I<callback> of the functions of the layer I<f> with the I<args>, | |
767 | or if there is no such callback, calls the I<base> version of the | |
768 | callback with the same args, or if the f is invalid, set errno to | |
769 | EBADF. | |
770 | ||
771 | Perl_PerlIO_or_fail_void(PerlIO* f, callback, args) either calls the | |
772 | I<callback> of the functions of the layer I<f> with the I<args>, or if | |
773 | there is no such callback, set errno to EINVAL. Or if the f is | |
774 | invalid, set errno to EBADF. | |
775 | ||
776 | =head2 Implementing PerlIO Layers | |
777 | ||
778 | If you find the implementation document unclear or not sufficient, | |
779 | look at the existing PerlIO layer implementations, which include: | |
780 | ||
781 | =over | |
782 | ||
783 | =item * C implementations | |
784 | ||
785 | The F<perlio.c> and F<perliol.h> in the Perl core implement the | |
786 | "unix", "perlio", "stdio", "crlf", "utf8", "byte", "raw", "pending" | |
787 | layers, and also the "mmap" and "win32" layers if applicable. | |
788 | (The "win32" is currently unfinished and unused, to see what is used | |
789 | instead in Win32, see L<PerlIO/"Querying the layers of filehandles"> .) | |
790 | ||
791 | PerlIO::encoding, PerlIO::scalar, PerlIO::via in the Perl core. | |
792 | ||
793 | PerlIO::gzip and APR::PerlIO (mod_perl 2.0) on CPAN. | |
794 | ||
795 | =item * Perl implementations | |
796 | ||
797 | PerlIO::via::QuotedPrint in the Perl core and PerlIO::via::* on CPAN. | |
798 | ||
799 | =back | |
800 | ||
801 | If you are creating a PerlIO layer, you may want to be lazy, in other | |
802 | words, implement only the methods that interest you. The other methods | |
803 | you can either replace with the "blank" methods | |
804 | ||
805 | PerlIOBase_noop_ok | |
806 | PerlIOBase_noop_fail | |
807 | ||
808 | (which do nothing, and return zero and -1, respectively) or for | |
809 | certain methods you may assume a default behaviour by using a NULL | |
810 | method. The Open method looks for help in the 'parent' layer. | |
811 | The following table summarizes the behaviour: | |
812 | ||
813 | method behaviour with NULL | |
814 | ||
815 | Clearerr PerlIOBase_clearerr | |
816 | Close PerlIOBase_close | |
817 | Dup PerlIOBase_dup | |
818 | Eof PerlIOBase_eof | |
819 | Error PerlIOBase_error | |
820 | Fileno PerlIOBase_fileno | |
821 | Fill FAILURE | |
822 | Flush SUCCESS | |
823 | Getarg SUCCESS | |
824 | Get_base FAILURE | |
825 | Get_bufsiz FAILURE | |
826 | Get_cnt FAILURE | |
827 | Get_ptr FAILURE | |
828 | Open INHERITED | |
829 | Popped SUCCESS | |
830 | Pushed SUCCESS | |
831 | Read PerlIOBase_read | |
832 | Seek FAILURE | |
833 | Set_cnt FAILURE | |
834 | Set_ptrcnt FAILURE | |
835 | Setlinebuf PerlIOBase_setlinebuf | |
836 | Tell FAILURE | |
837 | Unread PerlIOBase_unread | |
838 | Write FAILURE | |
839 | ||
840 | FAILURE Set errno (to EINVAL in UNIXish, to LIB$_INVARG in VMS) and | |
841 | return -1 (for numeric return values) or NULL (for pointers) | |
842 | INHERITED Inherited from the layer below | |
843 | SUCCESS Return 0 (for numeric return values) or a pointer | |
844 | ||
845 | =head2 Core Layers | |
846 | ||
847 | The file C<perlio.c> provides the following layers: | |
848 | ||
849 | =over 4 | |
850 | ||
851 | =item "unix" | |
852 | ||
853 | A basic non-buffered layer which calls Unix/POSIX C<read()>, C<write()>, | |
854 | C<lseek()>, C<close()>. No buffering. Even on platforms that distinguish | |
855 | between O_TEXT and O_BINARY this layer is always O_BINARY. | |
856 | ||
857 | =item "perlio" | |
858 | ||
859 | A very complete generic buffering layer which provides the whole of | |
860 | PerlIO API. It is also intended to be used as a "base class" for other | |
861 | layers. (For example its C<Read()> method is implemented in terms of | |
862 | the C<Get_cnt()>/C<Get_ptr()>/C<Set_ptrcnt()> methods). | |
863 | ||
864 | "perlio" over "unix" provides a complete replacement for stdio as seen | |
865 | via PerlIO API. This is the default for USE_PERLIO when system's stdio | |
866 | does not permit perl's "fast gets" access, and which do not | |
867 | distinguish between C<O_TEXT> and C<O_BINARY>. | |
868 | ||
869 | =item "stdio" | |
870 | ||
871 | A layer which provides the PerlIO API via the layer scheme, but | |
872 | implements it by calling system's stdio. This is (currently) the default | |
873 | if system's stdio provides sufficient access to allow perl's "fast gets" | |
874 | access and which do not distinguish between C<O_TEXT> and C<O_BINARY>. | |
875 | ||
876 | =item "crlf" | |
877 | ||
878 | A layer derived using "perlio" as a base class. It provides Win32-like | |
879 | "\n" to CR,LF translation. Can either be applied above "perlio" or serve | |
880 | as the buffer layer itself. "crlf" over "unix" is the default if system | |
881 | distinguishes between C<O_TEXT> and C<O_BINARY> opens. (At some point | |
882 | "unix" will be replaced by a "native" Win32 IO layer on that platform, | |
883 | as Win32's read/write layer has various drawbacks.) The "crlf" layer is | |
884 | a reasonable model for a layer which transforms data in some way. | |
885 | ||
886 | =item "mmap" | |
887 | ||
888 | If Configure detects C<mmap()> functions this layer is provided (with | |
889 | "perlio" as a "base") which does "read" operations by mmap()ing the | |
890 | file. Performance improvement is marginal on modern systems, so it is | |
891 | mainly there as a proof of concept. It is likely to be unbundled from | |
892 | the core at some point. The "mmap" layer is a reasonable model for a | |
893 | minimalist "derived" layer. | |
894 | ||
895 | =item "pending" | |
896 | ||
897 | An "internal" derivative of "perlio" which can be used to provide | |
898 | Unread() function for layers which have no buffer or cannot be | |
899 | bothered. (Basically this layer's C<Fill()> pops itself off the stack | |
900 | and so resumes reading from layer below.) | |
901 | ||
902 | =item "raw" | |
903 | ||
904 | A dummy layer which never exists on the layer stack. Instead when | |
905 | "pushed" it actually pops the stack removing itself, it then calls | |
906 | Binmode function table entry on all the layers in the stack - normally | |
907 | this (via PerlIOBase_binmode) removes any layers which do not have | |
908 | C<PERLIO_K_RAW> bit set. Layers can modify that behaviour by defining | |
909 | their own Binmode entry. | |
910 | ||
911 | =item "utf8" | |
912 | ||
913 | Another dummy layer. When pushed it pops itself and sets the | |
914 | C<PERLIO_F_UTF8> flag on the layer which was (and now is once more) | |
915 | the top of the stack. | |
916 | ||
917 | =back | |
918 | ||
919 | In addition F<perlio.c> also provides a number of C<PerlIOBase_xxxx()> | |
920 | functions which are intended to be used in the table slots of classes | |
921 | which do not need to do anything special for a particular method. | |
922 | ||
923 | =head2 Extension Layers | |
924 | ||
925 | Layers can made available by extension modules. When an unknown layer | |
926 | is encountered the PerlIO code will perform the equivalent of : | |
927 | ||
928 | use PerlIO 'layer'; | |
929 | ||
930 | Where I<layer> is the unknown layer. F<PerlIO.pm> will then attempt to: | |
931 | ||
932 | require PerlIO::layer; | |
933 | ||
934 | If after that process the layer is still not defined then the C<open> | |
935 | will fail. | |
936 | ||
937 | The following extension layers are bundled with perl: | |
938 | ||
939 | =over 4 | |
940 | ||
941 | =item ":encoding" | |
942 | ||
943 | use Encoding; | |
944 | ||
945 | makes this layer available, although F<PerlIO.pm> "knows" where to | |
946 | find it. It is an example of a layer which takes an argument as it is | |
947 | called thus: | |
948 | ||
949 | open( $fh, "<:encoding(iso-8859-7)", $pathname ); | |
950 | ||
951 | =item ":scalar" | |
952 | ||
953 | Provides support for reading data from and writing data to a scalar. | |
954 | ||
955 | open( $fh, "+<:scalar", \$scalar ); | |
956 | ||
957 | When a handle is so opened, then reads get bytes from the string value | |
958 | of I<$scalar>, and writes change the value. In both cases the position | |
959 | in I<$scalar> starts as zero but can be altered via C<seek>, and | |
960 | determined via C<tell>. | |
961 | ||
962 | Please note that this layer is implied when calling open() thus: | |
963 | ||
964 | open( $fh, "+<", \$scalar ); | |
965 | ||
966 | =item ":via" | |
967 | ||
968 | Provided to allow layers to be implemented as Perl code. For instance: | |
969 | ||
970 | use PerlIO::via::StripHTML; | |
971 | open( my $fh, "<:via(StripHTML)", "index.html" ); | |
972 | ||
973 | See L<PerlIO::via> for details. | |
974 | ||
975 | =back | |
976 | ||
977 | =head1 TODO | |
978 | ||
979 | Things that need to be done to improve this document. | |
980 | ||
981 | =over | |
982 | ||
983 | =item * | |
984 | ||
985 | Explain how to make a valid fh without going through open()(i.e. apply | |
986 | a layer). For example if the file is not opened through perl, but we | |
987 | want to get back a fh, like it was opened by Perl. | |
988 | ||
989 | How PerlIO_apply_layera fits in, where its docs, was it made public? | |
990 | ||
991 | Currently the example could be something like this: | |
992 | ||
993 | PerlIO *foo_to_PerlIO(pTHX_ char *mode, ...) | |
994 | { | |
995 | char *mode; /* "w", "r", etc */ | |
996 | const char *layers = ":APR"; /* the layer name */ | |
997 | PerlIO *f = PerlIO_allocate(aTHX); | |
998 | if (!f) { | |
999 | return NULL; | |
1000 | } | |
1001 | ||
1002 | PerlIO_apply_layers(aTHX_ f, mode, layers); | |
1003 | ||
1004 | if (f) { | |
1005 | PerlIOAPR *st = PerlIOSelf(f, PerlIOAPR); | |
1006 | /* fill in the st struct, as in _open() */ | |
1007 | st->file = file; | |
1008 | PerlIOBase(f)->flags |= PERLIO_F_OPEN; | |
1009 | ||
1010 | return f; | |
1011 | } | |
1012 | return NULL; | |
1013 | } | |
1014 | ||
1015 | =item * | |
1016 | ||
1017 | fix/add the documentation in places marked as XXX. | |
1018 | ||
1019 | =item * | |
1020 | ||
1021 | The handling of errors by the layer is not specified. e.g. when $! | |
1022 | should be set explicitly, when the error handling should be just | |
1023 | delegated to the top layer. | |
1024 | ||
1025 | Probably give some hints on using SETERRNO() or pointers to where they | |
1026 | can be found. | |
1027 | ||
1028 | =item * | |
1029 | ||
1030 | I think it would help to give some concrete examples to make it easier | |
1031 | to understand the API. Of course I agree that the API has to be | |
1032 | concise, but since there is no second document that is more of a | |
1033 | guide, I think that it'd make it easier to start with the doc which is | |
1034 | an API, but has examples in it in places where things are unclear, to | |
1035 | a person who is not a PerlIO guru (yet). | |
1036 | ||
1037 | =back | |
1038 | ||
1039 | =cut |