This is guile.info, produced by makeinfo version 6.8 from guile.texi.

This manual documents Guile version 3.0.11.

   Copyright (C) 1996-1997, 2000-2005, 2009-2023 Free Software
Foundation, Inc.
Copyright (C) 2021 Maxime Devos
Copyright (C) 2024 Tomas Volf

   Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.  A
copy of the license is included in the section entitled “GNU Free
Documentation License.”

   Additionally, the documentation of the SRFI 64 module is adapted from
its specification text, which is made available under the following
Expat license:

     Permission is hereby granted, free of charge, to any person
     obtaining a copy of this software and associated documentation
     files (the "Software"), to deal in the Software without
     restriction, including without limitation the rights to use, copy,
     modify, merge, publish, distribute, sublicense, and/or sell copies
     of the Software, and to permit persons to whom the Software is
     furnished to do so, subject to the following conditions:

     The above copyright notice and this permission notice shall be
     included in all copies or substantial portions of the Software.

     THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
     EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
     MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
     NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
     BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
     ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
     CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
     SOFTWARE.

   Additionally, the documentation of the SRFI 197 module is adapted
from its specification text, which is made available under the following
MIT license:

     Permission is hereby granted, free of charge, to any person
     obtaining a copy of this software and associated documentation
     files (the "Software"), to deal in the Software without
     restriction, including without limitation the rights to use, copy,
     modify, merge, publish, distribute, sublicense, and/or sell copies
     of the Software, and to permit persons to whom the Software is
     furnished to do so, subject to the following conditions:

     The above copyright notice and this permission notice shall be
     included in all copies or substantial portions of the Software.

     THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
     EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
     MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
     NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
     BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
     ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
     CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
     SOFTWARE.

INFO-DIR-SECTION The Algorithmic Language Scheme
START-INFO-DIR-ENTRY
* Guile Reference: (guile).     The Guile reference manual.
END-INFO-DIR-ENTRY


File: guile.info,  Node: Encoding,  Next: Textual I/O,  Prev: Binary I/O,  Up: Input and Output

6.12.3 Encoding
---------------

Textual input and output on Guile ports is layered on top of binary
operations.  To this end, each port has an associated character encoding
that controls how bytes read from the port are converted to characters,
and how characters written to the port are converted to bytes.

 -- Scheme Procedure: port-encoding port
 -- C Function: scm_port_encoding (port)
     Returns, as a string, the character encoding that PORT uses to
     interpret its input and output.

 -- Scheme Procedure: set-port-encoding! port enc
 -- C Function: scm_set_port_encoding_x (port, enc)
     Sets the character encoding that will be used to interpret I/O to
     PORT.  ENC is a string containing the name of an encoding.  Valid
     encoding names are those defined by IANA
     (http://www.iana.org/assignments/character-sets), for example
     ‘"UTF-8"’ or ‘"ISO-8859-1"’.

   When ports are created, they are assigned an encoding.  The usual
process to determine the initial encoding for a port is to take the
value of the ‘%default-port-encoding’ fluid.

 -- Scheme Variable: %default-port-encoding
     A fluid containing name of the encoding to be used by default for
     newly created ports (*note Fluids and Dynamic States::).  As a
     special case, the value ‘#f’ is equivalent to ‘"ISO-8859-1"’.

   The ‘%default-port-encoding’ itself defaults to the encoding
appropriate for the current locale, if ‘setlocale’ has been called.
*Note Locales::, for more on locales and when you might need to call
‘setlocale’ explicitly.

   Some port types have other ways of determining their initial locales.
String ports, for example, default to the UTF-8 encoding, in order to be
able to represent all characters regardless of the current locale.  File
ports can optionally sniff their file for a ‘coding:’ declaration; *Note
File Ports::.  Binary ports might be initialized to the ISO-8859-1
encoding in which each codepoint between 0 and 255 corresponds to a byte
with that value.

   Currently, the ports only work with _non-modal_ encodings.  Most
encodings are non-modal, meaning that the conversion of bytes to a
string doesn’t depend on its context: the same byte sequence will always
return the same string.  A couple of modal encodings are in common use,
like ISO-2022-JP and ISO-2022-KR, and they are not yet supported.

   Each port also has an associated conversion strategy, which
determines what to do when a Guile character can’t be converted to the
port’s encoded character representation for output.  There are three
possible strategies: to raise an error, to replace the character with a
hex escape, or to replace the character with a substitute character.
Port conversion strategies are also used when decoding characters from
an input port.

 -- Scheme Procedure: port-conversion-strategy port
 -- C Function: scm_port_conversion_strategy (port)
     Returns the behavior of the port when outputting a character that
     is not representable in the port’s current encoding.

     If PORT is ‘#f’, then the current default behavior will be
     returned.  New ports will have this default behavior when they are
     created.

 -- Scheme Procedure: set-port-conversion-strategy! port sym
 -- C Function: scm_set_port_conversion_strategy_x (port, sym)
     Sets the behavior of Guile when outputting a character that is not
     representable in the port’s current encoding, or when Guile
     encounters a decoding error when trying to read a character.  SYM
     can be either ‘error’, ‘substitute’, or ‘escape’.

     If PORT is an open port, the conversion error behavior is set for
     that port.  If it is ‘#f’, it is set as the default behavior for
     any future ports that get created in this thread.

   As with port encodings, there is a fluid which determines the initial
conversion strategy for a port.

 -- Scheme Variable: %default-port-conversion-strategy
     The fluid that defines the conversion strategy for newly created
     ports, and also for other conversion routines such as
     ‘scm_to_stringn’, ‘scm_from_stringn’, ‘string->pointer’, and
     ‘pointer->string’.

     Its value must be one of the symbols described above, with the same
     semantics: ‘error’, ‘substitute’, or ‘escape’.

     When Guile starts, its value is ‘substitute’.

     Note that ‘(set-port-conversion-strategy! #f SYM)’ is equivalent to
     ‘(fluid-set! %default-port-conversion-strategy SYM)’.

   As mentioned above, for an output port there are three possible port
conversion strategies.  The ‘error’ strategy will throw an error when a
nonconvertible character is encountered.  The ‘substitute’ strategy will
replace nonconvertible characters with a question mark (‘?’).  Finally
the ‘escape’ strategy will print nonconvertible characters as a hex
escape, using the escaping that is recognized by Guile’s string syntax.
Note that if the port’s encoding is a Unicode encoding, like ‘UTF-8’,
then encoding errors are impossible.

   For an input port, the ‘error’ strategy will cause Guile to throw an
error if it encounters an invalid encoding, such as might happen if you
tried to read ‘ISO-8859-1’ as ‘UTF-8’.  The error is thrown before
advancing the read position.  The ‘substitute’ strategy will replace the
bad bytes with a U+FFFD replacement character, in accordance with
Unicode recommendations.  When reading from an input port, the ‘escape’
strategy is treated as if it were ‘error’.


File: guile.info,  Node: Textual I/O,  Next: Simple Output,  Prev: Encoding,  Up: Input and Output

6.12.4 Textual I/O
------------------

This section describes Guile’s core textual I/O operations on characters
and strings.  *Note Binary I/O::, for input and output of bytes and
bytevectors.  *Note Encoding::, for more on how characters relate to
bytes.  To read general S-expressions from ports, *Note Scheme Read::.
*Note Scheme Write::, for interfaces that write generic Scheme datums.

   To use these routines, first include the textual I/O module:

     (use-modules (ice-9 textual-ports))

   Note that although this module’s name suggests that textual ports are
some different kind of port, that’s not the case: all ports in Guile are
both binary and textual ports.

 -- Scheme Procedure: get-char input-port
     Reads from INPUT-PORT, blocking as necessary, until a complete
     character is available from INPUT-PORT, or until an end of file is
     reached.

     If a complete character is available before the next end of file,
     ‘get-char’ returns that character and updates the input port to
     point past the character.  If an end of file is reached before any
     character is read, ‘get-char’ returns the end-of-file object.

 -- Scheme Procedure: lookahead-char input-port
     The ‘lookahead-char’ procedure is like ‘get-char’, but it does not
     update INPUT-PORT to point past the character.

   In the same way that it’s possible to "unget" a byte or bytes, it’s
possible to "unget" the bytes corresponding to an encoded character.

 -- Scheme Procedure: unget-char port char
     Place character CHAR in PORT so that it will be read by the next
     read operation.  If called multiple times, the unread characters
     will be read again in last-in first-out order.

 -- Scheme Procedure: unget-string port str
     Place the string STR in PORT so that its characters will be read
     from left-to-right as the next characters from PORT during
     subsequent read operations.  If called multiple times, the unread
     characters will be read again in last-in first-out order.

   Reading in a character at a time can be inefficient.  If it’s
possible to perform I/O over multiple characters at a time, via strings,
that might be faster.

 -- Scheme Procedure: get-string-n input-port count
     The ‘get-string-n’ procedure reads from INPUT-PORT, blocking as
     necessary, until COUNT characters are available, or until an end of
     file is reached.  COUNT must be an exact, non-negative integer,
     representing the number of characters to be read.

     If COUNT characters are available before end of file,
     ‘get-string-n’ returns a string consisting of those COUNT
     characters.  If fewer characters are available before an end of
     file, but one or more characters can be read, ‘get-string-n’
     returns a string containing those characters.  In either case, the
     input port is updated to point just past the characters read.  If
     no characters can be read before an end of file, the end-of-file
     object is returned.

 -- Scheme Procedure: get-string-n! input-port string start count
     The ‘get-string-n!’ procedure reads from INPUT-PORT in the same
     manner as ‘get-string-n’.  START and COUNT must be exact,
     non-negative integer objects, with COUNT representing the number of
     characters to be read.  STRING must be a string with at least
     $START + COUNT$ characters.

     If COUNT characters are available before an end of file, they are
     written into STRING starting at index START, and COUNT is returned.
     If fewer characters are available before an end of file, but one or
     more can be read, those characters are written into STRING starting
     at index START and the number of characters actually read is
     returned as an exact integer object.  If no characters can be read
     before an end of file, the end-of-file object is returned.

 -- Scheme Procedure: get-string-all input-port
     Reads from INPUT-PORT until an end of file, decoding characters in
     the same manner as ‘get-string-n’ and ‘get-string-n!’.

     If characters are available before the end of file, a string
     containing all the characters decoded from that data are returned.
     If no character precedes the end of file, the end-of-file object is
     returned.

 -- Scheme Procedure: get-line input-port
     Reads from INPUT-PORT up to and including the linefeed character or
     end of file, decoding characters in the same manner as
     ‘get-string-n’ and ‘get-string-n!’.

     If a linefeed character is read, a string containing all of the
     text up to (but not including) the linefeed character is returned,
     and the port is updated to point just past the linefeed character.
     If an end of file is encountered before any linefeed character is
     read, but some characters have been read and decoded as characters,
     a string containing those characters is returned.  If an end of
     file is encountered before any characters are read, the end-of-file
     object is returned.

   Finally, there are just two core procedures to write characters to a
port.

 -- Scheme Procedure: put-char port char
     Writes CHAR to the port.  The ‘put-char’ procedure returns an
     unspecified value.

 -- Scheme Procedure: put-string port string
 -- Scheme Procedure: put-string port string start
 -- Scheme Procedure: put-string port string start count
     Write the COUNT characters of STRING starting at index START to the
     port.

     START and COUNT must be non-negative exact integer objects.  STRING
     must have a length of at least START + COUNT.  START defaults to 0.
     COUNT defaults to ‘(string-length STRING)’ - START$.

     Calling ‘put-string’ is equivalent in all respects to calling
     ‘put-char’ on the relevant sequence of characters, except that it
     will attempt to write multiple characters to the port at a time,
     even if the port is unbuffered.

     The ‘put-string’ procedure returns an unspecified value.

   Textual ports have a textual position associated with them: a line
and a column.  Reading in characters or writing them out advances the
line and the column appropriately.

 -- Scheme Procedure: port-column port
 -- Scheme Procedure: port-line port
 -- C Function: scm_port_column (port)
 -- C Function: scm_port_line (port)
     Return the current column number or line number of PORT.

   Port lines and positions are represented as 0-origin integers, which
is to say that the first character of the first line is line 0, column
0.  However, when you display a line number, for example in an error
message, we recommend you add 1 to get 1-origin integers.  This is
because lines numbers traditionally start with 1, and that is what
non-programmers will find most natural.

 -- Scheme Procedure: set-port-column! port column
 -- Scheme Procedure: set-port-line! port line
 -- C Function: scm_set_port_column_x (port, column)
 -- C Function: scm_set_port_line_x (port, line)
     Set the current column or line number of PORT.


File: guile.info,  Node: Simple Output,  Next: Buffering,  Prev: Textual I/O,  Up: Input and Output

6.12.5 Simple Textual Output
----------------------------

Guile exports a simple formatted output function, ‘simple-format’.  For
a more capable formatted output facility, *Note Formatted Output::.

 -- Scheme Procedure: simple-format destination message . args
 -- C Function: scm_simple_format (destination, message, args)
     Write MESSAGE to DESTINATION, defaulting to the current output
     port.  MESSAGE can contain ‘~A’ and ‘~S’ escapes.  When printed,
     the escapes are replaced with corresponding members of ARGS: ‘~A’
     formats using ‘display’ and ‘~S’ formats using ‘write’.  If
     DESTINATION is ‘#t’, then use the current output port, if
     DESTINATION is ‘#f’, then return a string containing the formatted
     text.  Does not add a trailing newline.

   Somewhat confusingly, Guile binds the ‘format’ identifier to
‘simple-format’ at startup.  Once ‘(ice-9 format)’ loads, it actually
replaces the core ‘format’ binding, so depending on whether you or a
module you use has loaded ‘(ice-9 format)’, you may be using the simple
or the more capable version.


File: guile.info,  Node: Buffering,  Next: Random Access,  Prev: Simple Output,  Up: Input and Output

6.12.6 Buffering
----------------

Every port has associated input and output buffers.  You can think of
ports as being backed by some mutable store, and that store might be far
away.  For example, ports backed by file descriptors have to go all the
way to the kernel to read and write their data.  To avoid this
round-trip cost, Guile usually reads in data from the mutable store in
chunks, and then services small requests like ‘get-char’ out of that
intermediate buffer.  Similarly, small writes like ‘write-char’ first go
to a buffer, and are sent to the store when the buffer is full (or when
port is flushed).  Buffered ports speed up your program by reducing the
number of round-trips to the mutable store, and they do so in a way that
is mostly transparent to the user.

   There are two major ways, however, in which buffering affects program
semantics.  Building correct, performant programs requires understanding
these situations.

   The first case is in random-access read/write ports (*note Random
Access::).  These ports, usually backed by a file, logically operate
over the same mutable store when both reading and writing.  So, if you
read a character, causing the buffer to fill, then write a character,
the bytes you filled in your read buffer are now invalid.  Every time
you switch between reading and writing, Guile has to flush any pending
buffer.  If this happens frequently, the cost can be high.  In that case
you should reduce the amount that you buffer, in both directions.
Similarly, Guile has to flush buffers before seeking.  None of these
considerations apply to sockets, which don’t logically read from and
write to the same mutable store, and are not seekable.  Note also that
sockets are unbuffered by default.  *Note Network Sockets and
Communication::.

   The second case is the more pernicious one.  If you write data to a
buffered port, it probably doesn’t go out to the mutable store directly.
(This “probably” introduces some indeterminism in your program: what
goes to the store, and when, depends on how full the buffer is.  It is
something that the user needs to explicitly be aware of.)  The data is
written to the store later – when the buffer fills up due to another
write, or when ‘force-output’ is called, or when ‘close-port’ is called,
or when the program exits, or even when the garbage collector runs.  The
salient point is, _the errors are signaled then too_.  Buffered writes
defer error detection (and defer the side effects to the mutable store),
perhaps indefinitely if the port type does not need to be closed at GC.

   One common heuristic that works well for textual ports is to flush
output when a newline (‘\n’) is written.  This “line buffering” mode is
on by default for TTY ports.  Most other ports are “block buffered”,
meaning that once the output buffer reaches the block size, which
depends on the port and its configuration, the output is flushed as a
block, without regard to what is in the block.  Likewise reads are read
in at the block size, though if there are fewer bytes available to read,
the buffer may not be entirely filled.

   Note that binary reads or writes that are larger than the buffer size
go directly to the mutable store without passing through the buffers.
If your access pattern involves many big reads or writes, buffering
might not matter so much to you.

   To control the buffering behavior of a port, use ‘setvbuf’.

 -- Scheme Procedure: setvbuf port mode [size]
 -- C Function: scm_setvbuf (port, mode, size)
     Set the buffering mode for PORT.  MODE can be one of the following
     symbols:

     ‘none’
          non-buffered
     ‘line’
          line buffered
     ‘block’
          block buffered, using a newly allocated buffer of SIZE bytes.
          If SIZE is omitted, a default size will be used.

   Another way to set the buffering, for file ports, is to open the file
with ‘0’ or ‘l’ as part of the mode string, for unbuffered or
line-buffered ports, respectively.  *Note File Ports::, for more.

   Any buffered output data will be written out when the port is closed.
To make sure to flush it at specific points in your program, use
‘force-output’.

 -- Scheme Procedure: force-output [port]
 -- C Function: scm_force_output (port)
     Flush the specified output port, or the current output port if PORT
     is omitted.  The current output buffer contents, if any, are passed
     to the underlying port implementation.

     The return value is unspecified.

 -- Scheme Procedure: flush-all-ports
 -- C Function: scm_flush_all_ports ()
     Equivalent to calling ‘force-output’ on all open output ports.  The
     return value is unspecified.

   Similarly, sometimes you might want to switch from using Guile’s
ports to working directly on file descriptors.  In that case, for input
ports use ‘drain-input’ to get any buffered input from that port.

 -- Scheme Procedure: drain-input port
 -- C Function: scm_drain_input (port)
     This procedure clears a port’s input buffers, similar to the way
     that force-output clears the output buffer.  The contents of the
     buffers are returned as a single string, e.g.,

          (define p (open-input-file ...))
          (drain-input p) => empty string, nothing buffered yet.
          (unread-char (read-char p) p)
          (drain-input p) => initial chars from p, up to the buffer size.

   All of these considerations are very similar to those of streams in
the C library, although Guile’s ports are not built on top of C streams.
Still, it is useful to read what other systems do.  *Note
(libc)Streams::, for more discussion on C streams.


File: guile.info,  Node: Random Access,  Next: Line/Delimited,  Prev: Buffering,  Up: Input and Output

6.12.7 Random Access
--------------------

 -- Scheme Procedure: seek fd_port offset whence
 -- C Function: scm_seek (fd_port, offset, whence)
     Sets the current position of FD_PORT to the integer OFFSET.  For a
     file port, OFFSET is expressed as a number of bytes; for other
     types of ports, such as string ports, OFFSET is an abstract
     representation of the position within the port’s data, not
     necessarily expressed as a number of bytes.  OFFSET is interpreted
     according to the value of WHENCE.

     One of the following variables should be supplied for WHENCE:
      -- Variable: SEEK_SET
          Seek from the beginning of the file.
      -- Variable: SEEK_CUR
          Seek from the current position.
      -- Variable: SEEK_END
          Seek from the end of the file.

     On systems that support it, such as GNU/Linux, the following
     constants can be used for WHENCE to navigate “holes” in sparse
     files:
      -- Variable: SEEK_DATA
          Seek to the next location in the file greater than or equal to
          OFFSET containing data.  If OFFSET points to data, then the
          file offset is set to OFFSET.
      -- Variable: SEEK_HOLE
          Seek to the next hole in the file greater than or equal to the
          OFFSET.  If OFFSET points into the middle of a hole, then the
          file offset is set to OFFSET.  If there is no hole past
          OFFSET, then the file offset is adjusted to the end of the
          file—i.e., there is an implicit hole at the end of any file.

     If FD_PORT is a file descriptor, the underlying system call is
     ‘lseek’ (*note (libc)File Position Primitive::).  PORT may be a
     string port.

     The value returned is the new position in FD_PORT.  This means that
     the current position of a port can be obtained using:
          (seek port 0 SEEK_CUR)

 -- Scheme Procedure: ftell fd_port
 -- C Function: scm_ftell (fd_port)
     Return an integer representing the current position of FD_PORT,
     measured from the beginning.  Equivalent to:

          (seek port 0 SEEK_CUR)

 -- Scheme Procedure: truncate-file file [length]
 -- C Function: scm_truncate_file (file, length)
     Truncate FILE to LENGTH bytes.  FILE can be a filename string, a
     port object, or an integer file descriptor.  The return value is
     unspecified.

     For a port or file descriptor LENGTH can be omitted, in which case
     the file is truncated at the current position (per ‘ftell’ above).

     On most systems a file can be extended by giving a length greater
     than the current size, but this is not mandatory in the POSIX
     standard.


File: guile.info,  Node: Line/Delimited,  Next: Default Ports,  Prev: Random Access,  Up: Input and Output

6.12.8 Line Oriented and Delimited Text
---------------------------------------

The delimited-I/O module can be accessed with:

     (use-modules (ice-9 rdelim))

   It can be used to read or write lines of text, or read text delimited
by a specified set of characters.

 -- Scheme Procedure: read-line [port] [handle-delim]
     Return a line of text from PORT if specified, otherwise from the
     value returned by ‘(current-input-port)’.  Under Unix, a line of
     text is terminated by the first end-of-line character or by
     end-of-file.

     If HANDLE-DELIM is specified, it should be one of the following
     symbols:
     ‘trim’
          Discard the terminating delimiter.  This is the default, but
          it will be impossible to tell whether the read terminated with
          a delimiter or end-of-file.
     ‘concat’
          Append the terminating delimiter (if any) to the returned
          string.
     ‘peek’
          Push the terminating delimiter (if any) back on to the port.
     ‘split’
          Return a pair containing the string read from the port and the
          terminating delimiter or end-of-file object.

 -- Scheme Procedure: read-line! buf [port]
     Read a line of text into the supplied string BUF and return the
     number of characters added to BUF.  If BUF is filled, then ‘#f’ is
     returned.  Read from PORT if specified, otherwise from the value
     returned by ‘(current-input-port)’.

 -- Scheme Procedure: read-delimited delims [port] [handle-delim]
     Read text until one of the characters in the string DELIMS is found
     or end-of-file is reached.  Read from PORT if supplied, otherwise
     from the value returned by ‘(current-input-port)’.  HANDLE-DELIM
     takes the same values as described for ‘read-line’.

 -- Scheme Procedure: read-delimited! delims buf [port] [handle-delim]
          [start] [end]
     Read text into the supplied string BUF.

     If a delimiter was found, return the number of characters written,
     except if HANDLE-DELIM is ‘split’, in which case the return value
     is a pair, as noted above.

     As a special case, if PORT was already at end-of-stream, the EOF
     object is returned.  Also, if no characters were written because
     the buffer was full, ‘#f’ is returned.

     It’s something of a wacky interface, to be honest.

 -- Scheme Procedure: %read-delimited! delims str gobble [port [start
          [end]]]
 -- C Function: scm_read_delimited_x (delims, str, gobble, port, start,
          end)
     Read characters from PORT into STR until one of the characters in
     the DELIMS string is encountered.  If GOBBLE is true, discard the
     delimiter character; otherwise, leave it in the input stream for
     the next read.  If PORT is not specified, use the value of
     ‘(current-input-port)’.  If START or END are specified, store data
     only into the substring of STR bounded by START and END (which
     default to the beginning and end of the string, respectively).

     Return a pair consisting of the delimiter that terminated the
     string and the number of characters read.  If reading stopped at
     the end of file, the delimiter returned is the EOF-OBJECT; if the
     string was filled without encountering a delimiter, this value is
     ‘#f’.

 -- Scheme Procedure: %read-line [port]
 -- C Function: scm_read_line (port)
     Read a newline-terminated line from PORT, allocating storage as
     necessary.  The newline terminator (if any) is removed from the
     string, and a pair consisting of the line and its delimiter is
     returned.  The delimiter may be either a newline or the EOF-OBJECT;
     if ‘%read-line’ is called at the end of file, it returns the pair
     ‘(#<eof> . #<eof>)’.

 -- Scheme Procedure: write-line obj [port]
 -- C Function: scm_write_line (obj, port)
     Display OBJ and a newline character to PORT.  If PORT is not
     specified, ‘(current-output-port)’ is used.  This procedure is
     equivalent to:
          (display obj [port])
          (newline [port])

 -- Scheme Procedure: for-rdelim-from-port port proc rdelim-proc
          [#:stop-pred=eof-object?]
     For every unit provided by ‘(rdelim-proc port)’, provide this
     unit(rdelim) to PROC to be processed.  This will continue
     throughout PORT until STOP-PRED returns ‘#t’.  STOP-PRED is
     ‘eof-object?’ by default.  RDELIM-PROC has to advance through PORT
     with every call made to it.

 -- Scheme Procedure: for-delimited-from-port port proc
          [#:delims=''\n''] [#:handle-delim='trim]
     Call PROC for every line delimited by DELIMS from PORT.

 -- Scheme Procedure: for-line-in-file file proc [#:encoding=#f]
          [#:guess-encoding=#f]
     Call PROC for every line in FILE.  FILE must be a filename string.

     The line provided to PROC is guaranteed to be a string.


File: guile.info,  Node: Default Ports,  Next: Port Types,  Prev: Line/Delimited,  Up: Input and Output

6.12.9 Default Ports for Input, Output and Errors
-------------------------------------------------

 -- Scheme Procedure: current-input-port
 -- C Function: scm_current_input_port ()
     Return the current input port.  This is the default port used by
     many input procedures.

     Initially this is the “standard input” in Unix and C terminology.
     When the standard input is a TTY the port is unbuffered, otherwise
     it’s fully buffered.

     Unbuffered input is good if an application runs an interactive
     subprocess, since any type-ahead input won’t go into Guile’s buffer
     and be unavailable to the subprocess.

     Note that Guile buffering is completely separate from the TTY “line
     discipline”.  In the usual cooked mode on a TTY Guile only sees a
     line of input once the user presses <Return>.

 -- Scheme Procedure: current-output-port
 -- C Function: scm_current_output_port ()
     Return the current output port.  This is the default port used by
     many output procedures.

     Initially this is the “standard output” in Unix and C terminology.
     When the standard output is a TTY this port is unbuffered,
     otherwise it’s fully buffered.

     Unbuffered output to a TTY is good for ensuring progress output or
     a prompt is seen.  But an application which always prints whole
     lines could change to line buffered, or an application with a lot
     of output could go fully buffered and perhaps make explicit
     ‘force-output’ calls (*note Buffering::) at selected points.

 -- Scheme Procedure: current-error-port
 -- C Function: scm_current_error_port ()
     Return the port to which errors and warnings should be sent.

     Initially this is the “standard error” in Unix and C terminology.
     When the standard error is a TTY this port is unbuffered, otherwise
     it’s fully buffered.

 -- Scheme Procedure: set-current-input-port port
 -- Scheme Procedure: set-current-output-port port
 -- Scheme Procedure: set-current-error-port port
 -- C Function: scm_set_current_input_port (port)
 -- C Function: scm_set_current_output_port (port)
 -- C Function: scm_set_current_error_port (port)
     Change the ports returned by ‘current-input-port’,
     ‘current-output-port’ and ‘current-error-port’, respectively, so
     that they use the supplied PORT for input or output.

 -- Scheme Procedure: with-input-from-port port thunk
 -- Scheme Procedure: with-output-to-port port thunk
 -- Scheme Procedure: with-error-to-port port thunk
     Call THUNK in a dynamic environment in which ‘current-input-port’,
     ‘current-output-port’ or ‘current-error-port’ is rebound to the
     given PORT.

 -- C Function: void scm_dynwind_current_input_port (SCM port)
 -- C Function: void scm_dynwind_current_output_port (SCM port)
 -- C Function: void scm_dynwind_current_error_port (SCM port)
     These functions must be used inside a pair of calls to
     ‘scm_dynwind_begin’ and ‘scm_dynwind_end’ (*note Dynamic Wind::).
     During the dynwind context, the indicated port is set to PORT.

     More precisely, the current port is swapped with a ‘backup’ value
     whenever the dynwind context is entered or left.  The backup value
     is initialized with the PORT argument.


File: guile.info,  Node: Port Types,  Next: Venerable Port Interfaces,  Prev: Default Ports,  Up: Input and Output

6.12.10 Types of Port
---------------------

* Menu:

* File Ports:: Ports on an operating system file.
* Bytevector Ports:: Ports on a bytevector.
* String Ports:: Ports on a Scheme string.
* Custom Ports:: Ports whose implementation you control.
* Soft Ports:: A Guile-specific version of custom ports.
* Void Ports:: Ports on nothing at all.
* Low-Level Custom Ports:: Implementing new kinds of port.
* Low-Level Custom Ports in C:: A C counterpart to make-custom-port.


File: guile.info,  Node: File Ports,  Next: Bytevector Ports,  Up: Port Types

6.12.10.1 File Ports
....................

The following procedures are used to open file ports.  See also *note
open: Ports and File Descriptors, for an interface to the Unix ‘open’
system call.

   All file access uses the “LFS” large file support functions when
available, so files bigger than 2 gibibytes (2^31 bytes) can be read and
written on a 32-bit system.

   Most systems have limits on how many files can be open, so it’s
strongly recommended that file ports be closed explicitly when no longer
required (*note Ports::).

 -- Scheme Procedure: open-file filename mode [#:guess-encoding=#f]
          [#:encoding=#f]
 -- C Function: scm_open_file_with_encoding (filename, mode,
          guess_encoding, encoding)
 -- C Function: scm_open_file (filename, mode)
     Open the file whose name is FILENAME, and return a port
     representing that file.  The attributes of the port are determined
     by the MODE string.  The way in which this is interpreted is
     similar to C stdio.  The first character must be one of the
     following:

     ‘r’
          Open an existing file for input.
     ‘w’
          Open a file for output, creating it if it doesn’t already
          exist or removing its contents if it does.
     ‘a’
          Open a file for output, creating it if it doesn’t already
          exist.  All writes to the port will go to the end of the file.
          The "append mode" can be turned off while the port is in use
          *note fcntl: Ports and File Descriptors.

     The following additional characters can be appended:

     ‘b’
          Open the underlying file in binary mode, if supported by the
          system.  Also, open the file using the binary-compatible
          character encoding "ISO-8859-1", ignoring the default port
          encoding.
     ‘+’
          Open the port for both input and output.  E.g., ‘r+’: open an
          existing file for both input and output.
     ‘e’
          Mark the underlying file descriptor as close-on-exec, as per
          the ‘O_CLOEXEC’ flag.
     ‘0’
          Create an "unbuffered" port.  In this case input and output
          operations are passed directly to the underlying port
          implementation without additional buffering.  This is likely
          to slow down I/O operations.  The buffering mode can be
          changed while a port is in use (*note Buffering::).
     ‘l’
          Add line-buffering to the port.  The port output buffer will
          be automatically flushed whenever a newline character is
          written.
     ‘b’
          Use binary mode, ensuring that each byte in the file will be
          read as one Scheme character.

          To provide this property, the file will be opened with the
          8-bit character encoding "ISO-8859-1", ignoring the default
          port encoding.  *Note Ports::, for more information on port
          encodings.

          Note that while it is possible to read and write binary data
          as characters or strings, it is usually better to treat bytes
          as octets, and byte sequences as bytevectors.  *Note Binary
          I/O::, for more.

          This option had another historical meaning, for DOS
          compatibility: in the default (textual) mode, DOS reads a
          CR-LF sequence as one LF byte.  The ‘b’ flag prevents this
          from happening, adding ‘O_BINARY’ to the underlying ‘open’
          call.  Still, the flag is generally useful because of its port
          encoding ramifications.

     Unless binary mode is requested, the character encoding of the new
     port is determined as follows: First, if GUESS-ENCODING is true,
     the ‘file-encoding’ procedure is used to guess the encoding of the
     file (*note Character Encoding of Source Files::).  If
     GUESS-ENCODING is false or if ‘file-encoding’ fails, ENCODING is
     used unless it is also false.  As a last resort, the default port
     encoding is used.  *Note Ports::, for more information on port
     encodings.  It is an error to pass a non-false GUESS-ENCODING or
     ENCODING if binary mode is requested.

     If a file cannot be opened with the access requested, ‘open-file’
     throws an exception.

 -- Scheme Procedure: open-input-file filename [#:guess-encoding=#f]
          [#:encoding=#f] [#:binary=#f]

     Open FILENAME for input.  If BINARY is true, open the port in
     binary mode, otherwise use text mode.  ENCODING and GUESS-ENCODING
     determine the character encoding as described above for
     ‘open-file’.  Equivalent to
          (open-file FILENAME
                     (if BINARY "rb" "r")
                     #:guess-encoding GUESS-ENCODING
                     #:encoding ENCODING)

 -- Scheme Procedure: open-output-file filename [#:encoding=#f]
          [#:binary=#f]

     Open FILENAME for output.  If BINARY is true, open the port in
     binary mode, otherwise use text mode.  ENCODING specifies the
     character encoding as described above for ‘open-file’.  Equivalent
     to
          (open-file FILENAME
                     (if BINARY "wb" "w")
                     #:encoding ENCODING)

 -- Scheme Procedure: call-with-input-file filename proc
          [#:guess-encoding=#f] [#:encoding=#f] [#:binary=#f]
 -- Scheme Procedure: call-with-output-file filename proc
          [#:encoding=#f] [#:binary=#f]
     Open FILENAME for input or output, and call ‘(PROC port)’ with the
     resulting port.  Return the value returned by PROC.  FILENAME is
     opened as per ‘open-input-file’ or ‘open-output-file’ respectively,
     and an error is signaled if it cannot be opened.

     When PROC returns, the port is closed.  If PROC does not return
     (e.g. if it throws an error), then the port might not be closed
     automatically, though it will be garbage collected in the usual way
     if not otherwise referenced.

 -- Scheme Procedure: with-input-from-file filename thunk
          [#:guess-encoding=#f] [#:encoding=#f] [#:binary=#f]
 -- Scheme Procedure: with-output-to-file filename thunk [#:encoding=#f]
          [#:binary=#f]
 -- Scheme Procedure: with-error-to-file filename thunk [#:encoding=#f]
          [#:binary=#f]
     Open FILENAME and call ‘(THUNK)’ with the new port setup as
     respectively the ‘current-input-port’, ‘current-output-port’, or
     ‘current-error-port’.  Return the value returned by THUNK.
     FILENAME is opened as per ‘open-input-file’ or ‘open-output-file’
     respectively, and an error is signaled if it cannot be opened.

     When THUNK returns, the port is closed and the previous setting of
     the respective current port is restored.

     The current port setting is managed with ‘dynamic-wind’, so the
     previous value is restored no matter how THUNK exits (eg. an
     exception), and if THUNK is re-entered (via a captured
     continuation) then it’s set again to the FILENAME port.

     The port is closed when THUNK returns normally, but not when exited
     via an exception or new continuation.  This ensures it’s still
     ready for use if THUNK is re-entered by a captured continuation.
     Of course the port is always garbage collected and closed in the
     usual way when no longer referenced anywhere.

 -- Scheme Procedure: port-mode port
 -- C Function: scm_port_mode (port)
     Return the port modes associated with the open port PORT.  These
     will not necessarily be identical to the modes used when the port
     was opened, since modes such as "append" which are used only during
     port creation are not retained.

 -- Scheme Procedure: port-filename port
 -- C Function: scm_port_filename (port)
     Return the filename associated with PORT, or ‘#f’ if no filename is
     associated with the port.

     PORT must be open; ‘port-filename’ cannot be used once the port is
     closed.

 -- Scheme Procedure: set-port-filename! port filename
 -- C Function: scm_set_port_filename_x (port, filename)
     Change the filename associated with PORT, using the current input
     port if none is specified.  Note that this does not change the
     port’s source of data, but only the value that is returned by
     ‘port-filename’ and reported in diagnostic output.

 -- Scheme Procedure: file-port? obj
 -- C Function: scm_file_port_p (obj)
     Determine whether OBJ is a port that is related to a file.


File: guile.info,  Node: Bytevector Ports,  Next: String Ports,  Prev: File Ports,  Up: Port Types

6.12.10.2 Bytevector Ports
..........................

 -- Scheme Procedure: open-bytevector-input-port bv [transcoder]
 -- C Function: scm_open_bytevector_input_port (bv, transcoder)
     Return an input port whose contents are drawn from bytevector BV
     (*note Bytevectors::).

     The TRANSCODER argument is currently not supported.

 -- Scheme Procedure: open-bytevector-output-port [transcoder]
 -- C Function: scm_open_bytevector_output_port (transcoder)
     Return two values: a binary output port and a procedure.  The
     latter should be called with zero arguments to obtain a bytevector
     containing the data accumulated by the port, as illustrated below.

          (call-with-values
            (lambda ()
              (open-bytevector-output-port))
            (lambda (port get-bytevector)
              (display "hello" port)
              (get-bytevector)))

          ⇒ #vu8(104 101 108 108 111)

     The TRANSCODER argument is currently not supported.

 -- Scheme Procedure: call-with-output-bytevector proc
     Call the one-argument procedure PROC with a newly created
     bytevector output port.  When the function returns, the bytevector
     composed of the characters written into the port is returned.  PROC
     should not close the port.

 -- Scheme Procedure: call-with-input-bytevector bytevector proc
     Call the one-argument procedure PROC with a newly created input
     port from which BYTEVECTOR’s contents may be read.  The values
     yielded by the PROC is returned.


File: guile.info,  Node: String Ports,  Next: Custom Ports,  Prev: Bytevector Ports,  Up: Port Types

6.12.10.3 String Ports
......................

 -- Scheme Procedure: call-with-output-string proc
 -- C Function: scm_call_with_output_string (proc)
     Calls the one-argument procedure PROC with a newly created output
     port.  When the function returns, the string composed of the
     characters written into the port is returned.  PROC should not
     close the port.

 -- Scheme Procedure: call-with-input-string string proc
 -- C Function: scm_call_with_input_string (string, proc)
     Calls the one-argument procedure PROC with a newly created input
     port from which STRING’s contents may be read.  The value yielded
     by the PROC is returned.

 -- Scheme Procedure: with-output-to-string thunk
     Calls the zero-argument procedure THUNK with the current output
     port set temporarily to a new string port.  It returns a string
     composed of the characters written to the current output.

 -- Scheme Procedure: with-input-from-string string thunk
     Calls the zero-argument procedure THUNK with the current input port
     set temporarily to a string port opened on the specified STRING.
     The value yielded by THUNK is returned.

 -- Scheme Procedure: open-input-string str
 -- C Function: scm_open_input_string (str)
     Take a string and return an input port that delivers characters
     from the string.  The port can be closed by ‘close-input-port’,
     though its storage will be reclaimed by the garbage collector if it
     becomes inaccessible.

 -- Scheme Procedure: open-output-string
 -- C Function: scm_open_output_string ()
     Return an output port that will accumulate characters for retrieval
     by ‘get-output-string’.  The port can be closed by the procedure
     ‘close-output-port’, though its storage will be reclaimed by the
     garbage collector if it becomes inaccessible.

 -- Scheme Procedure: get-output-string port
 -- C Function: scm_get_output_string (port)
     Given an output port created by ‘open-output-string’, return a
     string consisting of the characters that have been output to the
     port so far.

     ‘get-output-string’ must be used before closing PORT, once closed
     the string cannot be obtained.

   With string ports, the port-encoding is treated differently than
other types of ports.  When string ports are created, they do not
inherit a character encoding from the current locale.  They are given a
default locale that allows them to handle all valid string characters.
Typically one should not modify a string port’s character encoding away
from its default.  *Note Encoding::.


File: guile.info,  Node: Custom Ports,  Next: Soft Ports,  Prev: String Ports,  Up: Port Types

6.12.10.4 Custom Ports
......................

Custom ports allow the user to provide input and handle output via
user-supplied procedures.  The most basic of these operates on the level
of bytes, calling user-supplied functions to supply bytes for input and
accept bytes for output.  In Guile, textual ports are built on top of
binary ports, encoding and decoding their codepoint sequences from the
bytes; the higher-level textual layer for custom ports allows users to
deal in characters instead of bytes.

   Before using these procedures, import the appropriate module:

     (use-modules (ice-9 binary-ports))
     (use-modules (ice-9 textual-ports))

 -- Scheme Procedure: make-custom-binary-input-port id read!
          get-position set-position! close
     Return a new custom binary input port named ID (a string) whose
     input is drained by invoking READ! and passing it a bytevector, an
     index where bytes should be written, and the number of bytes to
     read.  The ‘read!’ procedure must return an integer indicating the
     number of bytes read, or ‘0’ to indicate the end-of-file.

     Optionally, if GET-POSITION is not ‘#f’, it must be a thunk that
     will be called when ‘port-position’ is invoked on the custom binary
     port and should return an integer indicating the position within
     the underlying data stream; if GET-POSITION was not supplied, the
     returned port does not support ‘port-position’.

     Likewise, if SET-POSITION! is not ‘#f’, it should be a one-argument
     procedure.  When ‘set-port-position!’ is invoked on the custom
     binary input port, SET-POSITION! is passed an integer indicating
     the position of the next byte is to read.

     Finally, if CLOSE is not ‘#f’, it must be a thunk.  It is invoked
     when the custom binary input port is closed.

     The returned port is fully buffered by default, but its buffering
     mode can be changed using ‘setvbuf’ (*note Buffering::).

     Using a custom binary input port, the ‘open-bytevector-input-port’
     procedure (*note Bytevector Ports::) could be implemented as
     follows:

          (define (open-bytevector-input-port source)
            (define position 0)
            (define length (bytevector-length source))

            (define (read! bv start count)
              (let ((count (min count (- length position))))
                (bytevector-copy! source position
                                  bv start count)
                (set! position (+ position count))
                count))

            (define (get-position) position)

            (define (set-position! new-position)
              (set! position new-position))

            (make-custom-binary-input-port "the port" read!
                                            get-position set-position!
                                            #f))

          (read (open-bytevector-input-port (string->utf8 "hello")))
          ⇒ hello

 -- Scheme Procedure: make-custom-binary-output-port id write!
          get-position set-position! close
     Return a new custom binary output port named ID (a string) whose
     output is sunk by invoking WRITE! and passing it a bytevector, an
     index where bytes should be read from this bytevector, and the
     number of bytes to be “written”.  The ‘write!’ procedure must
     return an integer indicating the number of bytes actually written;
     when it is passed ‘0’ as the number of bytes to write, it should
     behave as though an end-of-file was sent to the byte sink.

     The other arguments are as for ‘make-custom-binary-input-port’.

 -- Scheme Procedure: make-custom-binary-input/output-port id read!
          write! get-position set-position! close
     Return a new custom binary input/output port named ID (a string).
     The various arguments are the same as for The other arguments are
     as for ‘make-custom-binary-input-port’ and
     ‘make-custom-binary-output-port’.  If buffering is enabled on the
     port, as is the case by default, input will be buffered in both
     directions; *Note Buffering::.  If the SET-POSITION! function is
     provided and not ‘#f’, then the port will also be marked as
     random-access, causing the buffer to be flushed between reads and
     writes.

 -- Scheme Procedure: make-custom-textual-input-port id read!
          get-position set-position! close
 -- Scheme Procedure: make-custom-textual-output-port id write!
          get-position set-position! close
 -- Scheme Procedure: make-custom-textual-input/output-port id read!
          write! get-position set-position! close
     Like their custom binary port counterparts, but for textual ports.
     Concretely this means that instead of being passed a bytevector,
     the READ function is passed a mutable string to fill, and likewise
     for the buffer supplied to WRITE.  Port positions are still
     expressed in bytes, however.

     If string ports were not supplied with Guile, we could implement
     them With custom textual ports:
          (define (open-string-input-port source)
            (define position 0)
            (define length (string-length source))

            (define (read! dst start count)
              (let ((count (min count (- length position))))
                (string-copy! dst start source position (+ position count))
                (set! position (+ position count))
                count))

            (make-custom-textual-input-port "strport" read! #f #f #f))

          (read (open-string-input-port "hello"))


File: guile.info,  Node: Soft Ports,  Next: Void Ports,  Prev: Custom Ports,  Up: Port Types

6.12.10.5 Soft Ports
....................

Soft ports are what Guile had before it had custom binary and textual
ports, and allow for customizable textual input and output.

   We recommend soft ports over R6RS custom textual ports because they
are easier to use while also being more expressive.  R6RS custom textual
ports operate under the principle that a port has a mutable string
buffer, and this is reflected in the ‘read’ and ‘write’ procedures which
take a buffer, offset, and length.  However in Guile as all ports have a
byte buffer rather than some having a string buffer, the R6RS interface
imposes overhead and complexity.

   Additionally, and unlike the R6RS interfaces, ‘make-soft-port’ from
the ‘(ice-9 soft-ports)’ module accepts keyword arguments, allowing for
its functionality to be extended over time.

   If you find yourself needing more power, notably the ability to seek,
probably you want to use low-level custom ports.  *Note Low-Level Custom
Ports::.

     (use-modules (ice-9 soft-ports))

 -- Scheme Procedure: make-soft-port [#:id] [#:read-string]
          [#:write-string] [#:input-waiting?] [#:close] [#:close-on-gc?]
     Return a new port.  If the READ-STRING keyword argument is present,
     the port will be an input port.  If WRITE-STRING is present, the
     port will be an output port.  If both are supplied, the port will
     be open for input and output.

     When the port’s internal buffers are empty, READ-STRING will be
     called with no arguments, and should return a string, or ‘#f’ to
     indicate end-of-stream.  Similarly when a port flushes its write
     buffer, the characters in that buffer will be passed to the
     WRITE-STRING procedure as its single argument.  WRITE-STRING
     returns unspecified values.

     If supplied, INPUT-WAITING? should return ‘#t’ if the soft port has
     input which would be returned directly by READ-STRING.

     If supplied, CLOSE will be called when the port is closed, with no
     arguments.  If CLOSE-ON-GC? is ‘#t’, CLOSE will additionally be
     called when the port becomes unreachable, after flushing any
     pending write buffers.

   With soft ports, the ‘open-string-input-port’ example from the
previous section is more simple:

     (define (open-string-input-port source)
       (define already-read? #f)

       (define (read-string)
         (cond
          (already-read? "")
          (else
           (set! already-read? #t)
           source)))

       (make-soft-port #:id "strport" #:read-string read-string))

   Note that there was an earlier form of ‘make-soft-port’ which was
exposed in Guile’s default environment, and which is still there.  Its
interface is more clumsy and its users historically expect unbuffered
input.  This interface will be deprecated, but we document it here.

 -- Scheme Procedure: deprecated-make-soft-port pv modes
     Return a port capable of receiving or delivering characters as
     specified by the MODES string (*note open-file: File Ports.).  PV
     must be a vector of length 5 or 6.  Its components are as follows:

       0. procedure accepting one character for output
       1. procedure accepting a string for output
       2. thunk for flushing output
       3. thunk for getting one character
       4. thunk for closing port (not by garbage collection)
       5. (if present and not ‘#f’) thunk for computing the number of
          characters that can be read from the port without blocking.

     For an output-only port only elements 0, 1, 2, and 4 need be
     procedures.  For an input-only port only elements 3 and 4 need be
     procedures.  Thunks 2 and 4 can instead be ‘#f’ if there is no
     useful operation for them to perform.

     If thunk 3 returns ‘#f’ or an ‘eof-object’ (*note eof-object?:
     (r5rs)Input.) it indicates that the port has reached end-of-file.
     For example:

          (define stdout (current-output-port))
          (define p (deprecated-make-soft-port
                     (vector
                      (lambda (c) (write c stdout))
                      (lambda (s) (display s stdout))
                      (lambda () (display "." stdout))
                      (lambda () (char-upcase (read-char)))
                      (lambda () (display "@" stdout)))
                     "rw"))

          (write p p) ⇒ #<input-output: soft 8081e20>


File: guile.info,  Node: Void Ports,  Next: Low-Level Custom Ports,  Prev: Soft Ports,  Up: Port Types

6.12.10.6 Void Ports
....................

This kind of port causes any data to be discarded when written to, and
always returns the end-of-file object when read from.

 -- Scheme Procedure: %make-void-port mode
 -- C Function: scm_sys_make_void_port (mode)
     Create and return a new void port.  A void port acts like
     ‘/dev/null’.  The MODE argument specifies the input/output modes
     for this port: see the documentation for ‘open-file’ in *note File
     Ports::.


File: guile.info,  Node: Low-Level Custom Ports,  Next: Low-Level Custom Ports in C,  Prev: Void Ports,  Up: Port Types

6.12.10.7 Low-Level Custom Ports
................................

This section describes how to implement a new kind of port using Guile’s
lowest-level, most primitive interfaces.  First, load the ‘(ice-9
custom-ports)’ module:

     (use-modules (ice-9 custom-ports))

   Then to make a new port, call ‘make-custom-port’:

 -- Scheme Procedure: make-custom-port [#:read] [#:write]
          [#:read-wait-fd] [#:write-wait-fd] [#:input-waiting?] [#:seek]
          [#:random-access?] [#:get-natural-buffer-sizes] [#:id]
          [#:print] [#:close] [#:close-on-gc?] [#:truncate] [#:encoding]
          [#:conversion-strategy]
     Make a new custom port.

     *Note Encoding::, for more on ‘#:encoding’ and
     ‘#:conversion-strategy’.

   A port has a number of associated procedures and properties which
collectively implement its behavior.  Creating a new custom port mostly
involves writing these procedures, which are passed as keyword arguments
to ‘make-custom-port’.

 -- Scheme Port Method: #:read port dst start count
     A port’s ‘#:read’ implementation fills read buffers.  It should
     copy bytes to the supplied bytevector DST, starting at offset START
     and continuing for COUNT bytes, and return the number of bytes that
     were read, or ‘#f’ to indicate that reading any bytes would block.

 -- Scheme Port Method: #:write port src start count
     A port’s ‘#:write’ implementation flushes write buffers to the
     mutable store.  It should write out bytes from the supplied
     bytevector SRC, starting at offset START and continuing for COUNT
     bytes, and return the number of bytes that were written, or ‘#f’ to
     indicate writing any bytes would block.

   If ‘make-custom-port’ is passed a ‘#:read’ argument, the port will be
an input port.  Passing a ‘#:write’ argument will make an output port,
and passing both will make an input-output port.

 -- Scheme Port Method: #:read-wait-fd port
 -- Scheme Port Method: #:write-wait-fd port
     If a port’s ‘#:read’ or ‘#:write’ method returns ‘#f’, that
     indicates that reading or writing would block, and that Guile
     should instead ‘poll’ on the file descriptor returned by the port’s
     ‘#:read-wait-fd’ or ‘#:write-wait-fd’ method, respectively, until
     the operation can complete.  *Note Non-Blocking I/O::, for a more
     in-depth discussion.

     These methods must be implemented if the ‘#:read’ or ‘#:write’
     method can return ‘#f’, and should return a non-negative integer
     file descriptor.  However they may be called explicitly by a user,
     for example to determine if a port may eventually be readable or
     writable.  If there is no associated file descriptor with the port,
     they should return ‘#f’.  The default implementation returns ‘#f’.

 -- Scheme Port Method: #:input-waiting? port
     In rare cases it is useful to be able to know whether data can be
     read from a port.  For example, if the user inputs ‘1 2 3’ at the
     interactive console, after reading and evaluating ‘1’ the console
     shouldn’t then print another prompt before reading and evaluating
     ‘2’ because there is input already waiting.  If the port can look
     ahead, then it should implement the ‘#:input-waiting?’ method,
     which returns ‘#t’ if input is available, or ‘#f’ reading the next
     byte would block.  The default implementation returns ‘#t’.

 -- Scheme Port Method: #:seek port offset whence
     Set or get the current byte position of the port.  Guile will flush
     read and/or write buffers before seeking, as appropriate.  The
     OFFSET and WHENCE parameters are as for the ‘seek’ procedure; *Note
     Random Access::.

     The ‘#:seek’ method returns the byte position after seeking.  To
     query the current position, ‘#:seek’ will be called with an OFFSET
     of 0 and ‘SEEK_CUR’ for WHENCE.  Other values of OFFSET and/or
     WHENCE will actually perform the seek.  The ‘#:seek’ method should
     throw an error if the port is not seekable, which is what the
     default implementation does.

 -- Scheme Port Method: #:truncate port
     Truncate the port data to be specified length.  Guile will flush
     buffers beforehand, as appropriate.  The default implementation
     throws an error, indicating that truncation is not supported for
     this port.

 -- Scheme Port Method: #:random-access? port
     Return ‘#t’ if PORT is open for random access, or ‘#f’ otherwise.

     Seeking on a random-access port with buffered input, or switching
     to writing after reading, will cause the buffered input to be
     discarded and Guile will seek the port back the buffered number of
     bytes.  Likewise seeking on a random-access port with buffered
     output, or switching to reading after writing, will flush pending
     bytes with a call to the ‘write’ procedure.  *Note Buffering::.

     Indicate to Guile that your port needs this behavior by returning
     true from your ‘#:random-access?’ method.  The default
     implementation of this function returns ‘#t’ if the port has a
     ‘#:seek’ implementation.

 -- Scheme Port Method: #:get-natural-buffer-sizes read-buf-size
          write-buf-size
     Guile will internally attach buffers to ports.  An input port
     always has a read buffer, and an output port always has a write
     buffer.  *Note Buffering::.  A port buffer consists of a
     bytevector, along with some cursors into that bytevector denoting
     where to get and put data.

     Port implementations generally don’t have to be concerned with
     buffering: a port’s ‘#:read’ or ‘#:write’ method will receive the
     buffer’s bytevector as an argument, along with an offset and a
     length into that bytevector, and should then either fill or empty
     that bytevector.  However in some cases, port implementations may
     be able to provide an appropriate default buffer size to Guile.
     For example file ports implement ‘#:get-natural-buffer-sizes’ to
     let the operating system inform Guile about the appropriate buffer
     sizes for the particular file opened by the port.

     This method returns two values, corresponding to the natural read
     and write buffer sizes for the ports.  The two parameters
     READ-BUF-SIZE and WRITE-BUF-SIZE are Guile’s guesses for what sizes
     might be good.  A custom ‘#:get-natural-buffer-sizes’ method could
     override Guile’s choices, or just pass them on, as the default
     implementation does.

 -- Scheme Port Method: #:print port out
     Called when the port PORT is written to OUT, e.g.  via ‘(write port
     out)’.

     If ‘#:print’ is not explicitly supplied, the default implementation
     prints something like ‘#<MODE:ID ADDRESS>’, where MODE is either
     ‘input’, ‘output’, or ‘input-output’, ID comes from the ‘#:id’
     keyword argument (defaulting to ‘"custom-port"’), and ADDRESS is a
     unique integer associated with the port.

 -- Scheme Port Method: #:close port
     Called when PORT is closed.  It should release any
     explicitly-managed resources used by the port.

   By default, ports that are garbage collected just go away without
closing or flushing any buffered output.  If your port needs to release
some external resource like a file descriptor, or needs to make sure
that its internal buffers are flushed even if the port is collected
while it was open, then pass ‘#:close-on-gc? #t’ to ‘make-custom-port’.
Note that in that case, the ‘#:close’ method will probably be called on
a separate thread.

   Note that calls to all of these methods can proceed in parallel and
concurrently and from any thread up until the point that the port is
closed.  The call to ‘close’ will happen when no other method is
running, and no method will be called after the ‘close’ method is
called.  If your port implementation needs mutual exclusion to prevent
concurrency, it is responsible for locking appropriately.


File: guile.info,  Node: Low-Level Custom Ports in C,  Prev: Low-Level Custom Ports,  Up: Port Types

6.12.10.8 Low-Level Custom Ports in C
.....................................

The ‘make-custom-port’ procedure described in the previous section has
similar functionality on the C level, though it is organized a bit
differently.

   In C, the mechanism is that one creates a new “port type object”.
The methods are then associated with the port type object instead of the
port itself.  The port type object is an opaque pointer allocated when
defining the port type, which serves as a key into the port API.

   Ports themselves have associated “stream” values.  The stream is a
pointer controlled by the user, which is set when the port is created.
Given a port, the ‘SCM_STREAM’ macro returns its associated stream
value, as a ‘scm_t_bits’.  Note that your port methods are only ever
called with ports of your type, so port methods can safely cast this
value to the expected type.  Contrast this to Scheme, which doesn’t need
access to the stream because the ‘make-custom-port’ methods can be
closures that share port-specific data directly.

   A port type is created by calling ‘scm_make_port_type’.

 -- Function: scm_t_port_type* scm_make_port_type (char *name, size_t
          (*read) (SCM port, SCM dst, size_t start, size_t count),
          size_t (*write) (SCM port, SCM src, size_t start, size_t
          count))
     Define a new port type.  The NAME parameter is like the ‘#:id’
     parameter to ‘make-custom-port’; and READ and WRITE are like
     ‘make-custom-port’’s ‘#:read’ and ‘#:write’, except that they
     should return ‘(size_t)-1’ if the read or write operation would
     block, instead of ‘#f’.

 -- Function: void scm_set_port_read_wait_fd (scm_t_port_type *type, int
          (*wait_fd) (SCM port))
 -- Function: void scm_set_port_write_wait_fd (scm_t_port_type *type,
          int (*wait_fd) (SCM port))
 -- Function: void scm_set_port_print (scm_t_port_type *type, int
          (*print) (SCM port, SCM dest_port, scm_print_state *pstate))
 -- Function: void scm_set_port_close (scm_t_port_type *type, void
          (*close) (SCM port))
 -- Function: void scm_set_port_needs_close_on_gc (scm_t_port_type
          *type, int needs_close_p)
 -- Function: void scm_set_port_seek (scm_t_port_type *type, scm_t_off
          (*seek) (SCM port, scm_t_off offset, int whence))
 -- Function: void scm_set_port_truncate (scm_t_port_type *type, void
          (*truncate) (SCM port, scm_t_off length))
 -- Function: void scm_set_port_random_access_p (scm_t_port_type *type,
          int (*random_access_p) (SCM port));
 -- Function: void scm_set_port_input_waiting (scm_t_port_type *type,
          int (*input_waiting) (SCM port));
 -- Function: void scm_set_port_get_natural_buffer_sizes
          (scm_t_port_type *type, void (*get_natural_buffer_sizes) (SCM,
          size_t *read_buf_size, size_t *write_buf_size))
     Port method definitions.  *Note Low-Level Custom Ports::, for more
     details on each of these methods.

   Once you have your port type, you can create ports with
‘scm_c_make_port’, or ‘scm_c_make_port_with_encoding’.

 -- Function: SCM scm_c_make_port_with_encoding (scm_t_port_type *type,
          unsigned long mode_bits, SCM encoding, SCM
          conversion_strategy, scm_t_bits stream)
 -- Function: SCM scm_c_make_port (scm_t_port_type *type, unsigned long
          mode_bits, scm_t_bits stream)
     Make a port with the given TYPE.  The STREAM indicates the private
     data associated with the port, which your port implementation may
     later retrieve with ‘SCM_STREAM’.  The mode bits should include one
     or more of the flags ‘SCM_RDNG’ or ‘SCM_WRTNG’, indicating that the
     port is an input and/or an output port, respectively.  The mode
     bits may also include ‘SCM_BUF0’ or ‘SCM_BUFLINE’, indicating that
     the port should be unbuffered or line-buffered, respectively.  The
     default is that the port will be block-buffered.  *Note
     Buffering::.

     As you would imagine, ENCODING and CONVERSION_STRATEGY specify the
     port’s initial textual encoding and conversion strategy.  Both are
     symbols.  ‘scm_c_make_port’ is the same as
     ‘scm_c_make_port_with_encoding’, except it uses the default port
     encoding and conversion strategy.

   At this point you may be wondering whether to implement your custom
port type in C or Scheme.  The answer is that probably you want to use
Scheme’s ‘make-custom-port’.  The speed is similar between C and Scheme,
and ports implemented in C have the disadvantage of not being
suspendable.  *Note Non-Blocking I/O::.


File: guile.info,  Node: Venerable Port Interfaces,  Next: Using Ports from C,  Prev: Port Types,  Up: Input and Output

6.12.11 Venerable Port Interfaces
---------------------------------

Over the 25 years or so that Guile has been around, its port system has
evolved, adding many useful features.  At the same time there have been
four major Scheme standards released in those 25 years, which also
evolve the common Scheme understanding of what a port interface should
be.  Alas, it would be too much to ask for all of these evolutionary
branches to be consistent.  Some of Guile’s original interfaces don’t
mesh with the later Scheme standards, and yet Guile can’t just drop old
interfaces.  Sadly as well, the R6RS and R7RS standards both part from a
base of R5RS, but end up in different and somewhat incompatible designs.

   Guile’s approach is to pick a set of port primitives that make sense
together.  We document that set of primitives, design our internal
interfaces around them, and recommend them to users.  As the R6RS I/O
system is the most capable standard that Scheme has yet produced in this
domain, we mostly recommend that; ‘(ice-9 binary-ports)’ and ‘(ice-9
textual-ports)’ are wholly modeled on ‘(rnrs io ports)’.  Guile does not
wholly copy R6RS, however; *Note R6RS Incompatibilities::.

   At the same time, we have many venerable port interfaces, lore handed
down to us from our hacker ancestors.  Most of these interfaces even
predate the expectation that Scheme should have modules, so they are
present in the default environment.  In Guile we support them as well
and we have no plans to remove them, but again we don’t recommend them
for new users.

 -- Scheme Procedure: char-ready? [port]
     Return ‘#t’ if a character is ready on input PORT and return ‘#f’
     otherwise.  If ‘char-ready?’ returns ‘#t’ then the next ‘read-char’
     operation on PORT is guaranteed not to hang.  If PORT is a file
     port at end of file then ‘char-ready?’ returns ‘#t’.

     ‘char-ready?’ exists to make it possible for a program to accept
     characters from interactive ports without getting stuck waiting for
     input.  Any input editors associated with such ports must make sure
     that characters whose existence has been asserted by ‘char-ready?’
     cannot be rubbed out.  If ‘char-ready?’ were to return ‘#f’ at end
     of file, a port at end of file would be indistinguishable from an
     interactive port that has no ready characters.

     Note that ‘char-ready?’ only works reliably for terminals and
     sockets with one-byte encodings.  Under the hood it will return
     ‘#t’ if the port has any input buffered, or if the file descriptor
     that backs the port polls as readable, indicating that Guile can
     fetch more bytes from the kernel.  However being able to fetch one
     byte doesn’t mean that a full character is available; *Note
     Encoding::.  Also, on many systems it’s possible for a file
     descriptor to poll as readable, but then block when it comes time
     to read bytes.  Note also that on Linux kernels, all file ports
     backed by files always poll as readable.  For non-file ports, this
     procedure always returns ‘#t’, except for soft ports, which have a
     ‘char-ready?’ handler.  *Note Soft Ports::.

     In short, this is a legacy procedure whose semantics are hard to
     provide.  However it is a useful check to see if any input is
     buffered.  *Note Non-Blocking I/O::.

 -- Scheme Procedure: read-char [port]
     The same as ‘get-char’, except that PORT defaults to the current
     input port.  *Note Textual I/O::.

 -- Scheme Procedure: peek-char [port]
     The same as ‘lookahead-char’, except that PORT defaults to the
     current input port.  *Note Textual I/O::.

 -- Scheme Procedure: unread-char cobj [port]
     The same as ‘unget-char’, except that PORT defaults to the current
     input port, and the arguments are swapped.  *Note Textual I/O::.

 -- Scheme Procedure: unread-string str [port]
 -- C Function: scm_unread_string (str, port)
     The same as ‘unget-string’, except that PORT defaults to the
     current input port, and the arguments are swapped.  *Note Textual
     I/O::.

 -- Scheme Procedure: newline [port]
     Send a newline to PORT.  If PORT is omitted, send to the current
     output port.  Equivalent to ‘(put-char port #\newline)’.

 -- Scheme Procedure: write-char chr [port]
     The same as ‘put-char’, except that PORT defaults to the current
     input port, and the arguments are swapped.  *Note Textual I/O::.


File: guile.info,  Node: Using Ports from C,  Next: Non-Blocking I/O,  Prev: Venerable Port Interfaces,  Up: Input and Output

6.12.12 Using Ports from C
--------------------------

Guile’s C interfaces provides some niceties for sending and receiving
bytes and characters in a way that works better with C.

 -- C Function: size_t scm_c_read (SCM port, void *buffer, size_t size)
     Read up to SIZE bytes from PORT and store them in BUFFER.  The
     return value is the number of bytes actually read, which can be
     less than SIZE if end-of-file has been reached.

     Note that as this is a binary input procedure, this function does
     not update ‘port-line’ and ‘port-column’ (*note Textual I/O::).

 -- C Function: void scm_c_write (SCM port, const void *buffer, size_t
          size)
     Write SIZE bytes at BUFFER to PORT.

     Note that as this is a binary output procedure, this function does
     not update ‘port-line’ and ‘port-column’ (*note Textual I/O::).

 -- C Function: size_t scm_c_read_bytes (SCM port, SCM bv, size_t start,
          size_t count)
 -- C Function: void scm_c_write_bytes (SCM port, SCM bv, size_t start,
          size_t count)
     Like ‘scm_c_read’ and ‘scm_c_write’, but reading into or writing
     from the bytevector BV.  COUNT indicates the byte index at which to
     start in the bytevector, and the read or write will continue for
     COUNT bytes.

 -- C Function: void scm_unget_bytes (const unsigned char *buf, size_t
          len, SCM port)
 -- C Function: void scm_unget_byte (int c, SCM port)
 -- C Function: void scm_ungetc (scm_t_wchar c, SCM port)
     Like ‘unget-bytevector’, ‘unget-byte’, and ‘unget-char’,
     respectively.  *Note Textual I/O::.

 -- C Function: void scm_c_put_latin1_chars (SCM port, const scm_t_uint8
          *buf, size_t len)
 -- C Function: void scm_c_put_utf32_chars (SCM port, const scm_t_uint32
          *buf, size_t len);
     Write a string to PORT.  In the first case, the ‘scm_t_uint8*’
     buffer is a string in the latin-1 encoding.  In the second, the
     ‘scm_t_uint32*’ buffer is a string in the UTF-32 encoding.  These
     routines will update the port’s line and column.


File: guile.info,  Node: Non-Blocking I/O,  Next: BOM Handling,  Prev: Using Ports from C,  Up: Input and Output

6.12.13 Non-Blocking I/O
------------------------

Most ports in Guile are “blocking”: when you try to read a character
from a port, Guile will block on the read until a character is ready, or
end-of-stream is detected.  Likewise whenever Guile goes to write
(possibly buffered) data to an output port, Guile will block until all
the data is written.

   Interacting with ports in blocking mode is very convenient: you can
write straightforward, sequential algorithms whose code flow reflects
the flow of data.  However, blocking I/O has two main limitations.

   The first is that it’s easy to get into a situation where code is
waiting on data.  Time spent waiting on data when code could be doing
something else is wasteful and prevents your program from reaching its
peak throughput.  If you implement a web server that sequentially
handles requests from clients, it’s very easy for the server to end up
waiting on a client to finish its HTTP request, or waiting on it to
consume the response.  The end result is that you are able to serve
fewer requests per second than you’d like to serve.

   The second limitation is related: a blocking parser over
user-controlled input is a denial-of-service vulnerability.  Indeed the
so-called “slow loris” attack of the early 2010s was just that: an
attack on common web servers that drip-fed HTTP requests, one character
at a time.  All it took was a handful of slow loris connections to
occupy an entire web server.

   In Guile we would like to preserve the ability to write
straightforward blocking networking processes of all kinds, but under
the hood to allow those processes to suspend their requests if they
would block.

   To do this, the first piece is to allow Guile ports to declare
themselves as being nonblocking.  This is currently supported only for
file ports, which also includes sockets, terminals, or any other port
that is backed by a file descriptor.  To do that, we use an arcane UNIX
incantation:

     (let ((flags (fcntl socket F_GETFL)))
       (fcntl socket F_SETFL (logior O_NONBLOCK flags)))

   Now the file descriptor is open in non-blocking mode.  If Guile tries
to read or write from this file and the read or write returns a result
indicating that more data can only be had by doing a blocking read or
write, Guile will block by polling on the socket’s ‘read-wait-fd’ or
‘write-wait-fd’, to preserve the illusion of a blocking read or write.
*Note Low-Level Custom Ports:: for more on those internal interfaces.

   So far we have just reproduced the status quo: the file descriptor is
non-blocking, but the operations on the port do block.  To go farther,
it would be nice if we could suspend the “thread” using delimited
continuations, and only resume the thread once the file descriptor is
readable or writable.  (*Note Prompts::).

   But here we run into a difficulty.  The ports code is implemented in
C, which means that although we can suspend the computation to some
outer prompt, we can’t resume it because Guile can’t resume delimited
continuations that capture the C stack.

   To overcome this difficulty we have created a compatible but entirely
parallel implementation of port operations.  To use this implementation,
do the following:

     (use-modules (ice-9 suspendable-ports))
     (install-suspendable-ports!)

   This will replace the core I/O primitives like ‘get-char’ and
‘put-bytevector’ with new versions that are exactly the same as the ones
in the standard library, but with two differences.  One is that when a
read or a write would block, the suspendable port operations call out
the value of the ‘current-read-waiter’ or ‘current-write-waiter’
parameter, as appropriate.  *Note Parameters::.  The default read and
write waiters do the same thing that the C read and write waiters do,
which is to poll.  User code can parameterize the waiters, though,
enabling the computation to suspend and allow the program to process
other I/O operations.  Because the new suspendable ports implementation
is written in Scheme, that suspended computation can resume again later
when it is able to make progress.  Success!

   The other main difference is that because the new ports
implementation is written in Scheme, it is slower than C, currently by a
factor of 3 or 4, though it depends on many factors.  For this reason we
have to keep the C implementations as the default ones.  One day when
Guile’s compiler is better, we can close this gap and have only one port
operation implementation again.

   Note that Guile does not currently include an implementation of the
facility to suspend the current thread and schedule other threads in the
meantime.  Before adding such a thing, we want to make sure that we’re
providing the right primitives that can be used to build schedulers and
other user-space concurrency patterns, and that the patterns that we
settle on are the right patterns.  In the meantime, have a look at 8sync
(<https://gnu.org/software/8sync>) for a prototype of an asynchronous
I/O and concurrency facility.

 -- Scheme Procedure: install-suspendable-ports!
     Replace the core ports implementation with suspendable ports, as
     described above.  This will mutate the values of the bindings like
     ‘get-char’, ‘put-u8’, and so on in place.

 -- Scheme Procedure: uninstall-suspendable-ports!
     Restore the original core ports implementation, un-doing the effect
     of ‘install-suspendable-ports!’.

 -- Scheme Parameter: current-read-waiter
 -- Scheme Parameter: current-write-waiter
     Parameters whose values are procedures of one argument, called when
     a suspendable port operation would block on a port while reading or
     writing, respectively.  The default values of these parameters do a
     blocking ‘poll’ on the port’s file descriptor.  The procedures are
     passed the port in question as their one argument.


File: guile.info,  Node: BOM Handling,  Prev: Non-Blocking I/O,  Up: Input and Output

6.12.14 Handling of Unicode Byte Order Marks
--------------------------------------------

This section documents the finer points of Guile’s handling of Unicode
byte order marks (BOMs).  A byte order mark (U+FEFF) is typically found
at the start of a UTF-16 or UTF-32 stream, to allow readers to reliably
determine the byte order.  Occasionally, a BOM is found at the start of
a UTF-8 stream, but this is much less common and not generally
recommended.

   Guile attempts to handle BOMs automatically, and in accordance with
the recommendations of the Unicode Standard, when the port encoding is
set to ‘UTF-8’, ‘UTF-16’, or ‘UTF-32’.  In brief, Guile automatically
writes a BOM at the start of a UTF-16 or UTF-32 stream, and
automatically consumes one from the start of a UTF-8, UTF-16, or UTF-32
stream.

   As specified in the Unicode Standard, a BOM is only handled specially
at the start of a stream, and only if the port encoding is set to
‘UTF-8’, ‘UTF-16’ or ‘UTF-32’.  If the port encoding is set to
‘UTF-16BE’, ‘UTF-16LE’, ‘UTF-32BE’, or ‘UTF-32LE’, then BOMs are _not_
handled specially, and none of the special handling described in this
section applies.

   • To ensure that Guile will properly detect the byte order of a
     UTF-16 or UTF-32 stream, you must perform a textual read before any
     writes, seeks, or binary I/O. Guile will not attempt to read a BOM
     unless a read is explicitly requested at the start of the stream.

   • If a textual write is performed before the first read, then an
     arbitrary byte order will be chosen.  Currently, big endian is the
     default on all platforms, but that may change in the future.  If
     you wish to explicitly control the byte order of an output stream,
     set the port encoding to ‘UTF-16BE’, ‘UTF-16LE’, ‘UTF-32BE’, or
     ‘UTF-32LE’, and explicitly write a BOM (‘#\xFEFF’) if desired.

   • If ‘set-port-encoding!’ is called in the middle of a stream, Guile
     treats this as a new logical “start of stream” for purposes of BOM
     handling, and will forget about any BOMs that had previously been
     seen.  Therefore, it may choose a different byte order than had
     been used previously.  This is intended to support multiple logical
     text streams embedded within a larger binary stream.

   • Binary I/O operations are not guaranteed to update Guile’s notion
     of whether the port is at the “start of the stream”, nor are they
     guaranteed to produce or consume BOMs.

   • For ports that support seeking (e.g.  normal files), the input and
     output streams are considered linked: if the user reads first, then
     a BOM will be consumed (if appropriate), but later writes will
     _not_ produce a BOM. Similarly, if the user writes first, then
     later reads will _not_ consume a BOM.

   • For ports that are not random access (e.g.  pipes, sockets, and
     terminals), the input and output streams are considered
     _independent_ for purposes of BOM handling: the first read will
     consume a BOM (if appropriate), and the first write will _also_
     produce a BOM (if appropriate).  However, the input and output
     streams will always use the same byte order.

   • Seeks to the beginning of a file will set the “start of stream”
     flags.  Therefore, a subsequent textual read or write will consume
     or produce a BOM. However, unlike ‘set-port-encoding!’, if a byte
     order had already been chosen for the port, it will remain in
     effect after a seek, and cannot be changed by the presence of a
     BOM. Seeks anywhere other than the beginning of a file clear the
     “start of stream” flags.


File: guile.info,  Node: Regular Expressions,  Next: LALR(1) Parsing,  Prev: Input and Output,  Up: API Reference

6.13 Regular Expressions
========================

A “regular expression” (or “regexp”) is a pattern that describes a whole
class of strings.  A full description of regular expressions and their
syntax is beyond the scope of this manual.

   If your system does not include a POSIX regular expression library,
and you have not linked Guile with a third-party regexp library such as
Rx, these functions will not be available.  You can tell whether your
Guile installation includes regular expression support by checking
whether ‘(provided? 'regex)’ returns true.

   The following regexp and string matching features are provided by the
‘(ice-9 regex)’ module.  Before using the described functions, you
should load this module by executing ‘(use-modules (ice-9 regex))’.

* Menu:

* Regexp Functions::            Functions that create and match regexps.
* Match Structures::            Finding what was matched by a regexp.
* Backslash Escapes::           Removing the special meaning of regexp
                                meta-characters.


File: guile.info,  Node: Regexp Functions,  Next: Match Structures,  Up: Regular Expressions

6.13.1 Regexp Functions
-----------------------

By default, Guile supports POSIX extended regular expressions.  That
means that the characters ‘(’, ‘)’, ‘+’ and ‘?’ are special, and must be
escaped if you wish to match the literal characters and there is no
support for “non-greedy” variants of ‘*’, ‘+’ or ‘?’.

   This regular expression interface was modeled after that implemented
by SCSH, the Scheme Shell.  It is intended to be upwardly compatible
with SCSH regular expressions.

   Zero bytes (‘#\nul’) cannot be used in regex patterns or input
strings, since the underlying C functions treat that as the end of
string.  If there’s a zero byte an error is thrown.

   Internally, patterns and input strings are converted to the current
locale’s encoding, and then passed to the C library’s regular expression
routines (*note (libc)Regular Expressions::).  The returned match
structures always point to characters in the strings, not to individual
bytes, even in the case of multi-byte encodings.  This ensures that the
match structures are correct when performing matching with characters
that have a multi-byte representation in the locale encoding.  Note,
however, that using characters which cannot be represented in the locale
encoding can lead to surprising results.

 -- Scheme Procedure: string-match pattern str [start]
     Compile the string PATTERN into a regular expression and compare it
     with STR.  The optional numeric argument START specifies the
     position of STR at which to begin matching.

     ‘string-match’ returns a “match structure” which describes what, if
     anything, was matched by the regular expression.  *Note Match
     Structures::.  If STR does not match PATTERN at all, ‘string-match’
     returns ‘#f’.

   Two examples of a match follow.  In the first example, the pattern
matches the four digits in the match string.  In the second, the pattern
matches nothing.

     (string-match "[0-9][0-9][0-9][0-9]" "blah2002")
     ⇒ #("blah2002" (4 . 8))

     (string-match "[A-Za-z]" "123456")
     ⇒ #f

   Each time ‘string-match’ is called, it must compile its PATTERN
argument into a regular expression structure.  This operation is
expensive, which makes ‘string-match’ inefficient if the same regular
expression is used several times (for example, in a loop).  For better
performance, you can compile a regular expression in advance and then
match strings against the compiled regexp.

 -- Scheme Procedure: make-regexp pat flag...
 -- C Function: scm_make_regexp (pat, flaglst)
     Compile the regular expression described by PAT, and return the
     compiled regexp structure.  If PAT does not describe a legal
     regular expression, ‘make-regexp’ throws a
     ‘regular-expression-syntax’ error.

     The FLAG arguments change the behavior of the compiled regular
     expression.  The following values may be supplied:

      -- Variable: regexp/icase
          Consider uppercase and lowercase letters to be the same when
          matching.

      -- Variable: regexp/newline
          If a newline appears in the target string, then permit the ‘^’
          and ‘$’ operators to match immediately after or immediately
          before the newline, respectively.  Also, the ‘.’ and ‘[^...]’
          operators will never match a newline character.  The intent of
          this flag is to treat the target string as a buffer containing
          many lines of text, and the regular expression as a pattern
          that may match a single one of those lines.

      -- Variable: regexp/basic
          Compile a basic (“obsolete”) regexp instead of the extended
          (“modern”) regexps that are the default.  Basic regexps do not
          consider ‘|’, ‘+’ or ‘?’ to be special characters, and require
          the ‘{...}’ and ‘(...)’ metacharacters to be backslash-escaped
          (*note Backslash Escapes::).  There are several other
          differences between basic and extended regular expressions,
          but these are the most significant.

      -- Variable: regexp/extended
          Compile an extended regular expression rather than a basic
          regexp.  This is the default behavior; this flag will not
          usually be needed.  If a call to ‘make-regexp’ includes both
          ‘regexp/basic’ and ‘regexp/extended’ flags, the one which
          comes last will override the earlier one.

 -- Scheme Procedure: regexp-exec rx str [start [flags]]
 -- C Function: scm_regexp_exec (rx, str, start, flags)
     Match the compiled regular expression RX against ‘str’.  If the
     optional integer START argument is provided, begin matching from
     that position in the string.  Return a match structure describing
     the results of the match, or ‘#f’ if no match could be found.

     The FLAGS argument changes the matching behavior.  The following
     flag values may be supplied, use ‘logior’ (*note Bitwise
     Operations::) to combine them,

      -- Variable: regexp/notbol
          Consider that the START offset into STR is not the beginning
          of a line and should not match operator ‘^’.

          If RX was created with the ‘regexp/newline’ option above, ‘^’
          will still match after a newline in STR.

      -- Variable: regexp/noteol
          Consider that the end of STR is not the end of a line and
          should not match operator ‘$’.

          If RX was created with the ‘regexp/newline’ option above, ‘$’
          will still match before a newline in STR.

     ;; Regexp to match uppercase letters
     (define r (make-regexp "[A-Z]*"))

     ;; Regexp to match letters, ignoring case
     (define ri (make-regexp "[A-Z]*" regexp/icase))

     ;; Search for bob using regexp r
     (match:substring (regexp-exec r "bob"))
     ⇒ ""                  ; no match

     ;; Search for bob using regexp ri
     (match:substring (regexp-exec ri "Bob"))
     ⇒ "Bob"               ; matched case insensitive

 -- Scheme Procedure: regexp? obj
 -- C Function: scm_regexp_p (obj)
     Return ‘#t’ if OBJ is a compiled regular expression, or ‘#f’
     otherwise.


 -- Scheme Procedure: list-matches regexp str [flags]
     Return a list of match structures which are the non-overlapping
     matches of REGEXP in STR.  REGEXP can be either a pattern string or
     a compiled regexp.  The FLAGS argument is as per ‘regexp-exec’
     above.

          (map match:substring (list-matches "[a-z]+" "abc 42 def 78"))
          ⇒ ("abc" "def")

 -- Scheme Procedure: fold-matches regexp str init proc [flags]
     Apply PROC to the non-overlapping matches of REGEXP in STR, to
     build a result.  REGEXP can be either a pattern string or a
     compiled regexp.  The FLAGS argument is as per ‘regexp-exec’ above.

     PROC is called as ‘(PROC match prev)’ where MATCH is a match
     structure and PREV is the previous return from PROC.  For the first
     call PREV is the given INIT parameter.  ‘fold-matches’ returns the
     final value from PROC.

     For example to count matches,

          (fold-matches "[a-z][0-9]" "abc x1 def y2" 0
                        (lambda (match count)
                          (1+ count)))
          ⇒ 2


   Regular expressions are commonly used to find patterns in one string
and replace them with the contents of another string.  The following
functions are convenient ways to do this.

 -- Scheme Procedure: regexp-substitute port match item ...
     Write to PORT selected parts of the match structure MATCH.  Or if
     PORT is ‘#f’ then form a string from those parts and return that.

     Each ITEM specifies a part to be written, and may be one of the
     following,

        • A string.  String arguments are written out verbatim.

        • An integer.  The submatch with that number is written
          (‘match:substring’).  Zero is the entire match.

        • The symbol ‘pre’.  The portion of the matched string preceding
          the regexp match is written (‘match:prefix’).

        • The symbol ‘post’.  The portion of the matched string
          following the regexp match is written (‘match:suffix’).

     For example, changing a match and retaining the text before and
     after,

          (regexp-substitute #f (string-match "[0-9]+" "number 25 is good")
                             'pre "37" 'post)
          ⇒ "number 37 is good"

     Or matching a YYYYMMDD format date such as ‘20020828’ and
     re-ordering and hyphenating the fields.

          (define date-regex
             "([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9])")
          (define s "Date 20020429 12am.")
          (regexp-substitute #f (string-match date-regex s)
                             'pre 2 "-" 3 "-" 1 'post " (" 0 ")")
          ⇒ "Date 04-29-2002 12am. (20020429)"

 -- Scheme Procedure: regexp-substitute/global port regexp target
          item...
     Write to PORT selected parts of matches of REGEXP in TARGET.  If
     PORT is ‘#f’ then form a string from those parts and return that.
     REGEXP can be a string or a compiled regex.

     This is similar to ‘regexp-substitute’, but allows global
     substitutions on TARGET.  Each ITEM behaves as per
     ‘regexp-substitute’, with the following differences,

        • A function.  Called as ‘(ITEM match)’ with the match structure
          for the REGEXP match, it should return a string to be written
          to PORT.

        • The symbol ‘post’.  This doesn’t output anything, but instead
          causes ‘regexp-substitute/global’ to recurse on the unmatched
          portion of TARGET.

          This _must_ be supplied to perform a global search and replace
          on TARGET; without it ‘regexp-substitute/global’ returns after
          a single match and output.

     For example, to collapse runs of tabs and spaces to a single hyphen
     each,

          (regexp-substitute/global #f "[ \t]+"  "this   is   the text"
                                    'pre "-" 'post)
          ⇒ "this-is-the-text"

     Or using a function to reverse the letters in each word,

          (regexp-substitute/global #f "[a-z]+"  "to do and not-do"
            'pre (lambda (m) (string-reverse (match:substring m))) 'post)
          ⇒ "ot od dna ton-od"

     Without the ‘post’ symbol, just one regexp match is made.  For
     example the following is the date example from ‘regexp-substitute’
     above, without the need for the separate ‘string-match’ call.

          (define date-regex
             "([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9])")
          (define s "Date 20020429 12am.")
          (regexp-substitute/global #f date-regex s
                                    'pre 2 "-" 3 "-" 1 'post " (" 0 ")")

          ⇒ "Date 04-29-2002 12am. (20020429)"


File: guile.info,  Node: Match Structures,  Next: Backslash Escapes,  Prev: Regexp Functions,  Up: Regular Expressions

6.13.2 Match Structures
-----------------------

A “match structure” is the object returned by ‘string-match’ and
‘regexp-exec’.  It describes which portion of a string, if any, matched
the given regular expression.  Match structures include: a reference to
the string that was checked for matches; the starting and ending
positions of the regexp match; and, if the regexp included any
parenthesized subexpressions, the starting and ending positions of each
submatch.

   In each of the regexp match functions described below, the ‘match’
argument must be a match structure returned by a previous call to
‘string-match’ or ‘regexp-exec’.  Most of these functions return some
information about the original target string that was matched against a
regular expression; we will call that string TARGET for easy reference.

 -- Scheme Procedure: regexp-match? obj
     Return ‘#t’ if OBJ is a match structure returned by a previous call
     to ‘regexp-exec’, or ‘#f’ otherwise.

 -- Scheme Procedure: match:substring match [n]
     Return the portion of TARGET matched by subexpression number N.
     Submatch 0 (the default) represents the entire regexp match.  If
     the regular expression as a whole matched, but the subexpression
     number N did not match, return ‘#f’.

     (define s (string-match "[0-9][0-9][0-9][0-9]" "blah2002foo"))
     (match:substring s)
     ⇒ "2002"

     ;; match starting at offset 6 in the string
     (match:substring
       (string-match "[0-9][0-9][0-9][0-9]" "blah987654" 6))
     ⇒ "7654"

 -- Scheme Procedure: match:start match [n]
     Return the starting position of submatch number N.

   In the following example, the result is 4, since the match starts at
character index 4:

     (define s (string-match "[0-9][0-9][0-9][0-9]" "blah2002foo"))
     (match:start s)
     ⇒ 4

 -- Scheme Procedure: match:end match [n]
     Return the ending position of submatch number N.

   In the following example, the result is 8, since the match runs
between characters 4 and 8 (i.e. the “2002”).

     (define s (string-match "[0-9][0-9][0-9][0-9]" "blah2002foo"))
     (match:end s)
     ⇒ 8

 -- Scheme Procedure: match:prefix match
     Return the unmatched portion of TARGET preceding the regexp match.

          (define s (string-match "[0-9][0-9][0-9][0-9]" "blah2002foo"))
          (match:prefix s)
          ⇒ "blah"

 -- Scheme Procedure: match:suffix match
     Return the unmatched portion of TARGET following the regexp match.

     (define s (string-match "[0-9][0-9][0-9][0-9]" "blah2002foo"))
     (match:suffix s)
     ⇒ "foo"

 -- Scheme Procedure: match:count match
     Return the number of parenthesized subexpressions from MATCH.  Note
     that the entire regular expression match itself counts as a
     subexpression, and failed submatches are included in the count.

 -- Scheme Procedure: match:string match
     Return the original TARGET string.

     (define s (string-match "[0-9][0-9][0-9][0-9]" "blah2002foo"))
     (match:string s)
     ⇒ "blah2002foo"


File: guile.info,  Node: Backslash Escapes,  Prev: Match Structures,  Up: Regular Expressions

6.13.3 Backslash Escapes
------------------------

Sometimes you will want a regexp to match characters like ‘*’ or ‘$’
exactly.  For example, to check whether a particular string represents a
menu entry from an Info node, it would be useful to match it against a
regexp like ‘^* [^:]*::’.  However, this won’t work; because the
asterisk is a metacharacter, it won’t match the ‘*’ at the beginning of
the string.  In this case, we want to make the first asterisk un-magic.

   You can do this by preceding the metacharacter with a backslash
character ‘\’.  (This is also called “quoting” the metacharacter, and is
known as a “backslash escape”.)  When Guile sees a backslash in a
regular expression, it considers the following glyph to be an ordinary
character, no matter what special meaning it would ordinarily have.
Therefore, we can make the above example work by changing the regexp to
‘^\* [^:]*::’.  The ‘\*’ sequence tells the regular expression engine to
match only a single asterisk in the target string.

   Since the backslash is itself a metacharacter, you may force a regexp
to match a backslash in the target string by preceding the backslash
with itself.  For example, to find variable references in a TeX program,
you might want to find occurrences of the string ‘\let\’ followed by any
number of alphabetic characters.  The regular expression
‘\\let\\[A-Za-z]*’ would do this: the double backslashes in the regexp
each match a single backslash in the target string.

 -- Scheme Procedure: regexp-quote str
     Quote each special character found in STR with a backslash, and
     return the resulting string.

   *Very important:* Using backslash escapes in Guile source code (as in
Emacs Lisp or C) can be tricky, because the backslash character has
special meaning for the Guile reader.  For example, if Guile encounters
the character sequence ‘\n’ in the middle of a string while processing
Scheme code, it replaces those characters with a newline character.
Similarly, the character sequence ‘\t’ is replaced by a horizontal tab.
Several of these “escape sequences” are processed by the Guile reader
before your code is executed.  Unrecognized escape sequences are
ignored: if the characters ‘\*’ appear in a string, they will be
translated to the single character ‘*’.

   This translation is obviously undesirable for regular expressions,
since we want to be able to include backslashes in a string in order to
escape regexp metacharacters.  Therefore, to make sure that a backslash
is preserved in a string in your Guile program, you must use _two_
consecutive backslashes:

     (define Info-menu-entry-pattern (make-regexp "^\\* [^:]*"))

   The string in this example is preprocessed by the Guile reader before
any code is executed.  The resulting argument to ‘make-regexp’ is the
string ‘^\* [^:]*’, which is what we really want.

   This also means that in order to write a regular expression that
matches a single backslash character, the regular expression string in
the source code must include _four_ backslashes.  Each consecutive pair
of backslashes gets translated by the Guile reader to a single
backslash, and the resulting double-backslash is interpreted by the
regexp engine as matching a single backslash character.  Hence:

     (define tex-variable-pattern (make-regexp "\\\\let\\\\=[A-Za-z]*"))

   The reason for the unwieldiness of this syntax is historical.  Both
regular expression pattern matchers and Unix string processing systems
have traditionally used backslashes with the special meanings described
above.  The POSIX regular expression specification and ANSI C standard
both require these semantics.  Attempting to abandon either convention
would cause other kinds of compatibility problems, possibly more severe
ones.  Therefore, without extending the Scheme reader to support strings
with different quoting conventions (an ungainly and confusing extension
when implemented in other languages), we must adhere to this cumbersome
escape syntax.


File: guile.info,  Node: LALR(1) Parsing,  Next: PEG Parsing,  Prev: Regular Expressions,  Up: API Reference

6.14 LALR(1) Parsing
====================

The ‘(system base lalr)’ module provides the ‘lalr-scm’ LALR(1) parser
generator by Dominique Boucher (https://github.com/schemeway/lalr-scm/).
‘lalr-scm’ uses the same algorithm as GNU Bison (*note Introduction to
Bison: (bison)Introduction.).  Parsers are defined using the
‘lalr-parser’ macro.

 -- Scheme Syntax: lalr-parser [OPTIONS] TOKENS RULES...
     Generate an LALR(1) syntax analyzer.  TOKENS is a list of symbols
     representing the terminal symbols of the grammar.  RULES are the
     grammar production rules.

     Each rule has the form ‘(NON-TERMINAL (RHS ...) : ACTION ...)’,
     where NON-TERMINAL is the name of the rule, RHS are the right-hand
     sides, i.e., the production rule, and ACTION is a semantic action
     associated with the rule.

     The generated parser is a two-argument procedure that takes a
     “tokenizer” and a “syntax error procedure”.  The tokenizer should
     be a thunk that returns lexical tokens as produced by
     ‘make-lexical-token’.  The syntax error procedure may be called
     with at least an error message (a string), and optionally the
     lexical token that caused the error.

   Please refer to the ‘lalr-scm’ documentation for details.


File: guile.info,  Node: PEG Parsing,  Next: Read/Load/Eval/Compile,  Prev: LALR(1) Parsing,  Up: API Reference

6.15 PEG Parsing
================

Parsing Expression Grammars (PEGs) are a way of specifying formal
languages for text processing.  They can be used either for matching
(like regular expressions) or for building recursive descent parsers
(like lex/yacc).  Guile uses a superset of PEG syntax that allows more
control over what information is preserved during parsing.

   Wikipedia has a clear and concise introduction to PEGs if you want to
familiarize yourself with the syntax:
<http://en.wikipedia.org/wiki/Parsing_expression_grammar>.

   The paper that introduced PEG contains a more detailed description of
how PEG works, and describes its syntax in detail:
<https://bford.info/pub/lang/peg.pdf>

   The ‘(ice-9 peg)’ module works by compiling PEGs down to lambda
expressions.  These can either be stored in variables at compile-time by
the define macros (‘define-peg-pattern’ and
‘define-peg-string-patterns’) or calculated explicitly at runtime with
the compile functions (‘compile-peg-pattern’ and ‘peg-string-compile’).

   They can then be used for either parsing (‘match-pattern’) or
searching (‘search-for-pattern’).  For convenience, ‘search-for-pattern’
also takes pattern literals in case you want to inline a simple search
(people often use regular expressions this way).

   The rest of this documentation consists of a syntax reference, an API
reference, and a tutorial.

* Menu:

* PEG Syntax Reference::
* PEG API Reference::
* PEG Tutorial::
* PEG Internals::


File: guile.info,  Node: PEG Syntax Reference,  Next: PEG API Reference,  Up: PEG Parsing

6.15.1 PEG Syntax Reference
---------------------------

Normal PEG Syntax:
..................

 -- PEG Pattern: sequence a b
     Parses A.  If this succeeds, continues to parse B from the end of
     the text parsed as A.  Succeeds if both A and B succeed.

     ‘"a b"’

     ‘(and a b)’

 -- PEG Pattern: ordered choice a b
     Parses A.  If this fails, backtracks and parses B.  Succeeds if
     either A or B succeeds.

     ‘"a/b"’

     ‘(or a b)’

 -- PEG Pattern: zero or more a
     Parses A as many times in a row as it can, starting each A at the
     end of the text parsed by the previous A.  Always succeeds.

     ‘"a*"’

     ‘(* a)’

 -- PEG Pattern: one or more a
     Parses A as many times in a row as it can, starting each A at the
     end of the text parsed by the previous A.  Succeeds if at least one
     A was parsed.

     ‘"a+"’

     ‘(+ a)’

 -- PEG Pattern: optional a
     Tries to parse A.  Succeeds if A succeeds.

     ‘"a?"’

     ‘(? a)’

 -- PEG Pattern: followed by a
     Makes sure it is possible to parse A, but does not actually parse
     it.  Succeeds if A would succeed.

     ‘"&a"’

     ‘(followed-by a)’

 -- PEG Pattern: not followed by a
     Makes sure it is impossible to parse A, but does not actually parse
     it.  Succeeds if A would fail.

     ‘"!a"’

     ‘(not-followed-by a)’

 -- PEG Pattern: string literal ``abc''
     Parses the string "ABC".  Succeeds if that parsing succeeds.

     ‘"'abc'"’

     ‘"abc"’

 -- PEG Pattern: any character
     Parses any single character.  Succeeds unless there is no more text
     to be parsed.

     ‘"."’

     ‘peg-any’

 -- PEG Pattern: character class a b
     Alternative syntax for “Ordered Choice A B” if A and B are
     characters.

     ‘"[ab]"’

     ‘(or "a" "b")’

 -- PEG Pattern: range of characters a z
     Parses any character falling between A and Z.

     ‘"[a-z]"’

     ‘(range #\a #\z)’

 -- PEG Pattern: inverse range of characters a z
     Parses any character not falling between A and Z.

     ‘"[^a-z]"’

     ‘(not-in-range #\a #\z)’

   Example:

     "(a !b / c &d*) 'e'+"

   Would be:

     (and
      (or
       (and a (not-followed-by b))
       (and c (followed-by (* d))))
      (+ "e"))

Extended Syntax
...............

There is some extra syntax for S-expressions.

 -- PEG Pattern: ignore a
     Ignore the text matching A

 -- PEG Pattern: capture a
     Capture the text matching A.

 -- PEG Pattern: peg a
     Embed the PEG pattern A using string syntax.

   Example:

     "!a / 'b'"

   Is equivalent to

     (or (peg "!a") "b")

   and

     (or (not-followed-by a) "b")


File: guile.info,  Node: PEG API Reference,  Next: PEG Tutorial,  Prev: PEG Syntax Reference,  Up: PEG Parsing

6.15.2 PEG API Reference
------------------------

Define Macros
.............

The most straightforward way to define a PEG is by using one of the
define macros (both of these macroexpand into ‘define’ expressions).
These macros bind parsing functions to variables.  These parsing
functions may be invoked by ‘match-pattern’ or ‘search-for-pattern’,
which return a PEG match record.  Raw data can be retrieved from this
record with the PEG match deconstructor functions.  More complicated
(and perhaps enlightening) examples can be found in the tutorial.

 -- Scheme Macro: define-peg-string-patterns peg-string
     Defines all the nonterminals in the PEG PEG-STRING.  More
     precisely, ‘define-peg-string-patterns’ takes a superset of PEGs.
     A normal PEG has a ‘<-’ between the nonterminal and the pattern.
     ‘define-peg-string-patterns’ uses this symbol to determine what
     information it should propagate up the parse tree.  The normal ‘<-’
     propagates the matched text up the parse tree, ‘<--’ propagates the
     matched text up the parse tree tagged with the name of the
     nonterminal, and ‘<’ discards that matched text and propagates
     nothing up the parse tree.  Also, nonterminals may include “-”
     character, while in normal PEG it is not allowed.

     For example, if we:
          (define-peg-string-patterns
            "as <- 'a'+
          bs <- 'b'+
          as-or-bs <- as/bs")
          (define-peg-string-patterns
            "as-tag <-- 'a'+
          bs-tag <-- 'b'+
          as-or-bs-tag <-- as-tag/bs-tag")
     Then:
          (match-pattern as-or-bs "aabbcc") ⇒
          #<peg start: 0 end: 2 string: aabbcc tree: aa>
          (match-pattern as-or-bs-tag "aabbcc") ⇒
          #<peg start: 0 end: 2 string: aabbcc tree: (as-or-bs-tag (as-tag aa))>

     Note that in doing this, we have bound 6 variables at the toplevel
     (AS, BS, AS-OR-BS, AS-TAG, BS-TAG, and AS-OR-BS-TAG).

 -- Scheme Macro: define-peg-pattern name capture-type peg-sexp
     Defines a single nonterminal NAME.  CAPTURE-TYPE determines how
     much information is passed up the parse tree.  PEG-SEXP is a PEG in
     S-expression form.

     Possible values for capture-type:

     ‘all’
          passes the matched text up the parse tree tagged with the name
          of the nonterminal.
     ‘body’
          passes the matched text up the parse tree.
     ‘none’
          passes nothing up the parse tree.

     For Example, if we:
          (define-peg-pattern as body (+ "a"))
          (define-peg-pattern bs body (+ "b"))
          (define-peg-pattern as-or-bs body (or as bs))
          (define-peg-pattern as-tag all (+ "a"))
          (define-peg-pattern bs-tag all (+ "b"))
          (define-peg-pattern as-or-bs-tag all (or as-tag bs-tag))
     Then:
          (match-pattern as-or-bs "aabbcc") ⇒
          #<peg start: 0 end: 2 string: aabbcc tree: aa>
          (match-pattern as-or-bs-tag "aabbcc") ⇒
          #<peg start: 0 end: 2 string: aabbcc tree: (as-or-bs-tag (as-tag aa))>

     Note that in doing this, we have bound 6 variables at the toplevel
     (AS, BS, AS-OR-BS, AS-TAG, BS-TAG, and AS-OR-BS-TAG).

Compile Functions
.................

It is sometimes useful to be able to compile anonymous PEG patterns at
runtime.  These functions let you do that using either syntax.

 -- Scheme Procedure: peg-string-compile peg-string capture-type
     Compiles the PEG pattern in PEG-STRING propagating according to
     CAPTURE-TYPE (capture-type can be any of the values from
     ‘define-peg-pattern’).

 -- Scheme Procedure: compile-peg-pattern peg-sexp capture-type
     Compiles the PEG pattern in PEG-SEXP propagating according to
     CAPTURE-TYPE (capture-type can be any of the values from
     ‘define-peg-pattern’).

   The functions return syntax objects, which can be useful if you want
to use them in macros.  If all you want is to define a new nonterminal,
you can do the following:

     (define exp '(+ "a"))
     (define as (compile (compile-peg-pattern exp 'body)))

   You can use this nonterminal with all of the regular PEG functions:

     (match-pattern as "aaaaa") ⇒
     #<peg start: 0 end: 5 string: aaaaa tree: aaaaa>

Parsing & Matching Functions
............................

For our purposes, “parsing” means parsing a string into a tree starting
from the first character, while “matching” means searching through the
string for a substring.  In practice, the only difference between the
two functions is that ‘match-pattern’ gives up if it can’t find a valid
substring starting at index 0 and ‘search-for-pattern’ keeps looking.
They are both equally capable of “parsing” and “matching” given those
constraints.

 -- Scheme Procedure: match-pattern nonterm string
     Parses STRING using the PEG stored in NONTERM.  If no match was
     found, ‘match-pattern’ returns false.  If a match was found, a PEG
     match record is returned.

     The ‘capture-type’ argument to ‘define-peg-pattern’ allows you to
     choose what information to hold on to while parsing.  The options
     are:

     ‘all’
          tag the matched text with the nonterminal
     ‘body’
          just the matched text
     ‘none’
          nothing

          (define-peg-pattern as all (+ "a"))
          (match-pattern as "aabbcc") ⇒
          #<peg start: 0 end: 2 string: aabbcc tree: (as aa)>

          (define-peg-pattern as body (+ "a"))
          (match-pattern as "aabbcc") ⇒
          #<peg start: 0 end: 2 string: aabbcc tree: aa>

          (define-peg-pattern as none (+ "a"))
          (match-pattern as "aabbcc") ⇒
          #<peg start: 0 end: 2 string: aabbcc tree: ()>

          (define-peg-pattern bs body (+ "b"))
          (match-pattern bs "aabbcc") ⇒
          #f

 -- Scheme Macro: search-for-pattern nonterm-or-peg string
     Searches through STRING looking for a matching subexpression.
     NONTERM-OR-PEG can either be a nonterminal or a literal PEG
     pattern.  When a literal PEG pattern is provided,
     ‘search-for-pattern’ works very similarly to the regular expression
     searches many hackers are used to.  If no match was found,
     ‘search-for-pattern’ returns false.  If a match was found, a PEG
     match record is returned.

          (define-peg-pattern as body (+ "a"))
          (search-for-pattern as "aabbcc") ⇒
          #<peg start: 0 end: 2 string: aabbcc tree: aa>
          (search-for-pattern (+ "a") "aabbcc") ⇒
          #<peg start: 0 end: 2 string: aabbcc tree: aa>
          (search-for-pattern "'a'+" "aabbcc") ⇒
          #<peg start: 0 end: 2 string: aabbcc tree: aa>

          (define-peg-pattern as all (+ "a"))
          (search-for-pattern as "aabbcc") ⇒
          #<peg start: 0 end: 2 string: aabbcc tree: (as aa)>

          (define-peg-pattern bs body (+ "b"))
          (search-for-pattern bs "aabbcc") ⇒
          #<peg start: 2 end: 4 string: aabbcc tree: bb>
          (search-for-pattern (+ "b") "aabbcc") ⇒
          #<peg start: 2 end: 4 string: aabbcc tree: bb>
          (search-for-pattern "'b'+" "aabbcc") ⇒
          #<peg start: 2 end: 4 string: aabbcc tree: bb>

          (define-peg-pattern zs body (+ "z"))
          (search-for-pattern zs "aabbcc") ⇒
          #f
          (search-for-pattern (+ "z") "aabbcc") ⇒
          #f
          (search-for-pattern "'z'+" "aabbcc") ⇒
          #f

PEG Match Records
.................

The ‘match-pattern’ and ‘search-for-pattern’ functions both return PEG
match records.  Actual information can be extracted from these with the
following functions.

 -- Scheme Procedure: peg:string match-record
     Returns the original string that was parsed in the creation of
     ‘match-record’.

 -- Scheme Procedure: peg:start match-record
     Returns the index of the first parsed character in the original
     string (from ‘peg:string’).  If this is the same as ‘peg:end’,
     nothing was parsed.

 -- Scheme Procedure: peg:end match-record
     Returns one more than the index of the last parsed character in the
     original string (from ‘peg:string’).  If this is the same as
     ‘peg:start’, nothing was parsed.

 -- Scheme Procedure: peg:substring match-record
     Returns the substring parsed by ‘match-record’.  This is equivalent
     to ‘(substring (peg:string match-record) (peg:start match-record)
     (peg:end match-record))’.

 -- Scheme Procedure: peg:tree match-record
     Returns the tree parsed by ‘match-record’.

 -- Scheme Procedure: peg-record? match-record
     Returns true if ‘match-record’ is a PEG match record, or false
     otherwise.

   Example:
     (define-peg-pattern bs all (peg "'b'+"))

     (search-for-pattern bs "aabbcc") ⇒
     #<peg start: 2 end: 4 string: aabbcc tree: (bs bb)>

     (let ((pm (search-for-pattern bs "aabbcc")))
        `((string ,(peg:string pm))
          (start ,(peg:start pm))
          (end ,(peg:end pm))
          (substring ,(peg:substring pm))
          (tree ,(peg:tree pm))
          (record? ,(peg-record? pm)))) ⇒
     ((string "aabbcc")
      (start 2)
      (end 4)
      (substring "bb")
      (tree (bs "bb"))
      (record? #t))

Miscellaneous
.............

 -- Scheme Procedure: context-flatten tst lst
     Takes a predicate TST and a list LST.  Flattens LST until all
     elements are either atoms or satisfy TST.  If LST itself satisfies
     TST, ‘(list lst)’ is returned (this is a flat list whose only
     element satisfies TST).

          (context-flatten (lambda (x) (and (number? (car x)) (= (car x) 1))) '(2 2 (1 1 (2 2)) (2 2 (1 1)))) ⇒
          (2 2 (1 1 (2 2)) 2 2 (1 1))
          (context-flatten (lambda (x) (and (number? (car x)) (= (car x) 1))) '(1 1 (1 1 (2 2)) (2 2 (1 1)))) ⇒
          ((1 1 (1 1 (2 2)) (2 2 (1 1))))

     If you’re wondering why this is here, take a look at the tutorial.

 -- Scheme Procedure: keyword-flatten terms lst
     A less general form of ‘context-flatten’.  Takes a list of terminal
     atoms ‘terms’ and flattens LST until all elements are either atoms,
     or lists which have an atom from ‘terms’ as their first element.
          (keyword-flatten '(a b) '(c a b (a c) (b c) (c (b a) (c a)))) ⇒
          (c a b (a c) (b c) c (b a) c a)

     If you’re wondering why this is here, take a look at the tutorial.


File: guile.info,  Node: PEG Tutorial,  Next: PEG Internals,  Prev: PEG API Reference,  Up: PEG Parsing

6.15.3 PEG Tutorial
-------------------

Parsing /etc/passwd
...................

This example will show how to parse /etc/passwd using PEGs.

   First we define an example /etc/passwd file:

     (define *etc-passwd*
       "root:x:0:0:root:/root:/bin/bash
     daemon:x:1:1:daemon:/usr/sbin:/bin/sh
     bin:x:2:2:bin:/bin:/bin/sh
     sys:x:3:3:sys:/dev:/bin/sh
     nobody:x:65534:65534:nobody:/nonexistent:/bin/sh
     messagebus:x:103:107::/var/run/dbus:/bin/false
     ")

   As a first pass at this, we might want to have all the entries in
/etc/passwd in a list.

   Doing this with string-based PEG syntax would look like this:
     (define-peg-string-patterns
       "passwd <- entry* !.
     entry <-- (! NL .)* NL*
     NL < '\n'")

   A ‘passwd’ file is 0 or more entries (‘entry*’) until the end of the
file (‘!.’ (‘.’ is any character, so ‘!.’ means “not anything”)).  We
want to capture the data in the nonterminal ‘passwd’, but not tag it
with the name, so we use ‘<-’.

   An entry is a series of 0 or more characters that aren’t newlines
(‘(! NL .)*’) followed by 0 or more newlines (‘NL*’).  We want to tag
all the entries with ‘entry’, so we use ‘<--’.

   A newline is just a literal newline (‘'\n'’).  We don’t want a bunch
of newlines cluttering up the output, so we use ‘<’ to throw away the
captured data.

   Here is the same PEG defined using S-expressions:
     (define-peg-pattern passwd body (and (* entry) (not-followed-by peg-any)))
     (define-peg-pattern entry all (and (* (and (not-followed-by NL) peg-any))
     			       (* NL)))
     (define-peg-pattern NL none "\n")

   Obviously this is much more verbose.  On the other hand, it’s more
explicit, and thus easier to build automatically.  However, there are
some tricks that make S-expressions easier to use in some cases.  One is
the ‘ignore’ keyword; the string syntax has no way to say “throw away
this text” except breaking it out into a separate nonterminal.  For
instance, to throw away the newlines we had to define ‘NL’.  In the
S-expression syntax, we could have simply written ‘(ignore "\n")’.
Also, for the cases where string syntax is really much cleaner, the
‘peg’ keyword can be used to embed string syntax in S-expression syntax.
For instance, we could have written:

     (define-peg-pattern passwd body (peg "entry* !."))

   However we define it, parsing ‘*etc-passwd*’ with the ‘passwd’
nonterminal yields the same results:

     (peg:tree (match-pattern passwd *etc-passwd*)) ⇒
     ((entry "root:x:0:0:root:/root:/bin/bash")
      (entry "daemon:x:1:1:daemon:/usr/sbin:/bin/sh")
      (entry "bin:x:2:2:bin:/bin:/bin/sh")
      (entry "sys:x:3:3:sys:/dev:/bin/sh")
      (entry "nobody:x:65534:65534:nobody:/nonexistent:/bin/sh")
      (entry "messagebus:x:103:107::/var/run/dbus:/bin/false"))

   However, here is something to be wary of:

     (peg:tree (match-pattern passwd "one entry")) ⇒
     (entry "one entry")

   By default, the parse trees generated by PEGs are compressed as much
as possible without losing information.  It may not look like this is
what you want at first, but uncompressed parse trees are an enormous
headache (there’s no easy way to predict how deep particular lists will
nest, there are empty lists littered everywhere, etc.  etc.).  One
side-effect of this, however, is that sometimes the compressor is too
aggressive.  No information is discarded when ‘((entry "one entry"))’ is
compressed to ‘(entry "one entry")’, but in this particular case it
probably isn’t what we want.

   There are two functions for easily dealing with this:
‘keyword-flatten’ and ‘context-flatten’.  The ‘keyword-flatten’ function
takes a list of keywords and a list to flatten, then tries to coerce the
list such that the first element of all sublists is one of the keywords.
The ‘context-flatten’ function is similar, but instead of a list of
keywords it takes a predicate that should indicate whether a given
sublist is good enough (refer to the API reference for more details).

   What we want here is ‘keyword-flatten’.
     (keyword-flatten '(entry) (peg:tree (match-pattern passwd *etc-passwd*))) ⇒
     ((entry "root:x:0:0:root:/root:/bin/bash")
      (entry "daemon:x:1:1:daemon:/usr/sbin:/bin/sh")
      (entry "bin:x:2:2:bin:/bin:/bin/sh")
      (entry "sys:x:3:3:sys:/dev:/bin/sh")
      (entry "nobody:x:65534:65534:nobody:/nonexistent:/bin/sh")
      (entry "messagebus:x:103:107::/var/run/dbus:/bin/false"))
     (keyword-flatten '(entry) (peg:tree (match-pattern passwd "one entry"))) ⇒
     ((entry "one entry"))

   Of course, this is a somewhat contrived example.  In practice we
would probably just tag the ‘passwd’ nonterminal to remove the ambiguity
(using either the ‘all’ keyword for S-expressions or the ‘<--’ symbol
for strings)..

     (define-peg-pattern tag-passwd all (peg "entry* !."))
     (peg:tree (match-pattern tag-passwd *etc-passwd*)) ⇒
     (tag-passwd
       (entry "root:x:0:0:root:/root:/bin/bash")
       (entry "daemon:x:1:1:daemon:/usr/sbin:/bin/sh")
       (entry "bin:x:2:2:bin:/bin:/bin/sh")
       (entry "sys:x:3:3:sys:/dev:/bin/sh")
       (entry "nobody:x:65534:65534:nobody:/nonexistent:/bin/sh")
       (entry "messagebus:x:103:107::/var/run/dbus:/bin/false"))
     (peg:tree (match-pattern tag-passwd "one entry"))
     (tag-passwd
       (entry "one entry"))

   If you’re ever uncertain about the potential results of parsing
something, remember the two absolute rules:
  1. No parsing information will ever be discarded.
  2. There will never be any lists with fewer than 2 elements.

   For the purposes of (1), "parsing information" means things tagged
with the ‘any’ keyword or the ‘<--’ symbol.  Plain strings will be
concatenated.

   Let’s extend this example a bit more and actually pull some useful
information out of the passwd file:

     (define-peg-string-patterns
       "passwd <-- entry* !.
     entry <-- login C pass C uid C gid C nameORcomment C homedir C shell NL*
     login <-- text
     pass <-- text
     uid <-- [0-9]*
     gid <-- [0-9]*
     nameORcomment <-- text
     homedir <-- path
     shell <-- path
     path <-- (SLASH pathELEMENT)*
     pathELEMENT <-- (!NL !C  !'/' .)*
     text <- (!NL !C  .)*
     C < ':'
     NL < '\n'
     SLASH < '/'")

   This produces rather pretty parse trees:
     (passwd
       (entry (login "root")
              (pass "x")
              (uid "0")
              (gid "0")
              (nameORcomment "root")
              (homedir (path (pathELEMENT "root")))
              (shell (path (pathELEMENT "bin") (pathELEMENT "bash"))))
       (entry (login "daemon")
              (pass "x")
              (uid "1")
              (gid "1")
              (nameORcomment "daemon")
              (homedir
                (path (pathELEMENT "usr") (pathELEMENT "sbin")))
              (shell (path (pathELEMENT "bin") (pathELEMENT "sh"))))
       (entry (login "bin")
              (pass "x")
              (uid "2")
              (gid "2")
              (nameORcomment "bin")
              (homedir (path (pathELEMENT "bin")))
              (shell (path (pathELEMENT "bin") (pathELEMENT "sh"))))
       (entry (login "sys")
              (pass "x")
              (uid "3")
              (gid "3")
              (nameORcomment "sys")
              (homedir (path (pathELEMENT "dev")))
              (shell (path (pathELEMENT "bin") (pathELEMENT "sh"))))
       (entry (login "nobody")
              (pass "x")
              (uid "65534")
              (gid "65534")
              (nameORcomment "nobody")
              (homedir (path (pathELEMENT "nonexistent")))
              (shell (path (pathELEMENT "bin") (pathELEMENT "sh"))))
       (entry (login "messagebus")
              (pass "x")
              (uid "103")
              (gid "107")
              nameORcomment
              (homedir
                (path (pathELEMENT "var")
                      (pathELEMENT "run")
                      (pathELEMENT "dbus")))
              (shell (path (pathELEMENT "bin") (pathELEMENT "false")))))

   Notice that when there’s no entry in a field (e.g.  ‘nameORcomment’
for messagebus) the symbol is inserted.  This is the “don’t throw away
any information” rule—we successfully matched a ‘nameORcomment’ of 0
characters (since we used ‘*’ when defining it).  This is usually what
you want, because it allows you to e.g.  use ‘list-ref’ to pull out
elements (since they all have known offsets).

   If you’d prefer not to have symbols for empty matches, you can
replace the ‘*’ with a ‘+’ and add a ‘?’ after the ‘nameORcomment’ in
‘entry’.  Then it will try to parse 1 or more characters, fail
(inserting nothing into the parse tree), but continue because it didn’t
have to match the nameORcomment to continue.

Embedding Arithmetic Expressions
................................

We can parse simple mathematical expressions with the following PEG:

     (define-peg-string-patterns
       "expr <- sum
     sum <-- (product ('+' / '-') sum) / product
     product <-- (value ('*' / '/') product) / value
     value <-- number / '(' expr ')'
     number <-- [0-9]+")

   Then:
     (peg:tree (match-pattern expr "1+1/2*3+(1+1)/2")) ⇒
     (sum (product (value (number "1")))
          "+"
          (sum (product
                 (value (number "1"))
                 "/"
                 (product
                   (value (number "2"))
                   "*"
                   (product (value (number "3")))))
               "+"
               (sum (product
                      (value "("
                             (sum (product (value (number "1")))
                                  "+"
                                  (sum (product (value (number "1")))))
                             ")")
                      "/"
                      (product (value (number "2")))))))

   There is very little wasted effort in this PEG. The ‘number’
nonterminal has to be tagged because otherwise the numbers might run
together with the arithmetic expressions during the string concatenation
stage of parse-tree compression (the parser will see “1” followed by “/”
and decide to call it “1/”).  When in doubt, tag.

   It is very easy to turn these parse trees into lisp expressions:

     (define (parse-sum sum left . rest)
       (if (null? rest)
           (apply parse-product left)
           (list (string->symbol (car rest))
     	    (apply parse-product left)
     	    (apply parse-sum (cadr rest)))))

     (define (parse-product product left . rest)
       (if (null? rest)
           (apply parse-value left)
           (list (string->symbol (car rest))
     	    (apply parse-value left)
     	    (apply parse-product (cadr rest)))))

     (define (parse-value value first . rest)
       (if (null? rest)
           (string->number (cadr first))
           (apply parse-sum (car rest))))

     (define parse-expr parse-sum)

   (Notice all these functions look very similar; for a more complicated
PEG, it would be worth abstracting.)

   Then:
     (apply parse-expr (peg:tree (match-pattern expr "1+1/2*3+(1+1)/2"))) ⇒
     (+ 1 (+ (/ 1 (* 2 3)) (/ (+ 1 1) 2)))

   But wait!  The associativity is wrong!  Where it says ‘(/ 1 (* 2
3))’, it should say ‘(* (/ 1 2) 3)’.

   It’s tempting to try replacing e.g.  ‘"sum <-- (product ('+' / '-')
sum) / product"’ with ‘"sum <-- (sum ('+' / '-') product) / product"’,
but this is a Bad Idea.  PEGs don’t support left recursion.  To see why,
imagine what the parser will do here.  When it tries to parse ‘sum’, it
first has to try and parse ‘sum’.  But to do that, it first has to try
and parse ‘sum’.  This will continue until the stack gets blown off.

   So how does one parse left-associative binary operators with PEGs?
Honestly, this is one of their major shortcomings.  There’s no
general-purpose way of doing this, but here the repetition operators are
a good choice:

     (use-modules (srfi srfi-1))

     (define-peg-string-patterns
       "expr <- sum
     sum <-- (product ('+' / '-'))* product
     product <-- (value ('*' / '/'))* value
     value <-- number / '(' expr ')'
     number <-- [0-9]+")

     ;; take a deep breath...
     (define (make-left-parser next-func)
       (lambda (sum first . rest) ;; general form, comments below assume
         ;; that we're dealing with a sum expression
         (if (null? rest) ;; form (sum (product ...))
           (apply next-func first)
           (if (string? (cadr first));; form (sum ((product ...) "+") (product ...))
     	  (list (string->symbol (cadr first))
     		(apply next-func (car first))
     		(apply next-func (car rest)))
               ;; form (sum (((product ...) "+") ((product ...) "+")) (product ...))
     	  (car
     	   (reduce ;; walk through the list and build a left-associative tree
     	    (lambda (l r)
     	      (list (list (cadr r) (car r) (apply next-func (car l)))
     		    (string->symbol (cadr l))))
     	    'ignore
     	    (append ;; make a list of all the products
                  ;; the first one should be pre-parsed
     	     (list (list (apply next-func (caar first))
     			 (string->symbol (cadar first))))
     	     (cdr first)
                  ;; the last one has to be added in
     	     (list (append rest '("done"))))))))))

     (define (parse-value value first . rest)
       (if (null? rest)
           (string->number (cadr first))
           (apply parse-sum (car rest))))
     (define parse-product (make-left-parser parse-value))
     (define parse-sum (make-left-parser parse-product))
     (define parse-expr parse-sum)

   Then:
     (apply parse-expr (peg:tree (match-pattern expr "1+1/2*3+(1+1)/2"))) ⇒
     (+ (+ 1 (* (/ 1 2) 3)) (/ (+ 1 1) 2))

   As you can see, this is much uglier (it could be made prettier by
using ‘context-flatten’, but the way it’s written above makes it clear
how we deal with the three ways the zero-or-more ‘*’ expression can
parse).  Fortunately, most of the time we can get away with only using
right-associativity.

Simplified Functions
....................

For a more tantalizing example, consider the following grammar that
parses (highly) simplified C functions:

     (define-peg-string-patterns
       "cfunc <-- cSP ctype cSP cname cSP cargs cLB cSP cbody cRB
     ctype <-- cidentifier
     cname <-- cidentifier
     cargs <-- cLP (! (cSP cRP) carg cSP (cCOMMA / cRP) cSP)* cSP
     carg <-- cSP ctype cSP cname
     cbody <-- cstatement *
     cidentifier <- [a-zA-z][a-zA-Z0-9_]*
     cstatement <-- (!';'.)*cSC cSP
     cSC < ';'
     cCOMMA < ','
     cLP < '('
     cRP < ')'
     cLB < '{'
     cRB < '}'
     cSP < [ \t\n]*")

   Then:
     (match-pattern cfunc "int square(int a) { return a*a;}") ⇒
     (32
      (cfunc (ctype "int")
             (cname "square")
             (cargs (carg (ctype "int") (cname "a")))
             (cbody (cstatement "return a*a"))))

   And:
     (match-pattern cfunc "int mod(int a, int b) { int c = a/b;return a-b*c; }") ⇒
     (52
      (cfunc (ctype "int")
             (cname "mod")
             (cargs (carg (ctype "int") (cname "a"))
                    (carg (ctype "int") (cname "b")))
             (cbody (cstatement "int c = a/b")
                    (cstatement "return a- b*c"))))

   By wrapping all the ‘carg’ nonterminals in a ‘cargs’ nonterminal, we
were able to remove any ambiguity in the parsing structure and avoid
having to call ‘context-flatten’ on the output of ‘match-pattern’.  We
used the same trick with the ‘cstatement’ nonterminals, wrapping them in
a ‘cbody’ nonterminal.

   The whitespace nonterminal ‘cSP’ used here is a (very) useful
instantiation of a common pattern for matching syntactically irrelevant
information.  Since it’s tagged with ‘<’ and ends with ‘*’ it won’t
clutter up the parse trees (all the empty lists will be discarded during
the compression step) and it will never cause parsing to fail.


File: guile.info,  Node: PEG Internals,  Prev: PEG Tutorial,  Up: PEG Parsing

6.15.4 PEG Internals
--------------------

A PEG parser takes a string as input and attempts to parse it as a given
nonterminal.  The key idea of the PEG implementation is that every
nonterminal is just a function that takes a string as an argument and
attempts to parse that string as its nonterminal.  The functions always
start from the beginning, but a parse is considered successful if there
is material left over at the end.

   This makes it easy to model different PEG parsing operations.  For
instance, consider the PEG grammar ‘"ab"’, which could also be written
‘(and "a" "b")’.  It matches the string “ab”.  Here’s how that might be
implemented in the PEG style:

     (define (match-and-a-b str)
       (match-a str)
       (match-b str))

   As you can see, the use of functions provides an easy way to model
sequencing.  In a similar way, one could model ‘(or a b)’ with something
like the following:

     (define (match-or-a-b str)
       (or (match-a str) (match-b str)))

   Here the semantics of a PEG ‘or’ expression map naturally onto
Scheme’s ‘or’ operator.  This function will attempt to run ‘(match-a
str)’, and return its result if it succeeds.  Otherwise it will run
‘(match-b str)’.

   Of course, the code above wouldn’t quite work.  We need some way for
the parsing functions to communicate.  The actual interface used is
below.

Parsing Function Interface
..........................

A parsing function takes three arguments - a string, the length of that
string, and the position in that string it should start parsing at.  In
effect, the parsing functions pass around substrings in pieces - the
first argument is a buffer of characters, and the second two give a
range within that buffer that the parsing function should look at.

   Parsing functions return either #f, if they failed to match their
nonterminal, or a list whose first element must be an integer
representing the final position in the string they matched and whose cdr
can be any other data the function wishes to return, or ’() if it
doesn’t have any more data.

   The one caveat is that if the extra data it returns is a list, any
adjacent strings in that list will be appended by ‘match-pattern’.  For
instance, if a parsing function returns ‘(13 ("a" "b" "c"))’,
‘match-pattern’ will take ‘(13 ("abc"))’ as its value.

   For example, here is a function to match “ab” using the actual
interface.

     (define (match-a-b str len pos)
        (and (<= (+ pos 2) len)
             (string= str "ab" pos (+ pos 2))
             (list (+ pos 2) '()))) ; we return no extra information

   The above function can be used to match a string by running
‘(match-pattern match-a-b "ab")’.

Code Generators and Extensible Syntax
.....................................

PEG expressions, such as those in a ‘define-peg-pattern’ form, are
interpreted internally in two steps.

   First, any string PEG is expanded into an s-expression PEG by the
code in the ‘(ice-9 peg string-peg)’ module.

   Then, the s-expression PEG that results is compiled into a parsing
function by the ‘(ice-9 peg codegen)’ module.  In particular, the
function ‘compile-peg-pattern’ is called on the s-expression.  It then
decides what to do based on the form it is passed.

   The PEG syntax can be expanded by providing ‘compile-peg-pattern’
more options for what to do with its forms.  The extended syntax will be
associated with a symbol, for instance ‘my-parsing-form’, and will be
called on all PEG expressions of the form
     (my-parsing-form ...)

   The parsing function should take two arguments.  The first will be a
syntax object containing a list with all of the arguments to the form
(but not the form’s name), and the second will be the ‘capture-type’
argument that is passed to ‘define-peg-pattern’.

   New functions can be registered by calling ‘(add-peg-compiler! symbol
function)’, where ‘symbol’ is the symbol that will indicate a form of
this type and ‘function’ is the code generating function described
above.  The function ‘add-peg-compiler!’ is exported from the ‘(ice-9
peg codegen)’ module.


File: guile.info,  Node: Read/Load/Eval/Compile,  Next: Memory Management,  Prev: PEG Parsing,  Up: API Reference

6.16 Reading and Evaluating Scheme Code
=======================================

This chapter describes Guile functions that are concerned with reading,
loading, evaluating, and compiling Scheme code at run time.

* Menu:

* Scheme Syntax::               Standard and extended Scheme syntax.
* Scheme Read::                 Reading Scheme code.
* Annotated Scheme Read::       Reading Scheme code, for the compiler.
* Scheme Write::                Writing Scheme values to a port.
* Fly Evaluation::              Procedures for on the fly evaluation.
* Compilation::                 How to compile Scheme files and procedures.
* Loading::                     Loading Scheme code from file.
* Load Paths::                  Where Guile looks for code.
* Character Encoding of Source Files:: Loading non-ASCII Scheme code from file.
* Delayed Evaluation::          Postponing evaluation until it is needed.
* Local Evaluation::            Evaluation in a local lexical environment.
* Local Inclusion::             Compile-time inclusion of one file in another.
* Sandboxed Evaluation::        Evaluation with limited capabilities.
* REPL Servers::                Serving a REPL over a socket.
* Cooperative REPL Servers::    REPL server for single-threaded applications.


File: guile.info,  Node: Scheme Syntax,  Next: Scheme Read,  Up: Read/Load/Eval/Compile

6.16.1 Scheme Syntax: Standard and Guile Extensions
---------------------------------------------------

* Menu:

* Expression Syntax::
* Comments::
* Block Comments::
* Case Sensitivity::
* Keyword Syntax::
* Reader Extensions::


File: guile.info,  Node: Expression Syntax,  Next: Comments,  Up: Scheme Syntax

6.16.1.1 Expression Syntax
..........................

An expression to be evaluated takes one of the following forms.

SYMBOL
     A symbol is evaluated by dereferencing.  A binding of that symbol
     is sought and the value there used.  For example,

          (define x 123)
          x ⇒ 123

(PROC ARGS...)
     A parenthesized expression is a function call.  PROC and each
     argument are evaluated, then the function (which PROC evaluated to)
     is called with those arguments.

     The order in which PROC and the arguments are evaluated is
     unspecified, so be careful when using expressions with side
     effects.

          (max 1 2 3) ⇒ 3

          (define (get-some-proc)  min)
          ((get-some-proc) 1 2 3) ⇒ 1

     The same sort of parenthesized form is used for a macro invocation,
     but in that case the arguments are not evaluated.  See the
     descriptions of macros for more on this (*note Macros::, and *note
     Syntax Rules::).

CONSTANT
     Number, string, character and boolean constants evaluate “to
     themselves”, so can appear as literals.

          123     ⇒ 123
          99.9    ⇒ 99.9
          "hello" ⇒ "hello"
          #\z     ⇒ #\z
          #t      ⇒ #t

     Note that an application must not attempt to modify literal
     strings, since they may be in read-only memory.

(quote DATA)
’DATA
     Quoting is used to obtain a literal symbol (instead of a variable
     reference), a literal list (instead of a function call), or a
     literal vector.  ’ is simply a shorthand for a ‘quote’ form.  For
     example,

          'x                   ⇒ x
          '(1 2 3)             ⇒ (1 2 3)
          '#(1 (2 3) 4)        ⇒ #(1 (2 3) 4)
          (quote x)            ⇒ x
          (quote (1 2 3))      ⇒ (1 2 3)
          (quote #(1 (2 3) 4)) ⇒ #(1 (2 3) 4)

     Note that an application must not attempt to modify literal lists
     or vectors obtained from a ‘quote’ form, since they may be in
     read-only memory.

(quasiquote DATA)
‘DATA
     Backquote quasi-quotation is like ‘quote’, but selected
     sub-expressions are evaluated.  This is a convenient way to
     construct a list or vector structure most of which is constant, but
     at certain points should have expressions substituted.

     The same effect can always be had with suitable ‘list’, ‘cons’ or
     ‘vector’ calls, but quasi-quoting is often easier.

     (unquote EXPR)
     ,EXPR
          Within the quasiquote DATA, ‘unquote’ or ‘,’ indicates an
          expression to be evaluated and inserted.  The comma syntax ‘,’
          is simply a shorthand for an ‘unquote’ form.  For example,

               `(1 2 (* 9 9) 3 4)       ⇒ (1 2 (* 9 9) 3 4)
               `(1 2 ,(* 9 9) 3 4)      ⇒ (1 2 81 3 4)
               `(1 (unquote (+ 1 1)) 3) ⇒ (1 2 3)
               `#(1 ,(/ 12 2))          ⇒ #(1 6)

     (unquote-splicing EXPR)
     ,@EXPR
          Within the quasiquote DATA, ‘unquote-splicing’ or ‘,@’
          indicates an expression to be evaluated and the elements of
          the returned list inserted.  EXPR must evaluate to a list.
          The “comma-at” syntax ‘,@’ is simply a shorthand for an
          ‘unquote-splicing’ form.

               (define x '(2 3))
               `(1 ,x 4)                           ⇒ (1 (2 3) 4)
               `(1 ,@x 4)                         ⇒ (1 2 3 4)
               `(1 (unquote-splicing (map 1+ x)))  ⇒ (1 3 4)
               `#(9 ,@x 9)                        ⇒ #(9 2 3 9)

          Notice ‘,@’ differs from plain ‘,’ in the way one level of
          nesting is stripped.  For ‘,@’ the elements of a returned list
          are inserted, whereas with ‘,’ it would be the list itself
          inserted.


File: guile.info,  Node: Comments,  Next: Block Comments,  Prev: Expression Syntax,  Up: Scheme Syntax

6.16.1.2 Comments
.................

Comments in Scheme source files are written by starting them with a
semicolon character (‘;’).  The comment then reaches up to the end of
the line.  Comments can begin at any column, and the may be inserted on
the same line as Scheme code.

     ; Comment
     ;; Comment too
     (define x 1)        ; Comment after expression
     (let ((y 1))
       ;; Display something.
       (display y)
     ;;; Comment at left margin.
       (display (+ y 1)))

   It is common to use a single semicolon for comments following
expressions on a line, to use two semicolons for comments which are
indented like code, and three semicolons for comments which start at
column 0, even if they are inside an indented code block.  This
convention is used when indenting code in Emacs’ Scheme mode.


File: guile.info,  Node: Block Comments,  Next: Case Sensitivity,  Prev: Comments,  Up: Scheme Syntax

6.16.1.3 Block Comments
.......................

In addition to the standard line comments defined by R5RS, Guile has
another comment type for multiline comments, called “block comments”.
This type of comment begins with the character sequence ‘#!’ and ends
with the characters ‘!#’.

   These comments are compatible with the block comments in the Scheme
Shell ‘scsh’ (*note The Scheme shell (scsh)::).  The characters ‘#!’
were chosen because they are the magic characters used in shell scripts
for indicating that the name of the program for executing the script
follows on the same line.

   Thus a Guile script often starts like this.

     #! /usr/local/bin/guile -s
     !#

   More details on Guile scripting can be found in the scripting section
(*note Guile Scripting::).

   Similarly, Guile (starting from version 2.0) supports nested block
comments as specified by R6RS and SRFI-30
(http://srfi.schemers.org/srfi-30/srfi-30.html):

     (+ 1 #| this is a #| nested |# block comment |# 2)
     ⇒ 3

   For backward compatibility, this syntax can be overridden with
‘read-hash-extend’ (*note ‘read-hash-extend’: Reader Extensions.).

   There is one special case where the contents of a comment can
actually affect the interpretation of code.  When a character encoding
declaration, such as ‘coding: utf-8’ appears in one of the first few
lines of a source file, it indicates to Guile’s default reader that this
source code file is not ASCII. For details see *note Character Encoding
of Source Files::.


File: guile.info,  Node: Case Sensitivity,  Next: Keyword Syntax,  Prev: Block Comments,  Up: Scheme Syntax

6.16.1.4 Case Sensitivity
.........................

Scheme as defined in R5RS is not case sensitive when reading symbols.
Guile, on the contrary is case sensitive by default, so the identifiers

     guile-whuzzy
     Guile-Whuzzy

   are the same in R5RS Scheme, but are different in Guile.

   It is possible to turn off case sensitivity in Guile by setting the
reader option ‘case-insensitive’.  For more information on reader
options, *Note Scheme Read::.

     (read-enable 'case-insensitive)

   It is also possible to disable (or enable) case sensitivity within a
single file by placing the reader directives ‘#!fold-case’ (or
‘#!no-fold-case’) within the file itself.


File: guile.info,  Node: Keyword Syntax,  Next: Reader Extensions,  Prev: Case Sensitivity,  Up: Scheme Syntax

6.16.1.5 Keyword Syntax
.......................


File: guile.info,  Node: Reader Extensions,  Prev: Keyword Syntax,  Up: Scheme Syntax

6.16.1.6 Reader Extensions
..........................

 -- Scheme Procedure: read-hash-extend chr proc
 -- C Function: scm_read_hash_extend (chr, proc)
     Install the procedure PROC for reading expressions starting with
     the character sequence ‘#’ and CHR.  PROC will be called with two
     arguments: the character CHR and the port to read further data
     from.  The object returned will be the return value of ‘read’.
     Passing ‘#f’ for PROC will remove a previous setting.


File: guile.info,  Node: Scheme Read,  Next: Annotated Scheme Read,  Prev: Scheme Syntax,  Up: Read/Load/Eval/Compile

6.16.2 Reading Scheme Code
--------------------------

 -- Scheme Procedure: read [port]
 -- C Function: scm_read (port)
     Read an s-expression from the input port PORT, or from the current
     input port if PORT is not specified.  Any whitespace before the
     next token is discarded.

   The behavior of Guile’s Scheme reader can be modified by manipulating
its read options.

 -- Scheme Procedure: read-options [setting]
     Display the current settings of the global read options.  If
     SETTING is omitted, only a short form of the current read options
     is printed.  Otherwise if SETTING is the symbol ‘help’, a complete
     options description is displayed.

   The set of available options, and their default values, may be had by
invoking ‘read-options’ at the prompt.

     scheme@(guile-user)> (read-options)
     (square-brackets keywords #f positions)
     scheme@(guile-user)> (read-options 'help)
     positions         yes   Record positions of source code expressions.
     case-insensitive  no    Convert symbols to lower case.
     keywords          #f    Style of keyword recognition: #f, 'prefix or 'postfix.
     r6rs-hex-escapes  no    Use R6RS variable-length character and string hex escapes.
     square-brackets   yes   Treat `[' and `]' as parentheses, for R6RS compatibility.
     hungry-eol-escapes no   In strings, consume leading whitespace after an
                             escaped end-of-line.
     curly-infix       no    Support SRFI-105 curly infix expressions.
     r7rs-symbols      no    Support R7RS |...| symbol notation.
     bytestrings       no    Support SRFI-207 #u8"\xce;\xbb; calculus" bytestrings

   Note that Guile also includes a preliminary mechanism for setting
read options on a per-port basis.  For instance, the ‘case-insensitive’
read option is set (or unset) on the port when the reader encounters the
‘#!fold-case’ or ‘#!no-fold-case’ reader directives.  Similarly, the
‘#!curly-infix’ reader directive sets the ‘curly-infix’ read option on
the port, and ‘#!curly-infix-and-bracket-lists’ sets ‘curly-infix’ and
unsets ‘square-brackets’ on the port (*note SRFI-105::).  There is
currently no other way to access or set the per-port read options.

   The boolean options may be toggled with ‘read-enable’ and
‘read-disable’.  The non-boolean ‘keywords’ option must be set using
‘read-set!’.

 -- Scheme Procedure: read-enable option-name
 -- Scheme Procedure: read-disable option-name
 -- Scheme Syntax: read-set! option-name value
     Modify the read options.  ‘read-enable’ should be used with boolean
     options and switches them on, ‘read-disable’ switches them off.

     ‘read-set!’ can be used to set an option to a specific value.  Due
     to historical oddities, it is a macro that expects an unquoted
     option name.

   For example, to make ‘read’ fold all symbols to their lower case
(perhaps for compatibility with older Scheme code), you can enter:

     (read-enable 'case-insensitive)

   For more information on the effect of the ‘r6rs-hex-escapes’ and
‘hungry-eol-escapes’ options, see (*note String Syntax::).

   For more information on the ‘r7rs-symbols’ option, see (*note Symbol
Read Syntax::).


File: guile.info,  Node: Annotated Scheme Read,  Next: Scheme Write,  Prev: Scheme Read,  Up: Read/Load/Eval/Compile

6.16.3 Reading Scheme Code, For the Compiler
--------------------------------------------

When something goes wrong with a Scheme program, the user will want to
know how to fix it.  This starts with identifying where the error
occurred: we want to associate a source location with each component
part of source code, and propagate that source location information
through to the compiler or interpreter.

   For that, Guile provides ‘read-syntax’.

 -- Scheme Procedure: read-syntax [port]
     Read an s-expression from the input port PORT, or from the current
     input port if PORT is not specified.

     If, after skipping white space and comments, no more bytes are
     available from PORT, return the end-of-file object.  *Note Binary
     I/O::.  Otherwise, return an annotated datum.  An annotated datum
     is a syntax object which associates a source location with a datum.
     For example:

          (call-with-input-string "  foo" read-syntax)
          ; ⇒ #<syntax:unknown file:1:2 foo>
          (call-with-input-string "(foo)" read-syntax)
          ; ⇒
          ; #<syntax:unknown file:1:0
          ;   (#<syntax unknown file:1:1 foo>)>

     As the second example shows, all fields of pairs and vectors are
     also annotated, recursively.

   Most users are familiar with syntax objects in the context of macros,
which use syntax objects to associate scope information with
identifiers.  *Note Macros::.  Here we use syntax objects to associate
source location information with any datum, but without attaching scope
information.  The Scheme compiler (‘compile’) and the interpreter
(‘eval’) can accept syntax objects directly as input, allowing them to
associate source information with resulting code.  *Note Compilation::,
and *Note Fly Evaluation::.

   Note that there is a legacy interface for getting source locations
into the Scheme compiler or interpreter, which is to use a side table
that associates “source properties” with each subdatum returned by
‘read’, instead of wrapping the datums directly as in ‘read-syntax’.
This has the disadvantage of not being able to annotate all kinds of
datums.  *Note Source Properties::, for more information.


File: guile.info,  Node: Scheme Write,  Next: Fly Evaluation,  Prev: Annotated Scheme Read,  Up: Read/Load/Eval/Compile

6.16.4 Writing Scheme Values
----------------------------

Any scheme value may be written to a port.  Not all values may be read
back in (*note Scheme Read::), however.

 -- Scheme Procedure: write obj [port]
     Send a representation of OBJ to PORT or to the current output port
     if not given.

     The output is designed to be machine readable, and can be read back
     with ‘read’ (*note Scheme Read::).  Strings are printed in double
     quotes, with escapes if necessary, and characters are printed in
     ‘#\’ notation.

 -- Scheme Procedure: display obj [port]
     Send a representation of OBJ to PORT or to the current output port
     if not given.

     The output is designed for human readability, it differs from
     ‘write’ in that strings are printed without double quotes and
     escapes, and characters are printed as per ‘write-char’, not in
     ‘#\’ form.

   As was the case with the Scheme reader, there are a few options that
affect the behavior of the Scheme printer.

 -- Scheme Procedure: print-options [setting]
     Display the current settings of the read options.  If SETTING is
     omitted, only a short form of the current read options is printed.
     Otherwise if SETTING is the symbol ‘help’, a complete options
     description is displayed.

   The set of available options, and their default values, may be had by
invoking ‘print-options’ at the prompt.

     scheme@(guile-user)> (print-options)
     (quote-keywordish-symbols reader highlight-suffix "}" highlight-prefix "{")
     scheme@(guile-user)> (print-options 'help)
     highlight-prefix          {       The string to print before highlighted values.
     highlight-suffix          }       The string to print after highlighted values.
     quote-keywordish-symbols  reader  How to print symbols that have a colon
                                       as their first or last character. The
                                       value '#f' does not quote the colons;
                                       '#t' quotes them; 'reader' quotes them
                                       when the reader option 'keywords' is
                                       not '#f'.
     escape-newlines           yes     Render newlines as \n when printing
                                       using `write'.
     r7rs-symbols              no      Escape symbols using R7RS |...| symbol
                                       notation.
     bytestrings               no      Render bytevectors as bytestrings (SRFI-207)

   These options may be modified with the print-set!  syntax.

 -- Scheme Syntax: print-set! option-name value
     Modify the print options.  Due to historical oddities, ‘print-set!’
     is a macro that expects an unquoted option name.


File: guile.info,  Node: Fly Evaluation,  Next: Compilation,  Prev: Scheme Write,  Up: Read/Load/Eval/Compile

6.16.5 Procedures for On the Fly Evaluation
-------------------------------------------

Scheme has the lovely property that its expressions may be represented
as data.  The ‘eval’ procedure takes a Scheme datum and evaluates it as
code.

 -- Scheme Procedure: eval exp module_or_state
 -- C Function: scm_eval (exp, module_or_state)
     Evaluate EXP, a list representing a Scheme expression, in the
     top-level environment specified by MODULE_OR_STATE.  While EXP is
     evaluated (using ‘primitive-eval’), MODULE_OR_STATE is made the
     current module.  The current module is reset to its previous value
     when ‘eval’ returns.  XXX - dynamic states.  Example: (eval ’(+ 1
     2) (interaction-environment))

 -- Scheme Procedure: interaction-environment
 -- C Function: scm_interaction_environment ()
     Return a specifier for the environment that contains
     implementation–defined bindings, typically a superset of those
     listed in the report.  The intent is that this procedure will
     return the environment in which the implementation would evaluate
     expressions dynamically typed by the user.

   *Note Environments::, for other environments.

   One does not always receive code as Scheme data, of course, and this
is especially the case for Guile’s other language implementations (*note
Other Languages::).  For the case in which all you have is a string, we
have ‘eval-string’.  There is a legacy version of this procedure in the
default environment, but you really want the one from ‘(ice-9
eval-string)’, so load it up:

     (use-modules (ice-9 eval-string))

 -- Scheme Procedure: eval-string string [#:module=#f] [#:file=#f]
          [#:line=#f] [#:column=#f] [#:lang=(current-language)]
          [#:compile?=#f]
     Parse STRING according to the current language, normally Scheme.
     Evaluate or compile the expressions it contains, in order,
     returning the last expression.

     If the MODULE keyword argument is set, save a module excursion
     (*note Module System Reflection::) and set the current module to
     MODULE before evaluation.

     The FILE, LINE, and COLUMN keyword arguments can be used to
     indicate that the source string begins at a particular source
     location.

     Finally, LANG is a language, defaulting to the current language,
     and the expression is compiled if COMPILE? is true or there is no
     evaluator for the given language.

 -- C Function: scm_eval_string (string)
 -- C Function: scm_eval_string_in_module (string, module)
     These C bindings call ‘eval-string’ from ‘(ice-9 eval-string)’,
     evaluating within MODULE or the current module.

 -- C Function: SCM scm_c_eval_string (const char *string)
     ‘scm_eval_string’, but taking a C string in locale encoding instead
     of an ‘SCM’.

 -- Scheme Procedure: apply proc arg ... arglst
 -- C Function: scm_apply_0 (proc, arglst)
 -- C Function: scm_apply_1 (proc, arg1, arglst)
 -- C Function: scm_apply_2 (proc, arg1, arg2, arglst)
 -- C Function: scm_apply_3 (proc, arg1, arg2, arg3, arglst)
 -- C Function: scm_apply (proc, arg, rest)
     Call PROC with arguments ARG ... and the elements of the ARGLST
     list.

     ‘scm_apply’ takes parameters corresponding to a Scheme level
     ‘(lambda (proc arg1 . rest) ...)’.  So ARG1 and all but the last
     element of the REST list make up ARG ..., and the last element of
     REST is the ARGLST list.  Or if REST is the empty list ‘SCM_EOL’
     then there’s no ARG ..., and (ARG1) is the ARGLST.

     ARGLST is not modified, but the REST list passed to ‘scm_apply’ is
     modified.

 -- C Function: scm_call_0 (proc)
 -- C Function: scm_call_1 (proc, arg1)
 -- C Function: scm_call_2 (proc, arg1, arg2)
 -- C Function: scm_call_3 (proc, arg1, arg2, arg3)
 -- C Function: scm_call_4 (proc, arg1, arg2, arg3, arg4)
 -- C Function: scm_call_5 (proc, arg1, arg2, arg3, arg4, arg5)
 -- C Function: scm_call_6 (proc, arg1, arg2, arg3, arg4, arg5, arg6)
 -- C Function: scm_call_7 (proc, arg1, arg2, arg3, arg4, arg5, arg6,
          arg7)
 -- C Function: scm_call_8 (proc, arg1, arg2, arg3, arg4, arg5, arg6,
          arg7, arg8)
 -- C Function: scm_call_9 (proc, arg1, arg2, arg3, arg4, arg5, arg6,
          arg7, arg8, arg9)
     Call PROC with the given arguments.

 -- C Function: scm_call (proc, ...)
     Call PROC with any number of arguments.  The argument list must be
     terminated by ‘SCM_UNDEFINED’.  For example:

          scm_call (scm_c_public_ref ("guile", "+"),
                    scm_from_int (1),
                    scm_from_int (2),
                    SCM_UNDEFINED);

 -- C Function: scm_call_n (proc, argv, nargs)
     Call PROC with the array of arguments ARGV, as a ‘SCM*’.  The
     length of the arguments should be passed in NARGS, as a ‘size_t’.

 -- Scheme Procedure: primitive-eval exp
 -- C Function: scm_primitive_eval (exp)
     Evaluate EXP in the top-level environment specified by the current
     module.


File: guile.info,  Node: Compilation,  Next: Loading,  Prev: Fly Evaluation,  Up: Read/Load/Eval/Compile

6.16.6 Compiling Scheme Code
----------------------------

The ‘eval’ procedure directly interprets the S-expression representation
of Scheme.  An alternate strategy for evaluation is to determine ahead
of time what computations will be necessary to evaluate the expression,
and then use that recipe to produce the desired results.  This is known
as “compilation”.

   While it is possible to compile simple Scheme expressions such as ‘(+
2 2)’ or even ‘"Hello world!"’, compilation is most interesting in the
context of procedures.  Compiling a lambda expression produces a
compiled procedure, which is just like a normal procedure except
typically much faster, because it can bypass the generic interpreter.

   Functions from system modules in a Guile installation are normally
compiled already, so they load and run quickly.

   Note that well-written Scheme programs will not typically call the
procedures in this section, for the same reason that it is often bad
taste to use ‘eval’.  By default, Guile automatically compiles any files
it encounters that have not been compiled yet (*note ‘--auto-compile’:
Invoking Guile.).  The compiler can also be invoked explicitly from the
shell as ‘guild compile foo.scm’.

   (Why are calls to ‘eval’ and ‘compile’ usually in bad taste?  Because
they are limited, in that they can only really make sense for top-level
expressions.  Also, most needs for “compile-time” computation are
fulfilled by macros and closures.  Of course one good counterexample is
the REPL itself, or any code that reads expressions from a port.)

   Automatic compilation generally works transparently, without any need
for user intervention.  However Guile does not yet do proper dependency
tracking, so that if file ‘A.scm’ uses macros from ‘B.scm’, and B.SCM
changes, ‘A.scm’ would not be automatically recompiled.  To forcibly
invalidate the auto-compilation cache, pass the ‘--fresh-auto-compile’
option to Guile, or set the ‘GUILE_AUTO_COMPILE’ environment variable to
‘fresh’ (instead of to ‘0’ or ‘1’).

   For more information on the compiler itself, see *note Compiling to
the Virtual Machine::.  For information on the virtual machine, see
*note A Virtual Machine for Guile::.

   The command-line interface to Guile’s compiler is the ‘guild compile’
command:

 -- Command: guild compile [option...] FILE...
     Compile FILE, a source file, and store bytecode in the compilation
     cache or in the file specified by the ‘-o’ option.  The following
     options are available:

     ‘-L DIR’
     ‘--load-path=DIR’
          Add DIR to the front of the module load path.

     ‘-o OFILE’
     ‘--output=OFILE’
          Write output bytecode to OFILE.  By convention, bytecode file
          names end in ‘.go’.  When ‘-o’ is omitted, the output file
          name is as for ‘compile-file’ (see below).

     ‘-x EXTENSION’
          Recognize EXTENSION as a valid source file name extension.

          For example, to compile R6RS code, you might want to pass ‘-x
          .sls’ so that files ending in ‘.sls’ can be found.

     ‘-W WARNING’
     ‘--warn=WARNING’
          Enable specific warning passes; use ‘-Whelp’ for a list of
          available options.  The default is ‘-W1’, which enables a
          number of common warnings.  Pass ‘-W0’ to disable all
          warnings.

     ‘-O OPT’
     ‘--optimize=OPT’
          Enable or disable specific compiler optimizations; use
          ‘-Ohelp’ for a list of available options.  The default is
          ‘-O2’, which enables most optimizations.  ‘-O0’ is recommended
          if compilation speed is more important than the speed of the
          compiled code.  Pass ‘-Ono-OPT’ to disable a specific compiler
          pass.  Any number of ‘-O’ options can be passed to the
          compiler, with later ones taking precedence.

     ‘--r6rs’
     ‘--r7rs’
          Compile in an environment whose default bindings, reader
          options, and load paths are adapted for specific Scheme
          standards.  *Note R6RS Support::, and *Note R7RS Support::.

     ‘-f LANG’
     ‘--from=LANG’
          Use LANG as the source language of FILE.  If this option is
          omitted, ‘scheme’ is assumed.

     ‘-t LANG’
     ‘--to=LANG’
          Use LANG as the target language of FILE.  If this option is
          omitted, ‘rtl’ is assumed.

     ‘-T TARGET’
     ‘--target=TARGET’
          Produce code for TARGET instead of %HOST-TYPE (*note
          %host-type: Build Config.).  Target must be a valid GNU
          triplet, such as ‘armv5tel-unknown-linux-gnueabi’ (*note
          (autoconf)Specifying Target Triplets::).

     Each FILE is assumed to be UTF-8-encoded, unless it contains a
     coding declaration as recognized by ‘file-encoding’ (*note
     Character Encoding of Source Files::).

   The compiler can also be invoked directly by Scheme code.  These
interfaces are in their own module:

     (use-modules (system base compile))

 -- Scheme Procedure: compile exp [#:env=#f] [#:from=(current-language)]
          [#:to=value] [#:opts='()]
          [#:optimization-level=(default-optimization-level)]
          [#:warning-level=(default-warning-level)]
     Compile the expression EXP in the environment ENV.  If EXP is a
     procedure, the result will be a compiled procedure; otherwise
     ‘compile’ is mostly equivalent to ‘eval’.

     For a discussion of languages and compiler options, *Note Compiling
     to the Virtual Machine::.

 -- Scheme Procedure: compile-file file [#:output-file=#f]
          [#:from=(current-language)] [#:to='rtl]
          [#:env=(default-environment from)] [#:opts='()]
          [#:optimization-level=(default-optimization-level)]
          [#:warning-level=(default-warning-level)]
          [#:canonicalization='relative]
     Compile the file named FILE.

     Output will be written to a OUTPUT-FILE.  If you do not supply an
     output file name, output is written to a file in the cache
     directory, as computed by ‘(compiled-file-name FILE)’.

     FROM and TO specify the source and target languages.  *Note
     Compiling to the Virtual Machine::, for more information on these
     options, and on ENV and OPTS.

     As with ‘guild compile’, FILE is assumed to be UTF-8-encoded unless
     it contains a coding declaration.

 -- Scheme Parameter: default-optimization-level
     The default optimization level, as an integer from 0 to 9.  The
     default is 2.
 -- Scheme Parameter: default-warning-level
     The default warning level, as an integer from 0 to 9.  The default
     is 1.

   *Note Parameters::, for more on how to set parameters.

 -- Scheme Procedure: compiled-file-name file
     Compute a cached location for a compiled version of a Scheme file
     named FILE.

     This file will usually be below the ‘$HOME/.cache/guile/ccache’
     directory, depending on the value of the ‘XDG_CACHE_HOME’
     environment variable.  The intention is that ‘compiled-file-name’
     provides a fallback location for caching auto-compiled files.  If
     you want to place a compile file in the ‘%load-compiled-path’, you
     should pass the OUTPUT-FILE option to ‘compile-file’, explicitly.

 -- Scheme Variable: %auto-compilation-options
     This variable contains the options passed to the ‘compile-file’
     procedure when auto-compiling source files.  By default, it enables
     useful compilation warnings.  It can be customized from ‘~/.guile’.


File: guile.info,  Node: Loading,  Next: Load Paths,  Prev: Compilation,  Up: Read/Load/Eval/Compile

6.16.7 Loading Scheme Code from File
------------------------------------

 -- Scheme Procedure: load filename [reader]
     Load FILENAME and evaluate its contents in the top-level
     environment.

     READER if provided should be either ‘#f’, or a procedure with the
     signature ‘(lambda (port) ...)’ which reads the next expression
     from PORT.  If READER is ‘#f’ or absent, Guile’s built-in ‘read’
     procedure is used (*note Scheme Read::).

     The READER argument takes effect by setting the value of the
     ‘current-reader’ fluid (see below) before loading the file, and
     restoring its previous value when loading is complete.  The Scheme
     code inside FILENAME can itself change the current reader procedure
     on the fly by setting ‘current-reader’ fluid.

     If the variable ‘%load-hook’ is defined, it should be bound to a
     procedure that will be called before any code is loaded.  See
     documentation for ‘%load-hook’ later in this section.

 -- Scheme Procedure: load-compiled filename
     Load the compiled file named FILENAME.

     Compiling a source file (*note Read/Load/Eval/Compile::) and then
     calling ‘load-compiled’ on the resulting file is equivalent to
     calling ‘load’ on the source file.

 -- Scheme Procedure: primitive-load filename
 -- C Function: scm_primitive_load (filename)
     Load the file named FILENAME and evaluate its contents in the
     top-level environment.  FILENAME must either be a full pathname or
     be a pathname relative to the current directory.  If the variable
     ‘%load-hook’ is defined, it should be bound to a procedure that
     will be called before any code is loaded.  See the documentation
     for ‘%load-hook’ later in this section.

 -- C Function: SCM scm_c_primitive_load (const char *filename)
     ‘scm_primitive_load’, but taking a C string instead of an ‘SCM’.

 -- Variable: current-reader
     ‘current-reader’ holds the read procedure that is currently being
     used by the above loading procedures to read expressions (from the
     file that they are loading).  ‘current-reader’ is a fluid, so it
     has an independent value in each dynamic root and should be read
     and set using ‘fluid-ref’ and ‘fluid-set!’ (*note Fluids and
     Dynamic States::).

     Changing ‘current-reader’ is typically useful to introduce local
     syntactic changes, such that code following the ‘fluid-set!’ call
     is read using the newly installed reader.  The ‘current-reader’
     change should take place at evaluation time when the code is
     evaluated, or at compilation time when the code is compiled:

          (eval-when (compile eval)
            (fluid-set! current-reader my-own-reader))

     The ‘eval-when’ form above ensures that the ‘current-reader’ change
     occurs at the right time.

 -- Variable: %load-hook
     A procedure to be called ‘(%load-hook FILENAME)’ whenever a file is
     loaded, or ‘#f’ for no such call.  ‘%load-hook’ is used by all of
     the loading functions (‘load’ and ‘primitive-load’, and
     ‘load-from-path’ and ‘primitive-load-path’ documented in the next
     section).

     For example an application can set this to show what’s loaded,

          (set! %load-hook (lambda (filename)
                             (format #t "Loading ~a ...\n" filename)))
          (load-from-path "foo.scm")
          ⊣ Loading /usr/local/share/guile/site/foo.scm ...

 -- Scheme Procedure: current-load-port
 -- C Function: scm_current_load_port ()
     Return the current-load-port.  The load port is used internally by
     ‘primitive-load’.


File: guile.info,  Node: Load Paths,  Next: Character Encoding of Source Files,  Prev: Loading,  Up: Read/Load/Eval/Compile

6.16.8 Load Paths
-----------------

The procedure in the previous section look for Scheme code in the file
system at specific location.  Guile also has some procedures to search
the load path for code.

 -- Variable: %load-path
     List of directories which should be searched for Scheme modules and
     libraries.  When Guile starts up, ‘%load-path’ is initialized to
     the default load path ‘(list (%library-dir) (%site-dir)
     (%global-site-dir) (%package-data-dir))’.  The ‘GUILE_LOAD_PATH’
     environment variable can be used to prepend or append additional
     directories (*note Environment Variables::).

     *Note Build Config::, for more on ‘%site-dir’ and related
     procedures.

 -- Scheme Procedure: load-from-path filename
     Similar to ‘load’, but searches for FILENAME in the load paths.
     Preferentially loads a compiled version of the file, if it is
     available and up-to-date.

   A user can extend the load path by calling ‘add-to-load-path’.

 -- Scheme Syntax: add-to-load-path dir
     Add DIR to the load path.

   For example, a script might include this form to add the directory
that it is in to the load path:

     (add-to-load-path (dirname (current-filename)))

   It’s better to use ‘add-to-load-path’ than to modify ‘%load-path’
directly, because ‘add-to-load-path’ takes care of modifying the path
both at compile-time and at run-time.

 -- Scheme Procedure: primitive-load-path filename
          [exception-on-not-found]
 -- C Function: scm_primitive_load_path (filename)
     Search ‘%load-path’ for the file named FILENAME and load it into
     the top-level environment.  If FILENAME is a relative pathname and
     is not found in the list of search paths, an error is signaled.
     Preferentially loads a compiled version of the file, if it is
     available and up-to-date.

     If FILENAME is a relative pathname and is not found in the list of
     search paths, one of three things may happen, depending on the
     optional second argument, EXCEPTION-ON-NOT-FOUND.  If it is ‘#f’,
     ‘#f’ will be returned.  If it is a procedure, it will be called
     with no arguments.  (This allows a distinction to be made between
     exceptions raised by loading a file, and exceptions related to the
     loader itself.)  Otherwise an error is signaled.

     For compatibility with Guile 1.8 and earlier, the C function takes
     only one argument, which can be either a string (the file name) or
     an argument list.

 -- Scheme Procedure: %search-load-path filename
 -- C Function: scm_sys_search_load_path (filename)
     Search ‘%load-path’ for the file named FILENAME, which must be
     readable by the current user.  If FILENAME is found in the list of
     paths to search or is an absolute pathname, return its full
     pathname.  Otherwise, return ‘#f’.  Filenames may have any of the
     optional extensions in the ‘%load-extensions’ list;
     ‘%search-load-path’ will try each extension automatically.

 -- Variable: %load-extensions
     A list of default file extensions for files containing Scheme code.
     ‘%search-load-path’ tries each of these extensions when looking for
     a file to load.  By default, ‘%load-extensions’ is bound to the
     list ‘("" ".scm")’.

   As mentioned above, when Guile searches the ‘%load-path’ for a source
file, it will also search the ‘%load-compiled-path’ for a corresponding
compiled file.  If the compiled file is as new or newer than the source
file, it will be loaded instead of the source file, using
‘load-compiled’.

 -- Variable: %load-compiled-path
     Like ‘%load-path’, but for compiled files.  By default, this path
     has two entries: one for compiled files from Guile itself, and one
     for site packages.  The ‘GUILE_LOAD_COMPILED_PATH’ environment
     variable can be used to prepend or append additional directories
     (*note Environment Variables::).

   When ‘primitive-load-path’ searches the ‘%load-compiled-path’ for a
corresponding compiled file for a relative path it does so by appending
‘.go’ to the relative path.  For example, searching for ‘ice-9/popen’
could find ‘/usr/lib/guile/3.0/ccache/ice-9/popen.go’, and use it
instead of ‘/usr/share/guile/3.0/ice-9/popen.scm’.

   If ‘primitive-load-path’ does not find a corresponding ‘.go’ file in
the ‘%load-compiled-path’, or the ‘.go’ file is out of date, it will
search for a corresponding auto-compiled file in the fallback path,
possibly creating one if one does not exist.

   *Note Installing Site Packages::, for more on how to correctly
install site packages.  *Note Modules and the File System::, for more on
the relationship between load paths and modules.  *Note Compilation::,
for more on the fallback path and auto-compilation.

   Finally, there are a couple of helper procedures for general path
manipulation.

 -- Scheme Procedure: parse-path path [tail]
 -- C Function: scm_parse_path (path, tail)
     Parse PATH, which is expected to be a colon-separated string, into
     a list and return the resulting list with TAIL appended.  If PATH
     is ‘#f’, TAIL is returned.

 -- Scheme Procedure: parse-path-with-ellipsis path base
 -- C Function: scm_parse_path_with_ellipsis (path, base)
     Parse PATH, which is expected to be a colon-separated string, into
     a list and return the resulting list with BASE (a list) spliced in
     place of the ‘...’ path component, if present, or else BASE is
     added to the end.  If PATH is ‘#f’, BASE is returned.

 -- Scheme Procedure: search-path path filename [extensions
          [require-exts?]]
 -- C Function: scm_search_path (path, filename, rest)
     Search PATH for a directory containing a file named FILENAME.  The
     file must be readable, and not a directory.  If we find one, return
     its full filename; otherwise, return ‘#f’.  If FILENAME is
     absolute, return it unchanged.  If given, EXTENSIONS is a list of
     strings; for each directory in PATH, we search for FILENAME
     concatenated with each EXTENSION.  If REQUIRE-EXTS? is true,
     require that the returned file name have one of the given
     extensions; if REQUIRE-EXTS? is not given, it defaults to ‘#f’.

     For compatibility with Guile 1.8 and earlier, the C function takes
     only three arguments.


File: guile.info,  Node: Character Encoding of Source Files,  Next: Delayed Evaluation,  Prev: Load Paths,  Up: Read/Load/Eval/Compile

6.16.9 Character Encoding of Source Files
-----------------------------------------

Scheme source code files are usually encoded in ASCII or UTF-8, but the
built-in reader can interpret other character encodings as well.  When
Guile loads Scheme source code, it uses the ‘file-encoding’ procedure
(described below) to try to guess the encoding of the file.  In the
absence of any hints, UTF-8 is assumed.  One way to provide a hint about
the encoding of a source file is to place a coding declaration in the
top 500 characters of the file.

   A coding declaration has the form ‘coding: XXXXXX’, where ‘XXXXXX’ is
the name of a character encoding in which the source code file has been
encoded.  The coding declaration must appear in a scheme comment.  It
can either be a semicolon-initiated comment, or the first block ‘#!’
comment in the file.

   The name of the character encoding in the coding declaration is
typically lower case and containing only letters, numbers, and hyphens,
as recognized by ‘set-port-encoding!’ (*note ‘set-port-encoding!’:
Ports.).  Common examples of character encoding names are ‘utf-8’ and
‘iso-8859-1’, as defined by IANA
(http://www.iana.org/assignments/character-sets).  Thus, the coding
declaration is mostly compatible with Emacs.

   However, there are some differences in encoding names recognized by
Emacs and encoding names defined by IANA, the latter being essentially a
subset of the former.  For instance, ‘latin-1’ is a valid encoding name
for Emacs, but it’s not according to the IANA standard, which Guile
follows; instead, you should use ‘iso-8859-1’, which is both understood
by Emacs and dubbed by IANA (IANA writes it uppercase but Emacs wants it
lowercase and Guile is case insensitive.)

   For source code, only a subset of all possible character encodings
can be interpreted by the built-in source code reader.  Only those
character encodings in which ASCII text appears unmodified can be used.
This includes ‘UTF-8’ and ‘ISO-8859-1’ through ‘ISO-8859-15’.  The
multi-byte character encodings ‘UTF-16’ and ‘UTF-32’ may not be used
because they are not compatible with ASCII.

   There might be a scenario in which one would want to read non-ASCII
code from a port, such as with the function ‘read’, instead of with
‘load’.  If the port’s character encoding is the same as the encoding of
the code to be read by the port, not other special handling is
necessary.  The port will automatically do the character encoding
conversion.  The functions ‘setlocale’ or by ‘set-port-encoding!’ are
used to set port encodings (*note Ports::).

   If a port is used to read code of unknown character encoding, it can
accomplish this in three steps.  First, the character encoding of the
port should be set to ISO-8859-1 using ‘set-port-encoding!’.  Then, the
procedure ‘file-encoding’, described below, is used to scan for a coding
declaration when reading from the port.  As a side effect, it rewinds
the port after its scan is complete.  After that, the port’s character
encoding should be set to the encoding returned by ‘file-encoding’, if
any, again by using ‘set-port-encoding!’.  Then the code can be read as
normal.

   Alternatively, one can use the ‘#:guess-encoding’ keyword argument of
‘open-file’ and related procedures.  *Note File Ports::.

 -- Scheme Procedure: file-encoding port
 -- C Function: scm_file_encoding (port)
     Attempt to scan the first few hundred bytes from the PORT for hints
     about its character encoding.  Return a string containing the
     encoding name or ‘#f’ if the encoding cannot be determined.  The
     port is rewound.

     Currently, the only supported method is to look for an Emacs-like
     character coding declaration (*note how Emacs recognizes file
     encoding: (emacs)Recognize Coding.).  The coding declaration is of
     the form ‘coding: XXXXX’ and must appear in a Scheme comment.
     Additional heuristics may be added in the future.


File: guile.info,  Node: Delayed Evaluation,  Next: Local Evaluation,  Prev: Character Encoding of Source Files,  Up: Read/Load/Eval/Compile

6.16.10 Delayed Evaluation
--------------------------

Promises are a convenient way to defer a calculation until its result is
actually needed, and to run such a calculation only once.  Also *note
SRFI-45::.

 -- syntax: delay expr
     Return a promise object which holds the given EXPR expression,
     ready to be evaluated by a later ‘force’.

 -- Scheme Procedure: promise? obj
 -- C Function: scm_promise_p (obj)
     Return true if OBJ is a promise.

 -- Scheme Procedure: force p
 -- C Function: scm_force (p)
     Return the value obtained from evaluating the EXPR in the given
     promise P.  If P has previously been forced then its EXPR is not
     evaluated again, instead the value obtained at that time is simply
     returned.

     During a ‘force’, an EXPR can call ‘force’ again on its own
     promise, resulting in a recursive evaluation of that EXPR.  The
     first evaluation to return gives the value for the promise.  Higher
     evaluations run to completion in the normal way, but their results
     are ignored, ‘force’ always returns the first value.


File: guile.info,  Node: Local Evaluation,  Next: Local Inclusion,  Prev: Delayed Evaluation,  Up: Read/Load/Eval/Compile

6.16.11 Local Evaluation
------------------------

Guile includes a facility to capture a lexical environment, and later
evaluate a new expression within that environment.  This code is
implemented in a module.

     (use-modules (ice-9 local-eval))

 -- syntax: the-environment
     Captures and returns a lexical environment for use with
     ‘local-eval’ or ‘local-compile’.

 -- Scheme Procedure: local-eval exp env
 -- C Function: scm_local_eval (exp, env)
 -- Scheme Procedure: local-compile exp env [opts=()]
     Evaluate or compile the expression EXP in the lexical environment
     ENV.

   Here is a simple example, illustrating that it is the variable that
gets captured, not just its value at one point in time.

     (define e (let ((x 100)) (the-environment)))
     (define fetch-x (local-eval '(lambda () x) e))
     (fetch-x)
     ⇒ 100
     (local-eval '(set! x 42) e)
     (fetch-x)
     ⇒ 42

   While EXP is evaluated within the lexical environment of
‘(the-environment)’, it has the dynamic environment of the call to
‘local-eval’.

   ‘local-eval’ and ‘local-compile’ can only evaluate expressions, not
definitions.

     (local-eval '(define foo 42)
                 (let ((x 100)) (the-environment)))
     ⇒ syntax error: definition in expression context

   Note that the current implementation of ‘(the-environment)’ only
captures “normal” lexical bindings, and pattern variables bound by
‘syntax-case’.  It does not currently capture local syntax transformers
bound by ‘let-syntax’, ‘letrec-syntax’ or non-top-level ‘define-syntax’
forms.  Any attempt to reference such captured syntactic keywords via
‘local-eval’ or ‘local-compile’ produces an error.


File: guile.info,  Node: Local Inclusion,  Next: Sandboxed Evaluation,  Prev: Local Evaluation,  Up: Read/Load/Eval/Compile

6.16.12 Local Inclusion
-----------------------

This section has discussed various means of linking Scheme code
together: fundamentally, loading up files at run-time using ‘load’ and
‘load-compiled’.  Guile provides another option to compose parts of
programs together at expansion-time instead of at run-time.

 -- Scheme Syntax: include file-name
     Open FILE-NAME, at expansion-time, and read the Scheme forms that
     it contains, splicing them into the location of the ‘include’,
     within a ‘begin’.

     If FILE-NAME is a relative path, it is searched for relative to the
     path that contains the file that the ‘include’ form appears in.

   If you are a C programmer, if ‘load’ in Scheme is like ‘dlopen’ in C,
consider ‘include’ to be like the C preprocessor’s ‘#include’.  When you
use ‘include’, it is as if the contents of the included file were typed
in instead of the ‘include’ form.

   Because the code is included at compile-time, it is available to the
macroexpander.  Syntax definitions in the included file are available to
later code in the form in which the ‘include’ appears, without the need
for ‘eval-when’.  (*Note Eval When::.)

   For the same reason, compiling a form that uses ‘include’ results in
one compilation unit, composed of multiple files.  Loading the compiled
file is one ‘stat’ operation for the compilation unit, instead of ‘2*N’
in the case of ‘load’ (once for each loaded source file, and once each
corresponding compiled file, in the best case).

   Unlike ‘load’, ‘include’ also works within nested lexical contexts.
It so happens that the optimizer works best within a lexical context,
because all of the uses of bindings in a lexical context are visible, so
composing files by including them within a ‘(let () ...)’ can sometimes
lead to important speed improvements.

   On the other hand, ‘include’ does have all the disadvantages of early
binding: once the code with the ‘include’ is compiled, no change to the
included file is reflected in the future behavior of the including form.

   Also, the particular form of ‘include’, which requires an absolute
path, or a path relative to the current directory at compile-time, is
not very amenable to compiling the source in one place, but then
installing the source to another place.  For this reason, Guile provides
another form, ‘include-from-path’, which looks for the source file to
include within a load path.

 -- Scheme Syntax: include-from-path file-name
     Like ‘include’, but instead of expecting ‘file-name’ to be an
     absolute file name, it is expected to be a relative path to search
     in the ‘%load-path’.

   ‘include-from-path’ is more useful when you want to install all of
the source files for a package (as you should!).  It makes it possible
to evaluate an installed file from source, instead of relying on the
‘.go’ file being up to date.


File: guile.info,  Node: Sandboxed Evaluation,  Next: REPL Servers,  Prev: Local Inclusion,  Up: Read/Load/Eval/Compile

6.16.13 Sandboxed Evaluation
----------------------------

Sometimes you would like to evaluate code that comes from an untrusted
party.  The safest way to do this is to buy a new computer, evaluate the
code on that computer, then throw the machine away.  However if you are
unwilling to take this simple approach, Guile does include a limited
“sandbox” facility that can allow untrusted code to be evaluated with
some confidence.

   To use the sandboxed evaluator, load its module:

     (use-modules (ice-9 sandbox))

   Guile’s sandboxing facility starts with the ability to restrict the
time and space used by a piece of code.

 -- Scheme Procedure: call-with-time-limit limit thunk limit-reached
     Call THUNK, but cancel it if LIMIT seconds of wall-clock time have
     elapsed.  If the computation is canceled, call LIMIT-REACHED in
     tail position.  THUNK must not disable interrupts or prevent an
     abort via a ‘dynamic-wind’ unwind handler.

 -- Scheme Procedure: call-with-allocation-limit limit thunk
          limit-reached
     Call THUNK, but cancel it if LIMIT bytes have been allocated.  If
     the computation is canceled, call LIMIT-REACHED in tail position.
     THUNK must not disable interrupts or prevent an abort via a
     ‘dynamic-wind’ unwind handler.

     This limit applies to both stack and heap allocation.  The
     computation will not be aborted before LIMIT bytes have been
     allocated, but for the heap allocation limit, the check may be
     postponed until the next garbage collection.

     Note that as a current shortcoming, the heap size limit applies to
     all threads; concurrent allocation by other unrelated threads
     counts towards the allocation limit.

 -- Scheme Procedure: call-with-time-and-allocation-limits time-limit
          allocation-limit thunk
     Invoke THUNK in a dynamic extent in which its execution is limited
     to TIME-LIMIT seconds of wall-clock time, and its allocation to
     ALLOCATION-LIMIT bytes.  THUNK must not disable interrupts or
     prevent an abort via a ‘dynamic-wind’ unwind handler.

     If successful, return all values produced by invoking THUNK.  Any
     uncaught exception thrown by the thunk will propagate out.  If the
     time or allocation limit is exceeded, an exception will be thrown
     to the ‘limit-exceeded’ key.

   The time limit and stack limit are both very precise, but the heap
limit only gets checked asynchronously, after a garbage collection.  In
particular, if the heap is already very large, the number of allocated
bytes between garbage collections will be large, and therefore the
precision of the check is reduced.

   Additionally, due to the mechanism used by the allocation limit (the
‘after-gc-hook’), large single allocations like ‘(make-vector #e1e7)’
are only detected after the allocation completes, even if the allocation
itself causes garbage collection.  It’s possible therefore for user code
to not only exceed the allocation limit set, but also to exhaust all
available memory, causing out-of-memory conditions at any allocation
site.  Failure to allocate memory in Guile itself should be safe and
cause an exception to be thrown, but most systems are not designed to
handle ‘malloc’ failures.  An allocation failure may therefore exercise
unexpected code paths in your system, so it is a weakness of the sandbox
(and therefore an interesting point of attack).

   The main sandbox interface is ‘eval-in-sandbox’.

 -- Scheme Procedure: eval-in-sandbox exp [#:time-limit 0.1]
          [#:allocation-limit #e10e6] [#:bindings all-pure-bindings]
          [#:module (make-sandbox-module bindings)] [#:sever-module? #t]
     Evaluate the Scheme expression EXP within an isolated "sandbox".
     Limit its execution to TIME-LIMIT seconds of wall-clock time, and
     limit its allocation to ALLOCATION-LIMIT bytes.

     The evaluation will occur in MODULE, which defaults to the result
     of calling ‘make-sandbox-module’ on BINDINGS, which itself defaults
     to ‘all-pure-bindings’.  This is the core of the sandbox: creating
     a scope for the expression that is “safe”.

     A safe sandbox module has two characteristics.  Firstly, it will
     not allow the expression being evaluated to avoid being canceled
     due to time or allocation limits.  This ensures that the expression
     terminates in a timely fashion.

     Secondly, a safe sandbox module will prevent the evaluation from
     receiving information from previous evaluations, or from affecting
     future evaluations.  All combinations of binding sets exported by
     ‘(ice-9 sandbox)’ form safe sandbox modules.

     The BINDINGS should be given as a list of import sets.  One import
     set is a list whose car names an interface, like ‘(ice-9 q)’, and
     whose cdr is a list of imports.  An import is either a bare symbol
     or a pair of ‘(OUT . IN)’, where OUT and IN are both symbols and
     denote the name under which a binding is exported from the module,
     and the name under which to make the binding available,
     respectively.  Note that BINDINGS is only used as an input to the
     default initializer for the MODULE argument; if you pass
     ‘#:module’, BINDINGS is unused.  If SEVER-MODULE? is true (the
     default), the module will be unlinked from the global module tree
     after the evaluation returns, to allow MOD to be garbage-collected.

     If successful, return all values produced by EXP.  Any uncaught
     exception thrown by the expression will propagate out.  If the time
     or allocation limit is exceeded, an exception will be thrown to the
     ‘limit-exceeded’ key.

   Constructing a safe sandbox module is tricky in general.  Guile
defines an easy way to construct safe modules from predefined sets of
bindings.  Before getting to that interface, here are some general notes
on safety.

  1. The time and allocation limits rely on the ability to interrupt and
     cancel a computation.  For this reason, no binding included in a
     sandbox module should be able to indefinitely postpone interrupt
     handling, nor should a binding be able to prevent an abort.  In
     practice this second consideration means that ‘dynamic-wind’ should
     not be included in any binding set.
  2. The time and allocation limits apply only to the ‘eval-in-sandbox’
     call.  If the call returns a procedure which is later called, no
     limit is “automatically” in place.  Users of ‘eval-in-sandbox’ have
     to be very careful to reimpose limits when calling procedures that
     escape from sandboxes.
  3. Similarly, the dynamic environment of the ‘eval-in-sandbox’ call is
     not necessarily in place when any procedure that escapes from the
     sandbox is later called.

     This detail prevents us from exposing ‘primitive-eval’ to the
     sandbox, for two reasons.  The first is that it’s possible for
     legacy code to forge references to any binding, if the
     ‘allow-legacy-syntax-objects?’ parameter is true.  The default for
     this parameter is true; *note Syntax Transformer Helpers:: for the
     details.  The parameter is bound to ‘#f’ for the duration of the
     ‘eval-in-sandbox’ call itself, but that will not be in place during
     calls to escaped procedures.

     The second reason we don’t expose ‘primitive-eval’ is that
     ‘primitive-eval’ implicitly works in the current module, which for
     an escaped procedure will probably be different than the module
     that is current for the ‘eval-in-sandbox’ call itself.

     The common denominator here is that if an interface exposed to the
     sandbox relies on dynamic environments, it is easy to mistakenly
     grant the sandboxed procedure additional capabilities in the form
     of bindings that it should not have access to.  For this reason,
     the default sets of predefined bindings do not depend on any
     dynamically scoped value.
  4. Mutation may allow a sandboxed evaluation to break some invariant
     in users of data supplied to it.  A lot of code culturally doesn’t
     expect mutation, but if you hand mutable data to a sandboxed
     evaluation and you also grant mutating capabilities to that
     evaluation, then the sandboxed code may indeed mutate that data.
     The default set of bindings to the sandbox do not include any
     mutating primitives.

     Relatedly, ‘set!’ may allow a sandbox to mutate a primitive,
     invalidating many system-wide invariants.  Guile is currently quite
     permissive when it comes to imported bindings and mutability.
     Although ‘set!’ to a module-local or lexically bound variable would
     be fine, we don’t currently have an easy way to disallow ‘set!’ to
     an imported binding, so currently no binding set includes ‘set!’.
  5. Mutation may allow a sandboxed evaluation to keep state, or make a
     communication mechanism with other code.  On the one hand this
     sounds cool, but on the other hand maybe this is part of your
     threat model.  Again, the default set of bindings doesn’t include
     mutating primitives, preventing sandboxed evaluations from keeping
     state.
  6. The sandbox should probably not be able to open a network
     connection, or write to a file, or open a file from disk.  The
     default binding set includes no interaction with the operating
     system.

   If you, dear reader, find the above discussion interesting, you will
enjoy Jonathan Rees’ dissertation, “A Security Kernel Based on the
Lambda Calculus”.

 -- Scheme Variable: all-pure-bindings
     All “pure” bindings that together form a safe subset of those
     bindings available by default to Guile user code.

 -- Scheme Variable: all-pure-and-impure-bindings
     Like ‘all-pure-bindings’, but additionally including mutating
     primitives like ‘vector-set!’.  This set is still safe in the sense
     mentioned above, with the caveats about mutation.

   The components of these composite sets are as follows:
 -- Scheme Variable: alist-bindings
 -- Scheme Variable: array-bindings
 -- Scheme Variable: bit-bindings
 -- Scheme Variable: bitvector-bindings
 -- Scheme Variable: char-bindings
 -- Scheme Variable: char-set-bindings
 -- Scheme Variable: clock-bindings
 -- Scheme Variable: core-bindings
 -- Scheme Variable: error-bindings
 -- Scheme Variable: fluid-bindings
 -- Scheme Variable: hash-bindings
 -- Scheme Variable: iteration-bindings
 -- Scheme Variable: keyword-bindings
 -- Scheme Variable: list-bindings
 -- Scheme Variable: macro-bindings
 -- Scheme Variable: nil-bindings
 -- Scheme Variable: number-bindings
 -- Scheme Variable: pair-bindings
 -- Scheme Variable: predicate-bindings
 -- Scheme Variable: procedure-bindings
 -- Scheme Variable: promise-bindings
 -- Scheme Variable: prompt-bindings
 -- Scheme Variable: regexp-bindings
 -- Scheme Variable: sort-bindings
 -- Scheme Variable: srfi-4-bindings
 -- Scheme Variable: string-bindings
 -- Scheme Variable: symbol-bindings
 -- Scheme Variable: unspecified-bindings
 -- Scheme Variable: variable-bindings
 -- Scheme Variable: vector-bindings
 -- Scheme Variable: version-bindings
     The components of ‘all-pure-bindings’.

 -- Scheme Variable: mutating-alist-bindings
 -- Scheme Variable: mutating-array-bindings
 -- Scheme Variable: mutating-bitvector-bindings
 -- Scheme Variable: mutating-fluid-bindings
 -- Scheme Variable: mutating-hash-bindings
 -- Scheme Variable: mutating-list-bindings
 -- Scheme Variable: mutating-pair-bindings
 -- Scheme Variable: mutating-sort-bindings
 -- Scheme Variable: mutating-srfi-4-bindings
 -- Scheme Variable: mutating-string-bindings
 -- Scheme Variable: mutating-variable-bindings
 -- Scheme Variable: mutating-vector-bindings
     The additional components of ‘all-pure-and-impure-bindings’.

   Finally, what do you do with a binding set?  What is a binding set
anyway?  ‘make-sandbox-module’ is here for you.

 -- Scheme Procedure: make-sandbox-module bindings
     Return a fresh module that only contains BINDINGS.

     The BINDINGS should be given as a list of import sets.  One import
     set is a list whose car names an interface, like ‘(ice-9 q)’, and
     whose cdr is a list of imports.  An import is either a bare symbol
     or a pair of ‘(OUT . IN)’, where OUT and IN are both symbols and
     denote the name under which a binding is exported from the module,
     and the name under which to make the binding available,
     respectively.

   So you see that binding sets are just lists, and
‘all-pure-and-impure-bindings’ is really just the result of appending
all of the component binding sets.


File: guile.info,  Node: REPL Servers,  Next: Cooperative REPL Servers,  Prev: Sandboxed Evaluation,  Up: Read/Load/Eval/Compile

6.16.14 REPL Servers
--------------------

The procedures in this section are provided by
     (use-modules (system repl server))

   When an application is written in Guile, it is often convenient to
allow the user to be able to interact with it by evaluating Scheme
expressions in a REPL.

   The procedures of this module allow you to spawn a “REPL server”,
which permits interaction over a local or TCP connection.  Guile itself
uses them internally to implement the ‘--listen’ switch, *note
Command-line Options::.

 -- Scheme Procedure: make-tcp-server-socket [#:host=#f] [#:addr]
          [#:port=37146]
     Return a stream socket bound to a given address ADDR and port
     number PORT.  If the HOST is given, and ADDR is not, then the HOST
     string is converted to an address.  If neither is given, we use the
     loopback address.

 -- Scheme Procedure: make-unix-domain-server-socket
          [#:path="/tmp/guile-socket"]
     Return a UNIX domain socket, bound to a given PATH.

 -- Scheme Procedure: run-server [server-socket]
 -- Scheme Procedure: spawn-server [server-socket]
     Create and run a REPL, making it available over the given
     SERVER-SOCKET.  If SERVER-SOCKET is not provided, it defaults to
     the socket created by calling ‘make-tcp-server-socket’ with no
     arguments.

     ‘run-server’ runs the server in the current thread, whereas
     ‘spawn-server’ runs the server in a new thread.

 -- Scheme Procedure: stop-server-and-clients!
     Closes the connection on all running server sockets.

     Please note that in the current implementation, the REPL threads
     are canceled without unwinding their stacks.  If any of them are
     holding mutexes or are within a critical section, the results are
     unspecified.


File: guile.info,  Node: Cooperative REPL Servers,  Prev: REPL Servers,  Up: Read/Load/Eval/Compile

6.16.15 Cooperative REPL Servers
--------------------------------

The procedures in this section are provided by
     (use-modules (system repl coop-server))

   Whereas ordinary REPL servers run in their own threads (*note REPL
Servers::), sometimes it is more convenient to provide REPLs that run at
specified times within an existing thread, for example in programs
utilizing an event loop or in single-threaded programs.  This allows for
safe access and mutation of a program’s data structures from the REPL,
without concern for thread synchronization.

   Although the REPLs are run in the thread that calls
‘spawn-coop-repl-server’ and ‘poll-coop-repl-server’, dedicated threads
are spawned so that the calling thread is not blocked.  The spawned
threads read input for the REPLs and to listen for new connections.

   Cooperative REPL servers must be polled periodically to evaluate any
pending expressions by calling ‘poll-coop-repl-server’ with the object
returned from ‘spawn-coop-repl-server’.  The thread that calls
‘poll-coop-repl-server’ will be blocked for as long as the expression
takes to be evaluated or if the debugger is entered.

 -- Scheme Procedure: spawn-coop-repl-server [server-socket]
     Create and return a new cooperative REPL server object, and spawn a
     new thread to listen for connections on SERVER-SOCKET.  Proper
     functioning of the REPL server requires that
     ‘poll-coop-repl-server’ be called periodically on the returned
     server object.

 -- Scheme Procedure: poll-coop-repl-server coop-server
     Poll the cooperative REPL server COOP-SERVER and apply a pending
     operation if there is one, such as evaluating an expression typed
     at the REPL prompt.  This procedure must be called from the same
     thread that called ‘spawn-coop-repl-server’.


File: guile.info,  Node: Memory Management,  Next: Modules,  Prev: Read/Load/Eval/Compile,  Up: API Reference

6.17 Memory Management and Garbage Collection
=============================================

Guile uses a _garbage collector_ to manage most of its objects.  While
the garbage collector is designed to be mostly invisible, you sometimes
need to interact with it explicitly.

   See *note Garbage Collection:: for a general discussion of how
garbage collection relates to using Guile from C.

* Menu:

* Garbage Collection Functions::
* Memory Blocks::
* Weak References::
* Guardians::


File: guile.info,  Node: Garbage Collection Functions,  Next: Memory Blocks,  Up: Memory Management

6.17.1 Function related to Garbage Collection
---------------------------------------------

 -- Scheme Procedure: gc
 -- C Function: scm_gc ()
     Finds all of the “live” ‘SCM’ objects and reclaims for further use
     those that are no longer accessible.  You normally don’t need to
     call this function explicitly.  Its functionality is invoked
     automatically as needed.

 -- C Function: SCM scm_gc_protect_object (SCM OBJ)
     Protects OBJ from being freed by the garbage collector, when it
     otherwise might be.  When you are done with the object, call
     ‘scm_gc_unprotect_object’ on the object.  Calls to
     ‘scm_gc_protect_object’/‘scm_gc_unprotect_object’ can be nested,
     and the object remains protected until it has been unprotected as
     many times as it was protected.  It is an error to unprotect an
     object more times than it has been protected.  Returns the SCM
     object it was passed.

     Note that storing OBJ in a C global variable has the same
     effect(1).

 -- C Function: SCM scm_gc_unprotect_object (SCM OBJ)

     Unprotects an object from the garbage collector which was protected
     by ‘scm_gc_unprotect_object’.  Returns the SCM object it was
     passed.

 -- C Function: SCM scm_permanent_object (SCM OBJ)

     Similar to ‘scm_gc_protect_object’ in that it causes the collector
     to always mark the object, except that it should not be nested
     (only call ‘scm_permanent_object’ on an object once), and it has no
     corresponding unpermanent function.  Once an object is declared
     permanent, it will never be freed.  Returns the SCM object it was
     passed.

 -- C Macro: void scm_remember_upto_here_1 (SCM obj)
 -- C Macro: void scm_remember_upto_here_2 (SCM obj1, SCM obj2)
     Create a reference to the given object or objects, so they’re
     certain to be present on the stack or in a register and hence will
     not be freed by the garbage collector before this point.

     Note that these functions can only be applied to ordinary C local
     variables (ie. “automatics”).  Objects held in global or static
     variables or some malloced block or the like cannot be protected
     with this mechanism.

 -- Scheme Procedure: gc-stats
 -- C Function: scm_gc_stats ()
     Return an association list of statistics about Guile’s current use
     of storage.

 -- Scheme Procedure: gc-live-object-stats
 -- C Function: scm_gc_live_object_stats ()
     Return an alist of statistics of the current live objects.

 -- Function: void scm_gc_mark (SCM X)
     Mark the object X, and recurse on any objects X refers to.  If X’s
     mark bit is already set, return immediately.  This function must
     only be called during the mark-phase of garbage collection,
     typically from a smob _mark_ function.

   ---------- Footnotes ----------

   (1) In Guile up to version 1.8, C global variables were not visited
by the garbage collector in the mark phase; hence,
‘scm_gc_protect_object’ was the only way in C to prevent a Scheme object
from being freed.


File: guile.info,  Node: Memory Blocks,  Next: Weak References,  Prev: Garbage Collection Functions,  Up: Memory Management

6.17.2 Memory Blocks
--------------------

In C programs, dynamic management of memory blocks is normally done with
the functions malloc, realloc, and free.  Guile has additional functions
for dynamic memory allocation that are integrated into the garbage
collector and the error reporting system.

   Memory blocks that are associated with Scheme objects (for example a
foreign object) should be allocated with ‘scm_gc_malloc’ or
‘scm_gc_malloc_pointerless’.  These two functions will either return a
valid pointer or signal an error.  Memory blocks allocated this way may
be released explicitly; however, this is not strictly needed, and we
recommend _not_ calling ‘scm_gc_free’.  All memory allocated with
‘scm_gc_malloc’ or ‘scm_gc_malloc_pointerless’ is automatically
reclaimed when the garbage collector no longer sees any live reference
to it(1).

   When garbage collection occurs, Guile will visit the words in memory
allocated with ‘scm_gc_malloc’, looking for live pointers.  This means
that if ‘scm_gc_malloc’-allocated memory contains a pointer to some
other part of the memory, the garbage collector notices it and prevents
it from being reclaimed(2).  Conversely, memory allocated with
‘scm_gc_malloc_pointerless’ is assumed to be “pointer-less” and is not
scanned for pointers.

   For memory that is not associated with a Scheme object, you can use
‘scm_malloc’ instead of ‘malloc’.  Like ‘scm_gc_malloc’, it will either
return a valid pointer or signal an error.  However, it will not assume
that the new memory block can be freed by a garbage collection.  The
memory must be explicitly freed with ‘free’.

   There is also ‘scm_gc_realloc’ and ‘scm_realloc’, to be used in place
of ‘realloc’ when appropriate, and ‘scm_gc_calloc’ and ‘scm_calloc’, to
be used in place of ‘calloc’ when appropriate.

   The function ‘scm_dynwind_free’ can be useful when memory should be
freed with libc’s ‘free’ when leaving a dynwind context, *Note Dynamic
Wind::.

 -- C Function: void * scm_malloc (size_t SIZE)
 -- C Function: void * scm_calloc (size_t SIZE)
     Allocate SIZE bytes of memory and return a pointer to it.  When
     SIZE is 0, return ‘NULL’.  When not enough memory is available,
     signal an error.  This function runs the GC to free up some memory
     when it deems it appropriate.

     The memory is allocated by the libc ‘malloc’ function and can be
     freed with ‘free’.  There is no ‘scm_free’ function to go with
     ‘scm_malloc’ to make it easier to pass memory back and forth
     between different modules.

     The function ‘scm_calloc’ is similar to ‘scm_malloc’, but
     initializes the block of memory to zero as well.

     These functions will (indirectly) call
     ‘scm_gc_register_allocation’.

 -- C Function: void * scm_realloc (void *MEM, size_t NEW_SIZE)
     Change the size of the memory block at MEM to NEW_SIZE and return
     its new location.  When NEW_SIZE is 0, this is the same as calling
     ‘free’ on MEM and ‘NULL’ is returned.  When MEM is ‘NULL’, this
     function behaves like ‘scm_malloc’ and allocates a new block of
     size NEW_SIZE.

     When not enough memory is available, signal an error.  This
     function runs the GC to free up some memory when it deems it
     appropriate.

     This function will call ‘scm_gc_register_allocation’.

 -- C Function: void * scm_gc_malloc (size_t SIZE, const char *WHAT)
 -- C Function: void * scm_gc_malloc_pointerless (size_t SIZE, const
          char *WHAT)
 -- C Function: void * scm_gc_realloc (void *MEM, size_t OLD_SIZE,
          size_t NEW_SIZE, const char *WHAT);
 -- C Function: void * scm_gc_calloc (size_t SIZE, const char *WHAT)
     Allocate SIZE bytes of automatically-managed memory.  The memory is
     automatically freed when no longer referenced from any live memory
     block.

     When garbage collection occurs, Guile will visit the words in
     memory allocated with ‘scm_gc_malloc’ or ‘scm_gc_calloc’, looking
     for pointers to other memory allocations that are managed by the
     GC. In contrast, memory allocated by ‘scm_gc_malloc_pointerless’ is
     not scanned for pointers.

     The ‘scm_gc_realloc’ call preserves the “pointerlessness” of the
     memory area pointed to by MEM.  Note that you need to pass the old
     size of a reallocated memory block as well.  See below for a
     motivation.

 -- C Function: void scm_gc_free (void *MEM, size_t SIZE, const char
          *WHAT)
     Explicitly free the memory block pointed to by MEM, which was
     previously allocated by one of the above ‘scm_gc’ functions.  This
     function is almost always unnecessary, except for codebases that
     still need to compile on Guile 1.8.

     Note that you need to explicitly pass the SIZE parameter.  This is
     done since it should normally be easy to provide this parameter
     (for memory that is associated with GC controlled objects) and help
     keep the memory management overhead very low.  However, in Guile
     2.x, SIZE is always ignored.

 -- C Function: void scm_gc_register_allocation (size_t SIZE)
     Informs the garbage collector that SIZE bytes have been allocated,
     which the collector would otherwise not have known about.

     In general, Scheme will decide to collect garbage only after some
     amount of memory has been allocated.  Calling this function will
     make the Scheme garbage collector know about more allocation, and
     thus run more often (as appropriate).

     It is especially important to call this function when large
     unmanaged allocations, like images, may be freed by small Scheme
     allocations, like foreign objects.

 -- C Function: void scm_dynwind_free (void *mem)
     Equivalent to ‘scm_dynwind_unwind_handler (free, MEM,
     SCM_F_WIND_EXPLICITLY)’.  That is, the memory block at MEM will be
     freed (using ‘free’ from the C library) when the current dynwind is
     left.

 -- Scheme Procedure: malloc-stats
     Return an alist ((WHAT .  N) ...)  describing number of malloced
     objects.  WHAT is the second argument to ‘scm_gc_malloc’, N is the
     number of objects of that type currently allocated.

     This function is only available if the ‘GUILE_DEBUG_MALLOC’
     preprocessor macro was defined when Guile was compiled.

   ---------- Footnotes ----------

   (1) In Guile up to version 1.8, memory allocated with ‘scm_gc_malloc’
_had_ to be freed with ‘scm_gc_free’.

   (2) In Guile up to 1.8, memory allocated with ‘scm_gc_malloc’ was
_not_ visited by the collector in the mark phase.  Consequently, the GC
had to be told explicitly about pointers to live objects contained in
the memory block, e.g., via SMOB mark functions (*note
‘scm_set_smob_mark’: Smobs.)


File: guile.info,  Node: Weak References,  Next: Guardians,  Prev: Memory Blocks,  Up: Memory Management

6.17.3 Weak References
----------------------

[FIXME: This chapter is based on Mikael Djurfeldt’s answer to a question
by Michael Livshin.  Any mistakes are not theirs, of course.  ]

   Weak references let you attach bookkeeping information to data so
that the additional information automatically disappears when the
original data is no longer in use and gets garbage collected.  In a weak
key hash, the hash entry for that key disappears as soon as the key is
no longer referenced from anywhere else.  For weak value hashes, the
same happens as soon as the value is no longer in use.  Entries in a
doubly weak hash disappear when either the key or the value are not used
anywhere else anymore.

   Object properties offer the same kind of functionality as weak key
hashes in many situations.  (*note Object Properties::)

   Here’s an example (a little bit strained perhaps, but one of the
examples is actually used in Guile):

   Assume that you’re implementing a debugging system where you want to
associate information about filename and position of source code
expressions with the expressions themselves.

   Hashtables can be used for that, but if you use ordinary hash tables
it will be impossible for the scheme interpreter to "forget" old source
when, for example, a file is reloaded.

   To implement the mapping from source code expressions to positional
information it is necessary to use weak-key tables since we don’t want
the expressions to be remembered just because they are in our table.

   To implement a mapping from source file line numbers to source code
expressions you would use a weak-value table.

   To implement a mapping from source code expressions to the procedures
they constitute a doubly-weak table has to be used.

* Menu:

* Weak hash tables::
* Weak vectors::


File: guile.info,  Node: Weak hash tables,  Next: Weak vectors,  Up: Weak References

6.17.3.1 Weak hash tables
.........................

 -- Scheme Procedure: make-weak-key-hash-table [size]
 -- Scheme Procedure: make-weak-value-hash-table [size]
 -- Scheme Procedure: make-doubly-weak-hash-table [size]
 -- C Function: scm_make_weak_key_hash_table (size)
 -- C Function: scm_make_weak_value_hash_table (size)
 -- C Function: scm_make_doubly_weak_hash_table (size)
     Return a weak hash table with SIZE buckets.  As with any hash
     table, choosing a good size for the table requires some caution.

     You can modify weak hash tables in exactly the same way you would
     modify regular hash tables, with the exception of the routines that
     act on handles.  Weak tables have a different implementation behind
     the scenes that doesn’t have handles.  *note Hash Tables::, for
     more on ‘hashq-ref’ et al.

   Note that in a weak-key hash table, the reference to the value is
strong.  This means that if the value references the key, even
indirectly, the key will never be collected, which can lead to a memory
leak.  The reverse is true for weak value tables.

 -- Scheme Procedure: weak-key-hash-table? obj
 -- Scheme Procedure: weak-value-hash-table? obj
 -- Scheme Procedure: doubly-weak-hash-table? obj
 -- C Function: scm_weak_key_hash_table_p (obj)
 -- C Function: scm_weak_value_hash_table_p (obj)
 -- C Function: scm_doubly_weak_hash_table_p (obj)
     Return ‘#t’ if OBJ is the specified weak hash table.  Note that a
     doubly weak hash table is neither a weak key nor a weak value hash
     table.


File: guile.info,  Node: Weak vectors,  Prev: Weak hash tables,  Up: Weak References

6.17.3.2 Weak vectors
.....................

 -- Scheme Procedure: make-weak-vector size [fill]
 -- C Function: scm_make_weak_vector (size, fill)
     Return a weak vector with SIZE elements.  If the optional argument
     FILL is given, all entries in the vector will be set to FILL.  The
     default value for FILL is the empty list.

 -- Scheme Procedure: weak-vector elem ...
 -- Scheme Procedure: list->weak-vector l
 -- C Function: scm_weak_vector (l)
     Construct a weak vector from a list: ‘weak-vector’ uses the list of
     its arguments while ‘list->weak-vector’ uses its only argument L (a
     list) to construct a weak vector the same way ‘list->vector’ would.

 -- Scheme Procedure: weak-vector? obj
 -- C Function: scm_weak_vector_p (obj)
     Return ‘#t’ if OBJ is a weak vector.

 -- Scheme Procedure: weak-vector-ref wvect k
 -- C Function: scm_weak_vector_ref (wvect, k)
     Return the Kth element of the weak vector WVECT, or ‘#f’ if that
     element has been collected.

 -- Scheme Procedure: weak-vector-set! wvect k elt
 -- C Function: scm_weak_vector_set_x (wvect, k, elt)
     Set the Kth element of the weak vector WVECT to ELT.


File: guile.info,  Node: Guardians,  Prev: Weak References,  Up: Memory Management

6.17.4 Guardians
----------------

Guardians provide a way to be notified about objects that would
otherwise be collected as garbage.  Guarding them prevents the objects
from being collected and cleanup actions can be performed on them, for
example.

   See R. Kent Dybvig, Carl Bruggeman, and David Eby (1993) "Guardians
in a Generation-Based Garbage Collector".  ACM SIGPLAN Conference on
Programming Language Design and Implementation, June 1993.

 -- Scheme Procedure: make-guardian
 -- C Function: scm_make_guardian ()
     Create a new guardian.  A guardian protects a set of objects from
     garbage collection, allowing a program to apply cleanup or other
     actions.

     ‘make-guardian’ returns a procedure representing the guardian.
     Calling the guardian procedure with an argument adds the argument
     to the guardian’s set of protected objects.  Calling the guardian
     procedure without an argument returns one of the protected objects
     which are ready for garbage collection, or ‘#f’ if no such object
     is available.  Objects which are returned in this way are removed
     from the guardian.

     You can put a single object into a guardian more than once and you
     can put a single object into more than one guardian.  The object
     will then be returned multiple times by the guardian procedures.

     An object is eligible to be returned from a guardian when it is no
     longer referenced from outside any guardian.

     There is no guarantee about the order in which objects are returned
     from a guardian.  If you want to impose an order on finalization
     actions, for example, you can do that by keeping objects alive in
     some global data structure until they are no longer needed for
     finalizing other objects.

     Being an element in a weak vector, a key in a hash table with weak
     keys, or a value in a hash table with weak values does not prevent
     an object from being returned by a guardian.  But as long as an
     object can be returned from a guardian it will not be removed from
     such a weak vector or hash table.  In other words, a weak link does
     not prevent an object from being considered collectible, but being
     inside a guardian prevents a weak link from being broken.

     A key in a weak key hash table can be thought of as having a strong
     reference to its associated value as long as the key is accessible.
     Consequently, when the key is only accessible from within a
     guardian, the reference from the key to the value is also
     considered to be coming from within a guardian.  Thus, if there is
     no other reference to the value, it is eligible to be returned from
     a guardian.


File: guile.info,  Node: Modules,  Next: Foreign Function Interface,  Prev: Memory Management,  Up: API Reference

6.18 Modules
============

When programs become large, naming conflicts can occur when a function
or global variable defined in one file has the same name as a function
or global variable in another file.  Even just a _similarity_ between
function names can cause hard-to-find bugs, since a programmer might
type the wrong function name.

   The approach used to tackle this problem is called _information
encapsulation_, which consists of packaging functional units into a
given name space that is clearly separated from other name spaces.

   The language features that allow this are usually called _the module
system_ because programs are broken up into modules that are compiled
separately (or loaded separately in an interpreter).

   Older languages, like C, have limited support for name space
manipulation and protection.  In C a variable or function is public by
default, and can be made local to a module with the ‘static’ keyword.
But you cannot reference public variables and functions from another
module with different names.

   More advanced module systems have become a common feature in recently
designed languages: ML, Python, Perl, and Modula 3 all allow the
_renaming_ of objects from a foreign module, so they will not clutter
the global name space.

   In addition, Guile offers variables as first-class objects.  They can
be used for interacting with the module system.

* Menu:

* General Information about Modules::  Guile module basics.
* Using Guile Modules::         How to use existing modules.
* Creating Guile Modules::      How to package your code into modules.
* Modules and the File System:: Installing modules in the file system.
* R6RS Version References::     Using version numbers with modules.
* R6RS Libraries::              The library and import forms.
* Variables::                   First-class variables.
* Module System Reflection::    First-class modules.
* Declarative Modules::         Allowing Guile to reason about modules.
* Accessing Modules from C::    How to work with modules with C code.
* provide and require::         The SLIB feature mechanism.
* Environments::                R5RS top-level environments.


File: guile.info,  Node: General Information about Modules,  Next: Using Guile Modules,  Up: Modules

6.18.1 General Information about Modules
----------------------------------------

A Guile module can be thought of as a collection of named procedures,
variables and macros.  More precisely, it is a set of “bindings” of
symbols (names) to Scheme objects.

   Within a module, all bindings are visible.  Certain bindings can be
declared “public”, in which case they are added to the module’s
so-called “export list”; this set of public bindings is called the
module’s “public interface” (*note Creating Guile Modules::).

   A client module “uses” a providing module’s bindings by either
accessing the providing module’s public interface, or by building a
custom interface (and then accessing that).  In a custom interface, the
client module can “select” which bindings to access and can also
algorithmically “rename” bindings.  In contrast, when using the
providing module’s public interface, the entire export list is available
without renaming (*note Using Guile Modules::).

   All Guile modules have a unique “module name”, for example ‘(ice-9
popen)’ or ‘(srfi srfi-11)’.  Module names are lists of one or more
symbols.

   When Guile goes to use an interface from a module, for example
‘(ice-9 popen)’, Guile first looks to see if it has loaded ‘(ice-9
popen)’ for any reason.  If the module has not been loaded yet, Guile
searches a “load path” for a file that might define it, and loads that
file.

   The following subsections go into more detail on using, creating,
installing, and otherwise manipulating modules and the module system.


File: guile.info,  Node: Using Guile Modules,  Next: Creating Guile Modules,  Prev: General Information about Modules,  Up: Modules

6.18.2 Using Guile Modules
--------------------------

To use a Guile module is to access either its public interface or a
custom interface (*note General Information about Modules::).  Both
types of access are handled by the syntactic form ‘use-modules’, which
accepts one or more interface specifications and, upon evaluation,
arranges for those interfaces to be available to the current module.
This process may include locating and loading code for a given module if
that code has not yet been loaded, following ‘%load-path’ (*note Modules
and the File System::).

   An “interface specification” has one of two forms.  The first
variation is simply to name the module, in which case its public
interface is the one accessed.  For example:

     (use-modules (ice-9 popen))

   Here, the interface specification is ‘(ice-9 popen)’, and the result
is that the current module now has access to ‘open-pipe’, ‘close-pipe’,
‘open-input-pipe’, and so on (*note Pipes::).

   Note in the previous example that if the current module had already
defined ‘open-pipe’, that definition would be overwritten by the
definition in ‘(ice-9 popen)’.  For this reason (and others), there is a
second variation of interface specification that not only names a module
to be accessed, but also selects bindings from it and renames them to
suit the current module’s needs.  For example:

     (use-modules ((ice-9 popen)
                   #:select ((open-pipe . pipe-open) close-pipe)
                   #:renamer (symbol-prefix-proc 'unixy:)))

or more simply:

     (use-modules ((ice-9 popen)
                   #:select ((open-pipe . pipe-open) close-pipe)
                   #:prefix unixy:))

   Here, the interface specification is more complex than before, and
the result is that a custom interface with only two bindings is created
and subsequently accessed by the current module.  The mapping of old to
new names is as follows:

     (ice-9 popen) sees:             current module sees:
     open-pipe                       unixy:pipe-open
     close-pipe                      unixy:close-pipe

   This example also shows how to use the convenience procedure
‘symbol-prefix-proc’.

   You can also directly refer to bindings in a module by using the ‘@’
syntax.  For example, instead of using the ‘use-modules’ statement from
above and writing ‘unixy:pipe-open’ to refer to the ‘pipe-open’ from the
‘(ice-9 popen)’, you could also write ‘(@ (ice-9 popen) open-pipe)’.
Thus an alternative to the complete ‘use-modules’ statement would be

     (define unixy:pipe-open (@ (ice-9 popen) open-pipe))
     (define unixy:close-pipe (@ (ice-9 popen) close-pipe))

   There is also ‘@@’, which can be used like ‘@’, but does not check
whether the variable that is being accessed is actually exported.  Thus,
‘@@’ can be thought of as the impolite version of ‘@’ and should only be
used as a last resort or for debugging, for example.

   Note that just as with a ‘use-modules’ statement, any module that has
not yet been loaded will be loaded when referenced by a ‘@’ or ‘@@’
form.

   You can also use the ‘@’ and ‘@@’ syntaxes as the target of a ‘set!’
when the binding refers to a variable.

 -- Scheme Procedure: symbol-prefix-proc prefix-sym
     Return a procedure that prefixes its arg (a symbol) with
     PREFIX-SYM.

 -- syntax: use-modules spec ...
     Resolve each interface specification SPEC into an interface and
     arrange for these to be accessible by the current module.  The
     return value is unspecified.

     SPEC can be a list of symbols, in which case it names a module
     whose public interface is found and used.

     SPEC can also be of the form:

           (MODULE-NAME [#:select SELECTION]
                        [#:hide HIDE]
                        [#:prefix PREFIX]
                        [#:renamer RENAMER])

     in which case a custom interface is newly created and used.
     MODULE-NAME is a list of symbols, as above; SELECTION is a list of
     selection-specs; HIDE is a list of bindings which should not be
     imported; PREFIX is a symbol that is prepended to imported names;
     and RENAMER is a procedure that takes a symbol and returns its new
     name.  A selection-spec is either a symbol or a pair of symbols
     ‘(ORIG . SEEN)’, where ORIG is the name in the used module and SEEN
     is the name in the using module.  Note that SEEN is also modified
     by PREFIX and RENAMER.

     The ‘#:select’, ‘#:hide’, ‘#:prefix’, and ‘#:renamer’ clauses are
     optional.  If all are omitted, this form behaves identically to the
     previous one.  If the ‘#:select’ clause is omitted, PREFIX and
     RENAMER operate on the used module’s public interface.

     The ‘#:hide’ operates on list of bindings in the module being
     imported, before any renaming is performed.  If both ‘#:select’ and
     ‘#:hide’ contain a binding, the ‘#:hide’ wins.

     In addition to the above, SPEC can also include a ‘#:version’
     clause, of the form:

           #:version VERSION-SPEC

     where VERSION-SPEC is an R6RS-compatible version reference.  An
     error will be signaled in the case in which a module with the same
     name has already been loaded, if that module specifies a version
     and that version is not compatible with VERSION-SPEC.  *Note R6RS
     Version References::, for more on version references.

     If the module name is not resolvable, ‘use-modules’ will signal an
     error.

     See also ‘scm_c_use_module’ for the C API.

 -- syntax: @ module-name binding-name
     Refer to the binding named BINDING-NAME in module MODULE-NAME.  The
     binding must have been exported by the module.

 -- syntax: @@ module-name binding-name
     Refer to the binding named BINDING-NAME in module MODULE-NAME.  The
     binding must not have been exported by the module.  This syntax is
     only intended for debugging purposes or as a last resort.  *Note
     Declarative Modules::, for some limitations on the use of ‘@@’.


File: guile.info,  Node: Creating Guile Modules,  Next: Modules and the File System,  Prev: Using Guile Modules,  Up: Modules

6.18.3 Creating Guile Modules
-----------------------------

When you want to create your own modules, you have to take the following
steps:

   • Create a Scheme source file and add all variables and procedures
     you wish to export, or which are required by the exported
     procedures.

   • Add a ‘define-module’ form at the beginning.

   • Export all bindings which should be in the public interface, either
     by using ‘define-public’ or ‘export’ (both documented below).

 -- syntax: define-module module-name option ...
     MODULE-NAME is a list of one or more symbols.

          (define-module (ice-9 popen))

     ‘define-module’ makes this module available to Guile programs under
     the given MODULE-NAME.

     OPTION ... are keyword/value pairs which specify more about the
     defined module.  The recognized options and their meaning are shown
     in the following table.

     ‘#:use-module INTERFACE-SPECIFICATION’
          Equivalent to a ‘(use-modules INTERFACE-SPECIFICATION)’ (*note
          Using Guile Modules::).

     ‘#:autoload MODULE SYMBOL-LIST’
          Load MODULE when any of SYMBOL-LIST are accessed.  For
          example,

               (define-module (my mod)
                 #:autoload (srfi srfi-1) (partition delete-duplicates))
               ...
               (when something
                 (set! foo (delete-duplicates ...)))

          When a module is autoloaded, only the bindings in SYMBOL-LIST
          become available(1).

          An autoload is a good way to put off loading a big module
          until it’s really needed, for instance for faster startup or
          if it will only be needed in certain circumstances.

     ‘#:export LIST’
          Export all identifiers in LIST which must be a list of symbols
          or pairs of symbols.  This is equivalent to ‘(export LIST)’ in
          the module body.

     ‘#:re-export LIST’
          Re-export all identifiers in LIST which must be a list of
          symbols or pairs of symbols.  The symbols in LIST must be
          imported by the current module from other modules.  This is
          equivalent to ‘re-export’ below.

     ‘#:replace LIST’
          Export all identifiers in LIST (a list of symbols or pairs of
          symbols) and mark them as “replacing bindings”.  In the module
          user’s name space, this will have the effect of replacing any
          binding with the same name that is not also “replacing”.
          Normally a replacement results in an “override” warning
          message, ‘#:replace’ avoids that.

          In general, a module that exports a binding for which the
          ‘(guile)’ module already has a definition should use
          ‘#:replace’ instead of ‘#:export’.  ‘#:replace’, in a sense,
          lets Guile know that the module _purposefully_ replaces a core
          binding.  It is important to note, however, that this binding
          replacement is confined to the name space of the module user.
          In other words, the value of the core binding in question
          remains unchanged for other modules.

          Note that although it is often a good idea for the replaced
          binding to remain compatible with a binding in ‘(guile)’, to
          avoid surprising the user, sometimes the bindings will be
          incompatible.  For example, SRFI-19 exports its own version of
          ‘current-time’ (*note SRFI-19 Time::) which is not compatible
          with the core ‘current-time’ function (*note Time::).  Guile
          assumes that a user importing a module knows what she is
          doing, and uses ‘#:replace’ for this binding rather than
          ‘#:export’.

          A ‘#:replace’ clause is equivalent to ‘(export! LIST)’ in the
          module body.

          The ‘#:duplicates’ (see below) provides fine-grain control
          about duplicate binding handling on the module-user side.

     ‘#:re-export-and-replace LIST’
          Like ‘#:re-export’, but also marking the bindings as
          replacements in the sense of ‘#:replace’.

     ‘#:version LIST’
          Specify a version for the module in the form of LIST, a list
          of zero or more exact, non-negative integers.  The
          corresponding ‘#:version’ option in the ‘use-modules’ form
          allows callers to restrict the value of this option in various
          ways.

     ‘#:duplicates LIST’
          Tell Guile to handle duplicate bindings for the bindings
          imported by the current module according to the policy defined
          by LIST, a list of symbols.  LIST must contain symbols
          representing a duplicate binding handling policy chosen among
          the following:

          ‘check’
               Raises an error when a binding is imported from more than
               one place.
          ‘warn’
               Issue a warning when a binding is imported from more than
               one place and leave the responsibility of actually
               handling the duplication to the next duplicate binding
               handler.
          ‘replace’
               When a new binding is imported that has the same name as
               a previously imported binding, then do the following:

                 1. If the old binding was said to be “replacing” (via
                    the ‘#:replace’ option above) and the new binding is
                    not replacing, the keep the old binding.
                 2. If the old binding was not said to be replacing and
                    the new binding is replacing, then replace the old
                    binding with the new one.
                 3. If neither the old nor the new binding is replacing,
                    then keep the old one.

          ‘warn-override-core’
               Issue a warning when a core binding is being overwritten
               and actually override the core binding with the new one.
          ‘first’
               In case of duplicate bindings, the firstly imported
               binding is always the one which is kept.
          ‘last’
               In case of duplicate bindings, the lastly imported
               binding is always the one which is kept.
          ‘noop’
               In case of duplicate bindings, leave the responsibility
               to the next duplicate handler.

          If LIST contains more than one symbol, then the duplicate
          binding handlers which appear first will be used first when
          resolving a duplicate binding situation.  As mentioned above,
          some resolution policies may explicitly leave the
          responsibility of handling the duplication to the next handler
          in LIST.

          If GOOPS has been loaded before the ‘#:duplicates’ clause is
          processed, there are additional strategies available for
          dealing with generic functions.  *Note Merging Generics::, for
          more information.

          The default duplicate binding resolution policy is given by
          the ‘default-duplicate-binding-handler’ procedure, and is

               (replace warn-override-core warn last)

     ‘#:pure’
          Create a “pure” module, that is a module which does not
          contain any of the standard procedure bindings except for the
          syntax forms.  This is useful if you want to create “safe”
          modules, that is modules which do not know anything about
          dangerous procedures.

 -- syntax: export variable ...
     Add all VARIABLEs (which must be symbols or pairs of symbols) to
     the list of exported bindings of the current module.  If VARIABLE
     is a pair, its ‘car’ gives the name of the variable as seen by the
     current module and its ‘cdr’ specifies a name for the binding in
     the current module’s public interface.

 -- syntax: define-public ...
     Equivalent to ‘(begin (define foo ...) (export foo))’.

 -- syntax: re-export variable ...
     Add all VARIABLEs (which must be symbols or pairs of symbols) to
     the list of re-exported bindings of the current module.  Pairs of
     symbols are handled as in ‘export’.  Re-exported bindings must be
     imported by the current module from some other module.

 -- syntax: export! variable ...
     Like ‘export’, but marking the exported variables as replacing.
     Using a module with replacing bindings will cause any existing
     bindings to be replaced without issuing any warnings.  See the
     discussion of ‘#:replace’ above.

   ---------- Footnotes ----------

   (1) In Guile 2.2 and earlier, _all_ the module bindings would become
available; SYMBOL-LIST was just the list of bindings that will first
trigger the load.


File: guile.info,  Node: Modules and the File System,  Next: R6RS Version References,  Prev: Creating Guile Modules,  Up: Modules

6.18.4 Modules and the File System
----------------------------------

Typical programs only use a small subset of modules installed on a Guile
system.  In order to keep startup time down, Guile only loads modules
when a program uses them, on demand.

   When a program evaluates ‘(use-modules (ice-9 popen))’, and the
module is not loaded, Guile searches for a conventionally-named file in
the “load path”.

   In this case, loading ‘(ice-9 popen)’ will eventually cause Guile to
run ‘(primitive-load-path "ice-9/popen")’.  ‘primitive-load-path’ will
search for a file ‘ice-9/popen’ in the ‘%load-path’ (*note Load
Paths::).  For each directory in ‘%load-path’, Guile will try to find
the file name, concatenated with the extensions from ‘%load-extensions’.
By default, this will cause Guile to ‘stat’ ‘ice-9/popen.scm’, and then
‘ice-9/popen’.  *Note Load Paths::, for more on ‘primitive-load-path’.

   If a corresponding compiled ‘.go’ file is found in the
‘%load-compiled-path’ or in the fallback path, and is as fresh as the
source file, it will be loaded instead of the source file.  If no
compiled file is found, Guile may try to compile the source file and
cache away the resulting ‘.go’ file.  *Note Compilation::, for more on
compilation.

   Once Guile finds a suitable source or compiled file is found, the
file will be loaded.  If, after loading the file, the module under
consideration is still not defined, Guile will signal an error.

   For more information on where and how to install Scheme modules,
*Note Installing Site Packages::.


File: guile.info,  Node: R6RS Version References,  Next: R6RS Libraries,  Prev: Modules and the File System,  Up: Modules

6.18.5 R6RS Version References
------------------------------

Guile’s module system includes support for locating modules based on a
declared version specifier of the same form as the one described in R6RS
(*note R6RS Library Form: (r6rs)Library form.).  By using the
‘#:version’ keyword in a ‘define-module’ form, a module may specify a
version as a list of zero or more exact, non-negative integers.

   This version can then be used to locate the module during the module
search process.  Client modules and callers of the ‘use-modules’
function may specify constraints on the versions of target modules by
providing a “version reference”, which has one of the following forms:

      (SUB-VERSION-REFERENCE ...)
      (and VERSION-REFERENCE ...)
      (or VERSION-REFERENCE ...)
      (not VERSION-REFERENCE)

   in which SUB-VERSION-REFERENCE is in turn one of:

      (SUB-VERSION)
      (>= SUB-VERSION)
      (<= SUB-VERSION)
      (and SUB-VERSION-REFERENCE ...)
      (or SUB-VERSION-REFERENCE ...)
      (not SUB-VERSION-REFERENCE)

   in which SUB-VERSION is an exact, non-negative integer as above.  A
version reference matches a declared module version if each element of
the version reference matches a corresponding element of the module
version, according to the following rules:

   • The ‘and’ sub-form matches a version or version element if every
     element in the tail of the sub-form matches the specified version
     or version element.

   • The ‘or’ sub-form matches a version or version element if any
     element in the tail of the sub-form matches the specified version
     or version element.

   • The ‘not’ sub-form matches a version or version element if the tail
     of the sub-form does not match the version or version element.

   • The ‘>=’ sub-form matches a version element if the element is
     greater than or equal to the SUB-VERSION in the tail of the
     sub-form.

   • The ‘<=’ sub-form matches a version element if the version is less
     than or equal to the SUB-VERSION in the tail of the sub-form.

   • A SUB-VERSION matches a version element if one is EQV? to the
     other.

   For example, a module declared as:

      (define-module (mylib mymodule) #:version (1 2 0))

   would be successfully loaded by any of the following ‘use-modules’
expressions:

      (use-modules ((mylib mymodule) #:version (1 2 (>= 0))))
      (use-modules ((mylib mymodule) #:version (or (1 2 0) (1 2 1))))
      (use-modules ((mylib mymodule) #:version ((and (>= 1) (not 2)) 2 0)))


File: guile.info,  Node: R6RS Libraries,  Next: Variables,  Prev: R6RS Version References,  Up: Modules

6.18.6 R6RS Libraries
---------------------

In addition to the API described in the previous sections, you also have
the option to create modules using the portable ‘library’ form described
in R6RS (*note R6RS Library Form: (r6rs)Library form.), and to import
libraries created in this format by other programmers.  Guile’s R6RS
library implementation takes advantage of the flexibility built into the
module system by expanding the R6RS library form into a corresponding
Guile ‘define-module’ form that specifies equivalent import and export
requirements and includes the same body expressions.  The library
expression:

       (library (mylib (1 2))
         (export mybinding)
         (import (otherlib (3))))

   is equivalent to the module definition:

       (define-module (mylib)
         #:version (1 2)
         #:use-module ((otherlib) #:version (3))
         #:export (mybinding))

   Central to the mechanics of R6RS libraries is the concept of import
and export “levels”, which control the visibility of bindings at various
phases of a library’s lifecycle — macros necessary to expand forms in
the library’s body need to be available at expand time; variables used
in the body of a procedure exported by the library must be available at
runtime.  R6RS specifies the optional ‘for’ sub-form of an _import set_
specification (see below) as a mechanism by which a library author can
indicate that a particular library import should take place at a
particular phase with respect to the lifecycle of the importing library.

   Guile’s library implementation uses a technique called “implicit
phasing” (first described by Abdulaziz Ghuloum and R. Kent Dybvig),
which allows the expander and compiler to automatically determine the
necessary visibility of a binding imported from another library.  As
such, the ‘for’ sub-form described below is ignored by Guile (but may be
required by Schemes in which phasing is explicit).

 -- Scheme Syntax: library name (export export-spec ...) (import
          import-spec ...) body ...
     Defines a new library with the specified name, exports, and
     imports, and evaluates the specified body expressions in this
     library’s environment.

     The library NAME is a non-empty list of identifiers, optionally
     ending with a version specification of the form described above
     (*note Creating Guile Modules::).

     Each EXPORT-SPEC is the name of a variable defined or imported by
     the library, or must take the form ‘(rename (internal-name
     external-name) ...)’, where the identifier INTERNAL-NAME names a
     variable defined or imported by the library and EXTERNAL-NAME is
     the name by which the variable is seen by importing libraries.

     Each IMPORT-SPEC must be either an “import set” (see below) or must
     be of the form ‘(for import-set import-level ...)’, where each
     IMPORT-LEVEL is one of:

            run
            expand
            (meta LEVEL)

     where LEVEL is an integer.  Note that since Guile does not require
     explicit phase specification, any IMPORT-SETs found inside of ‘for’
     sub-forms will be “unwrapped” during expansion and processed as if
     they had been specified directly.

     Import sets in turn take one of the following forms:

            LIBRARY-REFERENCE
            (library LIBRARY-REFERENCE)
            (only IMPORT-SET IDENTIFIER ...)
            (except IMPORT-SET IDENTIFIER ...)
            (prefix IMPORT-SET IDENTIFIER)
            (rename IMPORT-SET (INTERNAL-IDENTIFIER EXTERNAL-IDENTIFIER) ...)

     where LIBRARY-REFERENCE is a non-empty list of identifiers ending
     with an optional version reference (*note R6RS Version
     References::), and the other sub-forms have the following
     semantics, defined recursively on nested IMPORT-SETs:

        • The ‘library’ sub-form is used to specify libraries for import
          whose names begin with the identifier “library.”

        • The ‘only’ sub-form imports only the specified IDENTIFIERs
          from the given IMPORT-SET.

        • The ‘except’ sub-form imports all of the bindings exported by
          IMPORT-SET except for those that appear in the specified list
          of IDENTIFIERs.

        • The ‘prefix’ sub-form imports all of the bindings exported by
          IMPORT-SET, first prefixing them with the specified
          IDENTIFIER.

        • The ‘rename’ sub-form imports all of the identifiers exported
          by IMPORT-SET.  The binding for each INTERNAL-IDENTIFIER among
          these identifiers is made visible to the importing library as
          the corresponding EXTERNAL-IDENTIFIER; all other bindings are
          imported using the names provided by IMPORT-SET.

     Note that because Guile translates R6RS libraries into module
     definitions, an import specification may be used to declare a
     dependency on a native Guile module — although doing so may make
     your libraries less portable to other Schemes.

 -- Scheme Syntax: import import-spec ...
     Import into the current environment the libraries specified by the
     given import specifications, where each IMPORT-SPEC takes the same
     form as in the ‘library’ form described above.


File: guile.info,  Node: Variables,  Next: Module System Reflection,  Prev: R6RS Libraries,  Up: Modules

6.18.7 Variables
----------------

Each module has its own hash table, sometimes known as an “obarray”,
that maps the names defined in that module to their corresponding
variable objects.

   A variable is a box-like object that can hold any Scheme value.  It
is said to be “undefined” if its box holds a special Scheme value that
denotes undefined-ness (which is different from all other Scheme values,
including for example ‘#f’); otherwise the variable is “defined”.

   On its own, a variable object is anonymous.  A variable is said to be
“bound” when it is associated with a name in some way, usually a symbol
in a module obarray.  When this happens, the name is said to be bound to
the variable, in that module.

   (That’s the theory, anyway.  In practice, defined-ness and bound-ness
sometimes get confused, because Lisp and Scheme implementations have
often conflated — or deliberately drawn no distinction between — a name
that is unbound and a name that is bound to a variable whose value is
undefined.  We will try to be clear about the difference and explain any
confusion where it is unavoidable.)

   Variables do not have a read syntax.  Most commonly they are created
and bound implicitly by ‘define’ expressions: a top-level ‘define’
expression of the form

     (define NAME VALUE)

creates a variable with initial value VALUE and binds it to the name
NAME in the current module.  But they can also be created dynamically by
calling one of the constructor procedures ‘make-variable’ and
‘make-undefined-variable’.

 -- Scheme Procedure: make-undefined-variable
 -- C Function: scm_make_undefined_variable ()
     Return a variable that is initially unbound.

 -- Scheme Procedure: make-variable init
 -- C Function: scm_make_variable (init)
     Return a variable initialized to value INIT.

 -- Scheme Procedure: variable-bound? var
 -- C Function: scm_variable_bound_p (var)
     Return ‘#t’ if VAR is bound to a value, or ‘#f’ otherwise.  Throws
     an error if VAR is not a variable object.

 -- Scheme Procedure: variable-ref var
 -- C Function: scm_variable_ref (var)
     Dereference VAR and return its value.  VAR must be a variable
     object; see ‘make-variable’ and ‘make-undefined-variable’.

 -- Scheme Procedure: variable-set! var val
 -- C Function: scm_variable_set_x (var, val)
     Set the value of the variable VAR to VAL.  VAR must be a variable
     object, VAL can be any value.  Return an unspecified value.

 -- Scheme Procedure: variable-unset! var
 -- C Function: scm_variable_unset_x (var)
     Unset the value of the variable VAR, leaving VAR unbound.

 -- Scheme Procedure: variable? obj
 -- C Function: scm_variable_p (obj)
     Return ‘#t’ if OBJ is a variable object, else return ‘#f’.


File: guile.info,  Node: Module System Reflection,  Next: Declarative Modules,  Prev: Variables,  Up: Modules

6.18.8 Module System Reflection
-------------------------------

The previous sections have described a declarative view of the module
system.  You can also work with it programmatically by accessing and
modifying various parts of the Scheme objects that Guile uses to
implement the module system.

   At any time, there is a “current module”.  This module is the one
where a top-level ‘define’ and similar syntax will add new bindings.
You can find other module objects with ‘resolve-module’, for example.

   These module objects can be used as the second argument to ‘eval’.

 -- Scheme Procedure: current-module
 -- C Function: scm_current_module ()
     Return the current module object.

 -- Scheme Procedure: set-current-module module
 -- C Function: scm_set_current_module (module)
     Set the current module to MODULE and return the previous current
     module.

 -- Scheme Procedure: save-module-excursion thunk
     Call THUNK within a ‘dynamic-wind’ such that the module that is
     current at invocation time is restored when THUNK’s dynamic extent
     is left (*note Dynamic Wind::).

     More precisely, if THUNK escapes non-locally, the current module
     (at the time of escape) is saved, and the original current module
     (at the time THUNK’s dynamic extent was last entered) is restored.
     If THUNK’s dynamic extent is re-entered, then the current module is
     saved, and the previously saved inner module is set current again.

 -- Scheme Procedure: resolve-module name [autoload=#t] [version=#f]
          [#:ensure=#t]
 -- C Function: scm_resolve_module (name)
     Find the module named NAME and return it.  When it has not already
     been defined and AUTOLOAD is true, try to auto-load it.  When it
     can’t be found that way either, create an empty module if ENSURE is
     true, otherwise return ‘#f’.  If VERSION is true, ensure that the
     resulting module is compatible with the given version reference
     (*note R6RS Version References::).  The name is a list of symbols.

 -- Scheme Procedure: resolve-interface name [#:select=#f] [#:hide='()]
          [#:prefix=#f] [#:renamer=#f] [#:version=#f]
     Find the module named NAME as with ‘resolve-module’ and return its
     interface.  The interface of a module is also a module object, but
     it contains only the exported bindings.

 -- Scheme Procedure: module-uses module
     Return a list of the interfaces used by MODULE.

 -- Scheme Procedure: module-use! module interface
     Add INTERFACE to the front of the use-list of MODULE.  Both
     arguments should be module objects, and INTERFACE should very
     likely be a module returned by ‘resolve-interface’.

 -- Scheme Procedure: reload-module module
     Revisit the source file that corresponds to MODULE.  Raises an
     error if no source file is associated with the given module.

   As mentioned in the previous section, modules contain a mapping
between identifiers (as symbols) and storage locations (as variables).
Guile defines a number of procedures to allow access to this mapping.
If you are programming in C, *note Accessing Modules from C::.

 -- Scheme Procedure: module-variable module name
     Return the variable bound to NAME (a symbol) in MODULE, or ‘#f’ if
     NAME is unbound.

 -- Scheme Procedure: module-add! module name var
     Define a new binding between NAME (a symbol) and VAR (a variable)
     in MODULE.

 -- Scheme Procedure: module-ref module name
     Look up the value bound to NAME in MODULE.  Like ‘module-variable’,
     but also does a ‘variable-ref’ on the resulting variable, raising
     an error if NAME is unbound.

 -- Scheme Procedure: module-define! module name value
     Locally bind NAME to VALUE in MODULE.  If NAME was already locally
     bound in MODULE, i.e., defined locally and not by an imported
     module, the value stored in the existing variable will be updated.
     Otherwise, a new variable will be added to the module, via
     ‘module-add!’.

 -- Scheme Procedure: module-set! module name value
     Update the binding of NAME in MODULE to VALUE, raising an error if
     NAME is not already bound in MODULE.

   There are many other reflective procedures available in the default
environment.  If you find yourself using one of them, please contact the
Guile developers so that we can commit to stability for that interface.


File: guile.info,  Node: Declarative Modules,  Next: Accessing Modules from C,  Prev: Module System Reflection,  Up: Modules

6.18.9 Declarative Modules
--------------------------

The first-class access to modules and module variables described in the
previous subsection is very powerful and allows Guile users to build
many tools to dynamically learn things about their Guile systems.
However, as Scheme godparent Mathias Felleisen wrote in “On the
Expressive Power of Programming Languages”, a more expressive language
is necessarily harder to reason about.  There are transformations that
Guile’s compiler would like to make which can’t be done if every
top-level definition is subject to mutation at any time.

   Consider this module:

     (define-module (boxes)
       #:export (make-box box-ref box-set! box-swap!))

     (define (make-box x) (list x))
     (define (box-ref box) (car box))
     (define (box-set! box x) (set-car! box x))
     (define (box-swap! box x)
       (let ((y (box-ref box)))
         (box-set! box x)
         y))

   Ideally you’d like for the ‘box-ref’ in ‘box-swap!’ to be inlined to
‘car’.  Guile’s compiler can do this, but only if it knows that
‘box-ref’’s definition is what it appears to be in the text.  However,
in the general case it could be that a programmer could reach into the
‘(boxes)’ module at any time and change the value of ‘box-ref’.

   To allow Guile to reason about the values of top-levels from a
module, a module can be marked as “declarative”.  This flag applies only
to the subset of top-level definitions that are themselves declarative:
those that are defined within the compilation unit, and not assigned
(‘set!’) or redefined within the compilation unit.

   To explicitly mark a module as being declarative, pass the
‘#:declarative?’ keyword argument when declaring a module:

     (define-module (boxes)
       #:export (make-box box-ref box-set! box-swap!)
       #:declarative? #t)

   By default, modules are compiled declaratively if the
‘user-modules-declarative?’ parameter is true when the module is
compiled.

 -- Scheme Parameter: user-modules-declarative?
     A boolean indicating whether definitions in modules created by
     ‘define-module’ or implicitly as part of a compilation unit without
     an explicit module can be treated as declarative.

   Because it’s usually what you want, the default value of
‘user-modules-declarative?’ is ‘#t’.

Should I Mark My Module As Declarative?
.......................................

In the vast majority of use cases, declarative modules are what you
want.  However, there are exceptions.

   Consider the ‘(boxes)’ module above.  Let’s say you want to be able
to go in and change the definition of ‘box-set!’ at run-time:

     scheme@(guile-user)> (use-modules (boxes))
     scheme@(guile-user)> ,module boxes
     scheme@(boxes)> (define (box-set! x y) (set-car! x (pk y)))

   However, considering that ‘(boxes)’ is a declarative module, it could
be that ‘box-swap!’ inlined the call to ‘box-set!’ – so it may be that
you are surprised if you call ‘(box-swap! x y)’ and you don’t see the
new definition being used.  (Note, however, that Guile has no guarantees
about what definitions its compiler will or will not inline.)

   If you want to allow the definition of ‘box-set!’ to be changed and
to have all of its uses updated, then probably the best option is to
edit the module and reload the whole thing:

     scheme@(guile-user)> ,reload (boxes)

   The advantage of the reloading approach is that you maintain the
optimizations that declarative modules enable, while also being able to
live-update the code.  If the module keeps precious program state, those
definitions can be marked as ‘define-once’ to prevent reloads from
overwriting them.  *Note Top Level::, for more on ‘define-once’.
Incidentally, ‘define-once’ also prevents declarative-definition
optimizations, so if there’s a limited subset of redefinable bindings,
‘define-once’ could be an interesting tool to mark those definitions as
works-in-progress for interactive program development.

   To users, whether a module is declarative or not is mostly
immaterial: besides normal use via ‘use-modules’, users can reference
and redefine public or private bindings programmatically or
interactively.  The only difference is that changing a declarative
definition may not change all of its uses.  If this use-case is
important to you, and if reloading whole modules is insufficient, then
you can mark all definitions in a module as non-declarative by adding
‘#:declarative? #f’ to the module definition.

   The default of whether modules are declarative or not can be
controlled via the ‘(user-modules-declarative?)’ parameter mentioned
above, but care should be taken to set this parameter when the modules
are compiled, e.g.  via ‘(eval-when (expand) (user-modules-declarative?
#f))’.  *Note Eval When::.

   Alternately you can prevent declarative-definition optimizations by
compiling at the ‘-O1’ optimization level instead of the default ‘-O2’,
or via explicitly passing ‘-Ono-letrectify’ to the ‘guild compile’
invocation.  *Note Compilation::, for more on compiler options.

   One final note.  Currently, definitions from declarative modules can
only be inlined within the module they are defined in, and within a
compilation unit.  This may change in the future to allow Guile to
inline imported declarative definitions as well (cross-module inlining).
To Guile, whether a definition is inlinable or not is a property of the
definition, not its use.  We hope to improve compiler tooling in the
future to allow the user to identify definitions that are out of date
when a declarative binding is redefined.


File: guile.info,  Node: Accessing Modules from C,  Next: provide and require,  Prev: Declarative Modules,  Up: Modules

6.18.10 Accessing Modules from C
--------------------------------

The last sections have described how modules are used in Scheme code,
which is the recommended way of creating and accessing modules.  You can
also work with modules from C, but it is more cumbersome.

   The following procedures are available.

 -- C Function: SCM scm_c_call_with_current_module (SCM MODULE, SCM
          (*FUNC)(void *), void *DATA)
     Call FUNC and make MODULE the current module during the call.  The
     argument DATA is passed to FUNC.  The return value of
     ‘scm_c_call_with_current_module’ is the return value of FUNC.

 -- C Function: SCM scm_public_variable (SCM MODULE_NAME, SCM NAME)
 -- C Function: SCM scm_c_public_variable (const char *MODULE_NAME,
          const char *NAME)
     Find a the variable bound to the symbol NAME in the public
     interface of the module named MODULE_NAME.

     MODULE_NAME should be a list of symbols, when represented as a
     Scheme object, or a space-separated string, in the ‘const char *’
     case.  See ‘scm_c_define_module’ below, for more examples.

     Signals an error if no module was found with the given name.  If
     NAME is not bound in the module, just returns ‘#f’.

 -- C Function: SCM scm_private_variable (SCM MODULE_NAME, SCM NAME)
 -- C Function: SCM scm_c_private_variable (const char *MODULE_NAME,
          const char *NAME)
     Like ‘scm_public_variable’, but looks in the internals of the
     module named MODULE_NAME instead of the public interface.
     Logically, these procedures should only be called on modules you
     write.

 -- C Function: SCM scm_public_lookup (SCM MODULE_NAME, SCM NAME)
 -- C Function: SCM scm_c_public_lookup (const char *MODULE_NAME, const
          char *NAME)
 -- C Function: SCM scm_private_lookup (SCM MODULE_NAME, SCM NAME)
 -- C Function: SCM scm_c_private_lookup (const char *MODULE_NAME, const
          char *NAME)
     Like ‘scm_public_variable’ or ‘scm_private_variable’, but if the
     NAME is not bound in the module, signals an error.  Returns a
     variable, always.

          static SCM eval_string_var;

          /* NOTE: It is important that the call to 'my_init'
             happens-before all calls to 'my_eval_string'. */
          void my_init (void)
          {
            eval_string_var = scm_c_public_lookup ("ice-9 eval-string",
                                                   "eval-string");
          }

          SCM my_eval_string (SCM str)
          {
            return scm_call_1 (scm_variable_ref (eval_string_var), str);
          }

 -- C Function: SCM scm_public_ref (SCM MODULE_NAME, SCM NAME)
 -- C Function: SCM scm_c_public_ref (const char *MODULE_NAME, const
          char *NAME)
 -- C Function: SCM scm_private_ref (SCM MODULE_NAME, SCM NAME)
 -- C Function: SCM scm_c_private_ref (const char *MODULE_NAME, const
          char *NAME)
     Like ‘scm_public_lookup’ or ‘scm_private_lookup’, but additionally
     dereferences the variable.  If the variable object is unbound,
     signals an error.  Returns the value bound to NAME in MODULE_NAME.

   In addition, there are a number of other lookup-related procedures.
We suggest that you use the ‘scm_public_’ and ‘scm_private_’ family of
procedures instead, if possible.

 -- C Function: SCM scm_c_lookup (const char *NAME)
     Return the variable bound to the symbol indicated by NAME in the
     current module.  If there is no such binding or the symbol is not
     bound to a variable, signal an error.

 -- C Function: SCM scm_lookup (SCM NAME)
     Like ‘scm_c_lookup’, but the symbol is specified directly.

 -- C Function: SCM scm_c_module_lookup (SCM MODULE, const char *NAME)
 -- C Function: SCM scm_module_lookup (SCM MODULE, SCM NAME)
     Like ‘scm_c_lookup’ and ‘scm_lookup’, but the specified module is
     used instead of the current one.

 -- C Function: SCM scm_module_variable (SCM MODULE, SCM NAME)
     Like ‘scm_module_lookup’, but if the binding does not exist, just
     returns ‘#f’ instead of raising an error.

   To define a value, use ‘scm_define’:

 -- C Function: SCM scm_c_define (const char *NAME, SCM VAL)
     Bind the symbol indicated by NAME to a variable in the current
     module and set that variable to VAL.  When NAME is already bound to
     a variable, use that.  Else create a new variable.

 -- C Function: SCM scm_define (SCM NAME, SCM VAL)
     Like ‘scm_c_define’, but the symbol is specified directly.

 -- C Function: SCM scm_c_module_define (SCM MODULE, const char *NAME,
          SCM VAL)
 -- C Function: SCM scm_module_define (SCM MODULE, SCM NAME, SCM VAL)
     Like ‘scm_c_define’ and ‘scm_define’, but the specified module is
     used instead of the current one.

   In some rare cases, you may need to access the variable that
‘scm_module_define’ would have accessed, without changing the binding of
the existing variable, if one is present.  In that case, use
‘scm_module_ensure_local_variable’:

 -- C Function: SCM scm_module_ensure_local_variable (SCM MODULE, SCM
          SYM)
     Like ‘scm_module_define’, but if the SYM is already locally bound
     in that module, the variable’s existing binding is not reset.
     Returns a variable.

 -- C Function: SCM scm_module_reverse_lookup (SCM MODULE, SCM VARIABLE)
     Find the symbol that is bound to VARIABLE in MODULE.  When no such
     binding is found, return ‘#f’.

 -- C Function: SCM scm_c_define_module (const char *NAME, void
          (*INIT)(void *), void *DATA)
     Define a new module named NAME and make it current while INIT is
     called, passing it DATA.  Return the module.

     The parameter NAME is a string with the symbols that make up the
     module name, separated by spaces.  For example, ‘"foo bar"’ names
     the module ‘(foo bar)’.

     When there already exists a module named NAME, it is used
     unchanged, otherwise, an empty module is created.

 -- C Function: SCM scm_c_resolve_module (const char *NAME)
     Find the module name NAME and return it.  When it has not already
     been defined, try to auto-load it.  When it can’t be found that way
     either, create an empty module.  The name is interpreted as for
     ‘scm_c_define_module’.

 -- C Function: SCM scm_c_use_module (const char *NAME)
     Add the module named NAME to the uses list of the current module,
     as with ‘(use-modules NAME)’.  The name is interpreted as for
     ‘scm_c_define_module’.

 -- C Function: void scm_c_export (const char *NAME, ...)
     Add the bindings designated by NAME, ...  to the public interface
     of the current module.  The list of names is terminated by ‘NULL’.


File: guile.info,  Node: provide and require,  Next: Environments,  Prev: Accessing Modules from C,  Up: Modules

6.18.11 provide and require
---------------------------

Aubrey Jaffer, mostly to support his portable Scheme library SLIB,
implemented a provide/require mechanism for many Scheme implementations.
Library files in SLIB _provide_ a feature, and when user programs
_require_ that feature, the library file is loaded in.

   For example, the file ‘random.scm’ in the SLIB package contains the
line

     (provide 'random)

   so to use its procedures, a user would type

     (require 'random)

   and they would magically become available, _but still have the same
names!_  So this method is nice, but not as good as a full-featured
module system.

   When SLIB is used with Guile, provide and require can be used to
access its facilities.


File: guile.info,  Node: Environments,  Prev: provide and require,  Up: Modules

6.18.12 Environments
--------------------

Scheme, as defined in R5RS, does _not_ have a full module system.
However it does define the concept of a top-level “environment”.  Such
an environment maps identifiers (symbols) to Scheme objects such as
procedures and lists: *note About Closure::.  In other words, it
implements a set of “bindings”.

   Environments in R5RS can be passed as the second argument to ‘eval’
(*note Fly Evaluation::).  Three procedures are defined to return
environments: ‘scheme-report-environment’, ‘null-environment’ and
‘interaction-environment’ (*note Fly Evaluation::).

   In addition, in Guile any module can be used as an R5RS environment,
i.e., passed as the second argument to ‘eval’.

   Note: the following two procedures are available only when the
‘(ice-9 r5rs)’ module is loaded:

     (use-modules (ice-9 r5rs))

 -- Scheme Procedure: scheme-report-environment version
 -- Scheme Procedure: null-environment version
     VERSION must be the exact integer ‘5’, corresponding to revision 5
     of the Scheme report (the Revised^5 Report on Scheme).
     ‘scheme-report-environment’ returns a specifier for an environment
     that is empty except for all bindings defined in the report that
     are either required or both optional and supported by the
     implementation.  ‘null-environment’ returns a specifier for an
     environment that is empty except for the (syntactic) bindings for
     all syntactic keywords defined in the report that are either
     required or both optional and supported by the implementation.

     Currently Guile does not support values of VERSION for other
     revisions of the report.

     The effect of assigning (through the use of ‘eval’) a variable
     bound in a ‘scheme-report-environment’ (for example ‘car’) is
     unspecified.  Currently the environments specified by
     ‘scheme-report-environment’ are not immutable in Guile.


File: guile.info,  Node: Foreign Function Interface,  Next: Foreign Objects,  Prev: Modules,  Up: API Reference

6.19 Foreign Function Interface
===============================

Sometimes you need to use libraries written in C or Rust or some other
non-Scheme language.  More rarely, you might need to write some C to
extend Guile.  This section describes how to load these “foreign
libraries”, look up data and functions inside them, and so on.

* Menu:

* Foreign Libraries::              Dynamically linking to libraries.
* Foreign Extensions::             Extending Guile in C with loadable modules.
* Foreign Pointers::               Pointers to C data or functions.
* Foreign Types::                  Expressing C types in Scheme.
* Foreign Functions::              Simple calls to C procedures.
* Void Pointers and Byte Access::  Pointers into the ether.
* Foreign Structs::                Packing and unpacking structs.
* More Foreign Functions::         Advanced examples.


File: guile.info,  Node: Foreign Libraries,  Next: Foreign Extensions,  Up: Foreign Function Interface

6.19.1 Foreign Libraries
------------------------

Just as Guile can load up Scheme libraries at run-time, Guile can also
load some system libraries written in C or other low-level languages.
We refer to these as dynamically-loadable modules as “foreign
libraries”, to distinguish them from native libraries written in Scheme
or other languages implemented by Guile.

   Foreign libraries usually come in two forms.  Some foreign libraries
are part of the operating system, such as the compression library
‘libz’.  These shared libraries are built in such a way that many
programs can use their functionality without duplicating their code.
When a program written in C is built, it can declare that it uses a
specific set of shared libraries.  When the program is run, the
operating system takes care of locating and loading the shared
libraries.

   The operating system components that can dynamically load and link
shared libraries when a program is run are also available
programmatically during a program’s execution.  This is the interface
that’s most useful for Guile, and this is what we mean in Guile when we
refer to “dynamic linking”.  Dynamic linking at run-time is sometimes
called “dlopening”, to distinguish it from the dynamic linking that
happens at program start-up.

   The other kind of foreign library is sometimes known as a module,
plug-in, bundle, or an extension.  These foreign libraries aren’t meant
to be linked to by C programs, but rather only to be dynamically loaded
at run-time – they extend some main program with functionality, but
don’t stand on their own.  Sometimes a Guile library will implement some
of its functionality in a loadable module.

   In either case, the interface on the Guile side is the same.  You
load the interface using ‘load-foreign-library’.  The resulting foreign
library object implements a simple lookup interface whereby the user can
get addresses of data or code exported by the library.  There is no
facility to inspect foreign libraries; you have to know what’s in there
already before you look.

   Routines for loading foreign libraries and accessing their contents
are implemented in the ‘(system foreign-library)’ module.

     (use-modules (system foreign-library))

 -- Scheme Procedure: load-foreign-library [library]
          [#:extensions=system-library-extensions]
          [#:search-ltdl-library-path?=#t] [#:search-path=search-path]
          [#:search-system-paths?=#t] [#:lazy?=#t] [#:global=#f]
          [#:host-type-rename?=#t] [#:allow-dll-version-suffix?=#t]

     This procedure finds the shared library denoted by LIBRARY (a
     string) and dynamically links it into the running Guile
     application.  On success, the procedure returns a Scheme object
     suitable for representing the linked object file.  Otherwise an
     error is thrown.

     In the common usage, the LIBRARY parameter is a filename with no
     path and with no filename extension, such as ‘.so’, ‘.dylib’ or
     ‘.dll’.  The procedure will search for the library in a set of
     standard locations using the common filename extensions for the OS.
     The optional parameters can customize this behavior.

     When LIBRARY has directory elements or a filename extension, a more
     targeted search is performed.

     For each system, Guile has a default set of extensions that it will
     try.  On GNU systems, the default extension set is just ‘.so’; on
     Windows, just ‘.dll’; and on Darwin (Mac OS), it is ‘.bundle’,
     ‘.so’, and ‘.dylib’.  Pass ‘#:extensions EXTENSIONS’ to override
     the default extensions list.  If LIBRARY contains one of the
     extensions, no extensions are tried, so it is possible to specify
     the extension if you know exactly what file to load.

     Unless LIBRARY denotes an absolute file name or otherwise contains
     a directory separator (‘/’, and also ‘\’ on Windows), Guile will
     search for the library in the directories listed in SEARCH-PATHS.
     The default search path has three components, which can all be
     overridden by colon-delimited (semicolon on Windows) environment
     variables:

     ‘GUILE_EXTENSIONS_PATH’
          This is the environment variable for users to add directories
          containing Guile extensions to the search path.  The default
          value has no entries.  This environment variable was added in
          Guile 3.0.6.
     ‘LTDL_LIBRARY_PATH’
          When SEARCH-LTDL-LIBRARY-PATH? is true, this environment
          variable can also be used to add directories to the search
          path.  For each directory given in this environment variable,
          two directories are added to the search path: the given
          directory (for example, ‘D’) and a ‘.libs’ subdirectory
          (‘D/.libs’).

          For more information on the rationale, see the note below.
     ‘GUILE_SYSTEM_EXTENSIONS_PATH’
          The last path in Guile’s search path belongs to Guile itself,
          and defaults to the libdir and the extensiondir, in that
          order.  For example, if you install to ‘/opt/guile’, these
          would probably be ‘/opt/guile/lib’ and
          ‘/opt/guile/lib/guile/3.0/extensions’, respectively.  *Note
          Parallel Installations::, for more details on ‘extensionsdir’.

          For DLL-using systems, it searches bindir rather than libdir,
          so ‘/opt/guile/bin’ in this example.

     Finally, if no library is found in the search path, and if LIBRARY
     is not absolute and does not include directory separators, and if
     SEARCH-SYSTEM-PATHS? is true, the operating system may have its own
     logic for where to locate LIBRARY.  For example, on GNU, there will
     be a default set of paths (often ‘/usr/lib’ and ‘/lib’, though it
     depends on the system), and the ‘LD_LIBRARY_PATH’ environment
     variable can add additional paths.  On DLL-using systems, the
     ‘PATH’ is searched.  Other operating systems have other
     conventions.

     Falling back to the operating system for search is usually not a
     great thing; it is a recipe for making programs that work on one
     machine but not on others.  Still, when wrapping system libraries,
     it can be the only way to get things working at all.

     If LAZY? is true (the default), Guile will request the operating
     system to resolve symbols used by the loaded library as they are
     first used.  If GLOBAL? is true, symbols defined by the loaded
     library will be available when other modules need to resolve
     symbols; the default is ‘#f’, which keeps symbols local.

     If HOST-TYPE-RENAME? is true (the default) library names may be
     modified based on the current ‘%host-type’.  On Cygwin hosts, the
     search behavior is modified such that a filename that starts with
     “lib” will be searched for under the name “cyg”, as is customary
     for Cygwin.  Similarly, for MSYS hosts, “lib” becomes “msys-”.

     If DLL-VERSION-SUFFIX? is true (the default), the search behavior
     is modified such that when searching for a DLL, it will also search
     for DLLs with version suffixes.  For example, a search for
     ‘libtiff.dll’ will also allow ‘libtiff-1.dll’.  When the
     unversioned DLL is not found and multiple versioned DLLs exists, it
     will return the versioned DLL with the highest version.  Note that
     when searching, directories take precedence.  It does not return
     the highest versioned DLL among all search directories
     collectively; it returns the highest versioned in the first
     directory to have the DLL.

     If LIBRARY argument is omitted, it defaults to ‘#f’.  If ‘library’
     is false, the resulting foreign library gives access to all symbols
     available for dynamic linking in the currently running executable.

   The environment variables mentioned above are parsed when the
foreign-library module is first loaded and bound to parameters.  Null
path components, for example the three components of
‘GUILE_SYSTEM_EXTENSIONS_PATH="::"’, are ignored.

 -- Scheme Parameter: guile-extensions-path
 -- Scheme Parameter: ltdl-library-path
 -- Scheme Parameter: guile-system-extensions-path
     Parameters whose initial values are taken from
     ‘GUILE_EXTENSIONS_PATH’, ‘LTDL_LIBRARY_PATH’, and
     ‘GUILE_SYSTEM_EXTENSIONS_PATH’, respectively.  *Note Parameters::.
     The current values of these parameters are used when building the
     search path when ‘load-foreign-library’ is called, unless the
     caller explicitly passes a ‘#:search-path’ argument.

 -- Scheme Procedure: foreign-library? obj
     Return ‘#t’ if OBJ is a foreign library, or ‘#f’ otherwise.

   Before Guile 3.0.6, Guile loaded foreign libraries using ‘libltdl’,
the dynamic library loader provided by libtool.  This loader used
‘LTDL_LIBRARY_PATH’, and for backwards compatibility we still support
that path.

   However, ‘libltdl’ would not only open ‘.so’ (or ‘.dll’ and so on)
files, but also the ‘.la’ files created by libtool.  In installed
libraries – libraries that are in the target directories of ‘make
install’ – ‘.la’ files are never needed, to the extent that most
GNU/Linux distributions remove them entirely.  It is sufficient to just
load the ‘.so’ (or ‘.dll’ and so on) files, which are always located in
the same directory as the ‘.la’ files.

   But for uninstalled dynamic libraries, like those in a build tree,
the situation is a bit of a mess.  If you have a project that uses
libtool to build libraries – which is the case for Guile, and for most
projects using autotools – and you build ‘foo.so’ in directory ‘D’,
libtool will put ‘foo.la’ in ‘D’, but ‘foo.so’ gets put into ‘D/.libs’.

   Users were mostly oblivious to this situation, as ‘libltdl’ had
special logic to be able to read the ‘.la’ file to know where to find
the ‘.so’, even from an uninstalled build tree, preventing the existence
of ‘.libs’ from leaking out to the user.

   We don’t use libltdl now, essentially for flexibility and
error-reporting reasons.  But, to keep this old use-case working, if
SEARCH-LTDL-LIBRARY-PATH? is true, we add each entry of
‘LTDL_LIBRARY_PATH’ to the default extensions load path, additionally
adding the ‘.libs’ subdirectories for each entry, in case there are
‘.so’ files there instead of alongside the ‘.la’ files.

