Introduction to numericalio
===========================

numericalio is a collection of functions to read and write files and streams. Functions for plain-text input and output can read and write numbers (integer, float, or bigfloat), symbols, and strings. Functions for binary input and output can read and write only floating-point numbers.

If there already exists a list, matrix, or array object to store input data, numericalio input functions can write data into that object. Otherwise, numericalio can guess, to some degree, the structure of an object to store the data, and return that object.

Categories: File input File output Share packages Package numericalio

Plain-text input and output
---------------------------

In plain-text input and output, it is assumed that each item to read or write is an atom: an integer, float, bigfloat, string, or symbol, and not a rational or complex number or any other kind of nonatomic expression. The numericalio functions may attempt to do something sensible faced with nonatomic expressions, but the results are not specified here and subject to change.

Atoms in both input and output files have the same format as in Maxima batch files or the interactive console. In particular, strings are enclosed in double quotes, backslash \ prevents any special interpretation of the next character, and the question mark ? is recognized at the beginning of a symbol to mean a Lisp symbol (as opposed to a Maxima symbol). No continuation character (to join broken lines) is recognized.

Separator flag values for input
-------------------------------

The functions for plain-text input and output take an optional argument, SEPARATOR_FLAG, that tells what character separates data.

For plain-text input, these values of SEPARATOR_FLAG are recognized: comma for comma separated values, pipe for values separated by the vertical bar character |, semicolon for values separated by semicolon ;, and space for values separated by space or tab characters. Equivalently, the separator may be specified as a string of one character: "," (comma), "|" (pipe), ";" (semicolon), " " (space), or "    " (tab).

If the file name ends in .csv and SEPARATOR_FLAG is not specified, comma is assumed. If the file name ends in something other than .csv and separator_flag is not specified, space is assumed.

In plain-text input, multiple successive space and tab characters count as a single separator. However, multiple comma, pipe, or semicolon characters are significant. Successive comma, pipe, or semicolon characters (with or without intervening spaces or tabs) are considered to have false between the separators. For example, 1234,,Foo is treated the same as 1234,false,Foo.

Separator flag values for output
--------------------------------

For plain-text output, tab, for values separated by the tab character, is recognized as a value of SEPARATOR_FLAG, as well as comma, pipe, semicolon, and space.

In plain-text output, false atoms are written as such; a list [1234, false, Foo] is written 1234,false,Foo, and there is no attempt to collapse the output to 1234,,Foo.

Binary floating-point input and output
--------------------------------------

numericalio functions can read and write 8-byte IEEE 754 floating-point numbers. These numbers can be stored either least significant byte first or most significant byte first, according to the global flag set by assume_external_byte_order. If not specified, numericalio assumes the external byte order is most-significant byte first.

Other kinds of numbers are coerced to 8-byte floats; numericalio cannot read or write binary non-numeric data.

Some Lisp implementations do not recognize IEEE 754 special values (positive and negative infinity, not-a-number values, denormalized values). The effect of reading such values with numericalio is undefined.

numericalio includes functions to open a stream for reading or writing a stream of bytes.

Functions and Variables for plain-text input and output
=======================================================

 -- Function: read_matrix
     read_matrix (S)
     read_matrix (S, M)
     read_matrix (S, SEPARATOR_FLAG)
     read_matrix (S, M, SEPARATOR_FLAG)

     read_matrix(S) reads the source S and returns its entire content as a matrix. The size of the matrix is inferred from the input data; each line of the file becomes one row of the matrix. If some lines have different lengths, read_matrix complains.

     read_matrix(S, M) read the source S into the matrix M, until M is full or the source is exhausted. Input data are read into the matrix in row-major order; the input need not have the same number of rows and columns as M.

     The source S may be a file name or a stream which for example allows skipping the very first line of a file (that may be useful, if you read CSV data, where the first line often contains the description of the columns):

          s : openr("data.txt");
          readline(s);  /* skip the first line */
          M : read_matrix(s, 'comma);  /* read the following (comma-separated) lines into matrix M */
          close(s);

     The recognized values of SEPARATOR_FLAG are comma, pipe, semicolon, and space. Equivalently, the separator may be specified as a string of one character: "," (comma), "|" (pipe), ";" (semicolon), " " (space), or "    " (tab). If SEPARATOR_FLAG is not specified, the file is assumed space-delimited.

     See also openr read_array read_hashed_array read_list read_binary_matrix write_data and read_nested_list

     Categories: Package numericalio File input

 -- Function: read_array
     read_array (S, A)
     read_array (S, A, SEPARATOR_FLAG)

     Reads the source S into the array A, until A is full or the source is exhausted. Input data are read into the array in row-major order; the input need not conform to the dimensions of A.

     The source S may be a file name or a stream.

     The recognized values of SEPARATOR_FLAG are comma, pipe, semicolon, and space. Equivalently, the separator may be specified as a string of one character: "," (comma), "|" (pipe), ";" (semicolon), " " (space), or "    " (tab). If SEPARATOR_FLAG is not specified, the file is assumed space-delimited.

     See also openr read_matrix read_hashed_array read_list read_binary_array and read_nested_list

     Categories: Package numericalio File input

 -- Function: read_hashed_array
     read_hashed_array (S, A)
     read_hashed_array (S, A, SEPARATOR_FLAG)

     Reads the source S and returns its entire content as a hashed array The source S may be a file name or a stream.

     read_hashed_array treats the first item on each line as a hash key, and associates the remainder of the line (as a list) with the key. For example, the line 567 12 17 32 55 is equivalent to A[567]: [12, 17, 32, 55]$. Lines need not have the same numbers of elements.

     The recognized values of SEPARATOR_FLAG are comma, pipe, semicolon, and space. Equivalently, the separator may be specified as a string of one character: "," (comma), "|" (pipe), ";" (semicolon), " " (space), or "    " (tab). If SEPARATOR_FLAG is not specified, the file is assumed space-delimited.

     See also openr read_matrix read_array read_list and read_nested_list

     Categories: Package numericalio File input

 -- Function: read_nested_list
     read_nested_list (S)
     read_nested_list (S, SEPARATOR_FLAG)

     Reads the source S and returns its entire content as a nested list. The source S may be a file name or a stream.

     read_nested_list returns a list which has a sublist for each line of input. Lines need not have the same numbers of elements. Empty lines are not ignored: an empty line yields an empty sublist.

     The recognized values of SEPARATOR_FLAG are comma, pipe, semicolon, and space. Equivalently, the separator may be specified as a string of one character: "," (comma), "|" (pipe), ";" (semicolon), " " (space), or "    " (tab). If SEPARATOR_FLAG is not specified, the file is assumed space-delimited.

     See also openr read_matrix read_array read_list and read_hashed_array

     Categories: Package numericalio File input

 -- Function: read_list
     read_list (S)
     read_list (S, L)
     read_list (S, SEPARATOR_FLAG)
     read_list (S, L, SEPARATOR_FLAG)

     read_list(S) reads the source S and returns its entire content as a flat list.

     read_list(S, L) reads the source S into the list L, until L is full or the source is exhausted.

     The source S may be a file name or a stream.

     The recognized values of SEPARATOR_FLAG are comma, pipe, semicolon, and space. Equivalently, the separator may be specified as a string of one character: "," (comma), "|" (pipe), ";" (semicolon), " " (space), or "    " (tab). If SEPARATOR_FLAG is not specified, the file is assumed space-delimited.

     See also openr read_matrix read_array read_nested_list read_binary_list and read_hashed_array

     Categories: Package numericalio File input

 -- Function: write_data
     write_data (X, D)
     write_data (X, D, SEPARATOR_FLAG)

     Writes the object X to the destination D.

     write_data writes a matrix in row-major order, with one line per row.

     write_data writes an array created by array or make_array in row-major order, with a new line at the end of every slab. Higher-dimensional slabs are separated by additional new lines.

     write_data writes a hashed array with each key followed by its associated list on one line.

     write_data writes a nested list with each sublist on one line.

     write_data writes a flat list all on one line.

     The destination D may be a file name or a stream. When the destination is a file name, the global variable file_output_append governs whether the output file is appended or truncated. When the destination is a stream, no special action is taken by write_data after all the data are written; in particular, the stream remains open.

     The recognized values of SEPARATOR_FLAG are comma, pipe, semicolon, space, and tab. Equivalently, the separator may be specified as a string of one character: "," (comma), "|" (pipe), ";" (semicolon), " " (space), or "    " (tab). If SEPARATOR_FLAG is not specified, the file is assumed space-delimited.

     See also openw and read_matrix

     Categories: Package numericalio File output

Functions and Variables for binary input and output
===================================================

 -- Function: assume_external_byte_order (BYTE_ORDER_FLAG)
     Tells numericalio the byte order for reading and writing binary data. Two values of BYTE_ORDER_FLAG are recognized: lsb which indicates least-significant byte first, also called little-endian byte order; and msb which indicates most-significant byte first, also called big-endian byte order.

     If not specified, numericalio assumes the external byte order is most-significant byte first.

     Categories: Package numericalio

 -- Function: openr_binary (FILE_NAME)
     Returns an input stream of 8-bit unsigned bytes to read the file named by FILE_NAME.

     See also openw_binary and openr

     Categories: Package numericalio File input

 -- Function: openw_binary (FILE_NAME)
     Returns an output stream of 8-bit unsigned bytes to write the file named by FILE_NAME.

     See also openr_binary opena_binary and openw

     Categories: Package numericalio File output

 -- Function: opena_binary (FILE_NAME)
     Returns an output stream of 8-bit unsigned bytes to append the file named by FILE_NAME.

     Categories: Package numericalio File output

 -- Function: read_binary_matrix (S, M)
     Reads binary 8-byte floating point numbers from the source S into the matrix M until M is full, or the source is exhausted. Elements of M are read in row-major order.

     The source S may be a file name or a stream.

     The byte order in elements of the source is specified by assume_external_byte_order.

     See also read_matrix

     Categories: Package numericalio File input

 -- Function: read_binary_array (S, A)
     Reads binary 8-byte floating point numbers from the source S into the array A until A is full, or the source is exhausted. A must be an array created by array or make_array. Elements of A are read in row-major order.

     The source S may be a file name or a stream.

     The byte order in elements of the source is specified by assume_external_byte_order.

     See also read_array

     Categories: Package numericalio File input

 -- Function: read_binary_list
     read_binary_list (S)
     read_binary_list (S, L)

     read_binary_list(S) reads the entire content of the source S as a sequence of binary 8-byte floating point numbers, and returns it as a list. The source S may be a file name or a stream.

     read_binary_list(S, L) reads 8-byte binary floating point numbers from the source S until the list L is full, or the source is exhausted.

     The byte order in elements of the source is specified by assume_external_byte_order.

     See also read_list

     Categories: Package numericalio File input

 -- Function: write_binary_data (X, D)

     Writes the object X, comprising binary 8-byte IEEE 754 floating-point numbers, to the destination D. Other kinds of numbers are coerced to 8-byte floats. write_binary_data cannot write non-numeric data.

     The object X may be a list, a nested list, a matrix, or an array created by array or make_array; X cannot be a hashed array or any other type of object. write_binary_data writes nested lists, matrices, and arrays in row-major order.

     The destination D may be a file name or a stream. When the destination is a file name, the global variable file_output_append governs whether the output file is appended or truncated. When the destination is a stream, no special action is taken by write_binary_data after all the data are written; in particular, the stream remains open.

     The byte order in elements of the destination is specified by assume_external_byte_order.

     See also write_data

     Categories: Package numericalio File output