Module pl.data
Reading and querying simple tabular data.
data.read 'test.txt' ==> {{10,20},{2,5},{40,50},fieldnames={'x','y'},delim=','}
Provides a way of creating basic SQL-like queries.
require 'pl' local d = data.read('xyz.txt') local q = d:select('x,y,z where x > 3 and z < 2 sort by y') for x,y,z in q do print(x,y,z) end
See the Guide
Dependencies: pl.utils, pl.array2d (fallback methods)
Functions
| Data.column_by_name (name) | return a particular column as a list of values (method). |
| Data.select (condn) | return a query iterator on this data (method). |
| Data.select_row (condn) | return a row iterator on this data (method). |
| Data.copy_select (condn) | return a new data object based on this query (method). |
| Data.column_names () | return the field names of this data object (method). |
| Data.write_row (f) | write out a row (method). |
| Data.write (f) | write data out to file (method). |
| read (file, cnfg) | read a delimited file in a Lua table. |
| write (data, file[, fieldnames[, delim='\t']]) | write 2D data to a file. |
| new (d[, fieldnames]) | create a new dataset from a table of rows. |
| query (data, condn, context, return_row) | create a query iterator from a select string. |
| filter (Q, infile, outfile, dont_fail) | Filter input using a query. |
Functions
- Data.column_by_name (name)
-
return a particular column as a list of values (method).
Parameters:
- name either name of column, or numerical index.
- Data.select (condn)
-
return a query iterator on this data (method).
Parameters:
- condn string the query expression
See also:
- Data.select_row (condn)
-
return a row iterator on this data (method).
Parameters:
- condn string the query expression
- Data.copy_select (condn)
-
return a new data object based on this query (method).
Parameters:
- condn string the query expression
- Data.column_names ()
- return the field names of this data object (method).
- Data.write_row (f)
-
write out a row (method).
Parameters:
- f file-like object
- Data.write (f)
-
write data out to file (method).
Parameters:
- f file-like object
- read (file, cnfg)
-
read a delimited file in a Lua table.
By default, attempts to treat first line as separated list of fieldnames.
Parameters:
- file a filename or a file-like object
- cnfg parsing options
- delim string a string pattern to split fields
- fieldnames array (i.e. don't read from first line)
- no_convert boolean (default is to try conversion on first data line)
- convert table table of custom conversion functions with column keys
- numfields integer indices of columns known to be numbers
- last_field_collect boolean only split as many fields as fieldnames.
- thousands_dot integer thousands separator in Excel CSV is '.'
- csv boolean fields may be double-quoted and contain commas; Also, empty fields are considered to be equivalent to zero.
Returns:
-
data object, or
nil - error message. May be a file error, 'not a file-like object' or a conversion error
- write (data, file[, fieldnames[, delim='\t']])
-
write 2D data to a file.
Does not assume that the data has actually been
generated with new or read.
Parameters:
- data 2D array
- file filename or file-like object
- fieldnames {string} list of fields (optional) (optional)
- delim string delimiter (default tab) (default '\t')
Returns:
-
true or nil, error
- new (d[, fieldnames])
-
create a new dataset from a table of rows.
Can specify the fieldnames, else the table must have a field called
'fieldnames', which is either a string of delimiter-separated names,
or a table of names.
If the table does not have a field called 'delim', then an attempt will be made to guess it from the fieldnames string, defaults otherwise to tab.Parameters:
- d the table.
- fieldnames {string} optional fieldnames (optional)
Returns:
-
the table.
- query (data, condn, context, return_row)
-
create a query iterator from a select string.
Select string has this format:
FIELDLIST [ where LUA-CONDN [ sort by FIELD] ]
FIELDLIST is a comma-separated list of valid fields, or '*'.
The condition can also be a table, with fields 'fields' (comma-sep string or table), 'sort_by' (string) and 'where' (Lua expression string or function)Parameters:
- data table produced by read
- condn select string or table
- context a list of tables to be searched when resolving functions
- return_row if true, wrap the results in a row table
Returns:
- an iterator over the specified fields, or nil
- an error message
- filter (Q, infile, outfile, dont_fail)
-
Filter input using a query.
Parameters:
- Q string a query string
- infile filename or file-like object
- outfile filename or file-like object
- dont_fail boolean true if you want to return an error, not just fail