Read CSV file from path and load it into a dataframe.

Example

ghci> D.readCsv "./data/taxi.csv"

Read TSV (tab separated) file from path and load it into a dataframe.

Example

ghci> D.readTsv "./data/taxi.tsv"

Read CSV file from path and load it into a dataframe.

Example

ghci> D.readCsvWithOpts "./data/taxi.csv" (D.defaultReadOptions { dateFormat = "%d%-m%-Y" })

Read text file with specified delimiter into a dataframe.

Example

ghci> D.readSeparated (D.defaultReadOptions { columnSeparator = ';' }) "./data/taxi.txt"

Schema-driven CSV reader. Coerces each column to the type declared in Schema; columns absent from the schema fall back to inference and are still returned. To read only the schema's columns, pass schemaReadOptions to readCsvWithOpts.

import qualified DataFrame as D
df <- D.readCsvWithSchema schema "input.csv"

A reader the lazy scan can drive: readSeparated here, or fastReadCsvWithOpts from dataframe-fastcsv. The scan owns the ReadOptions, so every reader it accepts honours the projection it asks for.

Example

myReader :: CsvReader
myReader opts path = D.readCsvWithOpts opts path

Read options a scan derives from its Schema: the schema assigns the column types and selects the columns, so a scan reads only what its schema names — the same contract as scanParquet.

Example

ghci> schema = D.makeSchema [("id", D.schemaType @Int), ("name", D.schemaType @Text)]
ghci> D.readCsvWithOpts (schemaReadOptions schema) "./data/customers.csv"

Decode in-memory CSV bytes into a dataframe. The result is fully forced. (Note: unlike readSeparated, no UTF-8 BOM is stripped.)

Example

ghci> D.decodeSeparated D.defaultReadOptions "id,name\n1,Ada\n"

Parse a CSV string into a DataFrame using default options.

Example

ghci> D.fromCsv "id,name\n1,Ada\n"
Right (DataFrame ...)

Parse a lazy ByteString containing CSV data into a DataFrame using default options.

Example

ghci> D.fromCsvBytes "id,name\n1,Ada\n"

CSV read parameters.

How a reader decides each column's type.

Example

ghci> D.readCsvWithOpts D.defaultReadOptions{D.typeSpec = D.InferFromSample 500} "wide.csv"

The default ReadOptions: infer types from a 100-row sample, treat the first row as a header, read every row and every column.

Example

ghci> D.readCsvWithOpts D.defaultReadOptions{D.columnSeparator = ';'} "data.csv"

Where a reader's column names come from.

Example

ghci> D.readCsvWithOpts D.defaultReadOptions{D.headerSpec = D.ProvideNames ["a", "b"]} "no_header.csv"

How the fast reader should treat a row whose field count does not match the header row. Only consulted by dataframe-fastcsv; the pure Haskell reader always pads short rows with nulls and drops extras.

How the fast reader should treat an unclosed quoted field at EOF.

Whether a TypeSpec still infers types from a sample, once any SpecifyTypes fallback chain is followed to its end.

Example

>>> shouldInferFromSample (InferFromSample 100)
True
>>> shouldInferFromSample NoInference
False

The explicit column/type pairs a TypeSpec declares, if any.

Example

>>> :set -XTypeApplications
>>> M.toList (schemaTypeMap (SpecifyTypes [("id", schemaType @Int)] (InferFromSample 100)))
[("id",Int)]

The sample size a TypeSpec infers from, once any SpecifyTypes fallback chain is followed to its end. 0 when the chain ends in NoInference.

Example

>>> typeInferenceSampleSize (InferFromSample 100)
100
>>> typeInferenceSampleSize NoInference
0

Resolve readColumns against a file's header names: the columns to build, each paired with its field index within a row. Nothing keeps every column in file order; a selection keeps the order it was written in.

Example

>>> resolveSelection defaultReadOptions{readColumns = Just ["b", "a"]} ["a", "b", "c"]
[("b",1),("a",0)]
>>> resolveSelection defaultReadOptions ["a", "b"]
[("a",0),("b",1)]

A ReadOptions value that contradicts itself, with every problem listed. Thrown by validateReadOptions; see readOptionsErrors for the check without the exception.

Example

>>> show (InvalidReadOptions ["readColumns is an empty selection; omit it to read every column"])
"Invalid ReadOptions:\n  - readColumns is an empty selection; omit it to read every column"

Everything self-contradictory about opts, judged without looking at a file. A type or safe-read override naming an unread column is redundant, not contradictory, and is not reported.

Example

>>> readOptionsErrors defaultReadOptions{readColumns = Just []}
["readColumns is an empty selection; omit it to read every column"]
>>> readOptionsErrors defaultReadOptions
[]

Throw InvalidReadOptions if opts contradicts itself. Both readers call this before opening the file.

Example

ghci> D.validateReadOptions D.defaultReadOptions{D.readColumns = Just []}
*** Exception: Invalid ReadOptions:
  - readColumns is an empty selection; omit it to read every column

Write a dataframe to a comma-separated file.

Example

ghci> D.writeCsv "./out.csv" df

Write a dataframe to a tab-separated file.

Example

ghci> D.writeTsv "./out.tsv" df

Write a dataframe to a file using the given field separator.

Example

ghci> D.writeSeparated ';' "./out.txt" df

Strip one layer of double-quote delimiters from a field, if present.

Example

>>> stripQuotes "\"hello\""
"hello"
>>> stripQuotes "hello"
"hello"