This module implements encoding and decoding of comma-separated values (CSV) data. The implementation is RFC 4180 compliant, with the following extensions:

• Empty lines are ignored.
• Non-escaped fields may contain any characters except double-quotes, commas, carriage returns, and newlines.
• Escaped fields may contain any characters (but double-quotes need to be escaped).

Encoding standard Haskell types:

>>> :set -XOverloadedStrings
>>> import Data.Text (Text)
>>> encode [("John" :: Text, 27 :: Int), ("Jane", 28)]
"John,27\r\nJane,28\r\n"

Since we enabled the -XOverloadedStrings extension, string literals are polymorphic and we have to supply a type signature as the compiler couldn't deduce which string type (i.e. String, ShortText, or Text) we want to use. In most cases type inference will infer the type from the context and you can omit type signatures.

Decoding standard Haskell types:

>>> import Data.Vector (Vector)
>>> decode NoHeader "John,27\r\nJane,28\r\n" :: Either String (Vector (Text, Int))
Right [("John",27),("Jane",28)]

We pass NoHeader as the first argument to indicate that the CSV input data isn't preceded by a header.

In practice, the return type of decode rarely needs to be given, as it can often be inferred from the context.

To encode and decode your own data types you need to defined instances of either ToRecord and FromRecord or ToNamedRecord and FromNamedRecord. The former is used for encoding/decoding using the column index and the latter using the column name.

There are two ways to to define these instances, either by manually defining them or by using GHC generics to derive them automatically.

GHC.Generics-derived:

{-# LANGUAGE DeriveGeneric #-}

import Data.Text    (Text)
import GHC.Generics (Generic)

data Person = Person { name :: !Text , salary :: !Int }
    deriving (Generic, Show)

instance FromRecord Person
instance ToRecord Person

Manually defined:

import Control.Monad (mzero)

data Person = Person { name :: !Text , salary :: !Int }
    deriving (Show)

instance FromRecord Person where
    parseRecord v
        | length v == 2 = Person <$> v .! 0 <*> v .! 1
        | otherwise     = mzero
instance ToRecord Person where
    toRecord (Person name' age') = record [
        toField name', toField age']

We can now use e.g. encode and decode to encode and decode our data type.

Encoding:

>>> encode [Person ("John" :: Text) 27]
"John,27\r\n"

Decoding:

>>> decode NoHeader "John,27\r\n" :: Either String (Vector Person)
Right [Person {name = "John", salary = 27}]

GHC.Generics-derived:

{-# LANGUAGE DeriveGeneric #-}

import Data.Text    (Text)
import GHC.Generics (Generic)

data Person = Person { name :: !Text , salary :: !Int }
    deriving (Generic, Show)

instance FromNamedRecord Person
instance ToNamedRecord Person
instance DefaultOrdered Person

Manually defined:

data Person = Person { name :: !Text , salary :: !Int }
    deriving (Show)

instance FromNamedRecord Person where
    parseNamedRecord m = Person <$> m .: "name" <*> m .: "salary"
instance ToNamedRecord Person where
    toNamedRecord (Person name salary) = namedRecord [
        "name" .= name, "salary" .= salary]
instance DefaultOrdered Person where
    headerOrder _ = header ["name", "salary"]

We can now use e.g. encodeDefaultOrderedByName (or encodeByName with an explicit header order) and decodeByName to encode and decode our data type.

Encoding:

>>> encodeDefaultOrderedByName [Person ("John" :: Text) 27]
"name,salary\r\nJohn,27\r\n"

Decoding:

>>> decodeByName "name,salary\r\nJohn,27\r\n" :: Either String (Header, Vector Person)
Right (["name","salary"],[Person {name = "John", salary = 27}])

Demonstration of reading from a CSV file/ writing to a CSV file using the incremental API:

{-# LANGUAGE BangPatterns      #-}
{-# LANGUAGE DeriveGeneric     #-}
{-# LANGUAGE LambdaCase        #-}
{-# LANGUAGE OverloadedStrings #-}

-- from base
import GHC.Generics
import System.IO
import System.Exit (exitFailure)
-- from bytestring
import Data.ByteString (ByteString, hGetSome, empty)
import qualified Data.ByteString.Lazy as BL
-- from cassava
import Data.Csv.Incremental
import Data.Csv (FromRecord, ToRecord)

data Person = Person
  { name :: !ByteString
  , age  :: !Int
  } deriving (Show, Eq, Generic)

instance FromRecord Person
instance ToRecord Person

persons :: [Person]
persons = [Person "John Doe" 19, Person "Smith" 20]

writeToFile :: IO ()
writeToFile = do
  BL.writeFile "persons.csv" $ encode $
    foldMap encodeRecord persons

feed :: (ByteString -> Parser Person) -> Handle -> IO (Parser Person)
feed k csvFile = do
  hIsEOF csvFile >>= \case
    True  -> return $ k empty
    False -> k <$> hGetSome csvFile 4096

readFromFile :: IO ()
readFromFile = do
  withFile "persons.csv" ReadMode $ \ csvFile -> do
    let loop !_ (Fail _ errMsg) = do putStrLn errMsg; exitFailure
        loop acc (Many rs k)    = loop (acc <> rs) =<< feed k csvFile
        loop acc (Done rs)      = print (acc <> rs)
    loop [] (decode NoHeader)

main :: IO ()
main = do
  writeToFile
  readFromFile

Sometimes you might want to work with a CSV file which contents is unknown to you. For example, you might want remove the second column of a file without knowing anything about its content. To parse a CSV file to a generic representation, just convert each record to a Vector ByteString value, like so:

>>> import Data.ByteString (ByteString)
>>> decode NoHeader "John,27\r\nJane,28\r\n" :: Either String (Vector (Vector ByteString))
Right [["John","27"],["Jane","28"]]

As the example output above shows, all the fields are returned as uninterpreted ByteString values.

Most of the time the existing FromField and ToField instances do what you want. However, if you need to parse a different format (e.g. hex) but use a type (e.g. Int) for which there's already a FromField instance, you need to use a newtype. Example:

newtype Hex = Hex Int

parseHex :: ByteString -> Parser Int
parseHex = ...

instance FromField Hex where
    parseField s = Hex <$> parseHex s

Other than giving an explicit type signature, you can pattern match on the newtype constructor to indicate which type conversion you want to have the library use:

case decode NoHeader "0xff,0xaa\r\n0x11,0x22\r\n" of
    Left err -> putStrLn err
    Right v  -> forM_ v $ \ (Hex val1, Hex val2) ->
        print (val1, val2)

If a field might be in one several different formats, you can use a newtype to normalize the result:

newtype HexOrDecimal = HexOrDecimal Int

instance FromField DefaultToZero where
    parseField s = case runParser (parseField s :: Parser Hex) of
        Left err -> HexOrDecimal <$> parseField s  -- Uses Int instance
        Right n  -> pure $ HexOrDecimal n

You can use the unit type, (), to ignore a column. The parseField method for () doesn't look at the Field and thus always decodes successfully. Note that it lacks a corresponding ToField instance. Example:

case decode NoHeader "foo,1\r\nbar,22" of
    Left  err -> putStrLn err
    Right v   -> forM_ v $ \ ((), i) -> print (i :: Int)

If your input might contain invalid fields, you can write a custom FromField instance to deal with them. Example:

newtype DefaultToZero = DefaultToZero Int

instance FromField DefaultToZero where
    parseField s = case runParser (parseField s) of
        Left err -> pure $ DefaultToZero 0
        Right n  -> pure $ DefaultToZero n

Encoding and decoding is a two step process. To encode a value, it is first converted to a generic representation, using either ToRecord or ToNamedRecord. The generic representation is then encoded as CSV data. To decode a value the process is reversed and either FromRecord or FromNamedRecord is used instead. Both these steps are combined in the encode and decode functions.

These functions can be used to control how data is encoded and decoded. For example, they can be used to encode data in a tab-separated format instead of in a comma-separated format.

There are two ways to convert CSV records to and from and user-defined data types: index-based conversion and name-based conversion.

Index-based conversion lets you convert CSV records to and from user-defined data types by referring to a field's position (its index) in the record. The first column in a CSV file is given index 0, the second index 1, and so on.

Name-based conversion lets you convert CSV records to and from user-defined data types by referring to a field's name. The names of the fields are defined by the first line in the file, also known as the header. Name-based conversion is more robust to changes in the file structure e.g. to reording or addition of columns, but can be a bit slower.

The FromField and ToField classes define how to convert between Fields and values you care about (e.g. Ints). Most of the time you don't need to write your own instances as the standard ones cover most use cases.

There may be times that you do not want to manually write out class instances for record conversion, but you can't rely upon the default instances (e.g. you can't create field names that match the actual column names in expected data).

For example, consider you have a type MyType where you have prefixed certain columns with an underscore, but in the actual data they're not. You can then write:

myOptions :: Options
myOptions = defaultOptions { fieldLabelModifier = rmUnderscore }
  where
    rmUnderscore ('_':str) = str
    rmUnderscore str       = str

instance ToNamedRecord MyType where
  toNamedRecord = genericToNamedRecord myOptions

instance FromNamedRecord MyType where
  parseNamedRecord = genericParseNamedRecord myOptions

instance DefaultOrdered MyType where
  headerOrder = genericHeaderOrder myOptions

NOTE: Only the class names are exposed in order to make it possible to write type signatures referring to these classes