| License | MIT |
|---|---|
| Safe Haskell | None |
| Language | Haskell2010 |
DataFrame.IO.Parquet.Schema
Description
Helpers for converting a parquet schema (a list of SchemaElement) into an
empty DataFrame whose columns have the right types but no rows. Used by
declareColumnsFromParquetFile and by any tooling that wants
to inspect a parquet schema as a DataFrame.
Synopsis
- schemaToEmptyDataFrame :: Set Text -> [SchemaElement] -> DataFrame
- schemaElemToColumn :: Set Text -> SchemaElement -> (Text, Column)
- emptyColumnForType :: Maybe ThriftType -> Column
- emptyNullableColumnForType :: Maybe ThriftType -> Column
Documentation
schemaToEmptyDataFrame :: Set Text -> [SchemaElement] -> DataFrame Source #
Build an empty DataFrame from a flat list of parquet SchemaElements.
Only leaf elements (those with no children) become columns. Columns whose
name is in nullableCols are typed as Maybe a; the rest are typed as a.
schemaElemToColumn :: Set Text -> SchemaElement -> (Text, Column) Source #
Convert a single parquet SchemaElement into a named empty Column,
picking a nullable or non-nullable representation based on nullableCols.
emptyColumnForType :: Maybe ThriftType -> Column Source #
An empty Column of the given parquet physical type.
emptyNullableColumnForType :: Maybe ThriftType -> Column Source #
Like emptyColumnForType but produces a nullable Maybe a column.