| Safe Haskell | None |
|---|---|
| Language | Haskell2010 |
DataFrame.Expr.Serialize
Description
Persist dataframe expressions and pipelines, and load them back for inference in another process (or from Python — see DataFrame.IR.ExprJson for the shared wire format).
- A single
Exprround-trips viaencodeExpr/decodeExprAny(or the type-assertingdecodeExprAt) and the file helpers below. - A pipeline is an ordered
[— the same shape a fittedNamedExpr]Transformcarries.encodeNamedExprs/decodeNamedExprswrap it in a versioned envelope; replay a decoded pipeline for inference withderiveMany/applyTransform.
All functions are pure Either; the save helpers report an encode failure
before touching the file (no partial write), and the load helpers turn a
missing/unreadable file into a Left rather than throwing.
Limitations inherited from the wire format: CastWith / CastExprWith (they
hold closures) and collect (its list-typed output has no type tag) cannot be
serialized.
Synopsis
- data SomeExpr where
- SomeExpr :: forall a. Columnable a => TypeRep a -> Expr a -> SomeExpr
- encodeExpr :: Columnable a => Expr a -> Either String Value
- encodeExprToBytes :: Columnable a => Expr a -> Either String ByteString
- decodeExprAny :: Value -> Either String SomeExpr
- decodeExprAt :: Columnable a => Value -> Either String (Expr a)
- decodeExprFromBytes :: ByteString -> Either String SomeExpr
- encodeNamedExprs :: [NamedExpr] -> Either String Value
- decodeNamedExprs :: Value -> Either String [NamedExpr]
- saveExprToFile :: Columnable a => FilePath -> Expr a -> IO (Either String ())
- loadExprFromFile :: FilePath -> IO (Either String SomeExpr)
- loadExprAtFromFile :: Columnable a => FilePath -> IO (Either String (Expr a))
- savePipelineToFile :: FilePath -> [NamedExpr] -> IO (Either String ())
- loadPipelineFromFile :: FilePath -> IO (Either String [NamedExpr])
Single expressions (re-exported wire codec)
Existential wrapper around a typed expression decoded from JSON. TODO: mchavinda - Maybe consolidate with UExpr from the main package.
Constructors
| SomeExpr :: forall a. Columnable a => TypeRep a -> Expr a -> SomeExpr |
encodeExpr :: Columnable a => Expr a -> Either String Value Source #
encodeExprToBytes :: Columnable a => Expr a -> Either String ByteString Source #
Encode an Expr to a strict ByteString of JSON.
decodeExprAny :: Value -> Either String SomeExpr Source #
Decode a JSON value into a SomeExpr. The output type comes from the
out_type field on the root node.
decodeExprAt :: Columnable a => Value -> Either String (Expr a) Source #
Decode a JSON value, asserting the result type matches a.
decodeExprFromBytes :: ByteString -> Either String SomeExpr Source #
Decode a single expression from a strict JSON ByteString.
Pipelines
encodeNamedExprs :: [NamedExpr] -> Either String Value Source #
Encode an ordered pipeline (named output expressions) into the versioned
envelope { "version": 1, "outputs": [ { "name", "expr" }, … ] }.
decodeNamedExprs :: Value -> Either String [NamedExpr] Source #
Decode a pipeline produced by encodeNamedExprs.
File IO
saveExprToFile :: Columnable a => FilePath -> Expr a -> IO (Either String ()) Source #
Encode an expression and write it to a file. No file is written on failure.
loadExprFromFile :: FilePath -> IO (Either String SomeExpr) Source #
Load an expression of unknown output type from a file.
loadExprAtFromFile :: Columnable a => FilePath -> IO (Either String (Expr a)) Source #
Load an expression from a file, asserting its output type. Fails with
Left if the stored expression has a different output type.
savePipelineToFile :: FilePath -> [NamedExpr] -> IO (Either String ()) Source #
Encode a pipeline and write it to a file. No file is written on failure.
loadPipelineFromFile :: FilePath -> IO (Either String [NamedExpr]) Source #
Load a pipeline produced by savePipelineToFile.