| Safe Haskell | None |
|---|---|
| Language | Haskell2010 |
DataFrame.IO.Parquet.Seeking
Description
This module contains low-level utilities around file seeking.
later this module can be renamed / moved to an internal module.
Synopsis
- data SeekableHandle
- data SeekMode
- data FileBufferedOrSeekable
- type ForceNonSeekable = Maybe Bool
- mkFileBufferedOrSeekable :: ForceNonSeekable -> Handle -> IO FileBufferedOrSeekable
- mkSeekableHandle :: Handle -> IO (Maybe SeekableHandle)
- readLastBytes :: Integer -> FileBufferedOrSeekable -> IO ByteString
- withFileBufferedOrSeekable :: ForceNonSeekable -> FilePath -> IOMode -> (FileBufferedOrSeekable -> IO a) -> IO a
- fSeek :: FileBufferedOrSeekable -> SeekMode -> Integer -> IO ()
- fGet :: FileBufferedOrSeekable -> Int -> IO ByteString
Documentation
data SeekableHandle Source #
This handle carries a proof that it must be seekable. Note: Handle and SeekableHandle are not thread safe, should not be shared across threads, beaware when running parallel/concurrent code.
Not seekable: - stdin / stdout - pipes / FIFOs
But regular files are always seekable. Parquet fundamentally wants random access, a non-seekable source will not support effecient access without buffering the entire file.
A mode that determines the effect of hSeek hdl mode i.
Constructors
| AbsoluteSeek | the position of |
| RelativeSeek | the position of |
| SeekFromEnd | the position of |
Instances
| Enum SeekMode | Since: base-4.2.0.0 |
| Ix SeekMode | Since: base-4.2.0.0 |
Defined in GHC.IO.Device Methods range :: (SeekMode, SeekMode) -> [SeekMode] # index :: (SeekMode, SeekMode) -> SeekMode -> Int # unsafeIndex :: (SeekMode, SeekMode) -> SeekMode -> Int # inRange :: (SeekMode, SeekMode) -> SeekMode -> Bool # rangeSize :: (SeekMode, SeekMode) -> Int # unsafeRangeSize :: (SeekMode, SeekMode) -> Int # | |
| Read SeekMode | Since: base-4.2.0.0 |
| Show SeekMode | Since: base-4.2.0.0 |
| Eq SeekMode | Since: base-4.2.0.0 |
| Ord SeekMode | Since: base-4.2.0.0 |
Defined in GHC.IO.Device | |
data FileBufferedOrSeekable Source #
If we truely want to support non-seekable files, we need to also consider the case to buffer the entire file in memory.
Not thread safe, contains mutable reference (as Handle already is).
If we need concurrent / parallel parsing or something, we need to read into ByteString first, not sharing the same handle.
Constructors
| FileBuffered !(IORef Int64) !ByteString | |
| FileSeekable !SeekableHandle |
Instances
| RandomAccess LocalFile Source # | |
Defined in DataFrame.IO.Utils.RandomAccess Methods readBytes :: Range -> LocalFile ByteString Source # readRanges :: [Range] -> LocalFile [ByteString] Source # readSuffix :: Int -> LocalFile ByteString Source # | |
type ForceNonSeekable = Maybe Bool Source #
For testing only
mkFileBufferedOrSeekable :: ForceNonSeekable -> Handle -> IO FileBufferedOrSeekable Source #
Smart constructor for FileBufferedOrSeekable, tries to keep in the seekable case if possible.
mkSeekableHandle :: Handle -> IO (Maybe SeekableHandle) Source #
Smart constructor for SeekableHandle
readLastBytes :: Integer -> FileBufferedOrSeekable -> IO ByteString Source #
Read the last n bytes, useful for reading metadata without loading the
entire file. Uses hGet (not hGetContents) so the handle stays open for
the subsequent column-chunk reads.
withFileBufferedOrSeekable :: ForceNonSeekable -> FilePath -> IOMode -> (FileBufferedOrSeekable -> IO a) -> IO a Source #
With / bracket pattern for FileBufferedOrSeekable
Warning: do not return the FileBufferedOrSeekable outside the scope of the action as it will be closed.
fGet :: FileBufferedOrSeekable -> Int -> IO ByteString Source #