A space-efficient representation of a succession of Word8 vectors, supporting many efficient operations.

An effectful ByteString contains 8-bit bytes, or by using the operations from Data.ByteString.Streaming.Char8 it can be interpreted as containing 8-bit characters.

Smart constructor for Chunk.

The memory management overhead. Currently this is tuned for GHC only.

The chunk size used for I/O. Currently set to 32k, less the memory management overhead

Construct a succession of chunks from its Church encoding (compare GHC.Exts.build)

Resolve a succession of chunks into its Church encoding; this is not a safe operation; it is equivalent to exposing the constructors

Consume the chunks of an effectful ByteString with a natural right fold.

Consume the chunks of an effectful ByteString with a natural right monadic fold.

chunkFold is preferable to foldlChunks since it is an appropriate argument for Control.Foldl.purely which permits many folds and sinks to be run simulaneously on one bytestream.

chunkFoldM is preferable to foldlChunksM since it is an appropriate argument for Control.Foldl.impurely which permits many folds and sinks to be run simulaneously on one bytestream.

The recommended chunk size. Currently set to 4k, less the memory management overhead

Packing and unpacking from lists packBytes' :: Monad m => [Word8] -> ByteString m () packBytes' cs0 = packChunks 32 cs0 where packChunks n cs = case S.packUptoLenBytes n cs of (bs, []) -> Chunk bs (Empty ()) (bs, cs') -> Chunk bs (packChunks (min (n * 2) BI.smallChunkSize) cs') -- packUptoLenBytes :: Int -> [Word8] -> (ByteString, [Word8]) packUptoLenBytes len xs0 = unsafeDupablePerformIO (createUptoN' len $ p -> go p len xs0) where go !_ !n [] = return (len-n, []) go !_ !0 xs = return (len, xs) go !p !n (x:xs) = poke p x >> go (p plusPtr 1) (n-1) xs createUptoN' :: Int -> (Ptr Word8 -> IO (Int, a)) -> IO (S.ByteString, a) createUptoN' l f = do fp <- S.mallocByteString l (l', res) withForeignPtr fp $ p - f p assert (l' <= l) $ return (S.PS fp 0 l', res) {--}

Yield-style smart constructor for Chunk.

Reconceive an effect that results in an effectful bytestring as an effectful bytestring. Compare Streaming.mwrap. The closes equivalent of

>>> Streaming.wrap :: f (Stream f m r) -> Stream f m r

is here consChunk. mwrap is the smart constructor for the internal Go constructor.

Stream chunks from something that contains IO (Maybe ByteString) until it returns Nothing. reread is of particular use rendering io-streams input streams as byte streams in the present sense

Q.reread Streams.read             :: InputStream S.ByteString -> Q.ByteString IO ()
Q.reread (liftIO . Streams.read)  :: MonadIO m => InputStream S.ByteString -> Q.ByteString m ()

The other direction here is

Streams.unfoldM Q.unconsChunk     :: Q.ByteString IO r -> IO (InputStream S.ByteString)

Make the information in a bytestring available to more than one eliminating fold, e.g.

>>> Q.count 'l' $ Q.count 'o' $ Q.copy $ "hello\nworld"
3 :> (2 :> ())

>>> Q.length $ Q.count 'l' $ Q.count 'o' $ Q.copy $ Q.copy "hello\nworld"
11 :> (3 :> (2 :> ()))

>>> runResourceT $ Q.writeFile "hello2.txt" $ Q.writeFile "hello1.txt" $ Q.copy $ "hello\nworld\n"
>>> :! cat hello2.txt
hello
world
>>> :! cat hello1.txt
hello
world

This sort of manipulation could as well be acheived by combining folds - using Control.Foldl for example. But any sort of manipulation can be involved in the fold. Here are a couple of trivial complications involving splitting by lines:

>>> let doubleLines = Q.unlines . maps (<* Q.chunk "\n" ) . Q.lines
>>> let emphasize = Q.unlines . maps (<* Q.chunk "!" ) . Q.lines
>>> runResourceT $ Q.writeFile "hello2.txt" $ emphasize $ Q.writeFile "hello1.txt" $ doubleLines $ Q.copy $ "hello\nworld"
>>> :! cat hello2.txt
hello!
world!
>>> :! cat hello1.txt
hello

world

As with the parallel operations in Streaming.Prelude, we have

Q.effects . Q.copy       = id
hoist Q.effects . Q.copy = id

The duplication does not by itself involve the copying of bytestring chunks; it just makes two references to each chunk as it arises. This does, however double the number of constructors associated with each chunk.