lsm-tree
Copyright(c) 2023-2025 Cardano Development Foundation
LicenseApache-2.0
Stabilityexperimental
Portabilityportable
Safe HaskellSafe-Inferred
LanguageGHC2021

Database.LSMTree

Description

 
Synopsis

Usage Notes

This section focuses on the differences between the full API as defined in this module and the simple API as defined in Database.LSMTree.Simple. It assumes that the reader is familiar with Usage Notes for the simple API, which discusses crucial topics such as Resource Management, Concurrency, ACID properties, and Sharing.

Real and Simulated IO

 

Examples

The examples in this module use the preamble described in this section, which does three things:

  1. It imports this module qualified, as intended, as well as any other relevant modules.
  2. It defines types for keys, values, and BLOBs.
  3. It defines a helper function that runs examples with access to an open session and fresh table.

Importing Database.LSMTree

This module is intended to be imported qualified, to avoid name clashes with Prelude functions.

>>> import           Database.LSMTree (BlobRef, Cursor, RawBytes, ResolveValue (..), SerialiseKey (..), SerialiseValue (..), Session, Table)
>>> import qualified Database.LSMTree as LSMT

Defining key, value, and BLOB types

The examples in this module use the types Key, Value, and Blob for keys, values and BLOBs.

>>> import Data.ByteString (ByteString)
>>> import Data.ByteString.Short (ShortByteString)
>>> import Data.Proxy (Proxy)
>>> import Data.String (IsString)
>>> import Data.Word (Word64)

The type Key is a newtype wrapper around Word64. The required instance of SerialiseKey is derived by GeneralisedNewtypeDeriving from the preexisting instance for Word64.

>>> :{
newtype Key = Key Word64
  deriving stock (Eq, Ord, Show)
  deriving newtype (Num, SerialiseKey)
:}

The type Value is a newtype wrapper around ShortByteString. The required instance of SerialiseValue is derived by GeneralisedNewtypeDeriving from the preexisting instance for ShortByteString.

>>> :{
newtype Value = Value ShortByteString
  deriving stock (Eq, Show)
  deriving newtype (IsString, SerialiseValue)
:}

The type Value has an instance of ResolveValue which appends the new value to the old value separated by a space. It is sufficient to define either resolve or resolveSerialised, as each can be defined in terms of the other and serialiseValue/deserialiseValue. For optimal performance, you should always define resolveSerialised manually.

NOTE: The first argument of resolve and resolveSerialised is the new value and the second argument is the old value.

>>> :{
instance ResolveValue Value where
  resolve :: Value -> Value -> Value
  resolve (Value new) (Value old) = Value (new <> " " <> old)
  resolveSerialised :: Proxy Value -> RawBytes -> RawBytes -> RawBytes
  resolveSerialised _ new old = new <> " " <> old
:}

The type Blob is a newtype wrapper around ByteString, The required instance of SerialiseValue is derived by GeneralisedNewtypeDeriving from the preexisting instance for ByteString.

>>> :{
newtype Blob = Blob ByteString
  deriving stock (Eq, Show)
  deriving newtype (IsString, SerialiseValue)
:}

Defining a helper function to run examples

The examples in this module are wrapped in a call to runExample, which creates a temporary session directory and runs the example with access to an open Session and a fresh Table.

>>> import           Control.Exception (bracket, bracket_)
>>> import           Data.Foldable (traverse_)
>>> import qualified System.Directory as Dir
>>> import           System.FilePath ((</>))
>>> import           System.Process (getCurrentPid)
>>> :{
runExample :: (Session IO -> Table IO Key Value Blob -> IO a) -> IO a
runExample action = do
  tmpDir <- Dir.getTemporaryDirectory
  pid <- getCurrentPid
  let sessionDir = tmpDir </> "doctest_Database_LSMTree" </> show pid
  let createSessionDir = Dir.createDirectoryIfMissing True sessionDir
  let removeSessionDir = Dir.removeDirectoryRecursive sessionDir
  bracket_ createSessionDir removeSessionDir $ do
    LSMT.withOpenSessionIO mempty sessionDir $ \session -> do
      LSMT.withTable session $ \table ->
        action session table
:}

Sessions

data Session (m :: Type -> Type) #

Instances

Instances details
NFData (Session m) 
Instance details

Defined in Database.LSMTree.Internal.Types

Methods

rnf :: Session m -> () #

withOpenSession Source #

Arguments

:: forall m h a. (IOLike m, Typeable h) 
=> Tracer m LSMTreeTrace 
-> HasFS m h 
-> HasBlockIO m h 
-> Salt

The session salt.

-> FsPath

The session directory.

-> (Session m -> m a) 
-> m a 

Run an action with access to a session opened from a session directory.

If the session directory is empty, a new session is created using the given salt. Otherwise, the session directory is restored as an existing session ignoring the given salt.

If there are no open tables or cursors when the session terminates, then the disk I/O complexity of this operation is \(O(1)\). Otherwise, closeTable is called for each open table and closeCursor is called for each open cursor. Consequently, the worst-case disk I/O complexity of this operation depends on the merge policy of the open tables in the session. The following assumes all tables in the session have the same merge policy:

LazyLevelling
\(O(o \: T \log_T \frac{n}{B})\).

The variable \(o\) refers to the number of open tables and cursors in the session.

This function is exception-safe for both synchronous and asynchronous exceptions.

It is recommended to use this function instead of openSession and closeSession.

Throws the following exceptions:

SessionDirDoesNotExistError
If the session directory does not exist.
SessionDirLockedError
If the session directory is locked by another process.
SessionDirCorruptedError
If the session directory is malformed.

withOpenSessionIO :: Tracer IO LSMTreeTrace -> FilePath -> (Session IO -> IO a) -> IO a Source #

Variant of withOpenSession that is specialised to IO using the real filesystem.

withNewSession Source #

Arguments

:: forall m h a. (IOLike m, Typeable h) 
=> Tracer m LSMTreeTrace 
-> HasFS m h 
-> HasBlockIO m h 
-> Salt

The session salt.

-> FsPath

The session directory.

-> (Session m -> m a) 
-> m a 

Run an action with access to a new session.

The session directory must be empty.

If there are no open tables or cursors when the session terminates, then the disk I/O complexity of this operation is \(O(1)\). Otherwise, closeTable is called for each open table and closeCursor is called for each open cursor. Consequently, the worst-case disk I/O complexity of this operation depends on the merge policy of the open tables in the session. The following assumes all tables in the session have the same merge policy:

LazyLevelling
\(O(o \: T \log_T \frac{n}{B})\).

The variable \(o\) refers to the number of open tables and cursors in the session.

This function is exception-safe for both synchronous and asynchronous exceptions.

It is recommended to use this function instead of newSession and closeSession.

Throws the following exceptions:

SessionDirDoesNotExistError
If the session directory does not exist.
SessionDirLockedError
If the session directory is locked by another process.
SessionDirCorruptedError
If the session directory is malformed.

withRestoreSession Source #

Arguments

:: forall m h a. (IOLike m, Typeable h) 
=> Tracer m LSMTreeTrace 
-> HasFS m h 
-> HasBlockIO m h 
-> FsPath

The session directory.

-> (Session m -> m a) 
-> m a 

Run an action with access to a restored session.

The session directory must be non-empty: a session must have previously been opened and closed in this directory.

If there are no open tables or cursors when the session terminates, then the disk I/O complexity of this operation is \(O(1)\). Otherwise, closeTable is called for each open table and closeCursor is called for each open cursor. Consequently, the worst-case disk I/O complexity of this operation depends on the merge policy of the open tables in the session. The following assumes all tables in the session have the same merge policy:

LazyLevelling
\(O(o \: T \log_T \frac{n}{B})\).

The variable \(o\) refers to the number of open tables and cursors in the session.

This function is exception-safe for both synchronous and asynchronous exceptions.

It is recommended to use this function instead of restoreSession and closeSession.

Throws the following exceptions:

SessionDirDoesNotExistError
If the session directory does not exist.
SessionDirLockedError
If the session directory is locked by another process.
SessionDirCorruptedError
If the session directory is malformed.

openSession Source #

Arguments

:: forall m h. (IOLike m, Typeable h) 
=> Tracer m LSMTreeTrace 
-> HasFS m h 
-> HasBlockIO m h 
-> Salt

The session salt.

-> FsPath

The session directory.

-> m (Session m) 

Open a session from a session directory.

If the session directory is empty, a new session is created using the given salt. Otherwise, the session directory is restored as an existing session ignoring the given salt.

The worst-case disk I/O complexity of this operation is \(O(1)\).

Warning: Sessions hold open resources and must be closed using closeSession.

Throws the following exceptions:

SessionDirDoesNotExistError
If the session directory does not exist.
SessionDirLockedError
If the session directory is locked by another process.
SessionDirCorruptedError
If the session directory is malformed.

newSession Source #

Arguments

:: forall m h. (IOLike m, Typeable h) 
=> Tracer m LSMTreeTrace 
-> HasFS m h 
-> HasBlockIO m h 
-> Salt

The session salt.

-> FsPath

The session directory.

-> m (Session m) 

Create a new session.

The session directory must be empty.

The worst-case disk I/O complexity of this operation is \(O(1)\).

Warning: Sessions hold open resources and must be closed using closeSession.

Throws the following exceptions:

SessionDirDoesNotExistError
If the session directory does not exist.
SessionDirLockedError
If the session directory is locked by another process.
SessionDirCorruptedError
If the session directory is malformed.

restoreSession Source #

Arguments

:: forall m h. (IOLike m, Typeable h) 
=> Tracer m LSMTreeTrace 
-> HasFS m h 
-> HasBlockIO m h 
-> FsPath

The session directory.

-> m (Session m) 

Restore a session from a session directory.

The session directory must be non-empty: a session must have previously been opened (and closed) in this directory.

The worst-case disk I/O complexity of this operation is \(O(1)\).

Warning: Sessions hold open resources and must be closed using closeSession.

Throws the following exceptions:

SessionDirDoesNotExistError
If the session directory does not exist.
SessionDirLockedError
If the session directory is locked by another process.
SessionDirCorruptedError
If the session directory is malformed.

closeSession :: forall m. IOLike m => Session m -> m () Source #

Close a session.

If there are no open tables or cursors in the session, then the disk I/O complexity of this operation is \(O(1)\). Otherwise, closeTable is called for each open table and closeCursor is called for each open cursor. Consequently, the worst-case disk I/O complexity of this operation depends on the merge policy of the tables in the session. The following assumes all tables in the session have the same merge policy:

LazyLevelling
\(O(o \: T \log_T \frac{n}{B})\).

The variable \(o\) refers to the number of open tables and cursors in the session.

Closing is idempotent, i.e., closing a closed session does nothing. All other operations on a closed session will throw an exception.

Tables

data Table (m :: Type -> Type) k v b #

Instances

Instances details
NFData (Table m k v b) 
Instance details

Defined in Database.LSMTree.Internal.Types

Methods

rnf :: Table m k v b -> () #

withTable :: forall m k v b a. IOLike m => Session m -> (Table m k v b -> m a) -> m a Source #

Run an action with access to an empty table.

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(T \log_T \frac{n}{B})\).

This function is exception-safe for both synchronous and asynchronous exceptions.

It is recommended to use this function instead of newTable and closeTable.

Throws the following exceptions:

SessionClosedError
If the session is closed.

withTableWith :: forall m k v b a. IOLike m => TableConfig -> Session m -> (Table m k v b -> m a) -> m a Source #

Variant of withTable that accepts table configuration.

newTable :: forall m k v b. IOLike m => Session m -> m (Table m k v b) Source #

Create an empty table.

The worst-case disk I/O complexity of this operation is \(O(1)\).

Warning: Tables hold open resources and must be closed using closeTable.

Throws the following exceptions:

SessionClosedError
If the session is closed.

newTableWith :: forall m k v b. IOLike m => TableConfig -> Session m -> m (Table m k v b) Source #

Variant of newTable that accepts table configuration.

closeTable :: forall m k v b. IOLike m => Table m k v b -> m () Source #

Close a table.

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(T \log_T \frac{n}{B})\).

Closing is idempotent, i.e., closing a closed table does nothing. All other operations on a closed table will throw an exception.

Warning: Tables are ephemeral. Once you close a table, its data is lost forever. To persist tables, use snapshots.

Table Lookups

member :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v) => Table m k v b -> k -> m Bool Source #

Check if the key is a member of the table.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  print =<< LSMT.member table 0
:}
True

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(T \log_T \frac{n}{B})\).

Membership tests can be performed concurrently from multiple Haskell threads.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.
TableCorruptedError
If the table data is corrupted.

members :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v) => Table m k v b -> Vector k -> m (Vector Bool) Source #

Variant of member for batch membership tests. The batch of keys corresponds in-order to the batch of results.

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(b \: T \log_T \frac{n}{B})\).

The variable \(b\) refers to the length of the input vector.

The following property holds in the absence of races:

members table keys = traverse (member table) keys

data LookupResult v b Source #

Constructors

NotFound 
Found !v 
FoundWithBlob !v !b 

Instances

Instances details
Bifunctor LookupResult Source # 
Instance details

Defined in Database.LSMTree

Methods

bimap :: (a -> b) -> (c -> d) -> LookupResult a c -> LookupResult b d #

first :: (a -> b) -> LookupResult a c -> LookupResult b c #

second :: (b -> c) -> LookupResult a b -> LookupResult a c #

Foldable (LookupResult v) Source # 
Instance details

Defined in Database.LSMTree

Methods

fold :: Monoid m => LookupResult v m -> m #

foldMap :: Monoid m => (a -> m) -> LookupResult v a -> m #

foldMap' :: Monoid m => (a -> m) -> LookupResult v a -> m #

foldr :: (a -> b -> b) -> b -> LookupResult v a -> b #

foldr' :: (a -> b -> b) -> b -> LookupResult v a -> b #

foldl :: (b -> a -> b) -> b -> LookupResult v a -> b #

foldl' :: (b -> a -> b) -> b -> LookupResult v a -> b #

foldr1 :: (a -> a -> a) -> LookupResult v a -> a #

foldl1 :: (a -> a -> a) -> LookupResult v a -> a #

toList :: LookupResult v a -> [a] #

null :: LookupResult v a -> Bool #

length :: LookupResult v a -> Int #

elem :: Eq a => a -> LookupResult v a -> Bool #

maximum :: Ord a => LookupResult v a -> a #

minimum :: Ord a => LookupResult v a -> a #

sum :: Num a => LookupResult v a -> a #

product :: Num a => LookupResult v a -> a #

Traversable (LookupResult v) Source # 
Instance details

Defined in Database.LSMTree

Methods

traverse :: Applicative f => (a -> f b) -> LookupResult v a -> f (LookupResult v b) #

sequenceA :: Applicative f => LookupResult v (f a) -> f (LookupResult v a) #

mapM :: Monad m => (a -> m b) -> LookupResult v a -> m (LookupResult v b) #

sequence :: Monad m => LookupResult v (m a) -> m (LookupResult v a) #

Functor (LookupResult v) Source # 
Instance details

Defined in Database.LSMTree

Methods

fmap :: (a -> b) -> LookupResult v a -> LookupResult v b #

(<$) :: a -> LookupResult v b -> LookupResult v a #

(Show v, Show b) => Show (LookupResult v b) Source # 
Instance details

Defined in Database.LSMTree

(NFData v, NFData b) => NFData (LookupResult v b) Source # 
Instance details

Defined in Database.LSMTree

Methods

rnf :: LookupResult v b -> () #

(Eq v, Eq b) => Eq (LookupResult v b) Source # 
Instance details

Defined in Database.LSMTree

Methods

(==) :: LookupResult v b -> LookupResult v b -> Bool #

(/=) :: LookupResult v b -> LookupResult v b -> Bool #

getValue :: LookupResult v b -> Maybe v Source #

Get the field of type v from a LookupResult v b, if any.

getBlob :: LookupResult v b -> Maybe b Source #

Get the field of type b from a LookupResult v b, if any.

The following property holds:

isJust (getBlob result) <= isJust (getValue result)

lookup :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v) => Table m k v b -> k -> m (LookupResult v (BlobRef m b)) Source #

Look up the value associated with a key.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  print =<< LSMT.lookup table 0
:}
Found (Value "Hello")

If the key is not associated with any value, lookup returns NotFound.

>>> :{
runExample $ \session table -> do
  LSMT.lookup table 0
:}
NotFound

If the key has an associated BLOB, the result contains a BlobRef. The full BLOB can be retrieved by passing that BlobRef to retrieveBlob.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" (Just "World")
  print
    =<< traverse (LSMT.retrieveBlob session)
    =<< LSMT.lookup table 0
:}
FoundWithBlob (Value "Hello") (Blob "World")

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(T \log_T \frac{n}{B})\).

Lookups can be performed concurrently from multiple Haskell threads.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.
TableCorruptedError
If the table data is corrupted.

lookups :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v) => Table m k v b -> Vector k -> m (Vector (LookupResult v (BlobRef m b))) Source #

Variant of lookup for batch lookups. The batch of keys corresponds in-order to the batch of results.

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(b \: T \log_T \frac{n}{B})\).

The variable \(b\) refers to the length of the input vector.

The following property holds in the absence of races:

lookups table keys = traverse (lookup table) keys

data Entry k v b Source #

Constructors

Entry !k !v 
EntryWithBlob !k !v !b 

Instances

Instances details
Bifunctor (Entry k) Source # 
Instance details

Defined in Database.LSMTree

Methods

bimap :: (a -> b) -> (c -> d) -> Entry k a c -> Entry k b d #

first :: (a -> b) -> Entry k a c -> Entry k b c #

second :: (b -> c) -> Entry k a b -> Entry k a c #

Foldable (Entry k v) Source # 
Instance details

Defined in Database.LSMTree

Methods

fold :: Monoid m => Entry k v m -> m #

foldMap :: Monoid m => (a -> m) -> Entry k v a -> m #

foldMap' :: Monoid m => (a -> m) -> Entry k v a -> m #

foldr :: (a -> b -> b) -> b -> Entry k v a -> b #

foldr' :: (a -> b -> b) -> b -> Entry k v a -> b #

foldl :: (b -> a -> b) -> b -> Entry k v a -> b #

foldl' :: (b -> a -> b) -> b -> Entry k v a -> b #

foldr1 :: (a -> a -> a) -> Entry k v a -> a #

foldl1 :: (a -> a -> a) -> Entry k v a -> a #

toList :: Entry k v a -> [a] #

null :: Entry k v a -> Bool #

length :: Entry k v a -> Int #

elem :: Eq a => a -> Entry k v a -> Bool #

maximum :: Ord a => Entry k v a -> a #

minimum :: Ord a => Entry k v a -> a #

sum :: Num a => Entry k v a -> a #

product :: Num a => Entry k v a -> a #

Traversable (Entry k v) Source # 
Instance details

Defined in Database.LSMTree

Methods

traverse :: Applicative f => (a -> f b) -> Entry k v a -> f (Entry k v b) #

sequenceA :: Applicative f => Entry k v (f a) -> f (Entry k v a) #

mapM :: Monad m => (a -> m b) -> Entry k v a -> m (Entry k v b) #

sequence :: Monad m => Entry k v (m a) -> m (Entry k v a) #

Functor (Entry k v) Source # 
Instance details

Defined in Database.LSMTree

Methods

fmap :: (a -> b) -> Entry k v a -> Entry k v b #

(<$) :: a -> Entry k v b -> Entry k v a #

(Show k, Show v, Show b) => Show (Entry k v b) Source # 
Instance details

Defined in Database.LSMTree

Methods

showsPrec :: Int -> Entry k v b -> ShowS #

show :: Entry k v b -> String #

showList :: [Entry k v b] -> ShowS #

(NFData k, NFData v, NFData b) => NFData (Entry k v b) Source # 
Instance details

Defined in Database.LSMTree

Methods

rnf :: Entry k v b -> () #

(Eq k, Eq v, Eq b) => Eq (Entry k v b) Source # 
Instance details

Defined in Database.LSMTree

Methods

(==) :: Entry k v b -> Entry k v b -> Bool #

(/=) :: Entry k v b -> Entry k v b -> Bool #

getEntryKey :: Entry k v b -> k Source #

Get the field of type k from an Entry k v b.

getEntryValue :: Entry k v b -> v Source #

Get the field of type v from an Entry k v b.

getEntryBlob :: Entry k v b -> Maybe b Source #

Get the field of type b from an Entry k v b, if any.

rangeLookup :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v) => Table m k v b -> Range k -> m (Vector (Entry k v (BlobRef m b))) Source #

Look up a batch of values associated with keys in the given range.

The worst-case disk I/O complexity of this operation is \(O(T \log_T \frac{n}{B} + \frac{b}{P})\), where the variable \(b\) refers to the length of the output vector.

Range lookups can be performed concurrently from multiple Haskell threads.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.
TableCorruptedError
If the table data is corrupted.

Table Updates

insert :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v, SerialiseValue b) => Table m k v b -> k -> v -> Maybe b -> m () Source #

Insert associates the given value and BLOB with the given key in the table.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  print =<< LSMT.lookup table 0
:}
Found (Value "Hello")

Insert may optionally associate a BLOB value with the given key.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" (Just "World")
  print
    =<< traverse (retrieveBlob session)
    =<< LSMT.lookup table 0
:}
FoundWithBlob (Value "Hello") (Blob "World")

Insert overwrites any value and BLOB previously associated with the given key, even if the given BLOB is Nothing.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" (Just "World")
  LSMT.insert table 0 "Goodbye" Nothing
  print
    =<< traverse (retrieveBlob session)
    =<< LSMT.lookup table 0
:}
Found (Value "Goodbye")

The worst-case disk I/O complexity of this operation depends on the merge policy and the merge schedule of the table:

LazyLevelling/Incremental
\(O(\frac{1}{P} \: \log_T \frac{n}{B})\).
LazyLevelling/OneShot
\(O(\frac{n}{P})\).

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.

inserts :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v, SerialiseValue b) => Table m k v b -> Vector (k, v, Maybe b) -> m () Source #

Variant of insert for batch insertions.

The worst-case disk I/O complexity of this operation depends on the merge policy and the merge schedule of the table:

LazyLevelling/Incremental
\(O(b \: \frac{1}{P} \: \log_T \frac{n}{B})\).
LazyLevelling/OneShot
\(O(\frac{b}{P} \log_T \frac{b}{B} + \frac{n}{P})\).

The variable \(b\) refers to the length of the input vector.

The following property holds in the absence of races:

inserts table entries = traverse_ (uncurry $ insert table) entries

upsert :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v, SerialiseValue b) => Table m k v b -> k -> v -> m () Source #

If the given key is not a member of the table, upsert associates the given value with the given key in the table. Otherwise, upsert updates the value associated with the given key by combining it with the given value using resolve.

>>> :{
runExample $ \session table -> do
  LSMT.upsert table 0 "Hello"
  LSMT.upsert table 0 "Goodbye"
  print =<< LSMT.lookup table 0
:}
Found (Value "Goodbye Hello")

Warning: Upsert deletes any BLOB previously associated with the given key.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" (Just "World")
  LSMT.upsert table 0 "Goodbye"
  print
    =<< traverse (LSMT.retrieveBlob session)
    =<< LSMT.lookup table 0
:}
Found (Value "Goodbye Hello")

The worst-case disk I/O complexity of this operation depends on the merge policy and the merge schedule of the table:

LazyLevelling/Incremental
\(O(\frac{1}{P} \: \log_T \frac{n}{B})\).
LazyLevelling/OneShot
\(O(\frac{n}{P})\).

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.

The following property holds in the absence of races:

upsert table k v = do
  r <- lookup table k
  let v' = maybe v (resolve v) (getValue r)
  insert table k v' Nothing

upserts :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v, SerialiseValue b) => Table m k v b -> Vector (k, v) -> m () Source #

Variant of upsert for batch insertions.

The worst-case disk I/O complexity of this operation depends on the merge policy and the merge schedule of the table:

LazyLevelling/Incremental
\(O(b \: \frac{1}{P} \: \log_T \frac{n}{B})\).
LazyLevelling/OneShot
\(O(\frac{b}{P} \log_T \frac{b}{B} + \frac{n}{P})\).

The variable \(b\) refers to the length of the input vector.

The following property holds in the absence of races:

upserts table entries = traverse_ (uncurry $ upsert table) entries

delete :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v, SerialiseValue b) => Table m k v b -> k -> m () Source #

Delete a key from the table.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.delete table 0
  print =<< LSMT.lookup table 0
:}
NotFound

If the key is not a member of the table, the table is left unchanged.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.delete table 1
  print =<< LSMT.lookup table 0
:}
Found (Value "Hello")

The worst-case disk I/O complexity of this operation depends on the merge policy and the merge schedule of the table:

LazyLevelling/Incremental
\(O(\frac{1}{P} \: \log_T \frac{n}{B})\).
LazyLevelling/OneShot
\(O(\frac{n}{P})\).

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.

deletes :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v, SerialiseValue b) => Table m k v b -> Vector k -> m () Source #

Variant of delete for batch deletions.

The worst-case disk I/O complexity of this operation depends on the merge policy and the merge schedule of the table:

LazyLevelling/Incremental
\(O(b \: \frac{1}{P} \: \log_T \frac{n}{B})\).
LazyLevelling/OneShot
\(O(\frac{b}{P} \log_T \frac{b}{B} + \frac{n}{P})\).

The variable \(b\) refers to the length of the input vector.

The following property holds in the absence of races:

deletes table keys = traverse_ (delete table) keys

data Update v b Source #

Constructors

Insert !v !(Maybe b) 
Delete 
Upsert !v 

Instances

Instances details
(Show b, Show v) => Show (Update v b) Source # 
Instance details

Defined in Database.LSMTree

Methods

showsPrec :: Int -> Update v b -> ShowS #

show :: Update v b -> String #

showList :: [Update v b] -> ShowS #

(NFData v, NFData b) => NFData (Update v b) Source # 
Instance details

Defined in Database.LSMTree

Methods

rnf :: Update v b -> () #

(Eq b, Eq v) => Eq (Update v b) Source # 
Instance details

Defined in Database.LSMTree

Methods

(==) :: Update v b -> Update v b -> Bool #

(/=) :: Update v b -> Update v b -> Bool #

update :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v, SerialiseValue b) => Table m k v b -> k -> Update v b -> m () Source #

Update generalises insert, delete, and upsert.

The worst-case disk I/O complexity of this operation depends on the merge policy and the merge schedule of the table:

LazyLevelling/Incremental
\(O(\frac{1}{P} \: \log_T \frac{n}{B})\).
LazyLevelling/OneShot
\(O(\frac{n}{P})\).

The following properties hold:

update table k (Insert v mb) = insert table k v mb
update table k Delete = delete table k
update table k (Upsert v) = upsert table k v

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.

updates :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v, SerialiseValue b) => Table m k v b -> Vector (k, Update v b) -> m () Source #

Variant of update for batch updates.

The worst-case disk I/O complexity of this operation depends on the merge policy and the merge schedule of the table:

LazyLevelling/Incremental
\(O(b \: \frac{1}{P} \: \log_T \frac{n}{B})\).
LazyLevelling/OneShot
\(O(\frac{b}{P} \log_T \frac{b}{B} + \frac{n}{P})\).

The variable \(b\) refers to the length of the input vector.

The following property holds in the absence of races:

updates table entries = traverse_ (uncurry $ update table) entries

Table Duplication

withDuplicate :: forall m k v b a. IOLike m => Table m k v b -> (Table m k v b -> m a) -> m a Source #

Run an action with access to the duplicate of a table.

The duplicate is an independent copy of the given table. Subsequent updates to the original table do not affect the duplicate, and vice versa.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.withDuplicate table $ \table' -> do
    print =<< LSMT.lookup table' 0
    LSMT.insert table' 0 "Goodbye" Nothing
    print =<< LSMT.lookup table' 0
  LSMT.lookup table 0
  print =<< LSMT.lookup table 0
:}
Found (Value "Hello")
Found (Value "Goodbye")
Found (Value "Hello")

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(T \log_T \frac{n}{B})\).

This function is exception-safe for both synchronous and asynchronous exceptions.

It is recommended to use this function instead of duplicate and closeTable.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.

duplicate :: forall m k v b. IOLike m => Table m k v b -> m (Table m k v b) Source #

Duplicate a table.

The duplicate is an independent copy of the given table. Subsequent updates to the original table do not affect the duplicate, and vice versa.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  bracket (LSMT.duplicate table) LSMT.closeTable $ \table' -> do
    print =<< LSMT.lookup table' 0
    LSMT.insert table' 0 "Goodbye" Nothing
    print =<< LSMT.lookup table' 0
  LSMT.lookup table 0
  print =<< LSMT.lookup table 0
:}
Found (Value "Hello")
Found (Value "Goodbye")
Found (Value "Hello")

The worst-case disk I/O complexity of this operation is \(O(0)\).

Warning: The duplicate must be independently closed using closeTable.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.

Table Unions

withUnion :: forall m k v b a. IOLike m => ResolveValue v => Table m k v b -> Table m k v b -> (Table m k v b -> m a) -> m a Source #

Run an action with access to a table that contains the union of the entries of the given tables.

>>> :{
runExample $ \session table1 -> do
  LSMT.insert table1 0 "Hello" Nothing
  LSMT.withTable session $ \table2 -> do
    LSMT.insert table2 0 "World" Nothing
    LSMT.insert table2 1 "Goodbye" Nothing
    LSMT.withUnion table1 table2 $ \table3 -> do
      print =<< LSMT.lookup table3 0
      print =<< LSMT.lookup table3 1
    print =<< LSMT.lookup table1 0
    print =<< LSMT.lookup table2 0
:}
Found (Value "Hello World")
Found (Value "Goodbye")
Found (Value "Hello")
Found (Value "World")

The worst-case disk I/O complexity of this operation is \(O(\frac{n}{P})\).

This function is exception-safe for both synchronous and asynchronous exceptions.

It is recommended to use this function instead of union and closeTable.

Warning: Both input tables must be from the same Session.

Warning: This is a relatively expensive operation that may take some time to complete. See withIncrementalUnion for an incremental alternative.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.
TableUnionNotCompatibleError
If both tables are not from the same Session.

withUnions :: forall m k v b a. IOLike m => ResolveValue v => NonEmpty (Table m k v b) -> (Table m k v b -> m a) -> m a Source #

Variant of withUnions that takes any number of tables.

union :: forall m k v b. IOLike m => ResolveValue v => Table m k v b -> Table m k v b -> m (Table m k v b) Source #

Create a table that contains the union of the entries of the given tables.

If the given key is a member of a single input table, then the same key and value occur in the output table. Otherwise, the values for duplicate keys are combined using resolve from left to right. If the resolve function behaves like const, then this computes a left-biased union.

>>> :{
runExample $ \session table1 -> do
  LSMT.insert table1 0 "Hello" Nothing
  LSMT.withTable session $ \table2 -> do
    LSMT.insert table2 0 "World" Nothing
    LSMT.insert table2 1 "Goodbye" Nothing
    bracket (LSMT.union table1 table2) LSMT.closeTable $ \table3 -> do
      print =<< LSMT.lookup table3 0
      print =<< LSMT.lookup table3 1
    print =<< LSMT.lookup table1 0
    print =<< LSMT.lookup table2 0
:}
Found (Value "Hello World")
Found (Value "Goodbye")
Found (Value "Hello")
Found (Value "World")

The worst-case disk I/O complexity of this operation is \(O(\frac{n}{P})\).

Warning: The new table must be independently closed using closeTable.

Warning: Both input tables must be from the same Session.

Warning: This is a relatively expensive operation that may take some time to complete. See incrementalUnion for an incremental alternative.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.
TableUnionNotCompatibleError
If both tables are not from the same Session.

unions :: forall m k v b. IOLike m => ResolveValue v => NonEmpty (Table m k v b) -> m (Table m k v b) Source #

Variant of union that takes any number of tables.

withIncrementalUnion :: forall m k v b a. IOLike m => Table m k v b -> Table m k v b -> (Table m k v b -> m a) -> m a Source #

Run an action with access to a table that incrementally computes the union of the given tables.

>>> :{
runExample $ \session table1 -> do
  LSMT.insert table1 0 "Hello" Nothing
  LSMT.withTable session $ \table2 -> do
    LSMT.insert table2 0 "World" Nothing
    LSMT.insert table2 1 "Goodbye" Nothing
    LSMT.withIncrementalUnion table1 table2 $ \table3 -> do
      print =<< LSMT.lookup table3 0
      print =<< LSMT.lookup table3 1
    print =<< LSMT.lookup table1 0
    print =<< LSMT.lookup table2 0
:}
Found (Value "Hello World")
Found (Value "Goodbye")
Found (Value "Hello")
Found (Value "World")

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(T \log_T \frac{n}{B})\).

This function is exception-safe for both synchronous and asynchronous exceptions.

It is recommended to use this function instead of incrementalUnion and closeTable.

The created table has a union debt which represents the amount of computation that remains. See remainingUnionDebt. The union debt can be paid off by supplying union credit which performs an amount of computation proportional to the amount of union credit. See supplyUnionCredits. While a table has unresolved union debt, operations may become more expensive by a factor that scales with the number of unresolved unions.

Warning: Both input tables must be from the same Session.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.
TableUnionNotCompatibleError
If both tables are not from the same Session.

withIncrementalUnions :: forall m k v b a. IOLike m => NonEmpty (Table m k v b) -> (Table m k v b -> m a) -> m a Source #

Variant of withIncrementalUnion that takes any number of tables.

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(T \log_T \frac{n}{B} + b)\).

The variable \(b\) refers to the number of input tables.

incrementalUnion :: forall m k v b. IOLike m => Table m k v b -> Table m k v b -> m (Table m k v b) Source #

Create a table that incrementally computes the union of the given tables.

>>> :{
runExample $ \session table1 -> do
  LSMT.insert table1 0 "Hello" Nothing
  LSMT.withTable session $ \table2 -> do
    LSMT.insert table2 0 "World" Nothing
    LSMT.insert table2 1 "Goodbye" Nothing
    bracket (LSMT.incrementalUnion table1 table2) LSMT.closeTable $ \table3 -> do
      print =<< LSMT.lookup table3 0
      print =<< LSMT.lookup table3 1
    print =<< LSMT.lookup table1 0
    print =<< LSMT.lookup table2 0
:}
Found (Value "Hello World")
Found (Value "Goodbye")
Found (Value "Hello")
Found (Value "World")

The worst-case disk I/O complexity of this operation is \(O(1)\).

The created table has a union debt which represents the amount of computation that remains. See remainingUnionDebt. The union debt can be paid off by supplying union credit which performs an amount of computation proportional to the amount of union credit. See supplyUnionCredits. While a table has unresolved union debt, operations may become more expensive by a factor that scales with the number of unresolved unions.

Warning: The new table must be independently closed using closeTable.

Warning: Both input tables must be from the same Session.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.
TableUnionNotCompatibleError
If both tables are not from the same Session.

incrementalUnions :: forall m k v b. IOLike m => NonEmpty (Table m k v b) -> m (Table m k v b) Source #

Variant of incrementalUnion for any number of tables.

The worst-case disk I/O complexity of this operation is \(O(b)\), where the variable \(b\) refers to the number of input tables.

remainingUnionDebt :: forall m k v b. IOLike m => Table m k v b -> m UnionDebt Source #

Get an upper bound for the amount of remaining union debt. This includes the union debt of any table that was part of the union's input.

>>> :{
runExample $ \session table1 -> do
  LSMT.insert table1 0 "Hello" Nothing
  LSMT.withTable session $ \table2 -> do
    LSMT.insert table2 0 "World" Nothing
    LSMT.insert table2 1 "Goodbye" Nothing
    bracket (LSMT.incrementalUnion table1 table2) LSMT.closeTable $ \table3 -> do
      putStrLn . ("UnionDebt: "<>) . show =<< LSMT.remainingUnionDebt table3
:}
UnionDebt: 4

The worst-case disk I/O complexity of this operation is \(O(0)\).

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.

supplyUnionCredits :: forall m k v b. IOLike m => ResolveValue v => Table m k v b -> UnionCredits -> m UnionCredits Source #

Supply the given amount of union credits.

This reduces the union debt by at least the number of supplied union credits. It is therefore advisable to query remainingUnionDebt every once in a while to get an upper bound on the current debt.

This function returns any surplus of union credits as leftover credits when a union has finished. In particular, if the returned number of credits is positive, then the union is finished.

>>> :{
runExample $ \session table1 -> do
  LSMT.insert table1 0 "Hello" Nothing
  LSMT.withTable session $ \table2 -> do
    LSMT.insert table2 0 "World" Nothing
    LSMT.insert table2 1 "Goodbye" Nothing
    bracket (LSMT.incrementalUnion table1 table2) LSMT.closeTable $ \table3 -> do
      putStrLn . ("UnionDebt: "<>) . show =<< LSMT.remainingUnionDebt table3
      putStrLn . ("Leftovers: "<>) . show =<< LSMT.supplyUnionCredits table3 2
      putStrLn . ("UnionDebt: "<>) . show =<< LSMT.remainingUnionDebt table3
      putStrLn . ("Leftovers: "<>) . show =<< LSMT.supplyUnionCredits table3 4
:}
UnionDebt: 4
Leftovers: 0
UnionDebt: 2
Leftovers: 3

NOTE: The remainingUnionDebt functions gets an upper bound for the amount of remaning union debt. In the example above, the second call to remainingUnionDebt reports 2, but the union debt is 1. Therefore, the second call to supplyUnionCredits returns more leftovers than expected.

The worst-case disk I/O complexity of this operation is \(O(\frac{b}{P})\), where the variable \(b\) refers to the amount of credits supplied.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.

Blob References

data BlobRef (m :: Type -> Type) b #

Instances

Instances details
Show (BlobRef m b) 
Instance details

Defined in Database.LSMTree.Internal.Types

Methods

showsPrec :: Int -> BlobRef m b -> ShowS #

show :: BlobRef m b -> String #

showList :: [BlobRef m b] -> ShowS #

retrieveBlob :: forall m b. (IOLike m, SerialiseValue b) => Session m -> BlobRef m b -> m b Source #

Retrieve the blob value from a blob reference.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" (Just "World")
  print
    =<< traverse (LSMT.retrieveBlob session)
    =<< LSMT.lookup table 0
:}
FoundWithBlob (Value "Hello") (Blob "World")

The worst-case disk I/O complexity of this operation is \(O(1)\).

Warning: A blob reference is not stable. Any operation that modifies the table, cursor, or session that corresponds to a blob reference may cause it to be invalidated.

Throws the following exceptions:

SessionClosedError
If the session is closed.
BlobRefInvalidError
If the blob reference has been invalidated.

retrieveBlobs :: forall m b. (IOLike m, SerialiseValue b) => Session m -> Vector (BlobRef m b) -> m (Vector b) Source #

Variant of retrieveBlob for batch retrieval. The batch of blob references corresponds in-order to the batch of results.

The worst-case disk I/O complexity of this operation is \(O(b)\), where the variable \(b\) refers to the length of the input vector.

The following property holds in the absence of races:

retrieveBlobs session blobRefs = traverse (retrieveBlob session) blobRefs

Cursors

data Cursor (m :: Type -> Type) k v b #

Instances

Instances details
NFData (Cursor m k v b) 
Instance details

Defined in Database.LSMTree.Internal.Types

Methods

rnf :: Cursor m k v b -> () #

withCursor :: forall m k v b a. IOLike m => ResolveValue v => Table m k v b -> (Cursor m k v b -> m a) -> m a Source #

Run an action with access to a cursor.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  LSMT.withCursor table $ \cursor -> do
    traverse_ print
      =<< LSMT.take 32 cursor
:}
Entry (Key 0) (Value "Hello")
Entry (Key 1) (Value "World")

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(T \log_T \frac{n}{B})\).

This function is exception-safe for both synchronous and asynchronous exceptions.

It is recommended to use this function instead of newCursor and closeCursor.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.

withCursorAtOffset :: forall m k v b a. IOLike m => (SerialiseKey k, ResolveValue v) => Table m k v b -> k -> (Cursor m k v b -> m a) -> m a Source #

Variant of withCursor that starts at a given key.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  LSMT.withCursorAtOffset table 1 $ \cursor -> do
    traverse_ print
      =<< LSMT.take 32 cursor
:}
Entry (Key 1) (Value "World")

newCursor :: forall m k v b. IOLike m => ResolveValue v => Table m k v b -> m (Cursor m k v b) Source #

Create a cursor for the given table.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  bracket (LSMT.newCursor table) LSMT.closeCursor $ \cursor -> do
    traverse_ print
      =<< LSMT.take 32 cursor
:}
Entry (Key 0) (Value "Hello")
Entry (Key 1) (Value "World")

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(T \log_T \frac{n}{B})\).

Warning: Cursors hold open resources and must be closed using closeCursor.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.

newCursorAtOffset :: forall m k v b. IOLike m => (SerialiseKey k, ResolveValue v) => Table m k v b -> k -> m (Cursor m k v b) Source #

Variant of newCursor that starts at a given key.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  bracket (LSMT.newCursorAtOffset table 1) LSMT.closeCursor $ \cursor -> do
    traverse_ print
      =<< LSMT.take 32 cursor
:}
Entry (Key 1) (Value "World")

closeCursor :: forall m k v b. IOLike m => Cursor m k v b -> m () Source #

Close a cursor.

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(T \log_T \frac{n}{B})\).

Closing is idempotent, i.e., closing a closed cursor does nothing. All other operations on a closed cursor will throw an exception.

next :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v) => Cursor m k v b -> m (Maybe (Entry k v (BlobRef m b))) Source #

Read the next table entry from the cursor.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  LSMT.withCursor table $ \cursor -> do
    print =<< LSMT.next cursor
    print =<< LSMT.next cursor
    print =<< LSMT.next cursor
:}
Just (Entry (Key 0) (Value "Hello"))
Just (Entry (Key 1) (Value "World"))
Nothing

The worst-case disk I/O complexity of this operation is \(O(\frac{1}{P})\).

Throws the following exceptions:

SessionClosedError
If the session is closed.
CursorClosedError
If the cursor is closed.

take :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v) => Int -> Cursor m k v b -> m (Vector (Entry k v (BlobRef m b))) Source #

Read the next batch of table entries from the cursor.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  LSMT.withCursor table $ \cursor -> do
    traverse_ print
      =<< LSMT.take 32 cursor
:}
Entry (Key 0) (Value "Hello")
Entry (Key 1) (Value "World")

The worst-case disk I/O complexity of this operation is \(O(\frac{b}{P})\), where the variable \(b\) refers to the length of the output vector, which is at most equal to the given number. In practice, the length of the output vector is only less than the given number once the cursor reaches the end of the table.

The following property holds:

take n cursor = catMaybes <$> replicateM n (next cursor)

Throws the following exceptions:

SessionClosedError
If the session is closed.
CursorClosedError
If the cursor is closed.

takeWhile :: forall m k v b. IOLike m => (SerialiseKey k, SerialiseValue v, ResolveValue v) => Int -> (k -> Bool) -> Cursor m k v b -> m (Vector (Entry k v (BlobRef m b))) Source #

Variant of take that accepts an additional predicate to determine whether or not to continue reading.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  LSMT.withCursor table $ \cursor -> do
    traverse_ print
      =<< LSMT.takeWhile 32 (<1) cursor
:}
Entry (Key 0) (Value "Hello")

The worst-case disk I/O complexity of this operation is \(O(\frac{b}{P})\), where the variable \(b\) refers to the length of the output vector, which is at most equal to the given number. In practice, the length of the output vector is only less than the given number when the predicate returns false or the cursor reaches the end of the table.

The following properties hold:

takeWhile n (const True) cursor = take n cursor
takeWhile n (const False) cursor = pure empty

Throws the following exceptions:

SessionClosedError
If the session is closed.
CursorClosedError
If the cursor is closed.

Snapshots

saveSnapshot :: forall m k v b. IOLike m => SnapshotName -> SnapshotLabel -> Table m k v b -> m () Source #

Save the current state of the table to disk as a snapshot under the given snapshot name. This is the only mechanism that persists a table. Each snapshot must have a unique name, which may be used to restore the table from that snapshot using openTableFromSnapshot. Saving a snapshot does not close the table.

Saving a snapshot is relatively cheap when compared to opening a snapshot. However, it is not so cheap that one should use it after every operation.

>>> :{
runExample $ \session table -> do
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  LSMT.saveSnapshot "example" "Key Value Blob" table
:}

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(T \log_T \frac{n}{B})\).

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.
SnapshotExistsError
If a snapshot with the same name already exists.

withTableFromSnapshot :: forall m k v b a. IOLike m => ResolveValue v => Session m -> SnapshotName -> SnapshotLabel -> (Table m k v b -> m a) -> m a Source #

Run an action with access to a table from a snapshot.

>>> :{
runExample $ \session table -> do
  -- Save snapshot
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  LSMT.saveSnapshot "example" "Key Value Blob" table
  -- Open snapshot
  LSMT.withTableFromSnapshot @_ @Key @Value @Blob session "example" "Key Value Blob" $ \table' -> do
      LSMT.withCursor table' $ \cursor ->
        traverse_ print
          =<< LSMT.take 32 cursor
:}
Entry (Key 0) (Value "Hello")
Entry (Key 1) (Value "World")

The worst-case disk I/O complexity of this operation is \(O(\frac{n}{P})\).

This function is exception-safe for both synchronous and asynchronous exceptions.

It is recommended to use this function instead of openTableFromSnapshot and closeTable.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.
SnapshotDoesNotExistError
If no snapshot with the given name exists.
SnapshotCorruptedError
If the snapshot data is corrupted.
SnapshotNotCompatibleError
If the snapshot has a different label or is a different table type.

withTableFromSnapshotWith :: forall m k v b a. IOLike m => ResolveValue v => TableConfigOverride -> Session m -> SnapshotName -> SnapshotLabel -> (Table m k v b -> m a) -> m a Source #

openTableFromSnapshot :: forall m k v b. IOLike m => ResolveValue v => Session m -> SnapshotName -> SnapshotLabel -> m (Table m k v b) Source #

Open a table from a named snapshot.

>>> :{
runExample $ \session table -> do
  -- Save snapshot
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  LSMT.saveSnapshot "example" "Key Value Blob" table
  -- Open snapshot
  bracket
    (LSMT.openTableFromSnapshot @_ @Key @Value @Blob session "example" "Key Value Blob")
    LSMT.closeTable $ \table' -> do
      LSMT.withCursor table' $ \cursor ->
        traverse_ print
          =<< LSMT.take 32 cursor
:}
Entry (Key 0) (Value "Hello")
Entry (Key 1) (Value "World")

The worst-case disk I/O complexity of this operation is \(O(\frac{n}{P})\).

Warning: The new table must be independently closed using closeTable.

Throws the following exceptions:

SessionClosedError
If the session is closed.
TableClosedError
If the table is closed.
SnapshotDoesNotExistError
If no snapshot with the given name exists.
SnapshotCorruptedError
If the snapshot data is corrupted.
SnapshotNotCompatibleError
If the snapshot has a different label or is a different table type.

doesSnapshotExist :: forall m. IOLike m => Session m -> SnapshotName -> m Bool Source #

Check if the named snapshot exists.

>>> :{
runExample $ \session table -> do
  -- Save snapshot
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  LSMT.saveSnapshot "example" "Key Value Blob" table
  -- Check snapshots
  print =<< doesSnapshotExist session "example"
  print =<< doesSnapshotExist session "this_snapshot_does_not_exist"
:}
True
False

The worst-case disk I/O complexity of this operation is \(O(1)\).

Throws the following exceptions:

SessionClosedError
If the session is closed.

deleteSnapshot :: forall m. IOLike m => Session m -> SnapshotName -> m () Source #

Delete the named snapshot.

>>> :{
runExample $ \session table -> do
  -- Save snapshot
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  LSMT.saveSnapshot "example" "Key Value Blob" table
  -- Delete snapshot
  LSMT.deleteSnapshot session "example"
:}

The worst-case disk I/O complexity of this operation depends on the merge policy of the table:

LazyLevelling
\(O(T \log_T \frac{n}{B})\).

Throws the following exceptions:

SessionClosedError
If the session is closed.
SnapshotDoesNotExistError
If no snapshot with the given name exists.

listSnapshots :: forall m. IOLike m => Session m -> m [SnapshotName] Source #

List the names of all snapshots.

>>> :{
runExample $ \session table -> do
  -- Save snapshot
  LSMT.insert table 0 "Hello" Nothing
  LSMT.insert table 1 "World" Nothing
  LSMT.saveSnapshot "example" "Key Value Blob" table
  -- List snapshots
  traverse_ print
    =<< listSnapshots session
:}
"example"

The worst-case disk I/O complexity of this operation is \(O(s)\), where \(s\) refers to the number of snapshots in the session.

Throws the following exceptions:

SessionClosedError
If the session is closed.

Session Configuration

type Salt = Word64 #

Table Configuration

data MergePolicy #

Constructors

LazyLevelling 

Instances

Instances details
Show MergePolicy 
Instance details

Defined in Database.LSMTree.Internal.Config

NFData MergePolicy 
Instance details

Defined in Database.LSMTree.Internal.Config

Methods

rnf :: MergePolicy -> () #

Eq MergePolicy 
Instance details

Defined in Database.LSMTree.Internal.Config

data MergeSchedule #

Constructors

OneShot 
Incremental 

Instances

Instances details
Show MergeSchedule 
Instance details

Defined in Database.LSMTree.Internal.Config

NFData MergeSchedule 
Instance details

Defined in Database.LSMTree.Internal.Config

Methods

rnf :: MergeSchedule -> () #

Eq MergeSchedule 
Instance details

Defined in Database.LSMTree.Internal.Config

data SizeRatio #

Constructors

Four 

Instances

Instances details
Show SizeRatio 
Instance details

Defined in Database.LSMTree.Internal.Config

NFData SizeRatio 
Instance details

Defined in Database.LSMTree.Internal.Config

Methods

rnf :: SizeRatio -> () #

Eq SizeRatio 
Instance details

Defined in Database.LSMTree.Internal.Config

data DiskCachePolicy #

Instances

Instances details
Show DiskCachePolicy 
Instance details

Defined in Database.LSMTree.Internal.Config

NFData DiskCachePolicy 
Instance details

Defined in Database.LSMTree.Internal.Config

Methods

rnf :: DiskCachePolicy -> () #

Eq DiskCachePolicy 
Instance details

Defined in Database.LSMTree.Internal.Config

Override DiskCachePolicy TableConfig 
Instance details

Defined in Database.LSMTree.Internal.Config.Override

Override DiskCachePolicy SnapshotMetaData 
Instance details

Defined in Database.LSMTree.Internal.Config.Override

Methods

override :: DiskCachePolicy -> SnapshotMetaData -> SnapshotMetaData

Override DiskCachePolicy (SnapLevels SnapshotRun) 
Instance details

Defined in Database.LSMTree.Internal.Config.Override

Methods

override :: DiskCachePolicy -> SnapLevels SnapshotRun -> SnapLevels SnapshotRun

Table Configuration Overrides

Ranges

data Range k #

Constructors

FromToExcluding k k 
FromToIncluding k k 

Instances

Instances details
Functor Range 
Instance details

Defined in Database.LSMTree.Internal.Range

Methods

fmap :: (a -> b) -> Range a -> Range b #

(<$) :: a -> Range b -> Range a #

Show k => Show (Range k) 
Instance details

Defined in Database.LSMTree.Internal.Range

Methods

showsPrec :: Int -> Range k -> ShowS #

show :: Range k -> String #

showList :: [Range k] -> ShowS #

NFData k => NFData (Range k) 
Instance details

Defined in Database.LSMTree.Internal.Range

Methods

rnf :: Range k -> () #

Eq k => Eq (Range k) 
Instance details

Defined in Database.LSMTree.Internal.Range

Methods

(==) :: Range k -> Range k -> Bool #

(/=) :: Range k -> Range k -> Bool #

Union Credit and Debt

Key/Value Serialisation

newtype RawBytes #

Constructors

RawBytes (Vector Word8) 

Instances

Instances details
IsString RawBytes 
Instance details

Defined in Database.LSMTree.Internal.RawBytes

Monoid RawBytes 
Instance details

Defined in Database.LSMTree.Internal.RawBytes

Semigroup RawBytes 
Instance details

Defined in Database.LSMTree.Internal.RawBytes

IsList RawBytes 
Instance details

Defined in Database.LSMTree.Internal.RawBytes

Associated Types

type Item RawBytes #

Show RawBytes 
Instance details

Defined in Database.LSMTree.Internal.RawBytes

Hashable RawBytes 
Instance details

Defined in Database.LSMTree.Internal.RawBytes

Methods

hashSalt64 :: Salt -> RawBytes -> Hash #

NFData RawBytes 
Instance details

Defined in Database.LSMTree.Internal.RawBytes

Methods

rnf :: RawBytes -> () #

Eq RawBytes 
Instance details

Defined in Database.LSMTree.Internal.RawBytes

Ord RawBytes 
Instance details

Defined in Database.LSMTree.Internal.RawBytes

type Item RawBytes 
Instance details

Defined in Database.LSMTree.Internal.RawBytes

class SerialiseKey k where #

Instances

Instances details
SerialiseKey ByteArray 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey Void 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey Int16 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey Int32 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey Int64 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey Int8 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey Word16 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey Word32 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey Word64 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey Word8 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey ByteString 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey ByteString 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey ShortByteString 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey String 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey Int 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseKey Word 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

class SerialiseValue v where #

Instances

Instances details
SerialiseValue ByteArray 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue Void 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue Int16 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue Int32 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue Int64 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue Int8 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue Word16 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue Word32 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue Word64 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue Word8 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue ByteString 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue ByteString 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue ShortByteString 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue String 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue Int 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue Word 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue a => SerialiseValue (Sum a) 
Instance details

Defined in Database.LSMTree.Internal.Serialise.Class

SerialiseValue v => SerialiseValue (ResolveAsFirst v) 
Instance details

Defined in Database.LSMTree.Internal.Types

SerialiseValue v => SerialiseValue (ResolveViaSemigroup v) 
Instance details

Defined in Database.LSMTree.Internal.Types

Key/Value Serialisation Property Tests

Monoidal Value Resolution

Monoidal Value Resolution Property Tests

Errors

Traces

data Tracer (m :: Type -> Type) a #

This type describes some effect in m which depends upon some value of type a, for which the output value is not of interest (only the effects).

The motivating use case is to describe tracing, logging, monitoring, and similar features, in which the programmer wishes to provide some values to some other program which will do some real world side effect, such as writing to a log file or bumping a counter in some monitoring system.

The actual implementation of such a program will probably work on rather large, domain-agnostic types like Text, ByteString, JSON values for structured logs, etc.

But the call sites which ultimately invoke these implementations will deal with smaller, domain-specific types that concisely describe events, metrics, debug information, etc.

This difference is reconciled by the Contravariant instance for Tracer. contramap is used to change the input type of a tracer. This allows for a more general tracer to be used where a more specific one is expected.

Intuitively: if you can map your domain-specific type Event to a Text representation, then any Tracer m Text can stand in where a Tracer m Event is required.

eventToText :: Event -> Text

traceTextToLogFile :: Tracer m Text

traceEventToLogFile :: Tracer m Event
traceEventToLogFile = contramap eventToText traceTextToLogFile

Effectful tracers that actually do interesting stuff can be defined using emit, and composed via contramap.

The nullTracer can be used as a stand-in for any tracer, doing no side-effects and producing no interesting value.

To deal with branching, the arrow interface on the underlying Tracer should be used. Arrow notation can be helpful here.

For example, a common pattern is to trace only some variants of a sum type.

data Event = This Int | That Bool

traceOnlyThat :: Tracer m Int -> Tracer m Bool
traceOnlyThat tr = Tracer $ proc event -> do
  case event of
    This i -> use tr  -< i
    That _ -> squelch -< ()

The key point of using the arrow representation we have here is that this tracer will not necessarily need to force event: if the input tracer tr does not force its value, then event will not be forced. To elaborate, suppose tr is nullTracer. Then this expression becomes

classify (This i) = Left i
classify (That _) = Right ()

traceOnlyThat tr
= Tracer $ Pure classify >>> (squelch ||| squelch) >>> Pure (either id id)
= Tracer $ Pure classify >>> Pure (either (const (Left ())) (const (Right ()))) >>> Pure (either id id)
= Tracer $ Pure (classify >>> either (const (Left ())) (const (Right ())) >>> either id id)

So that when this tracer is run by traceWith we get

traceWith (traceOnlyThat tr) x
= traceWith (Pure _)
= pure ()

It is _essential_ that the computation of the tracing effects cannot itself have side-effects, as this would ruin the ability to short-circuit when it is known that no tracing will be done: the side-effects of a branch could change the outcome of another branch. This would fly in the face of a crucial design goal: you can leave your tracer calls in the program so they do not bitrot, but can also make them zero runtime cost by substituting nullTracer appropriately.

Instances

Instances details
Monad m => Contravariant (Tracer m) 
Instance details

Defined in Control.Tracer

Methods

contramap :: (a' -> a) -> Tracer m a -> Tracer m a' #

(>$) :: b -> Tracer m b -> Tracer m a #

Monad m => Monoid (Tracer m s) 
Instance details

Defined in Control.Tracer

Methods

mempty :: Tracer m s #

mappend :: Tracer m s -> Tracer m s -> Tracer m s #

mconcat :: [Tracer m s] -> Tracer m s #

Monad m => Semigroup (Tracer m s)

tr1 <> tr2 will run tr1 and then tr2 with the same input.

Instance details

Defined in Control.Tracer

Methods

(<>) :: Tracer m s -> Tracer m s -> Tracer m s #

sconcat :: NonEmpty (Tracer m s) -> Tracer m s #

stimes :: Integral b => b -> Tracer m s -> Tracer m s #

newtype SessionId #

Constructors

SessionId FsPath 

Instances

Instances details
Show SessionId 
Instance details

Defined in Database.LSMTree.Internal.Unsafe

Eq SessionId 
Instance details

Defined in Database.LSMTree.Internal.Unsafe

newtype TableId #

Constructors

TableId Int 

Instances

Instances details
Show TableId 
Instance details

Defined in Database.LSMTree.Internal.RunNumber

NFData TableId 
Instance details

Defined in Database.LSMTree.Internal.RunNumber

Methods

rnf :: TableId -> () #

Eq TableId 
Instance details

Defined in Database.LSMTree.Internal.RunNumber

Methods

(==) :: TableId -> TableId -> Bool #

(/=) :: TableId -> TableId -> Bool #

Ord TableId 
Instance details

Defined in Database.LSMTree.Internal.RunNumber

newtype CursorId #

Constructors

CursorId Int 

Instances

Instances details
Show CursorId 
Instance details

Defined in Database.LSMTree.Internal.RunNumber

NFData CursorId 
Instance details

Defined in Database.LSMTree.Internal.RunNumber

Methods

rnf :: CursorId -> () #

Eq CursorId 
Instance details

Defined in Database.LSMTree.Internal.RunNumber

Ord CursorId 
Instance details

Defined in Database.LSMTree.Internal.RunNumber