| Copyright | (c) Justin Le 2023 |
|---|---|
| License | BSD3 |
| Maintainer | justin@jle.im |
| Stability | experimental |
| Portability | non-portable |
| Safe Haskell | None |
| Language | Haskell2010 |
Numeric.Backprop.Internal
Description
Provides the types and instances used for the graph building/back-propagation for the library.
Do not import this unless you want to mess with the internals and know what you're doing. Otherwise, use Numeric.Backprop, Numeric.Backprop.Op, etc.
Since: 0.2.7.0
Synopsis
- data BVar s a = BV {}
- newtype W = W {
- wRef :: IORef (Int, [SomeTapeNode])
- backpropWithN :: forall (as :: [Type]) b. Rec ZeroFunc as -> (forall s. Reifies s W => Rec (BVar s) as -> BVar s b) -> Rec Identity as -> (b, b -> Rec Identity as)
- evalBPN :: forall (as :: [Type]) b. (forall s. Reifies s W => Rec (BVar s) as -> BVar s b) -> Rec Identity as -> b
- constVar :: a -> BVar s a
- liftOp :: forall (as :: [Type]) b s. Reifies s W => Rec AddFunc as -> Op as b -> Rec (BVar s) as -> BVar s b
- liftOp1 :: forall a b s. Reifies s W => AddFunc a -> Op '[a] b -> BVar s a -> BVar s b
- liftOp2 :: forall a b c s. Reifies s W => AddFunc a -> AddFunc b -> Op '[a, b] c -> BVar s a -> BVar s b -> BVar s c
- liftOp3 :: forall a b c d s. Reifies s W => AddFunc a -> AddFunc b -> AddFunc c -> Op '[a, b, c] d -> BVar s a -> BVar s b -> BVar s c -> BVar s d
- viewVar :: forall a b s. Reifies s W => AddFunc a -> ZeroFunc b -> Lens' b a -> BVar s b -> BVar s a
- setVar :: forall a b s. Reifies s W => AddFunc a -> AddFunc b -> ZeroFunc a -> Lens' b a -> BVar s a -> BVar s b -> BVar s b
- sequenceVar :: forall t a s. (Reifies s W, Traversable t) => AddFunc a -> ZeroFunc a -> BVar s (t a) -> t (BVar s a)
- collectVar :: forall t a s. (Reifies s W, Foldable t, Functor t) => AddFunc a -> ZeroFunc a -> t (BVar s a) -> BVar s (t a)
- previewVar :: forall b a s. Reifies s W => AddFunc a -> ZeroFunc b -> Traversal' b a -> BVar s b -> Maybe (BVar s a)
- toListOfVar :: forall b a s. Reifies s W => AddFunc a -> ZeroFunc b -> Traversal' b a -> BVar s b -> [BVar s a]
- coerceVar :: Coercible a b => BVar s a -> BVar s b
- newtype ZeroFunc a = ZF {
- runZF :: a -> a
- zfNum :: Num a => ZeroFunc a
- zeroFunc :: Backprop a => ZeroFunc a
- newtype AddFunc a = AF {
- runAF :: a -> a -> a
- afNum :: Num a => AddFunc a
- addFunc :: Backprop a => AddFunc a
- newtype OneFunc a = OF {
- runOF :: a -> a
- ofNum :: Num a => OneFunc a
- oneFunc :: Backprop a => OneFunc a
- debugSTN :: SomeTapeNode -> String
- debugIR :: InpRef a -> String
- data TapeNode a where
- data SomeTapeNode where
- STN :: forall a. {..} -> SomeTapeNode
- data BRef s
- data Runner s = R {}
- data InpRef a where
- initWengert :: IO W
- insertNode :: TapeNode a -> a -> W -> IO (BVar s a)
- bvConst :: BVar s a -> Maybe a
- forceBVar :: BVar s a -> ()
- forceInpRef :: InpRef a -> ()
- forceSomeTapeNode :: SomeTapeNode -> ()
- forceTapeNode :: TapeNode a -> ()
- fillWengert :: forall (as :: [Type]) b. (forall s. Reifies s W => Rec (BVar s) as -> BVar s b) -> Rec Identity as -> IO (Either Int (Int, [SomeTapeNode]), b)
- bumpMaybe :: a -> (a -> b -> b) -> (a -> b) -> Maybe b -> Maybe b
- initRunner :: (Int, [SomeTapeNode]) -> (Int, [Maybe (Any :: Type)]) -> ST s (Runner s)
- gradRunner :: b -> Runner s -> (Int, [SomeTapeNode]) -> ST s ()
Exported for re-use
A BVar s aa that can be "backpropagated".
Functions referring to BVars are tracked by the library and can be
automatically differentiated to get their gradients and results.
For simple numeric values, you can use its Num, Fractional, and
Floating instances to manipulate them as if they were the numbers they
represent.
If a contains items, the items can be accessed and extracted using
lenses. A Lens' b aa inside a b, using
^^. (viewVar):
(^.) :: a ->Lens'a b -> b (^^.) ::BVars a ->Lens'a b ->BVars b
There is also ^^? (previewVar), to use a Prism'
or Traversal' to extract a target that may or may not be present
(which can implement pattern matching), ^^..
(toListOfVar) to use a Traversal' to extract all
targets inside a BVar, and .~~ (setVar) to set and update values
inside a BVar.
If you have control over your data type definitions, you can also use
splitBV and joinBV to manipulate
data types by easily extracting fields out of a BVar of data types and
creating BVars of data types out of BVars of their fields. See
Numeric.Backprop for a tutorial on this use pattern.
For more complex operations, libraries can provide functions on BVars
using liftOp and related functions. This is how you
can create primitive functions that users can use to manipulate your
library's values. See
https://backprop.jle.im/08-equipping-your-library.html for a detailed
guide.
For example, the hmatrix library has a matrix-vector multiplication
function, #> :: L m n -> R n -> L m.
A library could instead provide a function #> :: , which the user can then use to manipulate their
BVar (L m n) -> BVar
(R n) -> BVar (R m)BVars of L m ns and R ns, etc.
See Numeric.Backprop and documentation for
liftOp for more information.
Instances
| IsoHKD (BVar s :: Type -> Type) (a :: Type) Source # | Since: 0.2.6.3 |
| BVGroup s ('[] :: [Type]) (K1 i a :: Type -> Type) (K1 i (BVar s a) :: Type -> Type) Source # | |
| (Backprop a, Reifies s W) => Backprop (BVar s a) Source # | Since: 0.2.2.0 |
| (Floating a, Reifies s W) => Floating (BVar s a) Source # | |
Defined in Numeric.Backprop.Internal Methods sqrt :: BVar s a -> BVar s a # (**) :: BVar s a -> BVar s a -> BVar s a # logBase :: BVar s a -> BVar s a -> BVar s a # asin :: BVar s a -> BVar s a # acos :: BVar s a -> BVar s a # atan :: BVar s a -> BVar s a # sinh :: BVar s a -> BVar s a # cosh :: BVar s a -> BVar s a # tanh :: BVar s a -> BVar s a # asinh :: BVar s a -> BVar s a # acosh :: BVar s a -> BVar s a # atanh :: BVar s a -> BVar s a # log1p :: BVar s a -> BVar s a # expm1 :: BVar s a -> BVar s a # | |
| (Num a, Reifies s W) => Num (BVar s a) Source # | |
| (Fractional a, Reifies s W) => Fractional (BVar s a) Source # | |
| NFData a => NFData (BVar s a) Source # | This will force the value inside, as well. |
Defined in Numeric.Backprop.Internal | |
| Eq a => Eq (BVar s a) Source # | Compares the values inside the Since: 0.1.5.0 |
| Ord a => Ord (BVar s a) Source # | Compares the values inside the Since: 0.1.5.0 |
Defined in Numeric.Backprop.Internal | |
| type HKD (BVar s :: Type -> Type) (a :: Type) Source # | |
An ephemeral Wengert Tape in the environment. Used internally to track of the computational graph of variables.
For the end user, one can just imagine Reifies s Ws that allows backpropagation to work.
Constructors
| W | Since: 0.2.7.1 |
Fields
| |
backpropWithN :: forall (as :: [Type]) b. Rec ZeroFunc as -> (forall s. Reifies s W => Rec (BVar s) as -> BVar s b) -> Rec Identity as -> (b, b -> Rec Identity as) Source #
backpropWithN, but with explicit zero and one.
Note that argument order changed in v0.2.4.
Since: 0.2.0.0
evalBPN :: forall (as :: [Type]) b. (forall s. Reifies s W => Rec (BVar s) as -> BVar s b) -> Rec Identity as -> b Source #
evalBP generalized to multiple inputs of different types. See
documentation for backpropN for more details.
constVar :: a -> BVar s a Source #
Lift a value into a BVar representing a constant value.
This value will not be considered an input, and its gradients will not be backpropagated.
liftOp :: forall (as :: [Type]) b s. Reifies s W => Rec AddFunc as -> Op as b -> Rec (BVar s) as -> BVar s b Source #
liftOp2 :: forall a b c s. Reifies s W => AddFunc a -> AddFunc b -> Op '[a, b] c -> BVar s a -> BVar s b -> BVar s c Source #
liftOp3 :: forall a b c d s. Reifies s W => AddFunc a -> AddFunc b -> AddFunc c -> Op '[a, b, c] d -> BVar s a -> BVar s b -> BVar s c -> BVar s d Source #
viewVar :: forall a b s. Reifies s W => AddFunc a -> ZeroFunc b -> Lens' b a -> BVar s b -> BVar s a Source #
setVar :: forall a b s. Reifies s W => AddFunc a -> AddFunc b -> ZeroFunc a -> Lens' b a -> BVar s a -> BVar s b -> BVar s b Source #
sequenceVar :: forall t a s. (Reifies s W, Traversable t) => AddFunc a -> ZeroFunc a -> BVar s (t a) -> t (BVar s a) Source #
sequenceVar, but with explicit add and zero.
collectVar :: forall t a s. (Reifies s W, Foldable t, Functor t) => AddFunc a -> ZeroFunc a -> t (BVar s a) -> BVar s (t a) Source #
collectVar, but with explicit add and zero.
previewVar :: forall b a s. Reifies s W => AddFunc a -> ZeroFunc b -> Traversal' b a -> BVar s b -> Maybe (BVar s a) Source #
previewVar, but with explicit add and zero.
toListOfVar :: forall b a s. Reifies s W => AddFunc a -> ZeroFunc b -> Traversal' b a -> BVar s b -> [BVar s a] Source #
toListOfVar, but with explicit add and zero.
coerceVar :: Coercible a b => BVar s a -> BVar s b Source #
Coerce a BVar contents. Useful for things like newtype wrappers.
Since: 0.1.5.2
Func wrappers
"Zero out" all components of a value. For scalar values, this should
just be const 0
Should be idempotent: Applying the function twice is the same as applying it just once.
Each type should ideally only have one ZeroFunc. This coherence
constraint is given by the typeclass Backprop.
Since: 0.2.0.0
Add together two values of a type. To combine contributions of gradients, so should ideally be information-preserving.
See laws for Backprop for the laws this should be expected to
preserve. Namely, it should be commutative and associative, with an
identity for a valid ZeroFunc.
Each type should ideally only have one AddFunc. This coherence
constraint is given by the typeclass Backprop.
Since: 0.2.0.0
One all components of a value. For scalar values, this should
just be const 1
Should be idempotent: Applying the function twice is the same as applying it just once.
Each type should ideally only have one OneFunc. This coherence
constraint is given by the typeclass Backprop.
Since: 0.2.0.0
Debug
debugSTN :: SomeTapeNode -> String Source #
Debugging string for a SomeTapeMode.
Only used internally within this module
data SomeTapeNode where Source #
Since: 0.2.7.1
Constructors
| STN | |
Fields
| |
Since: 0.2.7.1
Instances
Since: 0.2.7.1
Since: 0.2.7.2
initWengert :: IO W Source #
Since: 0.2.7.1
forceInpRef :: InpRef a -> () Source #
Since: 0.2.7.1
forceSomeTapeNode :: SomeTapeNode -> () Source #
Since: 0.2.7.1
forceTapeNode :: TapeNode a -> () Source #
Since: 0.2.7.1
fillWengert :: forall (as :: [Type]) b. (forall s. Reifies s W => Rec (BVar s) as -> BVar s b) -> Rec Identity as -> IO (Either Int (Int, [SomeTapeNode]), b) Source #
Since: 0.2.7.1