| Safe Haskell | Safe-Inferred |
|---|---|
| Language | Haskell2010 |
Synapse.NN.Layers.Initializers
Description
Allows to initialize values of layers parameters.
InitializerFn type alias represents functions that are able to initialize matrix with given size
and Initializer newtype wraps InitializerFns.
Synapse provides 4 types of initializers: * Non-random constant initializers * Random uniform distribution initializers * Random normal distribution initializers * Matrix-specific initializers
Synopsis
- type InitializerFn a = (Int, Int) -> Mat a
- newtype Initializer a = Initializer {}
- constants :: Num a => a -> InitializerFn a
- zeroes :: Num a => InitializerFn a
- ones :: Num a => InitializerFn a
- randomUniform :: (UniformRange a, RandomGen g) => (a, a) -> g -> InitializerFn a
- lecunUniform :: (UniformRange a, Floating a, RandomGen g) => g -> InitializerFn a
- heUniform :: (UniformRange a, Floating a, RandomGen g) => g -> InitializerFn a
- glorotUniform :: (UniformRange a, Floating a, RandomGen g) => g -> InitializerFn a
- randomNormal :: (UniformRange a, Floating a, Ord a, RandomGen g) => Maybe a -> a -> a -> g -> InitializerFn a
- lecunNormal :: (UniformRange a, Floating a, Ord a, RandomGen g) => g -> InitializerFn a
- heNormal :: (UniformRange a, Floating a, Ord a, RandomGen g) => g -> InitializerFn a
- glorotNormal :: (UniformRange a, Floating a, Ord a, RandomGen g) => g -> InitializerFn a
- identity :: Num a => InitializerFn a
- orthogonal :: (UniformRange a, Floating a, Ord a, RandomGen g) => g -> InitializerFn a
InitializerFn type alias and Initializer newtype
type InitializerFn a = (Int, Int) -> Mat a Source #
InitializerFn type alias represents functions that are able to initialize matrix with given size.
newtype Initializer a Source #
Initializer newtype wraps InitializerFns - functions that are able to initialize matrix with given size.
Constructors
| Initializer | |
Fields
| |
Non-random constant initializers
constants :: Num a => a -> InitializerFn a Source #
Initializes list that is filled with given constant.
zeroes :: Num a => InitializerFn a Source #
Initializes list that is filled with zeroes.
ones :: Num a => InitializerFn a Source #
Initializes list that is filled with ones.
Random uniform distribution initializers
randomUniform :: (UniformRange a, RandomGen g) => (a, a) -> g -> InitializerFn a Source #
Initializes list with samples from random uniform distribution in range.
This function does not preserve seed generator - split generator before calling this function.
lecunUniform :: (UniformRange a, Floating a, RandomGen g) => g -> InitializerFn a Source #
Initializes list with samples from random LeCun uniform distribution in range.
This function does not preserve seed generator - split generator before calling this function.
heUniform :: (UniformRange a, Floating a, RandomGen g) => g -> InitializerFn a Source #
Initializes list with samples from random He uniform distribution in range.
This function does not preserve seed generator - split generator before calling this function.
glorotUniform :: (UniformRange a, Floating a, RandomGen g) => g -> InitializerFn a Source #
Initializes list with samples from random Glorot uniform distribution in range.
This function does not preserve seed generator - split generator before calling this function.
Random normal distribution initializers
randomNormal :: (UniformRange a, Floating a, Ord a, RandomGen g) => Maybe a -> a -> a -> g -> InitializerFn a Source #
Initializes list with samples from random normal distribution in range which could be truncated.
This function does not preserve seed generator - split generator before calling this function.
lecunNormal :: (UniformRange a, Floating a, Ord a, RandomGen g) => g -> InitializerFn a Source #
Initializes list with samples from random LeCun normal distribution in range which is truncated for values more than two standard deviations from mean.
This function does not preserve seed generator - split generator before calling this function.
heNormal :: (UniformRange a, Floating a, Ord a, RandomGen g) => g -> InitializerFn a Source #
Initializes list with samples from random He normal distribution in range which is truncated for values more than two standard deviations from mean.
This function does not preserve seed generator - split generator before calling this function.
glorotNormal :: (UniformRange a, Floating a, Ord a, RandomGen g) => g -> InitializerFn a Source #
Initializes list with samples from random Glorot normal distribution in range which is truncated for values more than two standard deviations from mean.
This function does not preserve seed generator - split generator before calling this function.
Matrix-like initializers
identity :: Num a => InitializerFn a Source #
Initializes flat identity matrix. If dimensions do not represent square matrix, an error is thrown.
orthogonal :: (UniformRange a, Floating a, Ord a, RandomGen g) => g -> InitializerFn a Source #
Initializes float orthogonal matrix obtained from a random normal distribution that is truncated for values more than two standard deviations from mean.
This function does not preserve seed generator - split generator before calling this function.