Previous Chapter: Shaders

Main Page

This chapter will walk you through the important notions and tools for writing Materials.

A Material is esentially the shader that renders objects from the world onto a texture (generally a camera's texture).

A Material can be applied to an entity by inserting a WithMaterial component on it:

insert (WithMaterial @SpriteMaterial) bob

Each material has an associated VertexInput (VI), a VertexOutput (VO), and a set of Bindings (B).

VI is the input to the vertex shader. It contains a Uint, the index of the current vertex. (other types of draw calls will be supported in the future). This is the standard way of writing it:

newtype VertexInput f = VertexInput (BuiltIn f "vertex_index" Uint)
  deriving (Generic, ShaderParam)

VO is the output of the vertex shader and the input to the fragment shader. It always contains the builtin parameter position, indicating the world position of the vertex. The other parameters are added with Location and a unique index. A common VO looks something like this:

data VertexOutput f = VertexOutput
  { pos :: BuiltIn f "position" (Vec4 Float),
    uv :: Location f 0 (Vec2 Float)
  }
  deriving (Generic, ShaderParam)

The vertex and fragment shaders of material M are two functions that look like this:

vertex :: VI M GPU -> Shader M (VO M GPU)

fragment :: VO M GPU-> Shader M (Vec4 Float)

The GPU applied on the types is their shader representation. The above types become:

VertexInput(Uint)

VertexOutput {
  pos :: Vec4 Float,
  uv :: Vec2 Float
}

The Bindings of a Material are data that's passed into the shaders from outside. Bindings can contain Textures, Samplers and Buffers.

data Bindings f = Bindings {
    texture :: Binding f Texture
    sampler :: Binding f Texture,
    buffer :: Uniform f SomeBuffer
  } deriving (Generic, Bindable)

Each Material has a fetch function which is used to specify how you want to fetch the binding data from the world. This is done through a FetchBindings type which contains 3 functions:

camera :: CameraEntity -> Query System x

mesh :: MeshEntity -> Query System y

result :: x -> y -> b CPU

A function grabs some of the data from the camera, one from the mesh (i.e. sprite), and one which combines them to make the Bindings.

As an example, look at the Bindings and fetch function of SpriteMaterial:

data Bindings f = Bindings
  { matrices :: Uniform f (Matrices f),
    sprite :: Uniform f (SpriteData f),
    texture :: Binding f Texture,
    sampler :: Binding f Sampler
  }
  deriving (Generic, Bindable)

fetch =
  FetchBindings
    ( \camera -> do
        (DefaultSampler sampler) <- qres @DefaultSampler
        CameraMatrices matrices <- qget camera [qd|CameraMatrices|]
        pure (matrices, sampler)
    )
    ( \sprite -> do
        (Sprite {image}, buffer) <- qget sprite [qd|Sprite, (Buffer SpriteData)|]
        ImageTextureView texture <- qget image [qd|ImageTextureView|]
        pure (buffer, texture)
    )
    (\(matrices, sampler) (buffer, texture) -> Bindings {matrices, sampler, sprite = buffer, texture})

Inside a Shader M, you can request the bindings of material M using bindings:

b <- bindings

You can also get the bindings of any material which the current material extends using extension:

b <- extension @SpriteMaterial

Buffers are data types made of shader primitives, such as:

data SpriteData f = SpriteData
  { coords :: Field f (Vec3 Float),
    size :: Field f (Vec2 Float),
    slice :: Field f (Vec4 Float),
    flip :: Field f (Vec2 Int),
    rotation :: Field f (Mat 3 3)
  }
  deriving (Generic, Bufferable)

This is how you create buffers:

Just device <- res @RenderDevice
buf <- createBuffer @SpriteData device

And how you update them:

Just queue <- res @RenderQueue
uploadBuffer queue buf SpriteData {...}

When uploading, you must use the CPU representation of the type. This uses Linear types to represent vectors and matrices. For instance, SpriteData CPU looks like this:

SpriteData {
  coords :: V3 Float
  size :: V2 Float
  ...
}

Each buffer can be inserted on entities as a component, which makes it easy to fetch them.

If your buffer's CPU representation is also a component, you can schedule maintainBuffer. For instance: for instance:

instance Component (SpriteData CPU)

systems (maintainBuffer @SpriteMaterial)
  & schedule @RenderPrepare

For all entities that have SpriteData CPU, it will ensure a Buffer SpriteData component is on them and upload data into it each time it detects a change.

To 'run' a material, use one of the renderMaterial and renderSortedMaterial functions. These are typically scheduled to run each frame, in a Render schedule.

systems (renderMaterial @MyMaterial)
  & schedule @RenderUpdate

To make a fullscreen material, make a material that extends FullscreenMaterial and uses its vertex shader. For instance, to make a material which makes the screen red:

data Bindings f = Bindings deriving (Generic, Bindable)
data RedMaterial

instance Material RedMaterial where
  type VI RedMaterial = Fullscreen.VertexInput
  type VO RedMaterial = Fullscreen.VertexOutput
  type B RedMaterial = Bindings

  type Extends RedMaterial = '[FullscreenMaterial]

  vertex = liftExt . vertex @FullscreenMaterial
  fragment _ = pure $ vec4 1 0 0 1

  fetch =
    FetchBindings
      (const $ pure ())
      (const $ pure ())
      (const $ const $ Bindings)

Note that, when rendering, you need to set the number of vertices to 4:

renderMaterial @RedMaterial def {nvertices = 4}

A postprocess material is a fullscreen material which also takes its camera's texture as an input, modifying it.

You can turn the above material into a postprocess material by changing its bindings and its fetch function:

data Bindings f = Bindings
  { texture :: Binding f Texture,
    sampler :: Binding f Sampler
  }
  deriving (Generic, Bindable)

fetch =
  FetchBindings
    ( \camera -> do
        (DefaultSampler sampler) <- qres @DefaultSampler
        texture <- cameraSwapView camera
        pure (sampler, texture)
    )
    (const $ pure ())
    (\(sampler, texture) _ -> Bindings texture sampler)

You can now, for instance, multiply the entire camera's output with red:

fragment vo = do
  Bindings {texture, sampler} <- bindings
  pure $ sample texture sampler vo.uv * vec4 1 0 0 1