第十八章 Post-Processing
Post-processing是指在场景渲染之后,使用一些图形技术对场景进行处理。比如,把整个场景转换为grayscale(灰度)样式或使场景中明亮的区域发光。本章将编写一些post-processing effects,并集成到C++渲染引擎框架中。
Render Targets
到目前为止,所有的示例程序都是直接把场景渲染到back buffer,这是一个2D texture用于在渲染完成之后把图像显示到屏幕上。但是在post-processing应用程序中,首先把场景渲染到一个中间层的texture buffer中,然后把post-processing effect应用到该texture。最后使用一个全屏的四边形(由两个三角形组成,包含整个屏幕区域)渲染最终要显示到屏幕上的图像。
下面的步骤概括了post-processing的处理过程:
1、将一个off-screen render target(离屏渲染目标)绑定到管线的output-merger阶段。
2、在off-scrent render target上绘制场景。
3、恢复back buffer,并作为绑定到output-merger阶段的render target。
4、使用off-scrent render target的texture作为一种post-processing effect的输入buffer,绘制一个全屏的四边形。
为了更容易的执行这些步骤,首先创建一个FullScreenRenderTarget类,该类的声明代码如列表18.1所示。
列表18.1 Declaration of the FullScreenRenderTarget Class
#pragma once
#include "Common.h"
namespace Library
{
class Game;
class FullScreenRenderTarget
{
public:
FullScreenRenderTarget(Game& game);
~FullScreenRenderTarget();
ID3D11ShaderResourceView* OutputTexture() const;
ID3D11RenderTargetView* RenderTargetView() const;
ID3D11DepthStencilView* DepthStencilView() const;
void Begin();
void End();
private:
FullScreenRenderTarget();
FullScreenRenderTarget(const FullScreenRenderTarget& rhs);
FullScreenRenderTarget& operator=(const FullScreenRenderTarget& rhs);
Game* mGame;
ID3D11RenderTargetView* mRenderTargetView;
ID3D11DepthStencilView* mDepthStencilView;
ID3D11ShaderResourceView* mOutputTexture;
};
}
FullScreenRenderTarget类中的成员变量看起来非常熟悉,在Game类中已经包含了同样的数据类型ID3D11RenderTargetView和ID3D11DepthStencilView。这些类型的变量是用于把一个render target和depth-stencil buffer绑定到管线的output-merger阶段。与Game类不同的是,在FullScrentRenderTarget类中还包含有一个ID3D11ShaderResourceView类型的成员变量,表示render target底层的2D texture buffer。这种类型的输出texture可以作为post-processing shaders的输入数据。FullScreenRenderTarget::Begin()和FullScreenRenderTarget::End()函数分别用于把render target绑定以output-merger阶段和恢复back buffer。列表18.2列出了FullScrennRenderTarget类的实现代码。
列表18.2 Implementation of the FullScreenRenderTarget Class
#include "FullScreenRenderTarget.h"
#include "Game.h"
#include "GameException.h"
namespace Library
{
FullScreenRenderTarget::FullScreenRenderTarget(Game& game)
: mGame(&game), mRenderTargetView(nullptr), mDepthStencilView(nullptr), mOutputTexture(nullptr)
{
D3D11_TEXTURE2D_DESC fullScreenTextureDesc;
ZeroMemory(&fullScreenTextureDesc, sizeof(fullScreenTextureDesc));
fullScreenTextureDesc.Width = game.ScreenWidth();
fullScreenTextureDesc.Height = game.ScreenHeight();
fullScreenTextureDesc.MipLevels = 1;
fullScreenTextureDesc.ArraySize = 1;
fullScreenTextureDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
fullScreenTextureDesc.SampleDesc.Count = 1;
fullScreenTextureDesc.SampleDesc.Quality = 0;
fullScreenTextureDesc.BindFlags = D3D11_BIND_RENDER_TARGET | D3D11_BIND_SHADER_RESOURCE;
fullScreenTextureDesc.Usage = D3D11_USAGE_DEFAULT;
HRESULT hr;
ID3D11Texture2D* fullScreenTexture = nullptr;
if (FAILED(hr = game.Direct3DDevice()->CreateTexture2D(&fullScreenTextureDesc, nullptr, &fullScreenTexture)))
{
throw GameException("IDXGIDevice::CreateTexture2D() failed.", hr);
}
if (FAILED(hr = game.Direct3DDevice()->CreateShaderResourceView(fullScreenTexture, nullptr, &mOutputTexture)))
{
throw GameException("IDXGIDevice::CreateShaderResourceView() failed.", hr);
}
if (FAILED(hr = game.Direct3DDevice()->CreateRenderTargetView(fullScreenTexture, nullptr, &mRenderTargetView)))
{
ReleaseObject(fullScreenTexture);
throw GameException("IDXGIDevice::CreateRenderTargetView() failed.", hr);
}
ReleaseObject(fullScreenTexture);
D3D11_TEXTURE2D_DESC depthStencilDesc;
ZeroMemory(&depthStencilDesc, sizeof(depthStencilDesc));
depthStencilDesc.Width = game.ScreenWidth();
depthStencilDesc.Height = game.ScreenHeight();
depthStencilDesc.MipLevels = 1;
depthStencilDesc.ArraySize = 1;
depthStencilDesc.Format = DXGI_FORMAT_D24_UNORM_S8_UINT;
depthStencilDesc.SampleDesc.Count = 1;
depthStencilDesc.SampleDesc.Quality = 0;
depthStencilDesc.BindFlags = D3D11_BIND_DEPTH_STENCIL;
depthStencilDesc.Usage = D3D11_USAGE_DEFAULT;
ID3D11Texture2D* depthStencilBuffer = nullptr;
if (FAILED(hr = game.Direct3DDevice()->CreateTexture2D(&depthStencilDesc, nullptr, &depthStencilBuffer)))
{
throw GameException("IDXGIDevice::CreateTexture2D() failed.", hr);
}
if (FAILED(hr = game.Direct3DDevice()->CreateDepthStencilView(depthStencilBuffer, nullptr, &mDepthStencilView)))
{
ReleaseObject(depthStencilBuffer);
throw GameException("IDXGIDevice::CreateDepthStencilView() failed.", hr);
}
ReleaseObject(depthStencilBuffer);
}
FullScreenRenderTarget::~FullScreenRenderTarget()
{
ReleaseObject(mOutputTexture);
ReleaseObject(mDepthStencilView);
ReleaseObject(mRenderTargetView);
}
ID3D11ShaderResourceView* FullScreenRenderTarget::OutputTexture() const
{
return mOutputTexture;
}
ID3D11RenderTargetView* FullScreenRenderTarget::RenderTargetView() const
{
return mRenderTargetView;
}
ID3D11DepthStencilView* FullScreenRenderTarget::DepthStencilView() const
{
return mDepthStencilView;
}
void FullScreenRenderTarget::Begin()
{
mGame->Direct3DDeviceContext()->OMSetRenderTargets(1, &mRenderTargetView, mDepthStencilView);
}
void FullScreenRenderTarget::End()
{
mGame->ResetRenderTargets();
}
}
在FullScreenRenderTarget类的构造函数中,包含了该类的大部分实现代码。在该构造函数中,首先构建一个D3D11_TEXTURE2D_DESC结构体类型的实例,并用于创建render target的底层texture buffer。在Game类的初始化过程中这些步骤并不是显式定义的,因为在构建swap chain时就会隐含的创建back buffer。其中在D3D11_TEXTURE2D_DESC结构体实例化时指定了两个bind flags:D3D11_BIND_RENDER_TARGET和D3D_BIND_SHADER_RESOURCE。这两个flags值表示该texture可以同时作为一个render target和一个shader的输入数据。
创建完ID3D11Texture2D类型的texture之后,就可以使用该texture创建shader resource view和render target view。并对外提供了访问这两个views变量的公有函数接口;并且在这两个变量初始化之后就可以释放对texture的引用了。对于depth-stencil view变量的初始化使用类似的操作步骤。
在FullScrentRenderTarget::Begin()函数中,直接调用ID3D11DeviceContext::OMSetRenderTargets()函数把render target view和depth-stencil view绑定到管线的output-merger阶段。而FullScreenRenderTarget::End()函数中则是调用Game::ResetRenderTargets()函数,该函数同样是调用OMSetRenderTargets()函数重置绑定为Game类的中render target view和detph-stencil view变量。
对于FullScreenRenderTarget类的调用方式如下:
mRenderTarget->Begin();
// 1. Clear mRenderTarget->RenderTargetView()
// 2. Clear mRenderTarget->DepthStencilView()
// 3. Draw Objects
mRenderTarget->End();
这种方法首先把objects渲染到FullScreenRenderTarget实例下的2D texture,并且可以通过调用FullScreenRenderTarget::OutputTexture()函数访问该texture变量。最后调用FullScreenRenderTarget::End()函数,就可以把所有的渲染数据写到back buffer中。
A Full-Screen Quad Component
现在我们可以把场景渲染到一个off-screen texture中,接下来需要把一种effect应用到该texture并在屏幕上显示应用的输出结果。在这样一个系统结构中需要封装渲染部分的代码,否则在程序之间将会出现重复多余的代码,但是又要使渲染代码足够灵活,以及支持任意的post-processing effects。列表18.3列出了FullScreenQuad类的声明代码。
列表18.3 Declaration of the FullScreenQuad Class
#pragma once
#include <functional>
#include "DrawableGameComponent.h"
namespace Library
{
class Effect;
class Material;
class Pass;
class FullScreenQuad : public DrawableGameComponent
{
RTTI_DECLARATIONS(FullScreenQuad, DrawableGameComponent)
public:
FullScreenQuad(Game& game);
FullScreenQuad(Game& game, Material& material);
~FullScreenQuad();
Material* GetMaterial();
void SetMaterial(Material& material, const std::string& techniqueName, const std::string& passName);
void SetActiveTechnique(const std::string& techniqueName, const std::string& passName);
void SetCustomUpdateMaterial(std::function<void()> callback);
virtual void Initialize() override;
virtual void Draw(const GameTime& gameTime) override;
private:
FullScreenQuad();
FullScreenQuad(const FullScreenQuad& rhs);
FullScreenQuad& operator=(const FullScreenQuad& rhs);
Material* mMaterial;
Pass* mPass;
ID3D11InputLayout* mInputLayout;
ID3D11Buffer* mVertexBuffer;
ID3D11Buffer* mIndexBuffer;
UINT mIndexCount;
std::function<void()> mCustomUpdateMaterial;
};
}
在FullScreenQuad类中,成员变量mMaterial指向一个用于绘制四边形的material对象,并可以通过调用SetMaterial函数对该变量进行赋值。这样就可以在同一个FullScreenQuad实例对象的生存期内使用不同的materials。成员变量mPass和mInputLayout中存储了Draw()函数中使用的相关数据。另外两个非常熟悉的变量vertex和index buffers存储了该四边形对象的vertices和indices。在FullScreenQuad类中使用了一种新的数据类型std::function<T>,如果你之前没有见过的话,这是一种通用的函数封装语法,用于存储并调用函数,bind表达式,lambda表达式(回调以及关闭)。在这里是为了支持FullScreenQuad的调用者更新material shader中的变量。之所以这样做是因为在FullScreenQuad类中并不知道用于渲染四边形的是哪种material(因此也就无法知道material使用的shader输入变量)。FullScreenQuad类要完成的全部操作就是渲染一个四边形;而更新material变量的操作由该类对象的调用者完成。
列表18.4中列出了FullScreenQuad类的实现代码。
列表18.4 Implementation of the FullScreenQuad Class
#include "FullScreenQuad.h"
#include "Game.h"
#include "GameException.h"
#include "Material.h"
#include "VertexDeclarations.h"
namespace Library
{
RTTI_DEFINITIONS(FullScreenQuad)
FullScreenQuad::FullScreenQuad(Game& game)
: DrawableGameComponent(game),
mMaterial(nullptr), mPass(nullptr), mInputLayout(nullptr),
mVertexBuffer(nullptr), mIndexBuffer(nullptr), mIndexCount(0), mCustomUpdateMaterial(nullptr)
{
}
FullScreenQuad::FullScreenQuad(Game& game, Material& material)
: DrawableGameComponent(game),
mMaterial(&material), mPass(nullptr), mInputLayout(nullptr),
mVertexBuffer(nullptr), mIndexBuffer(nullptr), mIndexCount(0), mCustomUpdateMaterial(nullptr)
{
}
FullScreenQuad::~FullScreenQuad()
{
ReleaseObject(mIndexBuffer);
ReleaseObject(mVertexBuffer);
}
Material* FullScreenQuad::GetMaterial()
{
return mMaterial;
}
void FullScreenQuad::SetMaterial(Material& material, const std::string& techniqueName, const std::string& passName)
{
mMaterial = &material;
SetActiveTechnique(techniqueName, passName);
}
void FullScreenQuad::SetActiveTechnique(const std::string& techniqueName, const std::string& passName)
{
Technique* technique = mMaterial->GetEffect()->TechniquesByName().at(techniqueName);
assert(technique != nullptr);
mPass = technique->PassesByName().at(passName);
assert(mPass != nullptr);
mInputLayout = mMaterial->InputLayouts().at(mPass);
}
void FullScreenQuad::SetCustomUpdateMaterial(std::function<void()> callback)
{
mCustomUpdateMaterial = callback;
}
void FullScreenQuad::Initialize()
{
VertexPositionTexture vertices[] =
{
VertexPositionTexture(XMFLOAT4(-1.0f, -1.0f, 0.0f, 1.0f), XMFLOAT2(0.0f, 1.0f)),
VertexPositionTexture(XMFLOAT4(-1.0f, 1.0f, 0.0f, 1.0f), XMFLOAT2(0.0f, 0.0f)),
VertexPositionTexture(XMFLOAT4(1.0f, 1.0f, 0.0f, 1.0f), XMFLOAT2(1.0f, 0.0f)),
VertexPositionTexture(XMFLOAT4(1.0f, -1.0f, 0.0f, 1.0f), XMFLOAT2(1.0f, 1.0f)),
};
D3D11_BUFFER_DESC vertexBufferDesc;
ZeroMemory(&vertexBufferDesc, sizeof(vertexBufferDesc));
vertexBufferDesc.ByteWidth = sizeof(VertexPositionTexture)* ARRAYSIZE(vertices);
vertexBufferDesc.Usage = D3D11_USAGE_IMMUTABLE;
vertexBufferDesc.BindFlags = D3D11_BIND_VERTEX_BUFFER;
D3D11_SUBRESOURCE_DATA vertexSubResourceData;
ZeroMemory(&vertexSubResourceData, sizeof(vertexSubResourceData));
vertexSubResourceData.pSysMem = vertices;
if (FAILED(mGame->Direct3DDevice()->CreateBuffer(&vertexBufferDesc, &vertexSubResourceData, &mVertexBuffer)))
{
throw GameException("ID3D11Device::CreateBuffer() failed.");
}
UINT indices[] =
{
0, 1, 2,
0, 2, 3
};
mIndexCount = ARRAYSIZE(indices);
D3D11_BUFFER_DESC indexBufferDesc;
ZeroMemory(&indexBufferDesc, sizeof(indexBufferDesc));
indexBufferDesc.ByteWidth = sizeof(UINT)* mIndexCount;
indexBufferDesc.Usage = D3D11_USAGE_IMMUTABLE;
indexBufferDesc.BindFlags = D3D11_BIND_INDEX_BUFFER;
D3D11_SUBRESOURCE_DATA indexSubResourceData;
ZeroMemory(&indexSubResourceData, sizeof(indexSubResourceData));
indexSubResourceData.pSysMem = indices;
if (FAILED(mGame->Direct3DDevice()->CreateBuffer(&indexBufferDesc, &indexSubResourceData, &mIndexBuffer)))
{
throw GameException("ID3D11Device::CreateBuffer() failed.");
}
}
void FullScreenQuad::Draw(const GameTime& gameTime)
{
assert(mPass != nullptr);
assert(mInputLayout != nullptr);
ID3D11DeviceContext* direct3DDeviceContext = mGame->Direct3DDeviceContext();
direct3DDeviceContext->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
direct3DDeviceContext->IASetInputLayout(mInputLayout);
UINT stride = sizeof(VertexPositionTexture);
UINT offset = 0;
direct3DDeviceContext->IASetVertexBuffers(0, 1, &mVertexBuffer, &stride, &offset);
direct3DDeviceContext->IASetIndexBuffer(mIndexBuffer, DXGI_FORMAT_R32_UINT, 0);
if (mCustomUpdateMaterial != nullptr)
{
mCustomUpdateMaterial();
}
mPass->Apply(0, direct3DDeviceContext);
direct3DDeviceContext->DrawIndexed(mIndexCount, 0, 0);
}
}
首先分析FullScreenQuad::Initialize()函数,该函数的实现与之前示例中的代码基本相同,主要不同的地方是4个vertices所处的空间位置。在screen space中,(-1, -1)表示屏幕的左上角,(1, 1)表示右下角。因为vertices的坐标位置已经是处于screen space中,所以在vertext shader中不需要对这些坐标执行变换。
接下来,分析FullScreen::Draw()函数。除了执行std::function类型的mCustomUpdateMaterial()语句之外,在Draw()函数没有特别需要说明的地方。其中std::function<T>类是一个函数对象(或函数),并提供了公有的operator()函数(重载()运算符)。使用这种方式,看起来就像是调用了一个命名为mCustomUpdateMaterial的函数;实际上,只是调用了std::function<T>::operation()运算符重载函数,真正的目的是用于回调函数或lambda表达式。
最后,需要注意的是四边形的vertices由一个position和texture coordinates表示。这种表示方式限制了在FullScreenQuad类中能够使用的material种类,但是该依然具有多种应用。只需要对该类做一点点扩展,就可以根据具体的material动态的创建vertex buffer。