Rendering 3D Volumes Using Per-Pixel Displacement ... - Eric Risser

Abstract
Rendering 3D Volumes Using Per-Pixel Displacement Mapping
Rendering 3D Volumes Using Per-Pixel Displacement Mapping offers
a simple and practical solution to the problem of seamlessly
integrating many highly detailed 3D objects into a scene without
the need to render large sets of polygons or introduce the overhead
of an obtrusive scene-graph. This work takes advantage of modern
programmable GPU’s as well as recent related research in the
area of per-pixel displacement mapping to achieve view independent,
fully 3D rendering with per-pixel level of detail. To achieve
this, a box is used to bound texture-defined volumes. The box acts
as a surface to which the volume will be drawn on. By computing
a viewing ray from the camera to a point on the box and using that
point as a ray origin, the correct intersection with the texture volume
can be found using various per-pixel displacement mapping
techniques. Once the correct intersection is found, the final color
value for the corresponding point on the box can be computed. The
technique supports various effects taken both from established raycasting
and ray-tracing methods such as reflection, refraction, selfshadowing
on models, a simple animation scheme and an efficient
method for finding distances through volumes.
Keywords: per-pixel displacement mapping, image based rendering,
impostor rendering, volumetric rendering
∗ e-mail: erisser@cs.ucf.edu
Eric Risser ∗
University of Central Florida
1 Introduction
Image based techniques have eclipsed all others as a practical
means of adding profuse, highly detailed objects to an environment
during real-time rendering. Using conventional methods to achieve
this goal would require the use of huge sets of polygons to be rendered
as well as the extra overhead necessary to manage the added
scene complexity. Among the proposed alternatives, the impostor
method is at the forefront. An impostor is a two dimensional image
which is rendered onto a billboard. When incorporated into a scene
the billboard is meant to replace highly detailed geometry. Impostors
are pre-computed and stored in memory or they are generated
in real-time. In either case an impostor is only accurate for a specific
viewing direction. In practical implementations an accuracy
threshold is given. As the viewing direction changes the accuracy
of the impostor diminishes. Once the threshold is passed, the image
is replaced with a new image which is accurate in terms of the new
viewing direction.
This technique utilizes modern programmable graphics hardware in
order to achieve accurate geometric reproductions of 3D models for
any given viewing angle. It improves True Impostors [Risser. 2007]
as well as builds off of and relies heavily on concepts laid out in Relief
Mapping of Non-Height Field Surfaces [Policarpo et al. 2006]
and Parallax Occlusion Mapping [Tatarchuk. 2006]. Rendering 3D
Volumes Using Per-Pixel Displacement Mapping distinguishes itself
from previous per-pixel displacement mapping impostor methods
in two major areas.

1. By contributing a technique for rendering a texture volume
onto the three visible faces of a box, not only breaking free
of restrictions on the viewing direction but also achieving true
volumetric rendering.
2. By offering an alternative composition for texture-defined
volumes, allowing for a larger possibility of potential shapes
to be stored and rendered with no additional overhead.
2 Background
Parallax mapping [Kaneko et al. 2001] is a method for approximating
the parallax seen on uneven surfaces. Using the view ray transformed
to tangent space, parallax mapping samples a height texture
to find the approximate texture coordinate offset that will give the
illusion that a three dimensional structure is being rendered. Parallax
mapping is a crude approximation of displacement mapping. It
cannot simulate occlusion, self shadowing or silhouettes. Since it
requires only one additional texture read, it is a fast and therefore
relevant approximation for use in video games.
View dependent displacement mapping [Wang et al. 2003a] takes
a bi-directional texture function (BTF) approach to per-pixel displacement
mapping. This approach, much like impostors involves
pre-computing for each potential view direction an additional texture
map of the same dimensions as the source texture. The goal
of this method is to store the distance from the surface of our polygon
to the imaginary surface that is being viewed. The method
stores a five-dimensional map indexed by three position coordinates
and two angular coordinates. The pre-computed data is then compressed
and decompressed at runtime on the GPU. This method
produces good visual results but requires significant preprocessing
and storage to operate.
Relief mapping [Policarpo et al. 2005] can be considered an extension
of parallax mapping. Rather than a coarse approximation to
the solution, relief mapping performs a linear search or ”ray march”
along the view ray until it finds a collision with the surface, at which
point a binary search is used to locate the exact point of intersection.
Shell texture functions [Chen et al 2004], much like relief mapping
attempts to render the subsurface details of a polygon. However,
unlike relief mapping, which restricts itself to finding a new,
more detailed surface, shell texture functions attempts to ray trace a
complex volumetric data set. Shell texture functions produce highquality
visual results by accounting for sub-surface scattering. This
technique is notable due to the hybridization it attempts between
rasterization and ray tracing. It is, however, not viable for many
applications in the graphics community due to long pre-processing
times and a non-interactive frame rate.
Relief mapping of non-linear height-fields [Policarpo et al. 2006]
extends the concepts laid out in relief mapping by adding multiple
height-field data into the problem, creating distinct volumes.
The authors present an efficient method for determining whether
the view ray passes through one of these volumes.
Practical Dynamic Parallax occlusion mapping [Tatarchuk. 2006]
offers an alternative per-pixel displacement mapping solution. Similar
to relief mapping, a linear search is performed. Then a single
iteration of the secant method is used to fit a series of discrete
linear functions to the curve. This achieves a high rendering
speed by eliminating the need for branch dependent texture
lookups. Whereas the single iteration of the secant method does not
achieve the same level of accuracy as a true root finding method. in
practice there is very little rendering error, making this technique
suitable for real-time applications.
In True Impostors [Risser. 2007] a single texture can hold four
height-fields which can represent many volumes. It is possible to
add more texture data and extend this to any shape. This is illustrated
in Figure 2 where the image of a fish is defined by multiple
height-fields stored in a four-channel texture. This texture is shown
in image c where the red and blue color channels are each used to
store a height-field. Since the fish is a simple shape, it can be defined
by only two height-fields therefore the remaining two channels
are set to 1 as default. Images a and b show the 3D representation
of the texture function from image c projected into texture
coordinate space as a volume. More specifically, image a shows
that our algorithm transforms a billboard’s texture coordinates into
a 3D plane and rotates this plane around the texture volume. Image
b shows the perspective of the texture volume which would be
displayed on the billboard.
Figure 1: Visual representation of a quad’s texture coordinates
transformed into a 3D plane and rotated around the origin (a) the
corresponding image it produces (b) and the texture used to define
the fish shape (c).
Traditionally, in previous per-pixel displacement mapping techniques
such as Relief Mapping of Non Height Field Surfaces [Policarpo
et al. 2006], these height-fields would represent the surface
of a polygon, and the viewing vector would be transformed into the
polygon’s local coordinate frame with respect to its normal. However,
in this case the surface geometry and texture coordinates are
transformed with respect to the view direction; in other words, a
billboard is generated which can then be used as a window to view
the height fields from any given direction.
2.1 Per-Pixel Displacement Mapping Review
All of the methods presented as background, as well as True Impostors,
can be classified under per-pixel displacement mapping. It
is crucial that the reader have a solid grasp of this type of problem.
By its definition, the goal of per-pixel displacement mapping is to
displace or move a pixel from one point on the screen to another in
much the same way traditional vertex based displacement mapping

displaces the position of a vertex. This is a simple concept which
becomes difficult to implement due to the nature of rasterization and
the design of modern GPUs which has no mechanism for writing a
target pixels color value to a different pixel on the screen. Thus
a wealth of techniques have been proposed to solve this problem.
Generally, when defining the color for a pixel two factors must be
taken into account: the color of the material of the surface at that
pixel and the amount of light being reflected towards the viewer.
Texture maps are the data structure that tends to store all of this
information, either as color or normal values. Therefore, a practical
version of our earlier problem becomes how we can, for our current
pixel, find new texture coordinates that correspond to the point the
target pixel is actually representing.
To solve this problem, the view vector needs to be known from the
camera to each pixel and transformed to texture space using the
polygons normal and tangent as well as the true topology of the
object that is being drawn. The topology can take on many forms
stored in many ways. For simplicity, a single height-field texture is
used to define the true shape of the surface in figure 1.
Figure 2: View ray penetrating Geometry and corresponding
height-field.
Because the view ray projects itself as a line across the surface of
our polygon, it effectively takes a 1D slice out of the 2D function,
clarifying our problem to its true form. We must find the first intersection
of a ray with an arbitrary function. Once this intersection
has been found, the corresponding texture coordinates can be used
to find the desired color for the pixel.
3 Method
Multiple options exist with regard to storing the volume to be rendered.
A four-channel texture is the data structure used in this
technique; therefore, the model is rendered to a texture as a preprocessing
step. Although the texture can be organized in many
possible ways which might lend themselves to specific shapes, two
general schemes are presented in this paper which offer a practical
solution for storing most shapes with the least amount of texture
data needed.
The first, planar mapping, is effective for many shapes. As the name
suggests, the model is reduced to a set of height-maps stacked over
each other on the same plane (shown in Figure 1 c), where the two
height-maps are stored in the red and green color channels of a
single texture while the remaining two channels are set to one by
default. This method for representing volumes was first introduced
in Relief Mapping of Non-Linear Height-Fields [Policarpo et al.
2006] and used again in True Impostors [Risser. 2007].
The second method uses a cube map to produce a spherical height
field around an origin point. By rendering an environment map of
depth values from the center of a model to its surface, a volume is
defined. Multiple surfaces are stored in the multiple channels of the
texture map; figure 3 illustrates this process.
Figure 3: The camera is placed at the origin of the model. For each
of the six faces, a height map is rendered which stores the distance
from the origin to the surface of the mesh.
As opposed to an impostor which relies on the use of billboards, this
technique utilizes a box as the primary rendering surface offering
several advantages over the previous technique. Because a billboard
is a flat plane which rotates around its center, volumes can only be
viewed from within when the camera is looking towards the center
of the billboard. This deficiency leaves impostor methods ill-suited
for rendering volumetric data such as clouds and other participating
media. By using a box as the drawing surface and rendering only
the back-faces, no matter where the camera is within or without the
object, geometry will be in view for rendering, as shown in Figure
4.
Figure 4: Box geometry completely bounds the asteroid.
Performance concerns also favor the use of a box instead of a billboard.
Note that a billboard consists of two polygons whereas a box
consists of twelve. Although more geometry is processed when using
a box, the box can tightly bound the volume on three axes. This
is preferable, as the rendering algorithm is fill rate dependent and
the reduction in empty pixels is greatly reduced. Figure 5 illustrates
this by showing side by side a fish model bound by both a
box as discussed in this paper, and a billboard as recommended by

True Impostors [Risser. 2007]. It is clear that fewer empty pixels
are drawn using the box method. This is true for any object other
than a sphere, and the benefits of using a box become greater as the
target object becomes elongated in any given direction.
Figure 5: (left) fish model drawn on a box, (right) same fish model
drawn on a billboard.
Now that the benefits behind the box method have been discussed,
an efficient algorithm for rendering volumetric impostors within a
box will be given.
The bulk of this technique operates solely in the pixel pipeline of
the GPU. Once the model has been stored as a texture and a box
mesh has been set up, the vertex processor begins the rendering
process by transforming the vertices positions into world space and
creating for each vertex a view ray which passes from the camera
in world space to each vertex, as shown in Figure 6. Each vertex
also contains an origin point which is best described as a second
(x,y,z) position value which is local to the box itself where each
value takes on either a 1 or a -1 and therefore centers on the origin.
Figure 6: View rays are shown pointing away from the camera;
values are given for each vertex’s origin point.
Rasterization interpolates the view ray and origin point across each
pixel comprising the box. Recall that the back faces of the box
are the rendering surface. Thus, the view ray must be negated so
that it points inwards toward the volume. In most cases, the first
intersection with the volume needs to be found, not the last. Thus,
it is necessary to displace the origin point from the back of the box
to the front along the view ray. This is done in the pixel shader.
Figure 7: Efficient method for finding the intersection with a point
on a box.
Given the origin point (x, y) and the view vector (dx, dy) in this
two dimensional example, the method for finding the closest point
of intersection inside a box is to treat each axis independent of the
other and find the displacement along that axis from the origin point
to the bounding planes set to 1 and -1. Keep in mind that the view
vector is a normalized vector; so by dividing the displacement along
an axis by the same axial component of the vector, a scalar will be
produced which will stretch the view vector to the point of intersection,
this equation is shown in Figure 7. Also note that since texture
space is a cube, whatever scaling is multiplied to the box in world
space, the same scaling must be divided from the view vector in
texture space. Keep in mind that intersections occurring in the opposite
direction from where the view vector is pointing will result in
a negative scalar, thus the correct scalar is the maximum of the two
values. Once each axis has been tested independently, it is a simple
matter to find the minimum of the multiple values, being the other
intersection lying on the surface of the box. The new point can be
found by multiplying the scalar with the view vector and adding the
result back into the origin point. Once the correct origin point is
found, negate the view vector once again so that it is pointing away
from the camera and back towards the volume.
At this point, world space and the box mesh are no longer concerns;
the problem has been reduced to finding the point of intersection between
a ray and a volume, the volume being stored in a texture and
the ray being defined by the view vector and origin point. Various
ray-casting and ray-tracing techniques have been developed to solve
this problem in multiple ways for numerous material properties.
For opaque objects, ray-casting is the most efficient method for
quickly finding the point of intersection. The archetype technique
for this case was introduced in Relief Mapping of Non-Height-Field
Surface Details [Policarpo et al. 2006]. Finding the closest point of
intersection involves a linear followed by binary search, as shown
in Figure 8.
In figure 8, the ray march shows that points along the ray are sampled
at regular intervals. Once a sampled point that lays within the
volume is found, as shown in image a, a binary search repeatedly
halves the remaining interval until the exact point of collision is
found. This is shown in images b-d. Keep in mind that points lying

Figure 8: Streamlined ray-casting method.
on the vector will still take on values within the 1 to -1 range. For
each point tested, it is necessary to do a texture lookup to find the
height-field values. In the case of planar mapping, the values range
from -1 to 1; adding 1 and dividing by 2 will transform all three
axis values so that they lay between 0 and 1, corresponding to the
texture’s UV space. In the case of cube mapping, a vector from the
origin to the test point is used as a lookup into the cube-map.
A limited yet effective animation scheme is offered in which the
animation frames are tiled on a single texture, offering quick lookups
using the pixel shader.
Figure 9: Animated texture map with cooresponding models.
Rather than using the entire texture to store a single model, the image
is partitioned into discrete equally sized regions each storing a
frame of the desired animation, as seen in figure 9. Planar mapping
is shown in this example but any volume representation could be
accommodated. The animation can be looped through by passing
a global time variable into the shader and using it to select the current
target region. This would lend itself nicely for rendering large
herds of animals, schools of fish, and flocks of birds.
4 Results and Analysis
Nearly correct reproductions of geometry are rendered to the inside
surface of a box for any possible viewing direction. This is shown
in figure 10, where multiple dogs are displayed from arbitrary directions.
The use of a box as the rendering surface is favorable in
this instance as the shape of the dog can be tightly bound by the
shape of the box, resulting in both the reduction in empty pixels on
screen at any given time and the number of linear search steps that
need to be tested.
This method performs similarly to other per-pixel displacement
mapping techniques as the same ray-volume intersection algorithms
are shared among them. The main concern unique to this specific
rendition involves excessive overdraw. Performance is fill-rate dependent;
so when objects occlude other objects there is the potential
for the same pixel to be processed multiple times. Every precaution
is taken to avoid this situation. The CPU sorts objects front to
back so that filled pixels will be safe from overdraw. Empty pixels
are still at risk of being processed multiple times as parts of other
boxes. The use of the box limits the number of empty pixels drawn
for each object and keeps this shortcoming in check.
Using this method, it is possible to represent more geometry on
screen at once than could be achieved using standard polygonal
means. an example is shown on the first page and in figure 11,
where a model of Jupiter is rendered with a quarter of a million asteroids
forming the ring. The method discussed in this paper was
implemented in C++ using Direct X 9. As stated previously, performance
is fill-rate dependent. Therefore, performance is almost
entirely determined by how many pixels on screen are being drawn
as well as how much overdraw is occurring. Due to the nature of
this bottleneck, frame-rate measurements are somewhat superficial
and can drastically change with the view direction. However, given
ten linear search steps followed by eight binary search steps and
an object which fills up a 640 by 480 window, 180-190 fps was
achieved. In the Jupiter demo, which rendered a quarter of a million
asteroids in a 1024 by 768 window, 30 fps was achieved on
average. Both tests used a GeForce 7800 GT for benchmarking.
5 Discussion
This paper offers an effective and practical technique for generating
staggering amounts of faux-geometry to any rendered environment.
Impostor models are drawn with very little visual error in
comparison with their geometrical counterparts while offering perpixel
level of detail. A multitude of different rendering methods are
possible, offering techniques for animated opaque, reflective and
refractive objects, as well as the ground work for more custom and
potentially sophisticated volumetric rendering techniques such as
participating media.
This paper builds upon the concepts laid out in Relief Mapping of
Non-Height-Field Surface Details [Policarpo et al. 2006] and Parallax
Occlusion Mapping [Tatarchuk. 2006]. It extends upon True
Impostors [Risser. 2007] by offering a more efficient, streamlined
as well as robust rendering model. Various schemes are offered
as means to store and retrieve model data in the form of textures

which can be efficiently accessed by the GPU. This method adopts
advantages of both rasterization and ray-tracing, using a hybrid approach
which takes the best each has to offer. This method is ideal
for use in video games as it improves upon the already widely used
imposter concept.
References
BLINN, J. F. 1978. Simulation of wrinkled surfaces. In SIGGRAPH
’78: Proceedings of the 5th annual conference on Computer
graphics and interactive techniques, ACM Press, New York, NY,
USA, 286–292.
CHEN, Y., TONG, X., WANG, J., LIN, S., GUO, B., AND SHUM,
H. 2004. Shell texture functions. In Transaction of Grpahics -
Proceedings of SIGGRAPH 2004 23. 343-352.
COOK, R. L. 1984. Shade trees. In SIGGRAPH ’84: Proceedings
of the 11th annual conference on Computer graphics and interactive
techniques, ACM Press, New York, NY, USA, 223–231.
HART, J. C. 1996. Sphere tracing: A geometric method for the
antialiased ray tracing of implicit surfaces. The Visual Computer
12, 10, 527–545.
HIRCHE, J., EHLERT, A., GUTHE, S., AND DOGGETT, M. 2004.
Hardware accelerated per-pixel displacement mapping. In GI
’04: Proceedings of the 2004 conference on Graphics interface,
Canadian Human-Computer Communications Society, School of
Computer Science, University of Waterloo, Waterloo, Ontario,
Canada, 153–158.
KANEKO, T., TAKAHEI, T., INAMI, M., KAWAKAMI, N.,
YANAGIDA, Y., AND MAEDA, T. 2001. Detailed shape representation
with parallax mapping. In Proceedings of the ICAT
2001, 205–208.
KAUTZ, J., AND SEIDEL, H.-P. 2001. Hardware accelerated displacement
mapping for image based rendering. In GRIN’01: No
description on Graphics interface 2001, Canadian Information
Processing Society, Toronto, Ont., Canada, Canada, 61–70.
KOLB, A., AND REZK-SALAMA, C. 2005. Efficient empty space
skipping for per-pixel displacement mapping. In Proc. Vision,
Modeling and Visualization.
MACIEL, P.W.C., SHIRLEY, P. 1995. Visual navigation fo large
environments using textured clusters. In Proceedings of the 1995
symposium on Interactive 3D graphics. 95-102.
MAX, N. 1998. Horizon mapping: shadows for bump-mapped
surfaces. In The Visual Computer 4, 2, 109–117.
MCGUIRE, M. 2005. Steep parallax mapping. In I3D 2005 Poster.
OLIVEIRA, M. M., BISHOP, G., AND MCALLISTER, D. 2000.
Relief texture mapping. In SIGGRAPH ’00: Proceedings of the
27th annual conference on Computer graphics and interactive
techniques, ACM Press/Addison-Wesley Publishing Co., New
York, NY, USA, 359–368.
PATTERSON, J. W., HOGGAR, S. G., AND LOGIE, J. R. 1991.
Inverse displacement mapping. Comput. Graph. Forum 10, 2,
129–139.
POLICARPO, F., OLIVEIRA, M. M., AND COMBA, J. L. D. 2005.
Real-time relief mapping on arbitrary polygonal surfaces. In
SI3D ’05: Proceedings of the 2005 symposium on Interactive
3D graphics and games, ACM Press, New York, NY, USA, 155–
162.
POLICARPO, F., OLIVEIRA, M. M. 2006. Relief Mapping of Non-
Height-Field Surface Details. In SI3D ’06:ACM SIGGRAPH
2006 Symposium on Interactive 3D Graphics and Games, Redwood
City, CA, USA, 55–62.
PRESS, W., FLANNERY, B., TEUKOLSKY, S., AND VETTERLING,
W. 2002. Root finding and non-linear sets of equations. In
Numerical Recipes in C, 354–360.
SCHAUFLER, G., AND PRIGLINGER, M. 1999. Horizon mapping:
shadows for bump-mapped surfaces. In Efficient displacement
mapping by image warping, 175–186.
RISSER, E. 2007. True Impostors. In GPU Gems 3.
SLOAN, P., AND COHEN, M., 2000. Interactive horizon mapping.
TATARCHUK. 2006. Dynamic parallax occlusion mapping with
approximate soft shadows. In SI3D ’06: Proceedings of the
2006 symposium on Interactive 3D graphics and games, Redwood
City, California, 63–69.
WALSH. 2003. Parallax mapping with offset limiting. In Infiniscape
Tech Report.
WANG, L., WANG, X., TONG, X., LIN, S., HU, S., GUO, B.,
AND SHUM, H.-Y. 2003. View-dependent displacement mapping.
ACM Trans. Graph. 22, 3, 334–339.
WANG, X., TONG, X., LIN, S., HU, S., GUO, B., AND SHUM,
H.-Y. 2003. Generalized displacement maps. In Eurographics
Symposium on Rendering, 227–233.

Figure 10: The same box is shown from multiple arbitrary viewing directions.
Figure 11: Various images taken of a real time demo of the technique generating a quarter of a million asteroids.