Illustrative Rendering in

IllustrativeRendering inTeam Fortress 2Jason MitchellMoby FranckeDhabih Eng© 2007 Valve Corporation. All Rights Reserved.

Outline• Motivations and related work• Environments• Characters and interactive shading• Future work© 2007 Valve Corporation. All Rights Reserved.

Team Fortress 2• Class-based multiplayercombat game which will bereleased this fall• Unique visual style• Differentiation - multiplayercombat games tend toembrace a contemporaryphotorealistic look• Gameplay -Team Fortresshas always featuredcartoonish, over-the-topsituations• Readability - Classdifferentiation is the core ofTeam Fortress 2, hence weneeded to be able to clearlydifferentiate classes visually© 2007 Valve Corporation. All Rights Reserved.

Environment Design Principles• Value contrast• Simple forms• No unnecessarily off-kilter shapes• Minimize visual noise• Texture and geometric• Minimize repetition© 2007 Valve Corporation. All Rights Reserved.

Contrasting Team Properties• Red• Warm colors• Natural materials• Angular geometry• Blue• Cool colors• Industrial materials• Orthogonal forms© 2007 Valve Corporation. All Rights Reserved.

Blue base in 2fort map© 2007 Valve Corporation. All Rights Reserved.

Red base in 2fort map© 2007 Valve Corporation. All Rights Reserved.

World Rendering• Photorealistic techniques from our other games• Radiosity-generated light maps• Special effects such as reflection and refraction• Hand-painted textures with minimal noise,applied directly to 3D geometry• Loose details with visible brush strokes• Inherent solidity and frame-to-frame coherence• Hold up under magnification better thanphotoreference• Brush strokes appear in perspective, not in the2D image plane [Miyazaki02]• High frequency detail in photorealistic gamescan overpower designColor Palette© 2007 Valve Corporation. All Rights Reserved.

Miyazaki – Brush Width Foreshortened• Can easily imagine a 3D cameramove between these 2D views ofthe same space© 2007 Valve Corporation. All Rights Reserved.Background plates fromSpirited Away

Neutral Entities• Variations in hue and saturation areused to differentiate neutral entitiesin the game world• A hue other than red or blue createsdisassociation from either team color• Increased saturation makes theseimportant entities stand out in thedesaturated environment• Equally beneficial or dangerous toeither team• Beneficial green / cyan health pickups• Dangerous yellow train yard gates© 2007 Valve Corporation. All Rights Reserved.

© 2007 Valve Corporation. All Rights Reserved.

Character Design Goals• Easily visible against environment• Characters must be readable quickly by other players• Communicate shape via shading and silhouetteunder all lighting conditions© 2007 Valve Corporation. All Rights Reserved.

Gooch, 1998• Hue and luminance shifts indicate surfaceorientation relative to light• Blend between warm and cool based uponunclamped Lambertian term, underlying albedoand some free parameters• Extreme lights and darks are reserved for edgelines and highlightsConventionalShadingGoochShading⎛⎜⎝1212( )⎞( )⎛( )⎞nˆ⋅lˆ+ ⎟ k + α k + ⎜1−⎜ nˆ⋅lˆ+ ⎟⎟( k + βk)⎠blued⎛⎝⎝121 ⎞2 ⎠⎠yellowd© 2007 Valve Corporation. All Rights Reserved.

Lake, 2000• Lake used a 1D texture lookupbased upon the Lambertian termto simulate the limited colorpalette cartoonists use forpainting cels• Also allows for the inclusion of aview-independent pseudospecular highlight by including asmall number of bright texels atthe “lit” end of the 1D texture mapN·L© 2007 Valve Corporation. All Rights Reserved.

Barla, 2006• Barla has extended this technique by using a 2D texture lookup toincorporate view-dependent and level-of-detail effects.• Fresnel-like creates a hard “virtual backlight” which is essentially arim-lighting term, though this term is not designed to correspond toany particular lighting environment.|N·V| r|N·V| rN·LN·L© 2007 Valve Corporation. All Rights Reserved.

Early 20th Century Commercial Illustration• Chose to adopt specific conventions of thecommercial illustrator J. C. Leyendecker:• Shading obeys a warm-to-cool hue shift. Shadows go tocool, not black• Saturation increases at the terminator with respect to a givenlight source. The terminator is often reddened.• On characters, interior details such as clothing folds arechosen to echo silhouette shapes• Silhouettes are often emphasized with rim highlights ratherthan dark outlines© 2007 Valve Corporation. All Rights Reserved.

Rim HighlightsRed TerminatorJ.C. LeyendeckerJ.C. LeyendeckerArrow collar advertisement, 1929Swimmin’ Hole, 1935© 2007 Valve Corporation. All Rights Reserved.

Clothing FoldsJ.C. LeyendeckerThanksgiving 1628-1928© 2007 Valve Corporation. All Rights Reserved.J.C. LeyendeckerTally-Ho, 1930

Engineer Concept© 2007 Valve Corporation. All Rights Reserved.

Rim Highlighting: Before© 2007 Valve Corporation. All Rights Reserved.

Rim Highlighting: After© 2007 Valve Corporation. All Rights Reserved.

• Players must be able to quickly identify other players by team, classand selected weapon at a variety of distances and viewpoints• We think of this in terms of a visual “read hierarchy”• Design Goals• Team – Friend or Foe?• Color• Class – Run or Attack?• Distinctive silhouettes• Body proportions• Weapons• Shoes, hats and clothing folds• Selected weapon – What’s he packin’?• Highest contrast at chest level, where weapon is held• Gradient from dark feet to light chestColor Palette© 2007 Valve Corporation. All Rights Reserved.

Character Lighting EquationView independentkd⎡⎢a⎣⎤⎠⎥⎦L( ) ( ( ) ) ⎜⎛ +γnˆc w α nˆ⋅lˆβ ⎟⎞ ++∑i=1i⎝iL∑i=1[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)issirView-dependentrirr© 2007 Valve Corporation. All Rights Reserved.

kdView Independent Terms⎡⎢a⎣L( ) ( ( ) )⎤⎜⎛ +γnˆc w α nˆ⋅lˆβ ⎟⎞⎥ ⎦+∑i=1• Spatially-varying directionalambienti⎝i⎠© 2007 Valve Corporation. All Rights Reserved.

kdView Independent Terms⎡⎢a⎣L( ) ( ( ) )⎤⎜⎛ +γnˆc w α nˆ⋅lˆβ ⎟⎞⎥ ⎦+∑i=1• Spatially-varying directionalambient• Modified Lambertian termsi⎝i⎠© 2007 Valve Corporation. All Rights Reserved.

kdView Independent Terms⎡⎢a⎣L( ) ( ( ) )⎤⎜⎛ +γnˆc w α nˆ⋅lˆβ ⎟⎞⎥ ⎦+∑i=1• Spatially-varying directionalambient• Modified Lambertian terms• Unclamped Lambertian termi⎝i⎠© 2007 Valve Corporation. All Rights Reserved.

kdView Independent Terms⎡⎢a⎣L( ) ( ( ) )⎤⎜⎛ +γnˆc w α nˆ⋅lˆβ ⎟⎞⎥ ⎦+∑i=1• Spatially-varying directionalambient• Modified Lambertian terms• Unclamped Lambertian term• Scale, bias and exponenti⎝i⎠© 2007 Valve Corporation. All Rights Reserved.

View Independent Termskd⎡⎢a⎣L( ) ( ( ) )⎤⎜⎛ +γnˆc w α nˆ⋅lˆβ ⎟⎞⎥ ⎦+∑i=1• Spatially-varying directionalambient• Modified Lambertian terms• Unclamped Lambertian term• Scale, bias and exponent• Warping function• Albedo12i( nˆ⋅lˆ)+12⎝i⎠© 2007 Valve Corporation. All Rights Reserved.

kdView Independent Terms⎡⎢a⎣L( ) ( ( ) )⎤⎜⎛ +γnˆc w α nˆ⋅lˆβ ⎟⎞⎥ ⎦+∑i=1• Spatially-varying directionalambient• Modified Lambertian terms• Unclamped Lambertian term• Scale, bias and exponent• Warping functioni⎝i⎠© 2007 Valve Corporation. All Rights Reserved.

kdView Independent Terms⎡⎢a⎣L( ) ( ( ) )⎤⎜⎛ +γnˆc w α nˆ⋅lˆβ ⎟⎞⎥ ⎦+∑i=1• Spatially-varying directionalambient• Modified Lambertian terms• Unclamped Lambertian term• Scale, bias and exponent• Warping function• Albedoi⎝i⎠© 2007 Valve Corporation. All Rights Reserved.

kdView Independent Terms⎡⎢a⎣L( ) ( ( ) )⎤⎜⎛ +γnˆc w α nˆ⋅lˆβ ⎟⎞⎥ ⎦+∑i=1• Spatially-varying directionalambient• Modified Lambertian terms• Unclamped Lambertian term• Scale, bias and exponent• Warping function• Albedoi⎝i⎠© 2007 Valve Corporation. All Rights Reserved.

Ambient Cube• Grounds characters in game worlds• Pre-compute irradiance samplesthroughout the environment• Variable density irradiance volume[Greger98] where each sampledefines an irradiance environmentmap [Ramamoorthi01]• Directional ambient term whichincludes only indirect light• Lights beyond the first four can beadded to the ambient cube• Used in a novel way in rim lighting,which we’ll discuss in a momentDirectional AmbientConstant Ambient© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)issirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)is• Multiple Phong terms per lightsirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)is• Multiple Phong terms per light• k rimbroad, constant exponentsirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)is• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)sirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)is• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel termsirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)is• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4sirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)is• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4• k rrim mask texturesirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)is• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4• k rrim mask texture• k sspecular mask texturesirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)is• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4• k rrim mask texture• k sspecular mask texturesirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)is• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4• k rrim mask texture• k sspecular mask texturesirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)is• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4• k rrim mask texture• k sspecular mask texturesirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)iss• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4• k rrim mask texture• k sspecular mask texture• Dedicated rim lighting• a(v) Directional ambient evaluated with virrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)iss• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4• k rrim mask texture• k sspecular mask texture• Dedicated rim lighting• a(v) Directional ambient evaluated with v• k rsame rim maskirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)iss• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4• k rrim mask texture• k sspecular mask texture• Dedicated rim lighting• a(v) Directional ambient evaluated with v• k rsame rim mask• f rsame rim Fresnelirrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)iss• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4• k rrim mask texture• k sspecular mask texture• Dedicated rim lighting• a(v) Directional ambient evaluated with v• k rsame rim mask• f rsame rim Fresnel• n·u term that makes rim highlights tend tocome from above (u is up vector)irrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)iss• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4• k rrim mask texture• k sspecular mask texture• Dedicated rim lighting• a(v) Directional ambient evaluated with v• k rsame rim mask• f rsame rim Fresnel• n·u term that makes rim highlights tend tocome from above (u is up vector)irrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)iss• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4• k rrim mask texture• k sspecular mask texture• Dedicated rim lighting• a(v) Directional ambient evaluated with v• k rsame rim mask• f rsame rim Fresnel• n·u term that makes rim highlights tend tocome from above (u is up vector)irrirr© 2007 Valve Corporation. All Rights Reserved.

L∑i=1View-dependent Terms[ c k max( f ( vˆ⋅ rˆ) kf k ( v r ) )] spec, ˆ ⋅ ˆk rim+ ( nˆ⋅uˆ) f k a( vˆ)iss• Multiple Phong terms per light• k rimbroad, constant exponent• k specexponent (constant or texture)• f sartist tuned Fresnel term• f rrim Fresnel term, (1-(n·v)) 4• k rrim mask texture• k sspecular mask texture• Dedicated rim lighting• a(v) Directional ambient evaluated with v• k rsame rim mask• f rsame rim Fresnel• n·u term that makes rim highlights tend tocome from above (u is up vector)irrirr© 2007 Valve Corporation. All Rights Reserved.

Future Work• More flexible specular• Anisotropic highlights [Heidrich98][Gooch98]© 2007 Valve Corporation. All Rights Reserved.[Heidrich98]

Future Work• More flexible specular• Anisotropic highlights [Heidrich98][Gooch98]© 2007 Valve Corporation. All Rights Reserved.[Gooch98]

Future Work• More flexible specular• Anisotropic highlights [Heidrich98][Gooch98]• Shaping highlights [Anjyo03][Anjyo03]© 2007 Valve Corporation. All Rights Reserved.

Future Work• More flexible specular• Anisotropic highlights [Heidrich98][Gooch98]• Shaping highlights [Anjyo03]• More reliable rim term© 2007 Valve Corporation. All Rights Reserved.

Future Work• More flexible specular• Anisotropic highlights [Heidrich98][Gooch98]• Shaping highlights [Anjyo03]• More reliable rim term• Image-space contrastenhancement [Luft06]© 2007 Valve Corporation. All Rights Reserved.[Luft06]

Future Work• More flexible specular• Anisotropic highlights [Heidrich98][Gooch98]• Shaping highlights [Anjyo03]• More reliable rim term• Image-space contrastenhancement [Luft06]• Abstracted shadows [DeCoro07][DeCoro07]© 2007 Valve Corporation. All Rights Reserved.

Conclusion• Motivations and related work• Environments• Characters and interactiveshading• Future work© 2007 Valve Corporation. All Rights Reserved.