Segmentation & Constraints

Siggraph Course
Mesh Parameterization –
Theory and Practice
Segmentation & Constraints

Segmentation
• Necessary for closed and high genus meshes
• Reduce parametric distortion
Chartification
Texture Atlas

Segmentation – Goals
Large Charts
Low Distortion

Segmentation – Single Chart
Gu et al. 2002 Sheffer & Hart 2002
• High genus surface → topological disk
• Improve the cuts

Segmentation – Multiple Charts
MCGIM
LSCM
Iso-charts
D-charts

Iso-charts
• Fast and effective atlas generator
– large charts with bounded parametric distortion
• UVAtlas in DirectX 9
– D3DXUVAtlasCreate, D3DXUVAtlasPartition,
D3DXUVAtlasPack
• Light map compression, Halo

Iso-charts – Demo

Iso-charts – Overview
• Surface spectral analysis
• Stretch optimization
• Surface spectral clustering
• Optimize chart boundary
• Recursively split charts
– until stretch criterion is met
• Atlas packing

Iso-charts – Spectral Analysis
x N
y N
x 2
y 2
d 1 N
d 2 N
d12
x 1
min
N
N
i1 j1
(||
y
i
d
Geodesic Distance Distortion (GDD)
y
j
||
y 1
ij
2
)

Iso-charts – Spectral Analysis
d 1 N
x 1
d 2 N
d12
x N
x 2
D
N
d
d
d
2
11
2
21
2
N1
d
d
d
2
12
2
22
2
N 2
d
d
d
2
1N
2
2N
2
NN
• Step 1: Construct the matrix of squared distance
– Surazhsky et al. 2005

Iso-charts – Spectral Analysis
x N
1
d 1 N
d 2 N
d12
x 2
B
N
where J
N
2
J
I
N
D
1
N
N
J
T
11
N
x 1
• Step 2: Perform centering and normalization

Iso-charts – Spectral Analysis
x N
d 1 N
d 2 N
d12
x 2
T
1 v1
| | |
T
2 v2
y1
y 2 y N
| | |
T
n vn
x 1
• Step 3: Perform eigen-analysis to get the embedding
coordinates

Iso-charts – Spectral Analysis
Zigelman et al. 2002
• Produces triangle flips
• Only handles single-chart models

Iso-charts – Stretch Optimization
Sander et al. 2001
nonlinear optimization
2D texture domain
singular values: γ , Γ
surface in 3D

Iso-charts – Stretch Optimization
[Sander01], L 2 = 1.04, 222s
[Sander02], L 2 = 1.03, 39s
Spectral, L 2 = 1.04, 2s
Spectral+Optimization, L 2 = 1.03, 6s

Iso-charts – Spectral Clustering
Analysis
Clustering

Iso-charts – Boundary Optimization
S
T
• Create nonjaggy cut, through “crease” edges
• Minimize embedding distortion

Iso-charts – Atlas Packing
• A Tetris algorithm [Lévy02]
V
U

Iso-charts – Acceleration
• Landmark algorithm [Silva et al. 2003]
– reduce the size of the eigenanalysis
• Only compute the top 10 eigenvalues
– over 95% of the squared energy

Constraints
• Enforce specific point-to-point correspondences
3D mesh 2D image constrained
texture mapping

Constrained Texture Mapping
• Soft constraints [Lévy 2001]
– approximate constraints
– generate foldovers for large constraint sets
• Hard constraints
– Lagrange multipliers [Desbrun et al. 2002]
– Matchmaker [Kraevoy et al. 2003]
– TextureMontage [Zhou et al. 2005]

Matchmaker – Key Idea
• Reduce constrained
parameterization problem to
unconstrained one by
– Split mesh into patches s.t. constrained
vertices V F are on patch boundaries
4
1 2
5
3
– Split 2D domain into matching convex
pieces s.t. matching positions P F are
on their boundaries
1
5
2
– Map each patch to corresponding
convex using barycentric embedding
4
3

Matchmaker – Algorithm
1. Input: 3D mesh + (V F , P F )
2. Compute unconstrained
2D embedding M
3. Virtual boundary:
1. Triangulate region between
M and its bounding
rectangle
2. M*= M+new triangles
mesh+V F
P F
3. Add boundary vertices to V F
M
M*

Matchmaker – Algorithm
4. Find matching triangulations
– Adding non-intersecting edges/paths
M* T M* (V F )
P F T(P F )

Matchmaker – Algorithm
• Path matching
Compute all shortest paths between vertices in V F
While paths exist
Get shortest path s=(v i ,v j )
v k
p k
Test if s is legal
If legal
(p i ,p j ) does not intersect existing edges in T(P F )
(v i ,v j ) does not block future paths
Add s to T M* (V F ) & (p i ,p j ) to T(P F )
Recompute non-tested paths that intersect s
T M* (V F ) T(P F )

Matchmaker – Algorithm
• Introducing Steiner vertices
– allow the creation of a valid path
between any pairs of features
• Perform Dijkstra searches on
both the mesh vertices and the
edge midpoints
T(P F )
– Schreiner et al. 2004
T M* (V F )

Matchmaker – Algorithm
5. Map each patch S i T M* (V F ) to matching triangle
T i T(P F ) using barycentric coordinates
T M* (V F )
T(P F )
Barycentric
mapping M’
Texture using M’

Matchmaker – Algorithm
6. Constrained smoothing M’
– Minimize metric distortion compared to
original 3D mesh
– Maintain validity
M’
Smoothed M’

Matchmaker – Results
+
=

TextureMontage
• Texturing arbitrary surface from multiple images
Stanford Bunny
6 photographs of another bunny

TextureMontage – Key Issues
• Texturing arbitrary surface from multiple images
– Partition the model and images automatically and
simultaneously
– Ensure continuity across patch boundaries with few
correspondences
– Fill in non-texture areas

TextureMontage – Algorithm
1. Preprocessing
– Adjust exposure differences and color balances
– Segment background regions (e.g. LazySnapping [Li04])
– Compute and store the distance to the nearest non-background
pixel for each pixel

TextureMontage – Algorithm
2. Partition mesh and images
– Add valid path-edge pairs (similar to Matchmaker)
– Prevent path-edge pairs from crossing background regions

TextureMontage – Algorithm
3. Generate progressive mesh
– Half-edge collapse simplification [Hoppe 96]
v j
v i
v j

TextureMontage – Algorithm
4. Create coarse texture map
– Map triangles of the base mesh to corresponding texture
triangles in images

TextureMontage – Algorithm
5. Coarse-to-fine map construction
– Insert deleted vertices back one by one
v j v j v i
– For inner vertices, minimize geometric stretch energy
[Sander et al. 2001]
– For boundary vertices, minimize texture mismatch

TextureMontage – Algorithm
• Texture coords optimization for boundary vertices
v m
p
v j
v j
v i
v k
v l
v m
p v k
p v i
v j
G A (p)
v l
v j
G B (p)
p
v i
v k
Image 1 Image 2

TextureMontage – Algorithm
• Texture coords optimization for boundary vertices
– Evaluate texture mismatch energy
E
tex
n
'
'
I1
( sk
) I2(
sk
) (1 )
G ( sk
) G ( s
1
2 k
)
k1
– Minimize both geometric stretch and texture mismatch
– Fix one border and optimize the other
E
E geo
( 1
)
E tex

TextureMontage – Algorithm
• Texture coords optimization for boundary vertices
Original
Model
Unconstrained
Optimization
Color Matching
Enforced
Color and
Gradient

TextureMontage – Algorithm
6. Surface texture inpainting
– Unassigned region due to lack of correspondences

TextureMontage – Algorithm
6. Surface texture inpainting
– Smoothly fill in hole regions based on Poisson equation
s.t. | *
f d f f |
f
:
unknown color values of
vertices in hole regions
[Polthier and Preuss 2000]
[Pérez et al. 2003]
f
*
d
:
:
:
color values of vertices around
hole regions
Laplace operator
a controlling scalar field

TextureMontage – Algorithm
6. Surface texture inpainting
(a) (b) (c)

TextureMontage – Algorithm
7. Atlas packing (UVAtlas)

TextureMontage – Demo

Siggraph Course
Mesh Parameterization –
Theory and Practice
Segmentation & Constraints