HBASE SCHEMA DESIGN - ApacheCon

More documents

Recommendations

Info

MemStores • After data is written to the WAL the RegionServer saves KeyValues in memory store • Flush to disk based on size, see hbase.hregion.memstore.flush.size • Default size is 64MB • Uses snapshot mechanism to write flush to disk while still serving from it and accepting new data at the same time • Snapshots are released when flush has succeeded
Compactions • General Concepts • Two types: Minor and Major Compactions • Asynchronous and transparent to client • Manage file bloat from MemStore flushes • Minor Compactions • Combine last “few” flushes • Triggered by number of storage files • Major Compactions • Rewrite all storage files • Drop deleted data and those values exceeding TTL and/or number of versions • Triggered by time threshold • Cannot be scheduled automatically starting at a specific time (bummer!) • May (most definitely) tax overall HDFS IO performance Tip: Disable major compactions and schedule to run manually (e.g. cron) at off-peak times
Page 1 and 2: HBASE SCHEMA DESIGN and Cluster Siz
Page 3 and 4: Agenda • HBase Architecture • S
Page 5 and 6: HBase Tables
Page 7 and 8: HBase Tables
Page 9 and 10: Distribution
Page 11 and 12: HBase Architecture
Page 13: HBase Architecture (cont.) • Base
Page 17 and 18: Region Splits • Triggered by conf
Page 19 and 20: Auto Sharding and Distribution •
Page 21 and 22: Storage Separation • Column Famil
Page 23 and 24: SCHEMA DESIGN
Page 25 and 26: Key Cardinality • The best perfor
Page 27 and 28: Fold, Store, and Shift • Logical
Page 29 and 30: DDI • Stands for Denormalization,
Page 31 and 32: Tall-Narrow vs. Flat-Wide Tables
Page 33 and 34: Partial Key Scans Key - -- --- De
Page 35 and 36: Salting • Prefix row keys to gain
Page 37 and 38: Key Design
Page 39 and 40: Example: Facebook Insights • > 20
Page 41 and 42: Key Design • Reversed Domains •
Page 43 and 44: Summary • Design for Use-Case •
Page 45 and 46: CLUSTER SIZING
Page 47 and 48: Memory Sharing • By default every
Page 49 and 50: Block Cache • Use exported metric
Page 51 and 52: Write Performance • There are man
Page 53 and 54: Write-Ahead Log (cont.) • Size se
Page 55 and 56: Flushes • Every mutation call (pu
Page 57 and 58: Dependencies • Flushes happen acr
Page 59 and 60: Some More Numbers • Under real wo
Page 61 and 62: Cheat Sheet #1 • Ensure you have
Page 63 and 64: Example • Java Xmx heap at 10GB

HBASE SCHEMA DESIGN - ApacheCon

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?