Migrating Uber from MySQL to PostgreSQL

Recommendations

Info

Background Background (cont.) All of the parts of Uber that touch the database are written in Python. Most of the database interaction is through SQLAlchemy, with a handful of hand-written queries (i.e. cursor.execute()). Also, some business intelligence/MBA types issuing ad hoc queries directly to read slaves using an OS X MySQL GUI tool (Sequel Pro). Evan Klitzke (Uber, Inc.) Migrating Uber from MySQL to PostgreSQL March 13, 2013 4 / 59
The Objective Background The objective was to move our ∼50GB MySQL database to PostgreSQL. Why? A bunch of reasons, but one of the most important was the availability of PostGIS. Evan Klitzke (Uber, Inc.) Migrating Uber from MySQL to PostgreSQL March 13, 2013 5 / 59
Page 1 and 2: Migrating Uber from MySQL to Postgr
Page 3: Background Background Uber is a mob
Page 7 and 8: Obstacles cont. Background Addition
Page 9 and 10: Plan of Attack Background To do a m
Page 11 and 12: Background “Replication” Plan,
Page 13 and 14: Background Serialization Problem Th
Page 15 and 16: Conversion Background OK, how much
Page 17 and 18: Conversion Background It turns out
Page 19 and 20: Background MySQLisms (quoting) Quot
Page 21 and 22: Attempt 1 Background My first attem
Page 23 and 24: XML Attempt Background Next I disco
Page 25 and 26: XML Attempt Background New plan of
Page 27 and 28: XML Attempt Background Initial test
Page 29 and 30: XML Attempt Background It turns out
Page 31 and 32: xml2pgcopy Background The final imp
Page 33 and 34: Code Changes Code Changes The other
Page 35 and 36: Boolean Errors Code Changes As note
Page 37 and 38: Timezone Issues Code Changes Produc
Page 39 and 40: ENUMs Code Changes In MySQL, ENUMs
Page 41 and 42: Code Changes Implicit Ordering (con
Page 47 and 48: Code Changes GROUP BY (cont.) MySQL
Page 49 and 50: Case Sensitivity Code Changes Case
Page 51 and 52: Code Changes SQLAlchemy autoflush A
Page 53 and 54: The Actual Migration Production Set
Page 55 and 56:
First Attempt The Actual Migration
Page 57 and 58:
Interim The Actual Migration In ret
Page 59:
The Actual Migration xml2pgcopy, re
show all

Migrating Uber from MySQL to PostgreSQL

Create successful ePaper yourself

Delete template?

Save as template?