Migrating Uber from MySQL to PostgreSQL

Recommendations

Info

Downtime Plan The Actual Migration The actual real-time part of tracking cars and users is a Node.js thing that keeps state in memory and persists to Redis. We hacked up the Node.js thing to allow queueing trip data, to allow users/drivers to make trips while the Python API/database is unavailable. During “downtime”, most things work except signing up and actually charging credit cards. Evan Klitzke (Uber, Inc.) Migrating Uber from MySQL to PostgreSQL March 13, 2013 54 / 59
First Attempt The Actual Migration First conversion attempt was early February. Conversion takes about two hours. We turn on read-only queries against Postgres, and get a constant stream of errors like: OperationalError: server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. The errors were happening only on the master, not on slaves. After several hours of debugging, we ultimately and tragically had to roll back to MySQL before enabling writes. Much cursing was done. Evan Klitzke (Uber, Inc.) Migrating Uber from MySQL to PostgreSQL March 13, 2013 55 / 59
Page 1 and 2:
Migrating Uber from MySQL to Postgr
Page 3 and 4: Background Background Uber is a mob
Page 5 and 6: The Objective Background The object
Page 7 and 8: Obstacles cont. Background Addition
Page 9 and 10: Plan of Attack Background To do a m
Page 11 and 12: Background “Replication” Plan,
Page 13 and 14: Background Serialization Problem Th
Page 15 and 16: Conversion Background OK, how much
Page 17 and 18: Conversion Background It turns out
Page 19 and 20: Background MySQLisms (quoting) Quot
Page 21 and 22: Attempt 1 Background My first attem
Page 23 and 24: XML Attempt Background Next I disco
Page 25 and 26: XML Attempt Background New plan of
Page 27 and 28: XML Attempt Background Initial test
Page 29 and 30: XML Attempt Background It turns out
Page 31 and 32: xml2pgcopy Background The final imp
Page 33 and 34: Code Changes Code Changes The other
Page 35 and 36: Boolean Errors Code Changes As note
Page 37 and 38: Timezone Issues Code Changes Produc
Page 39 and 40: ENUMs Code Changes In MySQL, ENUMs
Page 41 and 42: Code Changes Implicit Ordering (con
Page 47 and 48: Code Changes GROUP BY (cont.) MySQL
Page 49 and 50: Case Sensitivity Code Changes Case
Page 51 and 52: Code Changes SQLAlchemy autoflush A
Page 53: The Actual Migration Production Set
Page 57 and 58: Interim The Actual Migration In ret
Page 59: The Actual Migration xml2pgcopy, re
show all

Migrating Uber from MySQL to PostgreSQL

Create successful ePaper yourself

Delete template?

Save as template?