Efficient Reliable Database Migration: A Legacy Innovation Story • Dave Thomas • GOTO 2024

This presentation was recorded at GOTO Chicago 2024. #GOTOcon #GOTOchgo
gotochgo.com/

Dave Thomas - Founder, Keynote Speaker & Software Industry Expert

RESOURCES
www.davethomas.net/
twitter.com/daveathomas
linkedin.com/in/davidathomas

ABSTRACT
Data migrations often fail due to underestimated complexity. This talk explores a critical, large-scale migration at a major ERP company that demanded high performance, reliability, and aggressive timelines.

We'll describe novel techniques including efficient bulk data transfer, table-driven approaches, functional prototyping, and a unique application of IV&V for robust testing. Learn how algorithms developed in distributed systems research solved real-world problems in legacy system modernization.

Discover strategies to implement reliable migration techniques and meet tight deadlines. This talk offers valuable insights for data engineers, architects, and development leaders facing migrations of any scale. [...]

TIMECODES
00:00 Intro
01:53 Reality of data migration
02:21 Case study: Critical legacy problem
05:38 Reporting problem
06:42 Reporting solution
07:45 Near-term solution
08:22 Long-term migration challenge
09:37 Conventional migration approach
10:52 Innovative solution
12:15 Logical & physical model
13:34 Essence of approach
15:37 Binary is better
16:09 Batch is better
16:43 Efficient serialization
17:49 Conservative implementation strategy to reduce risk
20:28 Verification & validation approach
29:53 Readers & writers?
32:53 Recap
37:39 Outro

Download slides and read the full abstract here:
gotochgo.com/2024/sessions/3362

RECOMMENDED BOOKS
Kevlin Henney • 97 Things Every Programmer Should Know • amzn.to/2Yahf9U
Henney & Monson-Haefel • 97 Things Every Software Architect Should Know • amzn.to/3pZuHsQ
Matthew Skelton & Manuel Pais • Team Topologies • amzn.to/3sVLyLQ
Michael Jackson • Software Requirements and Specifications • amzn.to/3ql2T14
Geoffrey West • Scale • amzn.to/3eKMbpc
Donald G. Reinertsen • The Principles of Product Development Flow • amzn.to/3hJ2Ye

bsky.app/profile/gotocon.com
twitter.com/GOTOcon
www.linkedin.com/company/goto-
www.instagram.com/goto_con
www.facebook.com/GOTOConferences
#DatabaseMigration #DataMigration #Legacy #TechnicalLeadership #DeveloperProductivity #Productivity #SoftwareEngineering #Programming #Teams #DaveThomas

CHANNEL MEMBERSHIP BONUS
Join this channel to get early access to videos & other perks:
youtube.com/channel/UCs_tLP3AiwYKwdUHpltJPuA/join

Looking for a unique learning experience?
Attend the next GOTO conference near you! Get your ticket at gotopia.tech/
Sign up for updates and specials at gotopia.tech/newsletter

SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.
youtube.com/user/GotoConferences/?sub_confirmation…