8 Steps of Successful Data Migration

Data migration is the process of transferring data between storage types, formats, or computer systems. But here we are specifically talking about database (i.e. Sybase, MySQL, DB2, SQL Server and Oracle) migration. Data migration is usually performed programmatically to achieve an automated migration.

Most of the times when we change the software system and implement a new database or change the database vendor then we will need to migrate the data from an existing database to a new database in order to continue our existing business. There is no doubt that we cannot ignore the existing data because of the historical data is really useful for our existing business or we cannot run the business without that data.

Data Migration is not easy as we may think. It is a complex process. Let us understand few real- time challenges that comes during the any data migration project.

• Database Structure/Functionality of Business Entity can be totally different in new system
• Data Field Length/type can be change that may create data integrity issues
• Size of Source Database
• Resource Involved in current software system
• Any type of conflict with production system
• Performance Issues
• Many More …

Now let‘s go through the 8 steps that we should consider in data migration projects.

Planning:  First step is really important because we have to be very clear about the data migration approach before starting any execution. First of all we have to make the strategy for data migration process, design, scope, timeline, resources, any utility tool require and detailed execution plan.

Analysis:  Analyze the existing historical data (i.e. source database), validate the business requirement and create a mapping. All analysis should be documented properly because it will be very useful during the execution plan.

ETL Process (Extract, Transform and Load): Transfer the data from existing database to intermediate database for data cleaning process.

Develop Migration Scripts:  Develop the Data Cleaning and Data Migration Scripts (like Package, Procedure and Functions etc.)

Data Cleaning: Clean the data in the intermediate database so that it will be ready for destination database.

Pre/Test Migration:  Do dress rehearsals for each planned release. Pre/Test Migration is required to verify migration procedures.

Business-User Testing of Migration: Complete data migration at test environment so that business users can do data validation, records count validation etc.

Go-Live Migration:  Final step or phase data migration at the Production environment.