Over the years we have been involved in various Data Migration projects either as a stand alone exercise or as part of a larger piece of project work. Migration has often been treated as a Cinderella part of a larger project but in reality it is crucial to the success of any project. Also interface projects to transfer data between systems. The underlying principles are true in all projects and over the years success has come from the following:-

• Planning - detailed data mapping is so important and an understanding of all transformations required.
• Cleansing - within existing systems it is best to ensure the data's integrity prior to migration. Also to ensure that data is kept in a good state it is sometimes important to tighten up validation on the legacy systems.
• Change Control - if an existing system has to be maintained during the Migration project then it is important to ensure that the migration project is kept up to date with what is happening in the other systems.
• Testing - A large number of test migrations should be performed and validated. An automated migration procedure can be defined to ensure a reliable process.
• Validation - both the target system has to be tested with migrated data and construction of reports to ensure that all data is transferred.

Typical projects performed over the year include the following:-

• Livingston Rental 1999 - Oracle using automated unix scripts to transfer and transform data to a new database structure. With automated scripts tests were run at least 3 times each week to create test data for the main project.
• BUPA 2000 - Data transfers between a cqcs system and mainframe and also a Point CRM system.
• Hays Mail Services - various projects to transfer data to SAP out of a system (written by Peter Chadburn in cqcs) also data links to CRM Point system. Also data transfer and re-structures into Sales management system.
• Express Dairies - various database re-design projects to improve performance which required data migration. The result was a reduction in weekend batch processing from 36 to 20 hours.
• NetSuite projects - formatting of data given by clients to ensure that their existing data was loaded in the correct format. Also the development using ASP/VBScript sub system to process XML exports from NetSuite and also to create XML import files that can be imported into NetSuite.
• Public Guardianship Office - Data Cleansing of existing old database to ensure quality addresses using the Quick Address with programs written to re-format data.