Data migration from old systems: a practical checklist
A checklist for moving data from spreadsheets or old software into a new system: what to keep, how to clean and map it, test runs, cutover and after.
By Sohail Hussain21 November 20265 min read
Migrate data from an old system in six stages: decide what to move and what to archive, inventory and export every source, clean and map each field, rehearse the import on a copy until counts and totals match, cut over with a short freeze and a rollback plan, then check again after go-live. Most of the effort sits in cleaning and mapping, not in the import itself. Migrations go wrong when that work is left to the final week.
Data migration is part of most projects in our client portals and internal tools service, and it is the part clients most often underestimate. It is one of the three things that make custom software go wrong in custom software vs off-the-shelf. The checklist below applies just as well to moving between two off-the-shelf products, such as one CRM and another.
What should you decide before moving any data?
- Scope. Which kinds of records move, and how far back. Open records and the history people actually look up, often two or three years, are usually enough.
- The archive. Everything older is kept as a complete, read-only export. Ask your accountant how long financial records must be kept, and your lawyer what privacy law in your markets, such as India's Digital Personal Data Protection Act or the GDPR, says about keeping personal data you no longer need.
- An owner for each data set, who answers questions and signs off the result. Developers cannot decide which of three "Sharma Traders" records is the real one.
- The cutover date, away from month-end, year-end and your busy season.
- The old system's future: read-only for how long, then switched off on what date.
How do you inventory and export the data?
- List every source. The old software, every spreadsheet, shared folders, email attachments, and the notebook one person keeps. For each, note the owner, the number of records, the format and how to get it out: a built-in export, an API, a database dump or a report saved to Excel.
- Export to CSV in UTF-8, so names in Hindi, Arabic or with accents survive. RFC 4180 describes the CSV format most tools expect, including how to quote commas and line breaks inside a field.
- Keep the original exports untouched. Date them, store them safely and work only on copies.
- Do not open exports in Excel and save them again. Excel drops leading zeros from codes and phone numbers, turns long numbers into scientific notation, and reformats dates. It also stores numbers to only 15 significant digits, per Microsoft's specifications, so a 16-digit account or card-style reference loses its last digit for good.
To look inside an export without Excel touching it, our CSV to JSON converter shows each row as named fields, and the JSON to CSV converter turns API output back into a sheet for review.
How do you clean and map the data?
Write a mapping table and agree it with each data owner. A few typical rows:
| Old field | New field | Transformation | If missing or invalid |
|---|---|---|---|
| Customer Name | Company name | Trim spaces; merge known duplicates | Reject the row for review |
| Phone | Contact phone | Remove spaces; add +91 to ten-digit Indian numbers | Leave empty |
| State | State code | Map names and misspellings to GST state codes | Flag for review |
| Status ("Done", "done", "Completed") | Order status | Map to a fixed list of values | Flag for review |
| Date ("12/03/24") | Order date | Parse as day/month/year, explicitly | Flag for review |
Then work through the usual problems:
- Duplicates. Agree the match rules (same GSTIN, same email, same phone) and which record wins when they disagree.
- Orphans. Orders whose customer no longer exists, invoices pointing at deleted items. Decide whether to fix, attach to a placeholder or leave behind.
- Free text. Columns where people typed anything. Map the common values and send the rest to the owner as a short list.
- Junk. Test records, abandoned drafts and "DO NOT USE" rows stay behind.
- Old IDs. Keep the old system's ID on every new record. It makes tracing, re-running and answering "where did this come from?" trivial.
How do you test a migration?
- Script it. The migration should be a program you can run again from scratch, not a sequence of manual imports. You will run it many times.
- Rehearse at least twice on a copy of the new system with a full, recent export.
- Reconcile counts and totals. Records per type should match, minus what you chose to leave. Money must match exactly: outstanding balances, stock value, open order totals.
- Spot-check with the people who know the data. Ask each owner to pick 20 to 30 records they know well and compare old and new side by side.
- Time the full run. It tells you how long the cutover freeze needs to be.
- Test real tasks on migrated data, not demo data. The odd records are where new systems break.
How do you plan the cutover?
- Freeze changes in the old system, or record the ones that happen during the window so they can be applied afterwards.
- Run the final export and the script, then reconcile exactly as in the rehearsals.
- Make a go or no-go decision against criteria agreed in advance, by a named person.
- Keep a rollback plan. If it is a no-go, the old system stays live and nothing has been deleted.
- Tell people what changes, when, and who to ask on the first day, and have someone available to answer.
What should you check after go-live?
- Reconcile again after a week, especially balances and stock, because real use exposes problems rehearsals miss.
- Collect "this looks wrong" reports in one place. Fix the record, and if the same issue appears twice, fix the script and the rule behind it.
- Keep the old system read-only for an agreed period, often three to six months, then archive a full export with a note describing its structure, and switch it off.
- Delete working copies that contain personal data once the archive is safe.
If the migration is part of replacing a spreadsheet, when to replace spreadsheets with a custom tool covers the steps before it; if it is a CRM move, HubSpot vs Zoho CRM vs Salesforce covers the choice. Either way, do the inventory first. A list of every source with its owner and record count is the best single input to an accurate quote.
Was this guide useful?