CRM Data Migration: A Method

Abstract

A CRM migration moves records, associations and history from one system into another and ends when both sides reconcile against agreed numbers — not when the import completes. The distinction sounds pedantic and is instead the whole method: nearly every recurring migration failure, from duplicated deals to orphaned associations to a sales team quietly keeping the old system open, traces at least in part to some step of the movement being treated as the import step. This reference sets out a sequence — source audit, remediation in the source, target model design, identity design, mapping policy, rehearsed import, reconciliation, cutover — and argues for the order as much as the steps. The argument draws on two bodies of research that practitioner guidance rarely touches: the data-quality literature, which established that quality is fitness for a use rather than an intrinsic property of records (Wang and Strong 1996; Strong, Lee, and Wang 1997), and the legacy-systems literature, which framed migration as a risk-management exercise decades before SaaS made it look routine (Bisbal et al. 1999). HubSpot-specific mechanics — import file constraints, per-object deduplication semantics, the unique-value import key — are given exactly, with the vendor's documentation linked as the authority. The reference closes with the case against migrating everything, and with its own limits.

1. The Migration Problem, Stated Precisely

The word migration is used loosely enough in commercial settings that a reference must begin by fixing it. A migration is the movement of an operating record system — the entities, their relationships, and some agreed portion of their history — from one platform to another, completed by reconciliation and adoption. An import is one mechanical step inside that movement. The gap between the two is where projects fail, because an import can succeed completely while the migration fails around it: rows land, counts look plausible, and yet deals are duplicated, associations point at nothing, and the team keeps the old tab open because it no longer trusts either system.

The information-systems literature took this problem seriously earlier than the CRM industry did, because the stakes were larger. Bisbal and colleagues (1999), writing about legacy systems, describe migration as a spectrum of strategies from wholesale replacement to gradual evolution, each a different distribution of risk — and their core observation transfers directly to a CRM move: the difficulty is rarely the data transfer and almost always the surrounding questions of what the target should be, what the source actually contains, and how the organisation crosses from one to the other without losing its operating record. A method, in this light, is a device for sequencing risk. Each step below exists to convert an unknown into a number before the next step depends on it.

One decision governs everything downstream, so it is stated here rather than buried: cleaning happens in the source, before anything moves. Cleaning in transit — fixing values inside the mapping while records flow — destroys the ability to reconcile, because a record that fails to arrive can no longer be distinguished from a record deliberately altered on the way through. The point is developed in section 4; it is previewed here because it is the method's least intuitive commitment and the one most often overridden by schedule pressure.

2. Definitions

Source audit — the measured description of what the source system actually contains, as distinct from what its administrators believe it contains.

Unique identifier — the column an import matches on to decide whether a row updates an existing record or creates a new one. In HubSpot this can be a custom property with unique values enabled, and section 6 argues it always should be.

Association — the link between records of different objects: a contact to its company, a deal to both. Associations are data, and they migrate — or fail to — separately from the records they connect.

Reconciliation — the comparison of agreed numbers, counts and sums on both sides after movement, investigated to zero unexplained difference.

Delta — the records created or changed in the source between the main export and the cutover freeze, moved as a final, small, verified import.

Cutover — the moment access switches: the source becomes read-only, the target becomes the system of record, and the migration's success becomes a question of adoption rather than data.

3. The Source Audit

The audit's purpose is to replace beliefs with counts, and its output is a short document of numbers that the rest of the project keeps referring back to. Count records per object. Then — the count that changes plans — count how many of those records have the fields the target model requires actually populated. A company object holding 40,000 records with 6,000 populated industry values is a different migration from one with 40,000, and the difference between the two may stay invisible until someone measures it. Then measure duplicates, orphaned records, and referential integrity — records associated to something that no longer exists. Finally, list the picklist values actually present rather than the ones configured, because years of free-text entry accumulate variants, and every variant becomes a distinct value on import unless a mapping says otherwise.

The data-quality literature explains why this step cannot be skipped even in an apparently tidy system. Wang and Strong (1996) established empirically that data consumers judge quality on many more dimensions than accuracy — completeness, consistency, timeliness, believability among them — and later work showed that quality problems are contextual (Strong, Lee, and Wang 1997): data adequate for the process it grew up in becomes inadequate when a new process, with new requirements, starts consuming it. A migration is precisely such a context change. The source data was fit for the old system's uses, or the firm would have fixed it earlier; whether it is fit for the target model is a fresh empirical question, and the audit is how it gets answered before the answer is expensive (Redman 1998).

4. Remediation in the Source System

Remediation follows the audit and happens where the data lives. The reasons to clean in the source rather than in transit are three, and each is an argument from reconcilability rather than tidiness. First, source-side cleaning preserves the migration's evidentiary chain: after movement, both sides can be compared record for record, and a discrepancy means something. Second, it is testable in place — a merged duplicate or repaired association can be verified inside a system the team already knows, instead of inside a mapping file nobody will read again. Third, it improves the source for however long the source remains in use, which in a phased migration is longer than anyone plans.

Duplicates deserve their own discipline, because remediation's most dangerous tool is the merge. Merging is destructive in most CRMs and permanent in HubSpot; the merge plan — which record survives, on what criterion — is therefore a checked artefact, and an export taken before any bulk merge is the only undo that exists. The economics of this work are better than they feel: the cost of defective records compounds through every process that touches them, which is why the data-quality literature keeps finding that prevention and source-side correction dominate downstream repair (Haug, Zachariassen, and van Liempd 2011; Batini et al. 2009). A migration is the one moment a firm is institutionally willing to pay that cost; the method spends the willingness where it works.

What remediation is not is redesign. The temptation to fix the source's model — its objects, its stages, its fields — while cleaning its data conflates two projects with different risks. The model is fixed in the target, next.

5. Target Model Design

Design the HubSpot model from the business process, and then map the source into it. The reverse — recreating the source's fields one for one — feels safe and imports a decade of accumulated decisions that nobody now remembers making or can justify. Enterprise-systems research gives this instinct a name and a warning: a platform embodies assumptions about how a business works, and the adopting organisation must consciously decide where to adapt itself and where to adapt the system, because unexamined defaults are still decisions (Davenport 1998; Soh and Sia 2004). A migration is the cheapest moment the firm will ever have to shed structure it does not want, and one of the method's quiet outputs is a written list of source fields deliberately left behind.

The design work itself is unglamorous: for each object, the properties the process requires, their types, their allowed values, and the pipelines and stages that reflect how the firm actually sells and serves — with the stage definitions written down, because a stage without an exit criterion is a place deals go to be miscounted. Where the source's history includes concepts the target does not model, the choice is explicit: model them with custom objects, flatten them into properties, or archive them outside the CRM. Declining to choose is choosing flattening, applied by whoever builds the mapping under deadline.

6. Identity and the Import Key

Before the first import runs, create a dedicated property on each migrated object to hold the source system's record identifier — a single-line text property with unique values enabled at creation, the same technique the QuickBooks integration study uses for cross-system identity. This property is what makes the migration safe to re-run. HubSpot supports a custom unique-value property as the import identifier, so every subsequent import of the same rows becomes an update rather than a duplication, and reconciliation can join source to target mechanically instead of by name.

The failure this prevents is specific enough to narrate. A first import creates records. Someone finds an error and re-imports a corrected file, matching on nothing in particular. Contacts, which HubSpot deduplicates on email address, mostly survive; companies deduplicate on domain, so companies without a website value do not; and deals carry no automatic deduplication at all. The portal that results has clean contacts, patchy companies and doubled deals, and the doubling is discovered when pipeline reporting shows twice the quarter anyone expected. Every part of that sequence is documented vendor behaviour; the method's contribution is refusing to let identity depend on it.

7. Mapping and Unmatched Values

The mapping document — source field, target property, transformation, and policy for the unexpected — is the migration's real specification, and its quality is measured by what it says about values it does not recognise. Free-text sources guarantee unmatched values, and there are exactly three honest policies for one: fail the row and surface it, write a documented default, or land the raw value in a holding property for review after go-live. Declining to decide hands the choice to the import tool's defaults, which is how "Bostn" becomes a lifecycle stage. The mapping also states, per field, what empty means — never populated, deliberately cleared, or not applicable — because the target's automation will treat all three identically unless told otherwise.

This is the step where the audit's picklist inventory pays for itself: each in-use variant is mapped once, in review with someone who owns the process, rather than discovered one support ticket at a time. It is also the step with the strongest claim to being boring, which is why it is the one most often delegated entirely to a tool. The tool applies the mapping; it cannot write the policy.

8. Import Mechanics

The mechanics are documented by the vendor and belong in the plan as constraints, not discoveries. HubSpot's import files are .csv, .xlsx or .xls, up to 512MB on paid subscriptions, with up to 250,000 rows and fewer than 1,000 columns per file, one sheet, and a header row whose columns correspond to properties. Large sources therefore migrate in planned batches, and the batch plan follows dependency: companies before contacts, both before deals, and every record before anything that associates to it, because an association cannot be created to a record that does not yet exist — and the error that failure produces is easy to overlook inside a large import's summary. After each run, the import history exposes the error rows; the method treats an unread error file as an unfinished import.

Two mechanical habits carry most of the safety. Imports match on the identifier property from section 6, always, including the very first run against an empty portal — a habit with no cost that makes every later re-run an update. And every import file is retained, unmodified, alongside its error file, because the pair is the migration's audit trail and the raw material of any repair.

9. Rehearsal and Reconciliation

The full sequence — export, transform, import, associate — runs first against a sandbox or test portal, at full volume where limits allow. Partial rehearsals rehearse partial problems: file limits, picklist collisions, association ordering and timezone handling all surface at scale or not at all. The rehearsal's exit criterion is not a completed run but a completed reconciliation, and the distinction deserves italics it will not get: a test migration whose only evidence is a green summary has demonstrated that the import ran, not that it was correct.

Reconciliation itself is a small, fixed report agreed before anyone exports anything: record counts per object on both sides; sums of load-bearing numeric fields — pipeline value by stage is the usual one — on both sides; association counts for the relationships that matter; and spot checks, by a person who knows the accounts, of a sample no automation chose. The data-quality literature's operational insight applies directly: assessment needs both objective metrics and subjective inspection, because each catches what the other cannot (Pipino, Lee, and Wang 2002). Every unexplained difference is investigated to a cause before cutover — the freeze window exists precisely so that stopping is still an option.

10. Cutover

Cutover is a sequence, and its order is the point: freeze the source to read-only, export the final delta, import it, reconcile again, and only then switch access. Reconciliation happens before access switches, not after, because afterwards the organisation is changing the target and the comparison means nothing. The freeze window is scheduled against the commercial calendar — never month-end, never mid-campaign — and its length is known from rehearsal rather than hoped.

What follows cutover is an adoption problem, and it deserves the same planning weight as the data. The technology-adoption research is unambiguous that early disappointment tends to convert to abandonment quickly and durably — users who meet wrong numbers or missing records in week one do not return in week three to check whether things improved (Speier and Venkatesh 2002). The migration's last deliverables are therefore organisational: the old system visibly read-only rather than switched off, a named channel where discrepancies get answered fast while trust is forming, and the implementation literature's standing reminder that training and executive attention, not data mechanics, are what separate adopted systems from technically successful failures (Umble, Haft, and Umble 2003).

11. The Case for Leaving History Behind

The method's most contested conversation is not how to migrate but how much, and the honest default is less than the firm first asks for. Complete history feels like safety; in practice, history's value tends to decay quickly while its migration cost — mapping retired fields, modelling retired processes, cleaning records nobody will open — is paid at full price. A defensible pattern migrates open work and the recent past in full, summarises the deep past into a handful of properties, and preserves the complete source export in cold storage where it satisfies the actual requirement, which is usually auditability rather than availability. The counterargument is real and sometimes wins: regulated industries, long sales cycles and service businesses with decade-old obligations may genuinely need deep history live. The method's demand is only that the decision be made against the audit's numbers, not against the feeling that deletion is loss.

12. Limits

This reference describes a method, and a method is not a warranty. It assumes a source that can export completely and a business that can tolerate a freeze window; systems that cannot export associations or history intact push the work toward APIs and raise its cost in ways this document does not size. It is written against HubSpot's documented import behaviour, and the specific constraints cited — file sizes, row limits, deduplication rules — change on the vendor's schedule; the linked documentation is authoritative, and this text is not. The research it draws on was developed largely on enterprise systems and general operational data rather than on CRM migrations specifically, so its application here is an argued transfer rather than a tested one. And the method's evidence from this consultancy's own engagements, such as the law-firm migration study, is practitioner reporting with the interests that implies — illustration, not proof.

13. Conclusion

A migration is a movement of trust between systems, and the data merely follows it. The method above is one long insistence on sequence: measure before cleaning, clean before moving, design identity before importing, rehearse before believing, reconcile before switching. None of its steps is sophisticated, and that is the finding — the projects that fail are rarely defeated by difficulty. They are defeated by an import that was mistaken for a migration, run in whatever order the deadline suggested. The order is the method, and the numbers at each step are what let a firm know, rather than hope, that its operating record survived the move.

References

Batini, Carlo, Cinzia Cappiello, Chiara Francalanci, and Andrea Maurino. 2009. "Methodologies for Data Quality Assessment and Improvement." ACM Computing Surveys 41 (3): Article 16. https://doi.org/10.1145/1541880.1541883

Bisbal, Jesús, Deirdre Lawless, Bing Wu, and Jane Grimson. 1999. "Legacy Information Systems: Issues and Directions." IEEE Software 16 (5): 103–111. https://doi.org/10.1109/52.795108

Davenport, Thomas H. 1998. "Putting the Enterprise into the Enterprise System." Harvard Business Review 76 (4): 121–131. https://hbr.org/1998/07/putting-the-enterprise-into-the-enterprise-system

Haug, Anders, Frederik Zachariassen, and Dennis van Liempd. 2011. "The Costs of Poor Data Quality." Journal of Industrial Engineering and Management 4 (2): 168–193. https://doi.org/10.3926/jiem.v4n2.p168-193

Pipino, Leo L., Yang W. Lee, and Richard Y. Wang. 2002. "Data Quality Assessment." Communications of the ACM 45 (4): 211–218. https://doi.org/10.1145/505248.506010

Redman, Thomas C. 1998. "The Impact of Poor Data Quality on the Typical Enterprise." Communications of the ACM 41 (2): 79–82. https://doi.org/10.1145/269012.269025

Soh, Christina, and Siew Kien Sia. 2004. "An Institutional Perspective on Sources of ERP Package–Organisation Misalignments." Journal of Strategic Information Systems 13 (4): 375–397. https://doi.org/10.1016/j.jsis.2004.11.001

Speier, Cheri, and Viswanath Venkatesh. 2002. "The Hidden Minefields in the Adoption of Sales Force Automation Technologies." Journal of Marketing 66 (3): 98–111. https://doi.org/10.1509/jmkg.66.3.98.18510

Strong, Diane M., Yang W. Lee, and Richard Y. Wang. 1997. "Data Quality in Context." Communications of the ACM 40 (5): 103–110. https://doi.org/10.1145/253769.253804

Umble, Elisabeth J., Ronald R. Haft, and M. Michael Umble. 2003. "Enterprise Resource Planning: Implementation Procedures and Critical Success Factors." European Journal of Operational Research 146 (2): 241–257. https://doi.org/10.1016/S0377-2217(02)00547-7

Wang, Richard Y., and Diane M. Strong. 1996. "Beyond Accuracy: What Data Quality Means to Data Consumers." Journal of Management Information Systems 12 (4): 5–33. https://doi.org/10.1080/07421222.1996.11518099