One of the reasons we host the CiviCRM Scotland Meetups is to create a space for the people who actually use CiviCRM to come together, share experiences, ask questions and learn from each other.
There are plenty of opportunities for developers and technical specialists to talk about CiviCRM. These meetups are a little different, they’re about the practical challenges organisations encounter when using the platform day-to-day, and how we can help each other solve them.
For our latest session, Aidan Saunders of Squiffle Consulting joined us to tackle a challenge almost every established CiviCRM database will encounter eventually: duplicate contacts.
Why talk about duplicate contacts?
One of CiviCRM’s greatest strengths is its ability to give organisations a 360-degree view of their contacts.
But duplicates can creep into CiviCRM surprisingly easily.
Someone signs up for an event using a different email address, joins a mailing list with their name spelled slightly differently, completes an anonymous form, or appears twice following a data import.
If someone’s membership is attached to one record, their event attendance to another and their mailing preferences to a third, that picture becomes fragmented. It can lead to incomplete searches, misleading statistics, confusing contact histories and even potential data protection issues.
So, rather than simply finding and deleting duplicates, Aidan’s session looked at how we can prevent them, identify them and safely bring records back together.
What was learned?
One of the biggest takeaways was that prevention is better than clean-up.
Where possible, having people sign into your website before submitting information or using checksum links in CiviCRM emails can help the system identify an existing contact rather than creating another record.
Where anonymous forms are necessary, your dedupe rules become particularly important.
Aidan broke down how these rules work using fields, weights and thresholds to determine whether two records could represent the same person. Crucially, there isn’t one perfect rule for every organisation or case.
An email address might seem like an obvious identifier, for example, until two members of the same household share one. Likewise, a matching surname and postcode doesn’t necessarily mean you’ve found the same person.
Your dedupe rules need to reflect the data you collect, the people in your database, and your specific use case.
Be cautious with automatic matching
The session also explored the difference between supervised and unsupervised deduplication.
With supervised matching, CiviCRM can flag a potential duplicate and allow someone to decide whether the records genuinely belong to the same person.
With unsupervised matching, that decision can happen automatically so the rules need to be much more cautious.
As Aidan explained, two records for the same person might be inconvenient. Combining information belonging to two completely different people is much harder to untangle.
A useful approach is to test new dedupe rules against your existing database first. Look at the matches they produce, identify any false positives and refine the rule before relying on it more widely.
Merge, don’t just delete
Finding a duplicate doesn’t necessarily mean one of the records should simply disappear.
One record might contain someone’s membership while another holds their event history. CiviCRM’s merge tools allow you to bring that information together into one surviving contact, including related information such as memberships, relationships and event participation.
Aidan demonstrated this process during the session, including how to review conflicting information and use CiviCRM’s batch merge functionality for larger sets of duplicates.
And if you’re planning a substantial clean-up, two particularly important pieces of advice came out of the discussion: back up your database first and consider enabling logging so you have a record of what changed when.
Watch the full CiviCRM deduplication session
There’s much more practical detail in the full meetup, including Aidan’s live demonstration of creating and testing dedupe rules, finding potential duplicates and safely merging contact records.
🎥 Watch the full session to see the process in CiviCRM and hear the Q&A with other CiviCRM users.
And that’s really what these meetups are for. They’re a chance for the CiviCRM community to ask questions, compare approaches and share the things we’ve learned from actually using the platform.
Our CiviCRM Scotland Meetups are open to people in Scotland, across the UK and further afield, and we’re always looking for ideas and speakers for future sessions.
If you’ve solved an interesting CiviCRM challenge or have something you’d like the community to explore, we’d love to hear from you.
Get In Touch
We’d love to hear what you’re up to. Please get in touch using hello@pooka.co or through the form linked below. Don’t be shy!
