Author: David Masri Founder & CEO

A few month ago I authored an article titled "Why are CRM systems so susceptible to bad data?" and in that article I laid out what I felt (and still do feel) are the root causes of bad CRM data. To quickly review the 6 root causes I identified are as follows (see the article for details):

1. Lots of manual data entry by non-data entry professionals.
2. Lots of Free text fields.
3. A UI focused Data Model
4. Circular references in CRM Data Models
5. CRM data goes stale faster than most other systems
6. Relative lack of importance of CRM data to an organization

More than a few people reached out to me and asked “Ok, great, what can we do about it?”. The sad answer is nothing, there is literally nothing you can do to stop the above from being true. (Maybe you can build and internal CRM system with a proper data model... but that’s not something I would recommend).

So, if we can’t stop the root caused of bad CRM data, does that mean all is lost?? NO! the answer is to declare a never-ending war on bad data, and continually fight an series of never ending battles, keeping your data clean.

Generally, this goes against everything we learn about building proper systems and processes. We a taught to be proactive in preventing things from going wrong or bad. What I am saying is that a proactive approach won’t work here, and that we must adopt a reactive approach of detecting bad data and fixing it as soon as we detect it.

Our Battle Plan (In 5 Steps)

1) When choosing new features to implement, prioritize end users features over managerial ones.

Often when designing a new CRM system, developers prioritize features requested by management over those requested by end users, simply because its management who is signing their checks. Now I'm not recommending developers go against management's wishes, but as good consultants we should advise our clients to build systems with end users in mind. Systems built with end users in mind get used enthusiastically, whereas getting users to use systems built for managers often requires quite a bit of arm twisting.

It's no secret that many CRM implementation struggle with user adoption. Building features that add value to end users is by far the best way to encourage adoption. When CRM systems add true value to end users, they take ownership of their data and proactively keep it clean (by being proactive in their reactiveness).

When the system is built for managers, end users don't care if the data is bad because the system adds no value to them. If they are even using it, it's because management is forcing them to. What's worse, is that often users will intentionally create bad data (ie. log fake phone calls or activities, or fudge time records) to make them look good on management reports.

2) Put in processes that automatically identify bad/stale data and classify records appropriately - and have users regularly audit records.

Classifying records with a cleanliness\completeness score

If we are going to reactively keep our data clean, we need a way to detect and classify bad data. We can write complex code using AI to do this, but most likely a simple formula field can do the job.

All we need to do is come up with some definition of what is a good or complete record (Such and such fields are completed to some minimal standard). This will (and should) vary by record type. The bar for data cleanliness is very different between an unqualified lead, a qualified lead, a vendor and a client record.

There is a very strong temptation to try and use validation rules or required fields force users into entering data. But we need to be very careful to NOT over leverage validation rules. Often users need to create a new record or promote one (from lead to contact) but don't because they are missing some required data point and the system won't let them. So now we still have the missing data point as well as a misclassified or missing record. Some users will even enter garbage data just to get passed a validation rule. (Email address required? - NoEmail@ScrewYou.com). We'll discuss this more on this in step 5.

Have users regularly validate\audit records.

Generally, if you a have a CRM systems you (or the sales team) is in regular contact with the people in the system. At least once a year (or so), when a call is made to an individual have the salesperson validate the info on the company and contact record, they should then update the record marking it "Validated" with the validation date, and of course correct the data if needed. Any record that has not been updated in some time should be classified as "Potentially Stale".

3) Change records ownership of records that are not being maintained, and demote stale records back to a "needs qualification" status.

There is no reason for a salesperson or an account manager to not be in regular contact with his or her clients, and if they are in regular contact with the client the data should remain in good health (and in a validated status). If you see that a salesperson is logging activities but not validating the data, a gentle reminder (or on-screen message) should do the trick. If they are not in contact with their assigned clients, you should take the client away from them, and assign the client to someone who will properly manage the account.This is not just a good idea for data cleanliness, it’s also good business.

The same goes for leads, if a qualified lead is not contacted, assign the lead to a salesperson who wants it. If a qualified lead goes stale, it should be demoted to "unqualified lead". If a "client" leaves, it should be demoted to "former client" - (which would convert back to a "lead" after some time.) Remember each of these classifications have their own standards for "clean".

4) Self-procuring data is king

Self-procuring data is the best you can get. Its one of the reasons I love LinkedIn so much, I always have up to date info for my personal contacts. If we have a mechanism to automatically keep data updated & clean - we should take advantage of it.

A few examples:

1. If we send out a marketing blast, and an email bounces, the contact record should automatically be flagged as "needs validation".


2. If we have current client contact data in the ERP system, we should integrate with it. Our ERP system is more likely to have correct data (again see my previous article here as to why).


3. If we are buying lead lists, make sure we save the lead source Id in Salesforce when importing the data, so the next time we buy the data from the same source, we can update the correct records, rather than creating a duplicates or running some ad-hoc matching process.


4. If it's cost effective, we can validate our data against public Databases or databases from professional organizations (FINRA, The American Bar Association, Healthcare professional databases, USPS address verification)

5) Strategically use validation rules and always use duplicate detection

Validation rules & Required Fields

Strategically use validation rules to prevent mistakes - not to force users to enter data they don’t have. Of course sometimes we have to force users to get data before we let them proceed, if we are going to ship something we need a shipping address first, but we should not be over zealous when making fields required – where the rule will prevent a user from doing his/her job. These rules are to help them do their job well, not to make life hard. It's Ok to force a user to enter notes with activities, but probably not a good idea to force them to enter "number of employees" when creating an Account, Lead or Opportunity record.

Systems users care about doing their job, not about passing our random validation rules to force them to do extra work so that some manager can run a meaning reports uses to micro-manage them. And users can get very creative in bypassing rules, sure entering bad data is one example, but take a look how these restaurant users are using the reservation system:

You can read the full story here: https://javlaskitsystem.se/2012/02/whats-the-waiter-doing-with-the-computer-screen/

Duplicate detection

Use of basic duplicate detection is a no brainer, I can't think of a single good reason to not use it. (Even for integrations, if our integration needs the ability to create duplicate records, then add an exception specifically for the integration user.)

This article is adapted from my book: Developing Data Migrations and Integrations with Salesforce.