Merge It and Purge It!

This article originally was published in Total Retail Report.

In a Direct Marketing business, a customer list, i.e., house file, is your most valuable asset. Therefore, it is important to manage your house file and to keep it segmented and always up to date.

To keep your house file current, you need to merge/purge it regularly. What does this mean? Merge/purge is a process that identifies and removes unwanted or duplicate names and combines those records into one record for every customer on your list. However, it is more than a deduping process which I will discuss later.

Eliminating duplicate records on your house file and when mailing to outside prospect names is important for several reasons, mostly financial. Mailing duplicate names is expensive. It can be avoided by running a merge/purge program. This is one of the services offered by a computer service bureau. How you instruct your service bureau to run the merge can affect your mailing results. How to assign list priorities, treat multi buyers, deal with family groups and more, are all important to the end result.

There are two types of duplicates that a merge identifies, which are as follows:

  1. Duplicates between files, which are known as Inter File Duplicates.
  2. Duplicates within a file are known as Intra File Duplicates.

When there are duplicates on the file, the merge can assign the surviving record randomly across all lists or based on a defined priority by list. In other words, the merge assigns which list will retain the record that appears on more than one list.

The three main options for the different levels a merge can be run are, (1) 1 per household, (2) 1 per address and (3) 1 per individual. I also want to mention that business-to-business merges have an additional level of complexity.

While the merge assigns the surviving record, it also creates a file of the duplicated records for future mailings. These are called multi buyers. They are names that have “hit” on one or more prospect lists. Normally, they are categorized by how many times the record appears in the merge, i.e., 2x, 3x and more. A two-time multi, for example, is a record that appeared in the merge twice. It was mailed for the first time under the list where it was assigned, and it can be mailed a second time in a future mailing for no additional list rental expense. Multis are only captured for outside lists since there is NO benefit to pulling house file names out of their R-F-M (recency, frequency, and monetary value) cells.

The surviving multi buyer information is also retained in the merge. When the merge assigns duplicates based on random allocation or on a prioritized allocation, the duplicate names that are retained are tagged as a multi buyers. Typically, the merge report identifies; “singles” that appear in the merge only once, multis that appear in the merge more than once (and were retained on that list) and “output,” which is the combination of the two.

Family groups are identified so that “same” lists do not falsely create a multi buyer. For example, if you take two different selects from the same list owner, they should be unduplicated before they are sent to your service bureau. If they are not, the duplicates between the files could be considered a multi buyer in the merge. In this example, they are not multis but are, instead, the same record being pulled twice. By identifying these in the same family group, the merge would consider these intra file duplicates, which can be deducted from your list rental invoice.

Always put your house file (your customers) in the top priority with the best performing R-F-M groups at the top of the priority and the lower performing R-F-M cells at the bottom of the priority. If your file has gone through an update, there will be very few duplicates. However, it is still important that any duplicates found are associated with the “best” possible R-F-M cells. There are times when you will only mail or re-mail a portion of your house file. If the duplicates within the house file were random, you might drop a record that is from the higher R-F-M priority. By prioritizing your house file, you will be sure to mail the best performers. Assign your house file buyers to the same family groups. Therefore, duplicates within the house file will be considered intra file duplicates. By doing so, you will know what multis credited to the house file will have also appeared on another list (other than the house file).

If you are not mailing all the house file records you put in the merge, make sure you do not drop what has been identified as multis. These are primarily records that came from an outside list that hit to your house file and they will perform well.

Unfortunately for “Direct” companies, your house file is an off-balance sheet asset that has no financing or borrowing leverage with lenders. Banks and other lending institutions don’t have a clue about the value of the house file. As a Direct Marketer it is important for you to understand and to appreciate the value of your house file.

Your merge should be used to maximize your mailing, not just to remove duplicate records. By considering how you would like duplicates and priorities assigned prior to the merge, you will get the most out of your merge.