Data Processing

It has been shown in cases too numerous to recount that data files, represented as being complete and correct, are flawed with overt error, or hampered by many types of subtle error conditions for which we have processes to define, identify and correct.

This examination process is provided at the expense of additional time ranging from one to two days, for most data files. Each and every word and term of the data file is examined, and every character verified. Last name words are verified to be those, and first name words to be those, and all business words recounted, as well as all coding and data types checked. This avoids unexpected printing errors when invalid data or characters are presented to image writers, or used in context within a presentation of data.

In general, input errors surface at the rate at least ten times greater than what any inexperienced customer would expect or anticipate.

Most experienced customers expect a 100 percent error condition and then use these processes to assure a better production ready status. Indeed this inherent doubt is what we must assume as your contract processor.

If you believe that your processing schedule cannot tolerate a single delay and wish us to proceed to use data presented from another processor as though it were ready and useable, no questions asked, then this acknowledgement will serve as your waiver of all the normal assurances that we provide to you as our customer. While we hope this is not going to be the case, for reasons mentioned above, your request to expedite all processes will be honored and we will continue to observe all that we can, down into the line of production, to see that there is no catastrophic failure on the part of your data such as we might plainly see.

Thank you for understanding our company's concern for this crucial quality control. And thank you for permitting us the opportunity to serve your data processing needs. We greatly appreciate that decision on your part.

The above form may be from time to time communicated to customers who have questions and requests regarding the expeditious advancement of a job in process.

**mail-production.com**--Corporate Offices
mail-production.com 1700 Venable Street Richmond, Virginia 23223-6308 Production 804.783.1000 open 24.7	Office Phone Numbers 804.780.1700 800.336.6245 Sales Office Fax 804.782.9876 Production Fax 804.783.2601

Multiplex MPX™ Merge|Purge

As authors of four merge|purge systems spanning two decades, we believe the latest advancement we made in the merge|purge sciences set us apart from any other process method. Largely the prior generations of merge|purge processes were spawned in search of line-code improvements to enhance functionality and improve correctness and multi-level job control. Sometimes the programs were rewritten to support a platform change. In the course of this lengthy history we used and monitored other authors' systems. This experience led us to directly develop our own. In 1998 it became apparent to the development that a major change in philosophy was going to be needed to accomplish a higher level of progress.

The ``next version'' was going to be something more than a group of improvements or fixes. Specifically, the declared objective stood to be an entire new approach to duplicate detection, and the all-important data isolations that figure into merge|purge mechanics. A calculus-based, fuzzy-set logic was chosen for match-truth decisioning, and a process engine built from that foundation. This method of reasoning is the closest approximation to human thought possible in non-analog (or digital) computers, but it maintains a precise data view, that which is essential to a parsing of minute terms–those too difficult in neuro-organic systems.

In the next several decades, depending on our achieving a few technical breakthroughs, all merge|purges will be run in systems analogous to natural thought processes, those rendering continuous signals, such as natural nerves, caused or based upon derivatives of all known, or learned, information. Today's information systems rather imitate that objective in a primitive and unnatural way, which is ``efficient'' and expedient to the state of developments of computing sciences as of the end of the Twentieth Century. This ``step'' is merely a forerunner to a similar design shift compelling the re-designs in the computers of the future. See the [pdf] article prepared which describes where computing sciencies have arrived, giving the choices our designers made during the first epoch of our long adventure into this realm.

The reason the shift is imminent is because the interpretive element in computing today is entirely too dependent upon human initiative. For example, a national security analyst could have come to work in 2001 and sleepily punched into his workstation, ``News: what's wrong?'', and the analog of his mission would have popped up on the screen telling him:

``There are nineteen to twenty-four foreign nationals, mostly Saudi Arabians, the bulk of whom have been enrolled in five flight schools learning to fly jumbo jets; and there is a 97 percent probability that they are under orders from Osama bin Laden [q.v.] to fly six or eight passenger aircraft into the towers of the World Trade Center, White House, Pentagon, NY Stock Exchange and two Nuclear Power Plants within the next three days. They are now all in the northeast US buying passenger tickets for various flights departing several airports the morning of September, 11th.

``Punch here for their names, locations, itineraries, phone numbers, public and confidential records, banking transactions, employment histories, bio-medical profiles and next of kin. (See FBI Field Report: 199-Eye WF 213 589, outlining the similar scenario.) There is a 0.33 percent probability they are mounting toward convergence at a Los Angeles County, CA, church picnic [q.v.], and this must not be ruled out of this question.''

Of course, the entire plan would have been accurately predicted weeks prior to this late hour.

This is a not-at-all atypical example of the important differences that analog systems offer their human counterparts, and it's the reason that important tasks requiring continuous analysis be placed on newly designed systems in the decades to come. This is the type of work they are best suited for, and the reason that chaotic systems must be replaced for serving this function. And another interesting part of this is, with the vista of molecular nano-circuitry looming just a few years off, all this computing power could be conveniently at hand within a device the size of a Barbie Doll's wristwatch.

It was finally decided, after a great deal of consternation, to throw out all the work we had accomplished over many years. This was because no degree of fixing and tweaking could begin to solve the problems that merge|purge programs face when they encounter improper and defective data. When a development is advanced to the point where it is trying to fix and deal with things so unusual that they're not supposed to be there in the data anyway, then that development has arrived at the iron wall of error where all other merge|purge systems reasonably draw the line.

To demonstrate the exceptional difference in the Multiplex MPX™ Merge|Purge, any merge|purge users from any source can send us their gross-name input file, or their net-name output file, and we will process it under our unique system and return you the resulting file and a summary for the first system's operations as compared to our demo process. This insight will reveal for you the unique properties of our advanced methods. The other nice part is our demo exam is totally free!

But, as far-fetched as it seemed to us, we wanted to find a way to proceed further into the cure and detection of hopeless and even absurd levels of input error. This, of course, had never been asked for, nor thought possible; and virtually all m|p customers use the former methods still today because they are all that are available from any publisher or private system. I will term that level of operation, however well advanced, the ``standard logic'' approach to merge|purge.

Standard logic means, in lieu of a more dramatic definition, that program should make certain valid assumptions. Those may be that a name field probably contains a name; an address line might contain a name or an address; that ZIP codes are where they're supposed to be, and so forth for many yards of ``assumptions.'' But what happens when they are not? This is where standard logic tends to break down, and rightfully so. It's as if you advertised a truck load of hamburgers, but got a shipment of hot dogs instead, and the hosted party ate the victuals anyway, without any notice as to a difference. Food is food, and these programs just eat it, only asking what flavor it is when they set up a menu.

Very few of them ``send the order back'' when it's prepared incorrectly. This is what I have thought of as ``industry standard'' processes. They are largely what is asked for, and they are what is delivered, as prescribed.

After many years, we were tired of saying, ``Gee, I wish some system could find that dupe,'' knowing full well its unique error made it too rare to program for, by species.

Then, when you add up all the nuances ``to rare to program for,'' you find that you have a large volume of little vexations, all troubling, but each too rare to pursue at the line level.

This is where we said, for all time, ``…in order to do this we have to look at this input as data, and not as names and addresses.'' If we assume it's anything at all, we're going to be wrong 0.3-, 0.5-, or 5 percent of the time. That's just too high.

What can we do?

As we began to do this, unlike the first programs, where we knew that merge|purge was theoretically possible, we went to the Land of Nod to see if what we had in mind ``might be able'' to work. I remember that after running some model algorithms thrown together during a few days work, just to see, I was more than surprised to see some things popping up I'd never dreamt possible. And yes, while the results were much worse than before (because the algorithms were totally out of control and weren't fully and effectively designed), they included some successes beyond our wildest dreams. We said, ``Wow. This is good stuff. Let's take it a bit further and get this thing settled down to see if we can control it (the overall process) and make it reliable.''

At that point we had decided we could detect duplicates of any type or description, without limits, but it had to exclude any overkill (a symptom of an overly aggressive or failing piece of logic). Labor intensified, and day after day tiny bits of control process were laid down and tested. Some engine components were rebuilt and improved; and finally, in about six months, we had a working ``alpha.''

That proceeded to a ``beta'' in one year, and thereafter we were running all production merge|purge's in a calculus-based processing environment, within a program and didn't know it was running a merge|purge.

It didn't have an exclusive set of ``names,'' and could care less what were addresses, or any other input bytes. It ran by reporting data findings through an interpretive and overlapping, or fuzzy, characteristic of types, or match truths.

Too, the new design incorporated another major design shift. Most systems, and ours as well, didn't really ``merge'' anything; they just merged their appearances in self-limiting indexes, not by a full location. This puts access loads on a retrieval system, and doesn't rectify the hesitant limitations of frequent revisits, thus relaxing the rigors of the most-complete and non-reluctant processes.

We said, for a couple of reasons, these data must be full sorted into a true merge set, plainly, for one reason, the algorithms won't run any other way. But, also, if this could be done with efficiency, all the after processes would be lying in close disk proximity within the same cluster, and marvelous efficiencies could then be realized.

That requirement set forth probably the first ``index-free'' program design in existence for merge|purge; as standard merge|purge's are actually indexing tricks and not real ``merges and purges'' of the input file data. This went better than expected; and relying on a low-level physical sort module so powerful as to drive a disk crazy (but one which never asked a disk to look outside its current sector to do anything far reaching), which is very, very desirable. This is done through the working with thousands of tiny sub-files or ``containers,'' instead of at any time using a large masterfile.

We had then managed to full sort the whole program data set, without any data abbreviation, and to do so quicker than it had before been merely indexed. This technique also requires a lot of data stability and an absolutely sterile and reliable data set, all of which was put into its preparation.

In the final analysis, should disk storage become virtual within static memory, we should still desire to operate this way because the design has proven over time to be inexorably efficient and graceful.

New merge|purge Dynamics

Because the heart of the Multiplex design is calculus, it inherits some impressive reporting capabilities from this build. The match sets are broken down into segregated types of sets, all with common or exact attributes. Then these leveled sets are ranked-out further into 1-to-1 dynamic ratios, with every member given its relative-to-one match-level. Example:

MATCH LEVEL

The address data are also factored into the reported match-level codes, and a full explanation of the coding system is provided with each completed job returned.

Some "certain" classes are rounded to 2-character levels, such as in the .99- and .98 notations seen above.

All flip/flopped names are easily reckoned with as the .98's of any match-set class.

"A" are the same addressee entity at the same address.

"H" are traditional household type match sets.

"F" are family type sets.

"B" "C" "D" have to do with specific address elements, etc.

Name and Address Hygienics

There are certain operations in existence that "claim" to be in the business of file repair and data cleanup, and so forth. If you have seen their results you will know why the word "claims" holds true for their operations. And I do not doubt that they can and do fix many things. The big challenge in data hygienics is in the science of finding what's wrong amid millions or even thousands of records. If you browse thoroughly just 10,000 records you can begin to appreciate the difficulty of the assignment.

Basically, anyone can fix anything when it's on their screen editor. (The preceding sentence is wholely false, but its intention seems instructive.) Getting the problem record there when it's supposed to be, and not getting it there when it's not is the big bugaboo. A human being cannot browse 1,000,000 records in time to do anything but be very late (almost in the obituarial sense); and his or her brain fries if he or she browses maybe 100,000 effectively. Thus the major task of data hygienics for mail files is in the diagnostics, and afterward in the level of automated versus manual repair tools that are turned loose on the file.

If my experience in this area is meaningful, then I will tell you that I have not seen a file repaired by any bureau that was not considerably damaged by their search-and-replace tinkering. Data correction is not, by its dynamics, a scan and replace business; but most all providers use that approach, in part at least.

We have developed, over thousands of hours, automated processes for finding and repairing file damage. And, these processes are very, very limited in their free rein. This is because there are few absolutes as to what is always wrong. A diligent person scanning a file can see a situation and say, "I cannot think of any case where this is correct," and thence proceed to sweep it. This is the hallmark fumble to a clean-up disaster. No one, no matter how diligent, can, without regret and error, make that decision over a large file data. The decision to do so will almost always prove unfortunate.

On the other hand, if a human is forced to pound keys all day long (and I mean long day), there are fatigue factors that creep in, in terms of accuracy. Balancing out the correct approach to file repair is the difficulty; and, since it's also very hard work, there are few sources that can perform it well. And there are only a few who purport to. But finding "their" mistakes is like finding a needle in the haystack, as well; and many of them get away with things that are done in error because any evidence is likely invisible.

Basically always asking for a change detail is helpful;, but even change details are beyond human scope on large repair projects. A change summary is virtually useless because the changes, while counted correctly, are often miscategorized and misrepresented in that summary.

Take, for instance, the address standardization of 1,000,000 address records. Well if the process changes 999,000 records, which is within reason, who will check that detail listing? We as processors have all had foul experiences with faulty address standardization, especially as the requirement for it was rather thrust upon us without the years of research needed to program it error free. Even today there are far too many address-correction errors. Some developers have even halted any attempt to correct those that remain within their systems. This leaves the diligent operator the task of knowing and retracting those errors, if he or she cannot prevent their insertion.

Mailfiles often have addresses that mix ``inside the building'' mailstop information with postal address information. For example,

A good mailfile layout includes not just an address field, but also a separate mailstop field. The address field should only include the information the Postal Service Mail Carrier requires for delivery, in other words, the data found in the USPS delivery sequence file (DSF). Any mailstop information should be isolated as ``inside the building'' delivery information. For example,

Please note the logical organization of the above address, going from a ``finer'' resolution at the top to a ``coarser'' resolution at the bottom. The mailstop is located in the company; the company is located at the street address; the street address is located in the city and state. Each address line is geographically ``within'' the line below it.

Printed labels or mail pieces might show the mailstop data in a number of possible print positions, but the actual mailing list datafile should store the USPS address and the mailstop information in separate fields which will enhance address matching and correction processes.

Advanced Data Services - AAC and ANC

What are Advanced Address Corrections and
Advanced Name Corrections and
Why They Are Needed

AAC and ANC are emerging data sciences being developed among America's most-conscientious process providers. If your processing entity has approached you regarding the use of these services, there is a good likelihood it has developed systems that have entered into data hygienics in earnest. Any casual reference to ``data cleanup'' can be regarded as puffery and false claims because these are major system developments requiring years of development and refinement for them to be of any validity in comparison with true methods. Since customers cannot readily determine if the services offered are advanced or not, it takes no real development investment for those who wish to pose as providers to make the claim.

It will take the leadership of those persons who are acutely aware of the data crisis that faces USA mailers for the development of advanced correction systems that go beyond the broad claims often seen. The systems must actually find and provide accurate solutions for error conditions that prevent standard processes such as DPV, M|P, and NCOA from operating at their highest capabilities. We welcome all effective accomplishments in this difficult arena, and especially remind sales and marketing personnel that, while they may never to able to explain to their customers exactly what ADS (AAC and ANC) actually is, their customers should be unreservedly assured that advanced operations are always needed.

The diagnostic operations are designed to run as part of our initial file processes, and if the customer believes the file needs no ADS, the diagnostics will confirm that belief by finding no present error conditions. Unless ADS corrections have been recently performed, we can not arrive at that conclusion.

We and others have found that after processes are improved at levels of greater than fourteen percent for all generalized list input. We normally do not see mailfiles with conditions ranging to extremely high degrees of error, beyond fifty percent, because the entities relying on and mailing such file data do not survive to provide them to us.

Your input file data will receive a complete AAC and ANC review prior to any after processes running, with the needed updates and corrections implemented. This assures you that the important processes being run are capable of operating for their indented purposes, freed of the error conditions which have heretofore blocked their successful completion.

It is not our intention to give away all of the tricks of the trade here, but is it, without doubt, a trade of many tricks. The editing mechanism must employ many keystroke macros to reduce key press and to heighten accuracy. But no matter who repairs your files, always weight the merit of what they did with a true regard to what "they did."

Example: If someone fixed case "1" 1000 times, and goofed it up once, where is the logic of complaining about that? We should know our facts, and while every error is bad, every fix is better, and over the entirety things should have balance.

Tip Two. Never ask anyone to "check behind" another processor. That is patently unfair. In doing so, all the good things the first process did are discounted as being meaningless (or without merit), and therein may reside the majority value of that person's work. Too, any two excellent processes are better than the single one of them, else one is not excellent, lest it be identical, which can by chance not be the case.

If any comparison is needed, and any advertised error is not a factor, ask each to do his job on the same source data, then get a fair and competent third party to enumerate all the differences. Done this way you'll have in hand a fair comparison which was done under identical awareness conditions and time constraints. Ask any of the competitors to analyze the results and that's fatal for obvious reasons. MPX analysis

I guess that's why Miss America isn't scored that way, ending in the female version of the world wrestling title bout. Sometimes we all like the runner up, or our own state's entrant better, as well, so tastes and preferences weigh in heavily in matters of beauty and style.

Tip Three. Anyone involved in hygienics work must possess a complete skill in words evaluations, including all misspellings thereof. The definition of all business words is crucial, and the gender assignment over all words must be in line with the client's needs.

Tip Four. There are too many cases to document where there is no known correct answer, and your selected bureau must have skills that enable a third authoritative answer.

A simple example of this is seen in the name Todd Brian as opposed to Brian Todd.

Or

Wm Smit vs
Wm Smits vs
Wm Smith vs
Wm Smithson vs
Smithsonian Women
Mitchell Williams

And even here we don't know if this is another person named Walker M Smith because, for some reason, too many typists are entering two-initial names without including a needed space between the two which permits their correct punctuation.

Hygienics work involves the active undoing, as well as the active doing to get to that elusive Dallas Morning News; and here you should be looking at a ranking for hits on all three business words. We have all derogatory and profane words cataloged and prepared for removal as well, and it is surprising how often these types of words are found.

We've met many Brian's. Thus we have countless Brain's in mail files, which are 99 percent invalid,

And, as well, about ninety percent of keying errors involve missing the intended key by one keyboard position. Thus you won't be often wrong to think that Billu Williams may well be Billy, and the slight discovery that that 'u' and 'y' are one key apart just about settles it.

Patterns of errors are all important. Hygienics involves the discovery of error patterns unique to a certain source file. When she does it wrong five time, there's a high probability she did it wrong thirty-five times, too, and this must be ruled out. OCR pattern errors are common threads that must be traced as discovered.

Program errors and table converstions often involve massive trouble, such as our finding that all the "sisters" have suddenly become "seniors," as the SR notation is used in different ways.

And while all of this may be of some supposed interest, to quote Ed Poe I will say, "that I must perforce draw this paper to a close," as there are truly an endless supply of worthy and frightful examples. This fact is why we have this "hygienics" field anyway, because not everyone can do it, and after too furtive a study hereof, you may begin to wonder if anyone can.

USPS Official Abbreviations

MPX™ Official Business Word Abbreviations

Click below to review the most common business-word abbreviations for United States domestic mail.

| a | b | c | d | e | f | g | h | i | j | k | l | m | n | o | p | q | r | s | t | u | v | w | x | y | z |

The MPX™ Standard Listings fully incorporate the USPS Appendix `G' Business Word Lists, with the addition of other popular requests drawn from the 150,000-business-word mail-production.com/ Multiplex diagnostic file, and from our other active dictionaries of more than 830,000 name-, dictionary-, and addreviation-word trackings.

Visit our complete business abbreviation pages for current lookups for the most common standardized business-word abbreviations.

Disagree? There's no flaming permitted here! We are somewhat `stuck' with the longstanding decisions on abbreviations that have been `set' by our postal authorities; and we have endeavored to follow those patterns to help avoid confusion. But, if you want to ramp-up another opinion, email us at admin@cms-mpc.com and we will list your revised or new word and its abbreviation format on the internet pages as an alternative form, or new listing. Thanks for taking your time to help support this growing project. Your input is always considered important to us.

Any disagreements can be fairly resolved on the authority of smart people with words http://www.wordsmith.org/; and you may ask its editor if he has a spare moment to help decide the matter at the Editor's Address. Your request can be submitted to them, along with a suggested support contribution on your part, all in view of that organization's non-commercial standing and situation as the final word authority for this enterprise.

and hundreds more.

Foundation and its abbreviations throw many into a whirl. As likewise with

ASS
ASN
ASSOC
ASSN
ASSTS
ASSOCIATE
ASSOCIATES
ASSOCIATION

If your job requires the extraction of business titles from within names, and the correct isolation and standardization of them you have your hands full.

Also, the effective standardization of business names is a field unto itself and sometimes requires a wordwise ranking and restoration technique, with which we have many years of execution experience. We may have invented it, for all I know. But it's the only conceivable way to tackling some massive business name messes.

The subject of dual names is exhaustive, and if you permit them, they will come, to paraphrase a movie script.

There hundreds of variations of them, each requiring a judgement which is both difficult and fraught with dangers.

Could all those `Ma' ladies really be keyboard errors involving the position of the `s' key?

Mr&Ms Kathy and Tom Jones
DM Thomas Peters
Dr Peter S Thomas
Dr and Mrs Peter S T
Thomas Peters MD
MR Thomas Peters MD
Dr Thomas Peters /Family
PAUL FAMILY
PAUL FAM
FAM PAUL
PUAL
PAUL
THE PAULS
TOM/SUE TOMPSOM
SUSAN THOMASJR
SR SUS AN V THOMPSON JS
MR LI NWOOD WU BAD CUSTOMER
BAD CREDIT
PUT ON THE PORCH
SUSAN AN?????
MR SEENMORE ASS

Most input files contain many CHRISTOPHE names, as well as some CHRISTOPH, and CHRISTOP words too.

Some files have all the name data following column 30 truncated. Some quit having it right after 40, or some other point. All this must be analyzed and handled.

***Providing Solutions...***
*In summary* we will say there is no more difficult field in data processing than that of data hygienics. And within this field there is no greater opportunity for America's marketing personnel than `warming up' to what these processes entail--energizing them to put forth the services for customer use, having a complete awareness of the field's essential benefits. Until there is a better understand of the field itself, direct-mail users will remain reluctant to call for it, and to appreciated the skill-level requirements necessary to bring these processes into standard file bearing.	Because the field is endlessly elaborate and tedious, many list owners continually forego their entrance to its use; and this is unfortunate because it is a maintenance of a one-time exercise, and many-times benefit. Once a name and address are repaired to a top-flight condition, that mailing record stays set, and is used over and over, at no additional cost. Thought of in this way, we can see that data hygienics is an initial data-acquisitions step, but is thereafter an economical reward of long duration, and one for which there is no reasonable substitute.

Many of the examples herein were recalled from the author's recent memory. Had we used an actual production name-change report, you'd be flabbergasted and perhaps more concerned than ever over the presence of this kind of error content within all varieties of mail files. Ask for one upon the completion of your order.

But wait! We have decided to post some real examples. Take a look from your web browser.

I would advise every customer's finding out if its current data-cleanup provider is not, in fact, a pretender to the throne.

**mail-production.com**--Processing Services, Including...
Name Isolations Dual Name Isolations Salutations Creation Dual Name Salutations Business Word and Name Location Bank and Legal Status Words Removal Finance Upgrades Processes Check Digit and Match Key Generations Bar Code Generations Splits and Keys and Codings Postal processing BMC and SCF Destinations Processing Accurate Sort Divisions Advanced Address Correction Advanced Name Correction Data Security Services	Applications Programming Scientific Samplings Demographic Assignments File-Crossing Attributes Duplicates Identification Merge\|Purge File and Data Associations File Reports Generation Data Summaries Data Analysis Data Mining Data Enhancements Errors Detection Errors Correction USPS NCOA Updating USPS DPVS Updating

Direct Mail is today a $900 billion industry which employs nine million people and accounts for eight percent of the gross domestic product. The USPS alone leases 26,594

mail facilities and pays almost $2 billion in employee salaries and benefits every two weeks. Direct Mail Marketing is ``The Gateway to the US Household,'' and our system handles more than 40% of the world's card and letter mail volume, delivering more than 200 billion pieces of mail a year. The industry transports mail by airplane, truck, railroad, boat and even by mule to and from the bottom of the Grand Canyon, and pays more than $5 billion a year in air and highway transportation costs. An average letter carrier delivers over 2,300 pieces of mail a day to approximately 500 addresses.

Data Processing Expedite Acknowledgement