Tuesday, November 8, 2011

Data Mining in Banks and Financial Institutions

With the lattest news showing clients of large banks fleeing to smaller credit unions and local banks and as banking competition becomes more and more global and intense, banks have to fight more creatively and proactively to gain or even maintain market shares. Data mining is becoming strategically important area for many business organizations including banking sector. It is a process of analyzing the data from various perspectives and summarizing it into valuable information. Data mining assists the banks to look for hidden pattern in a group and discover unknown relationship in the data.

Banks which still rely on reactive customer service techniques and conventional mass marketing are doomed to failure or atrophy. The banks of the future will use one asset, knowledge and not financial resources, as their leverage for survival and excellence. Surprisingly, most of this knowledge are currently in the banking system and generated by daily transactions and operations. This valuable information need not be gathered by intrusive customer surveys or expensive market research programs. The only problem is that this storehouse of data has to be mined for useful information.

 Normally unmined and unappreciated, these terabytes of transaction data are collected, generated, printed, stored, only to be filed and discarded after they have served their short-lived purposes as audit trails and paper trails. Most data generated by the bank's information systems, manual or automated like ATM's and credit card processing, were designed to support or track transactions, satisfy internal and external audit requirements, and meet government or central bank regulations. Few are gathered intentionally and originally to generate useful management reports.

Current information systems are not designed as decision support systems (DSS) that would help management make effective decisions to manage resources, compete successfully, and enhance customer satisfaction and service.

Consequently, adhoc or even the most basic management reports have to be extracted excruciatingly from scattered and autonomous data centers or islands of automation that use incompatible formats. The results are management reports that are perennially late, inaccurate, and incomplete. Executive decisions based on these misleading reports can lead to millions of dollars in short and long term losses and lost opportunities and markets.

The tremendous increase in the power of information technology will enable banks to tap existing information systems, also known as legacy systems, and mine useful management information and insights from the data stored in them. This process can be done without the need to change the current systems and the data they generate. But before data mining can proceed, a data warehouse will have to be created first. Data warehousing is the process of extracting, cleaning, transforming, and standardizing incompatible data from the bank's current systems so that these data can be mined and analyzed for useful patterns, relationships, and associations.

The data warehouse need not be updated as regularly or daily as the transaction based systems. Data warehouses can be updated and mined as infrequently as the need for management reports and decisions dictate, i.e., monthly, quarterly, or on a ad hoc basis. Data warehousing and mining can run parallel with banking transaction information systems, without intrusion and interruptions.

What are the benefits and application of data mining in the banking industry? One of the earliest application of data mining was in retail supermarket. Mining the volumes of point of sale (POS) data generated daily by cash registers, the store management analyzed the housewife's shopping basket, and discovered which items were often bought together. This knowledge led to changes in store layout the brought the related items physically closer and better promotions that packaged and sold the related items together. The knowledge discovered also led to better stocking and inventory management. Retailers like WalMart have experienced sales increase as much as 20% after extensively applying data mining. Some frequently bought item pairs discovered by data mining may be obvious, like toothbrush and toothpaste, wine and cheese, chips and soda. Some were unexpected and bizarre like disposable diapers and beer on Friday nights.

 In banking, the questions data mining can possibly answer are:

1. What transactions does a customer do before shifting to a competitor bank? (to prevent attrition)

2. What is the profile of an ATM customer and what type of products is he likely to buy? (to cross sell)

3. Which bank products are often availed of together by which groups of customers? (to cross sell and do target marketing)

4. What patterns in credit transactions lead to fraud? (to detect and deter fraud)

5. What is the profile of a high-risk borrower? (to prevent defaults, bad loans, and improve screening)

6. What services and benefits would current customers likely desire? (To increase loyalty and customer retention)

Note that data mining does not start with a hypothesis that has to be proven or disproven. It is an exploratory process aimed at "knowledge discovery" rather than the traditional "knowledge verification". Knowledge verification DSS otherwise known as OLAP (on line analytical processing) would ask straighforward questions like "how many card holders defaulted this month compared to the same month last year?" or "how many of our ATM customers are also borrowers?" While OLAP queries are useful, they are not as insightful, powerful, and as focused as data mining queries, especially in preempting competition or preventing customer attrition.

The data miner does not have a priori knowledge or assumptions. The data mining software will usually reveal unexpected patterns and opportunities and make its own hypothesis. Data mining will be the cornerstone of the competitive if not the survival strategy for the next millennium in banking. Banks which ignore it are giving away their future to competitors which today are busy mining.