The Potential for Machine Learning: Thoughts from the Cassandra Summit


I had the honor of presenting at the Cassandra Summit in Santa Clara, California, earlier this fall as an expert of machine learning in financial services. I was among many distinguished professionals, including big data developers, data scientists, system architects and more in the field of banking and financial services industries. With the rise of big data, many of the participants came from prestigious and innovative financial institutions. In my time there, I encouraged many in the financial sector to consider the option of incorporating machine learning as a way to improve fraud detection at key levels of any transaction.

Understanding the distinction of transaction fraud

During my presentation, I pointed out that there is good reason for machine learning to be of particular importance to financial institutions. Big data technologies like Cassandra give you the ability to store and process very large amounts of data. What institutions find challenging is what to do with all that information. Machine learning gives you the brains to process all that data, in particular for fraud prevention. Interest is high for the technology, for these organizations are regularly subject to payment fraud. The Association of Financial Professionals’ most recent survey found that 62 percent of all financial organizations experienced payments fraud attempts in the last year. There are distinct challenges that banks, credit unions and financial service companies face when fighting fraud at the transactional level. The four characteristics that distinguish the tracking of fraudulent transactions from other illicit commercial activity are:

  • Real-time activity. Fraudulent transactions happen live at any time. Commercial activity can occur at any hour, especially with e-commerce, and fraud is no different.
  • Adversarial motives. The instigators of fraudulent activity aren’t competitors you expect to follow the rules of engagement, but criminals who purposely cause harm by breaking those rules and the law. They do so by actively circumventing the system.
  • Long-tail damage. Many attacks are so-called “corner cases” that individually do not cause a lot of damage but can create harm over time as they stack. Additionally, they’re far too costly to catch through human-based intervention.
  • Omnichannel sources. Hackers and criminals will attack using the same avenues as regular law-abiding customers. The questions then are, how can you tell the difference between fraudulent and regular activity, and how do you prevent false flags from happening?

With all these pain points, you would think that there is coordination in fraud prevention. However, there is often conflict between the technology used and the people using it, usually fraud analysts and customer service representatives. You want to prevent as much fraud as possible by being more aggressive on flagging and stopping transactions while at the same time not disturbing or annoying genuine customers. Legacy fraud-prevention systems suffer from a lack of balance in this respect. Their rules are either too aggressive or too lenient, which translates to too many false alarms or a lot of fraudulent activity, respectively. .

Looking at patterns of behavior

It is in this conflict between security and simplicity that machine learning comes into play. Machine learning is essentially pattern recognition and involves finding distinct signatures that deviate from the norm within the data automatically, and adjusting its responses in kind. The concept contains three core elements:

  1. Profiles. These are the individual, segmented behavior signatures of unique entities, including merchants and customers.
  2. Models. This is the representation of aggregated data of payment patterns transformed into structural adaptations and predictions of behavior.
  3. Explanations. A semantic layer, this turns all this data and analysis into descriptions that anyone can read without resorting to machine logic from the whitebox.

Without these mechanisms in place through automation, it would take weeks or months to sort through a database to find a pattern, if it’s found at all.

After machine learning establishes a normal pattern in fraud detection, it can reliably predict if a new transaction is fraudulent or not. From there, a feedback loop develops that improves learning over time, enabling identification of previously-unseen patterns.  This process helps fight adversarial criminals who constantly seek new ways to outsmart systems.

Ensemble methods: The wisdom of the crowd

I feel that where machine learning shines is the ability to incorporate multiple models to provide reliable information. Much like the collective wisdom of a team can make better decisions than an individual’s opinion, a machine that utilizes different algorithms or collections of decision elements that work in parallel can overcome singular biases to achieve superior performance. The ensemble method enables the views of different prediction patterns to combine, showing a clearer picture of the situation. This is extremely helpful in the fraud prevention environment, since you’ll be able to train the initial models in far less time than you would training humans.

While machine learning remains a relatively new concept to many financial institutions, its potential to mitigate and prevent fraud is worthy of everyone’s attention. My hope is that in presenting its capabilities at the Cassandra Summit, banks and financial services companies came out of it interested in seeing where it fits in their business models.