“Don’t Be Creepy” And Other Lessons from Last Night’s REAL MACHINE

Last night, a slew of AI enthusiasts came together at Wells Fargo in San Francisco for a panel discussion on AI. It was the latest installment in Feedzai’s REAL MACHINE event series. Sitting on the panel were Petros Zerfos (the Next Generation Applications Group at IBM), Pedro Bizarro (Chief Science Officer of Feedzai), and Eric Greene (Lead AI Architect at Wells Fargo).

The theme of the night had to do with the question: Why is this a special moment for AI? “AI is not new,” said Pedro Bizarro. “It’s from the 1950s. The difference is that now, for the first time, AI is attainable and affordable. We have clients who are using AI like they are turning on the lights.”

With this theme comes another. “Once your capability gets stronger, your security needs to get stronger, too.” That was Eric Greene speaking. Pedro would later add: “We are seeing an arm’s race. The fraudsters are employing this tech themselves, so we need to have it, to be more clever, organized, and centralized than the bad guys. It’s something we need to prepare for.”

In the Q&A portion of the evening, an audience member, who turned out to be a top data scientist with Uber, challenged Pedro’s assertion that fraudsters are employing machine learning algorithms themselves to conduct fraud. “Do you have a use case of a criminal using machine learning? Because I have not seen that myself.”

Pedro said that he did not have a smoking gun. But he did describe a peculiar instance, where a European tourism website hosted an online vote where users could select their favorite city. Pedro said that the data from this online contest suggested that someone was rigging the voting using machine learning.

“The fraudster was using machine learning to mimic the time a human might take when browsing websites, before proceeding to this website to vote on a city. And if they are doing that on a small tourist site, imagine where else they are doing it. These fraudsters are hiding in plain sight.”

Another theme, funnily enough, was humans. It turns out that we’re not obsolete just yet. Petros said, “This is about extending and accelerating human cognitive power. Not replacing it. Educating users about this fact is important to drive adoption. The aim is not to remove the humans. On the contrary, humans are necessary to provide supervision, and leverage the power of AI. The human will always be in the loop. They have the final decision.”

One thing a human can do better than a machine is not be creepy by mistake. Eric said, “People are comfortable with having systems look at their data to make better predictions. But you have to be careful not to make it seem like a creepy experience.” Pedro gave an example. “The user doesn’t mind a god-like AI system if it’s curing cancer. But the user does mind if that god-like system is following him around the supermarket to change prices and ads according to personal data. So there’s always the context. What we want to do with the system depends on how powerful it should be.”

After the panel discussion, I went and chatted with one of the audience members, a data scientist at a large corporation. “It sounds like we’re at a real inflection point,” I said. “It will be cool to see what happens with AI in the next few years.” He corrected me. “It’s not happening in the next few years. It’s happening now.”