Automation, Algorithms, Predictive Models and All That

Earlier this week, I was one of the speakers at a panel that discussed how automation, algorithms, predictive models, and related technology have changed our lives.

The event was kicked off Christopher Steiner, author of Automate This: How Algorithms Came to Rule Our World, who talked about some of the ways that algorithms are changing our lives, ranging from high speed trading to medical diagnoses.

In addition to myself, the panel included:

  • Rayid Ghani, Chief Scientist of the Obama for America 2012 campaign
  • George John, Founder and CEO, rocketfuel
  • Keary Philips, Allstate Insurance Company
  • Rishad Tobaccowala, Chief Strategy and Innovation Officer, VivaKi

Rayid Ghani spoke about some of the ways that predictive analytics was used to help persuade some of those who may not have voted to actually register to vote and later to show up at the polls and to vote.

I discussed that as important as algorithms are, they are sometimes best thought of in the context of: i) what are the Concepts and abstractions used to model the problem; ii) what are the Algorithms used to compute with these abstractions; iii) and what are the Devices that the algorithms run on? CAD for short.

It is interesting to look at big data from the perspective of the concepts, algorithms and devices. We are better at predictive analytics today not just because we have better algorithms, but also because we have made significant progress on the concepts and abstractions that underlie predictive analytics and on the devices we use.

For example, 20 years ago with big data and predictive analytics, the focus was on building a single statistical model and looking for knowledge; we generally used regression algorithms to analyze data; and we used high end workstations for the computations. Today, with big data, we tend to think of collections of models (ensembles, cubes of models, etc.) and focus the actions (not the knowledge) that are possible; we would more typically use algorithms that compute trees or support vector machines; and we do computations over clusters of workstations.

There is more about CAD in Chapter 1 of my book on the Structure of Digital Computing.

This entry was posted in analytic models, big data, Blog. Bookmark the permalink.