Big Data Has A Big Diversity Problem

February 8, 2016 Author: Mario X. Carrasco

Big data continues to be a central point of conversation in the market research world. Whether it is touted as the end of market research as we know it or the beginning of the new market researcher that weaves big data and custom research into a projection of the future, one thing is for sure, big data is here to stay.

The conversation in the market research world revolves around how big data in a silo is of no use to companies since it is a lot of chatter with no structure, and custom research coalesces this chatter into meaningful data that weaves a narrative with custom insights. This issue has been thoroughly explored in 2015 but I see a bigger issue that has yet to be tackled for 2016, diversity.

Let’s take a look at the main sources of big data:

  • Activity Data (smart phone collecting usage, credit card company collecting data where you shop)
  • Conversation Data (email, social media, etc.)
  • Photo and Video Image Data
  • Sensor Data (smart phone global positioning sensor)

This is by no means an exhaustive list as the list is growing every day, but these are the most critical sources of data for market researchers. If we were to hone in on the single most important big data point it would be credit card transactions. This would seem to be the holy grail of data for market researchers and marketers alike as it captures what people are buying and where.

However, a large piece of the puzzle is missing here and it comprises the second largest number of transactions at retailers, cash. Looking at the most recent Federal Reserve Payments study, debit cards and cash continued to account for the two largest shares of consumer payments in 2013 (31.1% and 26.3%, respectively). That means, as a whole, we are losing over a quarter of big data insights into transactions to cash payments.

Digging deeper into demographic transactions patterns, we see those making less than $25,000 annually use cash for a much wider variety of transactions than do those with higher household incomes. These consumers typically have a higher number of monthly cash transactions with a ticket size valued at $25 or more than does any other household income group. Specifically, those making less than $25,000 annually have an average of 5.1 such transactions a month versus 3.2 for households earning more than $25,000. Consistent with this, the total value of low-income consumers’ cash spending, at $558 per month, is much higher than any other income group’s cash spending. This likely is due, at least in part, to less access to banking and financial products. This finding is not surprising in light of various studies of "unbanked and underbanked" U.S. consumers, many of whom also fall into the lower-income category. For example, as a study by the FDIC found, low-income consumers use cash because they view most mainstream banking and financial products as too expensive or too difficult to obtain.

Big data is failing to capture a large amount of cash transactions. It is easy to assume that cash transactions are minimal and constitute small amount of transactions, yet the data points elsewhere as low income consumers are spending over 25% of their monthly income via cash transactions.

Big data’s diversity problem becomes clear when we look at preference of transaction type by ethnicity:

Demographics by Credit Preference

The stark difference lies in the preference for credit with Hispanics preferring credit only 6.7% of the time vs. 85.4% for non-Hispanic whites. Looking at African-Americans, the preference is even less, at 4.4%. This data points to the fact that the picture that big data is painting is lacking color.

It looks as though big data is going through the same issues market research at large did a decade ago. It is failing to capture one of the largest opportunities for growth in its insights, multicultural consumers. While the promise of big data is an exciting one, it is important to keep its points of weakness at the forefront as decisions made from this seemingly bulletproof source of information has some potentially dangerous holes.

This blog post was originally published on MediaPost - Engage: Hispanics