Retail in Asia

In Trends

Speaker in the spotlight: Managing the Big Data

"Opportunities multiply as they are seized." Sun Tzu, The Art of War. It is true for many things. However opportunity is nearly limitless when it comes to the amount of information that retailers are able to acquire about their customers and the different devices those customers use to shop with. The information is already present in its simplest form on a receipt when items are bought. The opportunity is gained when those receipts are collected and patterns are established.

Tim Young, VP Netezza Big Data Appliances of IBM Asia Pacific, spoke about such in his presentation "Profiting from the Wealth of Customer Data Collected in Multi-Channel Retailing" at Retail Asia Expo’s retail technology seminar yesterday. Young shares with Retail in Asia some highlights of his presentation and talks about the methods and resources available for retailers to improve sales and maximise profit.

RIA: What technology challenges are Asian retailers facing in managing consumer database nowadays?

Tim Young (TY): Retailers face what has become one of the greatest technology challenges of the last decade: Big Data. Big Data is a loosely defined term that describes data sets that are so large and complex that they become difficult to work with using traditional database technologies. The Big Data challenge exists for several reasons including: the rise of social media and the opportunity this creates for retailers who can extract meaningful information from it such as buying intentions or complaints; the prevalence of electronic commerce and the ease with which data can be collected about customer behaviour, such as web clickstream analysis or video surveillance; the desire of retailers to deeply mine this data in order to understand and predict customer behaviour in order to identify appropriate actions.

Big Data is at the heart of effective analytics such as market basket and proximity analytics and as Retail Management increasingly becomes a science rather than an art, getting on top of this Big Data challenge is a priority for today’s retailer. IBM describes Big Data in terms of the volume, velocity and variety of data.

RIA: How to manage these challenges?

TY: While Big Data is a technology challenge, technology also provides the answer. Data warehouse appliances and Apache Hadoop are two such technologies. Data Warehouse Appliances are purpose built data analytics engines that combine server, storage and database into a system optimised for high-performance query analytics against huge volumes of data. In 2010, IBM acquired data warehouse appliance pioneer, Netezza, but all major database vendors, including Oracle, EMC and Teradata, have data warehouse appliances as part of their portfolio.

Apache Hadoop provides an open source software framework that supports data-intensive analytics applications under a free license. Originally based on work conducted by Google and Yahoo to cope with their own Big Data analytics challenge, Hadoop is being used (or at least investigated) by hundreds of IT organisations. IBM offer services, server technology for Hadoop as well as software innovation (IBM InfoSphere BigInsights) that makes Hadoop enterprise ready, such as enhanced security and administration features.

RIA: What top tips have you shared with retailers at your presentation "Profiting from the Wealth of Customer Data Collected in Multi-Channel Retailing" at Retail Asia Expo’s retail technology seminar?

TY: The presentation introduces the concepts of Big Data, outlines the technology challenges faced and also offers a pragmatic approach to address Big Data analytics without ignoring the considerable investment that retailers already have in their IT infrastructure. While it would be nice to start with a clean sheet of paper, few retailers have this luxury, particularly in the global financial crisis. The tips shared are based on real-world experience and include enhancing the existing data warehouse infrastructure with data warehouse appliances, deploying Hadoop to address the challenge of unstructured data, such as Twitter and then using specialist technology from IBM – InfoSphere Streams – to extrapolate real-time insight from "data in motion".

RIA: Where do you see this database technology going in the next two to five years in multi-channel retailing?

TY: Big Data has led to resurgence in database technology and what was becoming a commodity technology 10 or 15 years ago, is now centre stage in the technology discussion. The Big Data trend means that databases must not only evolve to cope with huge volumes of data, but they must also now perform complex analytics instantly that allows the retailers to take the next best action while the customer is captive. Appliances combine server, storage and database technology in a massively parallel processing architecture designed for high performance analytics against vast volumes of data. Embedding complex retail analytics into the database is also a key trend. Embedded analytics allows complex models (for example, those that predict buying intentions based on consumer behaviour) to be created and then executed routinely within the database as part of normal business operations. This precludes the need to move data into specialised analytics tools and allows complex predictive analysis to be performed quickly and routinely.


Retail Asia Expo 2012 is a three-day event held from 12 to 14 June, at the Hong Kong Convention and Exhibition Centre.