Login
Industries
Unique opportunities, challenges, and benefits across Financial Services, Retail & Distribution, Government & Public Sector, Communications, and Media & Entertainment industries
Big Data and Analytics
in Financial Services
Big data trends, risk performance and measurement, and regulatory compliance are the driving market forces behind business analytics investments in financial organizations. Moving to big data solutions is seen as a years-long journey, but many that see the opportunities are well on their way to transformation.
Financial firms are addressing the requirements of big data within the context of their traditional data infrastructures, but IDC expects this effort to grow in complexity as big data technologies mature.
More big data projects will be implemented as firms better articulate big data's business potential, identify ideal use cases, receive further buy-in from decision makers, and, most importantly, strategize and plan to incorporate and retain the necessary talent and skills for big data projects.
Looking at the reasons for adopting big data solutions, the two primary uses at this stage of maturity for big data technologies, according to survey respondents from the banking industry, are analysis of operations-related data and analysis of online customer behavior-related data (see chart).
Specific applications of big data technologies most common among financial organizations include the following:
Operational Data for Fraud detection and Investigations
Using third-party public and private data to develop and analyze entity relationships and discover hard-to-detect patterns of behaviors for internal and external fraud identification and investigation
Voice of the customer
Using social analytics to gauge sentiment toward key product and service initiatives
Up-sell and cross-sell with predictive analytics
Using geo-location data to create merchant intelligence and assist in optimizing offers and pricing to retail customers, driving mobile banking growth
Trend-spotting for investment bankers
Using services of private sector satellite companies to track parking lots and fill rates of major retailers (fill rates and trends inform investment research)
To make these applications work and reap the full benefits, as well as succeed generally with big data projects, financial organizations should consider the following guidance:
- Big data is not solely a technology issue. Financial executives must recognize that big data is more about how to use the growing velocity, variety, and volume of information to make material change and improvements in the way financial enterprises interact with customers, partners, regulators, and employees; manage risk and run efficient operations.
- Big data is a journey. The big data journey is unique to each organization, dependent on many factors, including the maturity of an organization's current data infrastructure, the type of business, and the availability of skills within the organization.
- Big data is an incremental process. Big data deployments need to start small and build out incrementally. Programs should be incorporated within business-as-usual activities and be part of an overall data management and analysis road map.
- Big data competencies need to be built within an analytics strategy. Analytics excellence is core to innovation across the financial industry. Business executives in the financial industry must view analytics as an important capability that will ultimately distinguish those that thrive in uncertain and uneven markets from those that fumble.
Big Data and Analytics in Retail
Opportunities for business intelligence and analytic insights abound in retail. They range from new product development and introduction, localization of assortments, pricing, and promotion to customer acquisition, retention, and lifetime value, offer orchestration, and optimized operations. Retailers have pursued these opportunities in earnest for over a decade and at a quickening pace—aided and abetted by packaged application software, purpose-specific business warehouse appliances, and data management tools.
Still for most retailers gleaning actionable insight from sales, customer, product, and market data remains elusive. Only 35% of retailers report that their investment in business intelligence and analytics have paid off. Common challenges include high technology costs, lengthy design and deployment time, insufficient computing resources, lack of IT and analytical skill sets, poor data quality, and complex data integration.
Enter big data. Despite its attendant buzz, big data remains the land of the unknown unknowns. Only about one in 10 of IDC survey respondents can identify big data's common characteristics — the "three V's" that are volume, velocity, and variety.
Volume.
While retailers are no stranger to handling large amounts of data, notably POS transactions and customer records, social data and streaming data from scanners and sensors now present petabyte- and soon exabyte-scale challenges. Social media data alone is growing at a rate of 10.2PB annually.
Velocity.
High-volume data is by definition hyper-fast data, accelerating the clock speed of retail analytics.
Variety.
Big data is diverse, ranging from new sources of structured data such as social network data to unstructured data types in product descriptions, emails, ratings and reviews, and the semantics of Facebook updates and blog posts.
Twenty-eight percent of retailers with big data initiatives underway say they've yet to encounter any challenges. Yet, in reality, gleaning insight from big data presents the same BI challenges mentioned above. Moreover 27% of retailers struggling with the three V's are challenged in finding and retaining relevant data, and 13% cannot garner line of business support.
There's also a fourth V in big data — value. Big data can be monetized in the same dimensions delivered by traditional BI investment, with potentially much higher returns in product and service innovation, optimized operations, pricing and promotions, localized assortment, customer acquisition, retention, and lifetime value, and hyper-personalized marketing dialogues and orchestration of offers that match each customer's circumstances.
Bottom line: the potential for big data aligns with the business initiatives that drive IT investment — increasing revenue, reducing costs, improving products and services, and earning customer loyalty and vocal advocacy amplified in social media. Toward this end, retailers are most likely to point to the analysis of online customer behavior data as an important driver for their big data initiatives.
The path to monetizing the potential of big data requires enhancing a mix of competencies in data, talent, technology, and process intentionally coordinated through strategy, sponsorship, and justification. Maturation of these dimensions from beginner to expert stages will move the goalposts of big data efforts from data analysis and explanation to market leadership and sustained competitive advantage. Guidelines along the way include understanding that big data is a journey and incremental process within an overarching business-driven analytics strategy and not solely a technology issue.
Big Data and Analytics
in Government
Big data represents a big opportunity for government executives. Government data continues to be generated and digitally archived in increasing rates because of technology adoption, including the rapid growth of mobile devices and applications, smart sensors and devices, and citizen-facing portals. The use of mobile-device applications, and participation in social networking sites, is driving the creation of unstructured data, as well as fueling citizen expectations of access to immediate relevant information and government interaction.
The fact is, government organizations must explore and invest in big data technologies or risk being overwhelmed by the challenges of data management and analytics in the intelligent economy. Indeed, according to IDC Government Insights, agencies already report that they are "swimming in sensors and drowning in data." Couple these challenges with the fact that there are many government decision makers with different drivers when it comes to deploying big data.
While some government entities have invested in high-performance computing, widespread adoption of big data technologies outside of high-performance computing is relatively nascent. However, some segments of government have begun to invest in big data hardware, software, and services to access, store, track, model, and analyze information. For example, the Defense Advanced Research Projects Agency (DARPA) Insight program has executed a field test and demonstrated end-to-end system functionality and the capability to perform sequence-neutral (i.e., out-of-order) fusion of data from multiple intelligence sources, as well as graph-based fusion of these sources.
The field test also produced a unique, multimodality, high-fidelity data set that is available to researchers across the Department of Defense (DOD) and the Intelligence Community. This data set, combined with the foundational data set collected in the fall of 2010, provides an unparalleled 135-terabyte resource to more than 240 users across government, industry, and academia.
IDC Government Insights expects government spending on big data solutions to expand rapidly. What is different now is that systems that were previously affordable and available only to the largest government agencies are now more affordable because of a combination of open source software and decreasing hardware prices. Government organizations that have begun to embrace big data technology and approaches are demonstrating that they can better serve mission-making decisions by taking action based on timely, relevant, complete, and accurate information.
Capital cost reduction.
A reduction in software, hardware, and other infrastructure costs
Operational efficiency.
A reduction in labor costs and an increase in employee productivity due to more efficient methods for data integration, management, analysis, and delivery
Mission enhancements.
Easy and timely retrieval and analysis of related and unrelated information, which is crucial for government to meet and improve mission requirements such as protecting war fighters, providing national security, serving citizens, ensuring public safety, and providing human/social services.
As government leaders across the spectrum strive to become a data-driven organization to successfully accomplish their missions, they are laying the groundwork to correlate dependencies across events and track dependencies across people, processes, and information.
Agencies also need to understand the underlying computational resources required, and plan accordingly to accomplish their mission successfully.
Big Data and Analytics
in Communications
Communications service providers (CSPs) are investing in big data and business analytics software, tools, and technologies because of the pressing need for customer, operational, and network insights. For many CSPs, data is everywhere but actionable insights are nowhere to be found. CSPs generally recognize the many benefits that can be gained throughout their organization by deploying big data solutions. However, not all CSPs understand that big data is a critical, multiyear journey, and one that must be started immediately. In the communications industry, information is emerging as the new currency of business, and the basis for true competitive differentiation, innovation, value creation, growth, and risk mitigation.
Accordingly, CSPs need to begin the big data journey now by investing in solutions that will assist them in managing customers and services throughout their respective lifecycles. From customer acquisition to support and bad debt management, big data solutions can enable CSPs to reduce both operational and capital expenditures and increase revenue and profitability. Some of the use cases and potential benefits CSPs can enjoy by harnessing the latent value of big data include the following:
For big data projects to succeed, CSPs should enlist the assistance of a trusted partner with the expertise and capabilities required to enable the organization to fully capitalize on all of the potential benefits that can be derived from big data. Specifically, CSPs will need to identify the top-priority use cases, work with key stakeholders throughout the organization to tap into available data sources and, most importantly, strategize and plan to incorporate new business insights into the very fabric of the organization and throughout virtually all essential business processes and departments.
IDC expects CSPs to embrace the many potential benefits that can be achievable by leveraging big data, and forecasts that communications industry spending on big data and business analytics software, services, and hardware will grow over 10% from 2012 to $7.3 billion in 2013, clearly demonstrating the investment CSPs are expected to make in big data solutions.
Big Data and Analytics
in Media and Entertainment
In the consumer-driven industry of media and entertainment, customer retention is as important — and sometimes even more important — than new-customer acquisition. Having the right information can make a big difference between success and failure of customer service, as well as of sales and marketing efforts. That’s one reason why the trend toward evidence-based decision-making has taken root in media and entertainment organizations.
Other drivers, too, such as increased competition due to changing business models, deregulation or, in some cases, increased regulation in the form of new compliance requirements, are prompting media and entertainment companies of all sizes to adopt big data technologies and services to either automate or support decision-making processes. These organizations are applying big data solutions across three key areas of the media and entertainment business:
- Customers
- Advertising
- Digital supply chain
In the area of customer analytics, companies are using big data technologies to improve forecasting accuracy and refine pricing decisions based on price-sensitivity analysis to better understand customers’ behavior. By using advanced analytics to target its marketing campaigns more precisely, one media and entertainment company saw a return on investment of 200% to 300%, with the best-performing customer segments realizing 50% overall campaign performance improvement.
The company was also able to respond to changing market dynamics more quickly using event-based marketing (EBM) tools, which are software-based rules and triggers that digitally detect, analyze, and interpret customer activities, transactions, and interactions. For example, the company could react faster to customer requests thanks to text analysis capabilities. Incoming mail categorization work was reduced by 70%, while the error rate for more than 50 different categories was lowered to less than 5%.
Advertising analytics enable companies to gain deeper insight into consumer behavior on branded social media applications. Media and entertainment companies are able to monitor and optimize branded social applications and social media campaigns, gathering large amounts of information about user behavior and the effectiveness of various engagement tactics. Advanced analytics let companies develop segments for online targeted advertising for multiple online properties.
Perhaps the most exciting application of big data for media and entertainment companies is managing the inventory and distribution of content in the digital supply chain. Companies are using enterprise-wide digital asset management systems to ensure that profitable content is always available and being delivered to customers, as well as to track ad placement, effectiveness, versioning, and usage rights. Companies are also using big data tools to better manage content inventory by eliminating or bridging gaps among siloed content repositories.
In one media company, for example, the relevance of publication topics is tied directly to the latest industry “buzz,” but determining what that buzz is required pouring over billions of records compiled from blogs and job postings. With a big data solution, queries that once took 10 hours to run are now completed in only 6 minutes. This has enabled the company to run more analysis cycles in significantly less time, identify rising trends and hot topics, and deliver more relevant, timely publications to market.
The bottom line is that the potential for big data in media and entertainment organizations aligns with the business initiatives that drive IT investment — increasing revenue, reducing costs, improving products and services, and earning customer loyalty while reducing churn.