The prospect, a leading provider of big data analytics solutions, had identified a need to capture, curate, categorize and visualize the massive amounts of web usage data sets stemming from mobile service providers’ millions of subscribers in order to develop turnkey analytics offerings which its customers can utilize for cross-selling opportunities that exceed traditional business intelligence offerings.
Profile
Leading big data analytics solution provider (“Data Analytics Provider”)
Industry
Big Data, Subscriber Analytics
Deployment Requirement
An on-disk SDK deployment in a data analytics provider’s data center supporting extremely high volume URL query requirements associated with analyzing the web activity for tens of millions of subscribers.
“The ability to deploy a highly accurate URL database that had broad web coverage and very high volume URL query performance was extremely important to us and our service provider customers.”
– Managing Director, Data Analytics Provider
The Problem
The Data Analytics Provider needed to capture and interpret mobile service providers’ subscriber web activity for use with a big data analytics solution that generated subscriber-specific cross- and up-marketing in a near real-time fashion. The Data Analytics Provider identified numerous issues to overcome in order to effectively analyze the large amounts of web usage data conducted by the millions of the service providers’ mobile subscribers located worldwide.
In order to develop robust and scalable analytics solutions, the Data Analytics Provider required a URL database with excellent accuracy, broad web coverage and the ability to quickly categorize content for new active web traffic from mobile subscribers for use in identifying and predicting subscriber-specific behaviors and patterns for use in advertising, marketing and sales opportunities.
| The business insights provided by the Data Analytics Provider’s big data offering would help its mobile service providers’ customers: | |||
|---|---|---|---|
| Profile subscribers – the primary objective was to identify behavioral trends to help mobile service providers improve their offerings, optimize the pricing structure of their data plans and to tailor their marketing offers based on individual subscriber’s interests. | |||
| Identify and prevent network abuse – another objective was to identify and stop network abuse, such as spamming, conducting fraud, distributing malware or disseminating incriminating web content, even though specific terms of service may be in place to prevent subscribers from conducting this type of activity. | |||
| Streamline data usage – mobile carriers are constantly faced with balancing data usage and as a result many are turning towards tiered plans. The analytics intelligence solution would help monitor peak data usage hours and would allow plans to be balanced and streamlined to help ensure optimal connectivity. | |||
| Improve customer care and retention – a deeper understanding of how a mobile carrier’s network is used to access varying content types helps improve marketing offers and provides other insights to improve customer support to help retain subscribers. | |||
| Online privacy assurance – in certain areas of the world, mobile carriers are required to provide “do not track” mechanisms to ensure online privacy that can apply to the personal consumption of religious and political content, and can include violence, sex and hate content as well. | |||
| Real-time malicious website protection – another key objective was to provide real-time protection against malicious and compromised websites, particularly those hosting spyware, malware, botnets, phishing, fraud and other exploits. Also important was to “clear” legit websites within minutes after being cleaned. | |||
| Broad language support – dynamic content categorization support of the most popular languages was also identified as a requirement. | |||
| Lightweight API integration – a lightweight API was required that could be easily be integrated into the analytics offering to support high-speed URL queries. | |||
| Scalability – a solution that could accommodate a rapid growth of subscriber network traffic was desired. | |||
| Flexible pricing model – a pricing structure that mirrored the Data Analytics Provider’s business model was required and one that could adapt to subscriber-based, bandwidth-based or usage-based business models that the Data Analytics Provider had established with its service providers’ customers. | |||
| Custom URL category mapping – the flexibility to map the categories of the URL database to the Data Analytics Provider’s custom category sets was also a requirement, especially in markets where abiding to regulatory entities’ guidelines was mandatory. | |||
The Data Analytics Provider’s requirement was for a high performance, highly accurate URL database with broad web coverage and excellent content categorization capabilities, from a vendor with the ability to provide multiple, flexible pricing models to accommodate existing business relationships.
“Our service providers are laser-focused on finding any advantage for service differentiation, to increase margins, and reduce subscriber churn – and the ability to utilize accurate URL content categorization capabilities as part of our big-data analytics offerings provide them with this critical edge.”
– COO, Data Analytics Provider
The Evaluation
The Data Analytics Provider conducted a comprehensive evaluation of the zveloDB®, with specific emphasis on coverage, accuracy and performance for various types of web content, to include the detection of adult and inappropriate content and the real-time detection of malicious websites.
| The evaluation criteria included: | |||
|---|---|---|---|
| Accuracy – an evaluation of the accuracy of the zveloDB URL database was conducted, with an in-depth analysis of over 100,000 URLs entailing a wide variety of web content types and spanning some of the most popular languages. | |||
| Performance – an evaluation of the URL query performance of the zveloDB SDK was conducted using actual subscriber data usage stemming from North America, Europe and Asia-Pacific regions to ensure satisfactory query response times. | |||
| Integration – integration of the on-disk zveloDB SDK was assessed to verify that the deployment approach would satisfy the fast and high volume query speeds of the mobile analytics solution. | |||
| Adult/Porn Detection & Coverage – a test was performed of the zveloDB URL database for coverage of porn, sex, violence and other objectionable web content in various languages to help abide to online privacy guidelines in the key markets being tested. | |||
| Malicious website detection – an evaluation of zvelo coverage and detection of malicious websites, particularly those targeting mobile devices, was performed, using a range of proprietary and third-party feeds and blacklists. Specific tests were also performed that evaluated the speed at which zvelo revisited websites previously detected as malicious to determine the revisit logic as/when websites are cleaned up or taken offline. | |||
| Category Mapping – the Data Analytics Provider evaluated zvelo ability to provide the required mapping flexibility to a number of industry and custom category sets by testing a range of URLs for proper category responses. | |||
“The zveloDB provided the ideal combination of performance, accuracy and coverage and exceeded our expectations for each of our criteria.”
– CTO, Data Analytics Provider
Following the thorough evaluation, the Data Analytics Provider and zvelo agreed to a partnership for the integration and deployment of the zveloDB URL database in the Data Analytics Provider’s mobile analytics big data solutions.
Integration option one: zveloDB URL database deployed within service provider data center
Integration option two: zveloDB URL database deployed within the big data analytics solutions provider’s data center
Data Analytics Provider’s Solution in Action
The Data Analytics Provider’s application queries the zveloDB URL database running on servers deployed in the mobile service provider’s data center and, for new active web traffic, conducts asynchronous zveloNET® cloud queries for content categorization. Subscriber’s web traffic is categorized appropriately and incorporated into the Data Analytic Provider’s big data analytics application for subscriber-specific advertising, marketing and sales efforts.
zveloNET’s ability to harness the collective web activity of all of the users across all of zvelo’s customers provides the basis for the extremely high coverage of the ActiveWeb*. Each additional user increases the breadth of ActiveWeb sites visited and categorized, thereby further increasing the coverage and malicious website detection for all of the collective users.
Results and Benefits of zveloDB URL Database and zveloNET Cloud
The zveloDB’s 99.99% coverage of the ActiveWeb provided the required website coverage, accuracy and performance to keep pace with the flood of big data received and processed by the Data Analytics Provider’s big data mobile analytics solutions. zvelo flexibility with deployment options, category mapping and pricing also were factors for the Data Analytics Provider. The Data Analytics Provider further benefited from the real-time content categorization and malicious website detection capabilities providing protection from inappropriate, dangerous and malicious websites and other web-based threats. The Data Analytics Provider quickly integrated the zveloDB SDK and launched the big data mobile analytics solutions required by its mobile service provider customers.
“zvelo has been an excellent partner. Their business responsiveness has been exceptional and their products surpass our expectations.”
– CTO, Data Analytics Provider
*ActiveWeb – websites and web content visited by actual users.
