The Convergence of Network Intelligence and Business Intelligence

Believe it or not, I am not the only one to blog on the topic of Network Intelligence (NI). Several analysts at Yankee Group have posted entries on this topic: see http://blogs.yankeegroup.com/category/network-intelligence. As a matter of fact, Yankee have been pioneering the concept of Network Intelligence in the telco space and I have had some good discussions with them. They just published an interesting report entitled “The Convergence of Network Intelligence and Business Intelligence”, with Jon Paisner as the main author, and co-authors David Vorhaus and Sheryl Kingstone. The document describes the similarities and synergies between Network Intelligence and Business Intelligence; I highly recommend it to all those interested in Network Intelligence technology as an enabler of smart pipe solutions. It also confirms the key role played by Network Intelligence-gathering technologies such as deep packet inspection and its various evolutions for protection, optimizing and monetizing network information.

Jerome

Breaking the high speed barrier

I have recently had several interesting discussions with solution vendors who are faced with the serious challenge of making their systems work at multi-Gbps line rates. To break this “high-speed barrier”, they can either 1) spend considerable time and resources redeveloping their products to work at higher speeds, or 2) optimize their solutions with the help of Network Intelligence.

The challenge is especially acute for vendors in the mobile data environment, where bandwidth is growing at an exponential rate. Vendors who sell (for example) mobile service quality monitoring need to continue offering the same level of functionality at speeds in excess of 1 Gbps. However, it is not as easy to do well at high speeds what they used to do well at lower speeds… Of course, some vendors may be tempted to reduce functionality in order to handle the increased bandwidth, but this is not an option : it is clearly not an acceptable solution for mobile operators and their clients.

So how can Network Intelligence accelerate existing mobile solutions?

As we all know, mobile data traffic is literally exploding, due to the combination of new iPhone-type handsets, new broadband infrastructure (3G, HSDPA, HSUPA) and attractive pricing schemes. At the same time, video streaming, P2P, and social networking applications consume a large percentage of this traffic, pushing bandwidth at the core of mobile networks to multi-Gbps. This creates a need to generate useful data at workable bandwidths, and only retain essential information (e.g. IP metadata or IPDRs) for lower priority services (P2P, etc.).

Example 1: TroubleShooting

In order for customer service to respond to customer complaints, mobile operators often need to check service quality for certain subscribers. In this case, a Qosmos probe can filter selected IMSIs and forward only relevant traffic to an existing troubleshooting solution, at manageable speeds. No change is necessary to the existing solution.

Example 2: Subscriber Knowledge

Mobile operators carry massive amounts of data traffic from services outside their walled-gardens. This traffic can represent more than 90% of total traffic (see this Webcast for a concrete example). To optimize networks and service offerings, MNOs require better network intelligence to answer questions like: what mobile devices are being used (PC/handheld)? Which applications drive data growth? What are user the behavior patterns? All these questions can be answered with Qosmos Network Intelligence technology.

Example 3: Lawful Intercept

Beyond mobile solutions, similar challenges exist in the area of Lawful Intercept, where many intercept probes have not been designed to handle very high bandwidth. Here again, Network Intelligence technology saves the day by processing the high-bandwidth raw traffic and carrying out optimized dispatching of information to an existing lawful intercept system that can continue to process efficiently.

In summary, Network Intelligence technology can be used to enable existing solutions to continue performing efficiently even at very high speeds! Even though total bandwidth exceeds several Gbps, the traffic selected for forwarding can be handled easily by existing applications. The benefit for solution vendors: by adding a layer of Network Intelligence, they can avoid costly development and continue to sell existing solutions even as bandwidth grows exponentially!

I should also point out that our approach is different from “load balancing” provided by some suppliers, since Network Intelligence implements smart traffic filtering based on deeply embedded criteria (e.g. IMSI, MIME type, email sender, etc.), and can generate CDRs even for discarded traffic.

Jerome

The Perfect Storm

We have all heard and read about the dramatic increase in mobile data usage. For me it became real when I talked to an employee at a mobile operator in Europe. He described as a "perfect storm": the combination of flat rate tariffs + the deployment of new infrastructure (3G, HSDPA, HSUPA) + new "iPhone like" phones have created an explosion in mobile data traffic in his country: + 50% over the past 3 months! He worries that his network would reach maximum capacity within 6 months...

Not only is he running out of bandwidth, but he is also blind: 98% of the subscriber traffic bypasses his own portal and goes directly to the (mobile) Web; he can't see the traffic. The key questions which are keeping him awake at night: - are people using mobile phones or PCs with 3G cards? Which services are used? What is the typical usage pattern?

For him, the solution was to install a Qosmos ixMachine behind the GGSN. The traffic data is extracted in real-time from the network, structured into .CSV files and fed into a BI software (QlickView in this case) which generates detailed reports. This creates complete visibility on devices (phones/PCs), services and usage patterns. Armed with this detailed network intelligence, he is now able to adjust pricing schemes, develop new services, optimize network traffic and even envisage partnerships with selected Web players.

I suspect most mobile operators face similar storms - luckily network intelligence technology is now available to help them ;-)

Jerome

The Power of Metadata

Metadata is information about communication: who communicates with whom, how, when, and where. For example VoIP caller/called party, Email / Webmail sender / receiver / subject, IM contact list / status / sender / receiver, route update in routing protocol, etc.

With the exponential increase in IP communications, metadata has a great untapped potential for mapping communication patterns, especially for protection purposes as in the case of lawful intercept. It also solves the problem of ballooning storage requirements. In fact, the metadata approach may even be the ONLY way to handle lawful intercept in the future!

So the opportunity is to leverage metadata for intelligence gathering, for ex. to reconstruct links between people, to understand which virtual IDs the same person is using (starting from physical IP/IMSI) or to identify intentionally hidden information (“the Dark Web”).

However, IP communication metadata is not readily available on operator servers, or resides on third-party Web servers, outside the control of a given telco (think P2P,social networking, etc.). Therefore, metadata has to be extracted directly from the network.

So how can buyers of LI solutions leverage IP metadata?

1. Use the rich set of information available with IP metadata to build completely new types of intelligence solutions - as a complement to content-focused solutions. This gives a better understanding of potential threats (the macro-view)
2. Take advantage of more computer-based information processing 1.5 billion Internet users, thousands of Web applications… the number of analysts cannot increase at the same rate as the number of IP communications, the number of Web applications and the amount of content generated > need to think differently?!
   

The solution is to implement automatic detection of suspicious behavioral communication patterns and create real-time view of the threat situation based on continuous streaming of metadata and network information. There is also an opportunity to minimize storage and post-processing time by extracting significant information and structuring it as soon as each metadata is available

Long live metadata!

Jerome

Network Intelligence: a key trend in IT networking

I took advantage of the Christmas break to catch up on some reading, like this article on Forbes.com that a friend sent me. It was interesting to see that the author Dan Woods cleverly identified a key trend in IT networking: networks aren't just highways!

As Dan puts it, "This week the JargonSpy looks at the gradual transformation of the network as transport, to the network as a source of information. As the network comes alive as a center of intelligence, the way that many applications do their jobs will change for the better."

It is great to see that Network Intelligence is becoming a discussion topic even in mainstream business media!

Wishing everyone a happy new year,

Jerome

A Network-based Approach to Enterprise Compliance

This time, I'd like to talk about a new, network-based approach to Compliance. All kinds of enterprises need to comply with regulations such as SOX, PCI DSS, HIPAA; at the same time, they absolutely want to avoid costly upgrades and maintain service.

The challenges are many: What information should be stored? What upgrades arerequired? Where can the information be found? Is it accessible and meaningful? Are different information sources compatible? Is the information complete? reliable? tamper-proof ?...usable?

If you've ever had to sort out several jigsaw puzzles that have been mixed together and scattered about your home, you quickly realize the impracticality of log-based solutions to such challenges. And this is where a network-based approach comes in.

Qosmos network intelligence technology queries an IP network in real-time as though it were a database. This enables you to see in real-time what actually happens, wherever it happens, and to record the information in a single data format (e.g. SMTP for emails), and with a single database entry for each session. And you can do all this from one point on your network: it doesn't matter what your existing IT system is, and there's no impact on existing management policies, systems or services. Furthermore, Qosmos filters and structures relevant data to optimize storage and speed up post-processing, and extracts information directly from the network, independently of servers, and even in cases where they do not have access to (third party) logs or databases (such as for Webmail or public IM).

Judging from some of my recent discussions, it is clear that Qosmos has an interesting value proposition: we can work in partnership with suppliers of Compliance solutions, to provide them with information extraction components on which they can build complete systems – with all the advantages of a network-based approach!

Jerome

Time for Change

You've certainly caught some of the buzz generated by the recent Obama - Verizon controversy.

It's clear that telecom operators’ databases contain sensitive customer information, ranging from subscriber phone numbers and addresses, to private bank details, call history and geo-localization data. And you don't need to be president-elect to understand that preventing the theft or misuse of this confidential information is of critical importance. Knowing who accesses what, where, when and why are the cardinal points of confidence in the information age.

Data protection represents, therefore, both a strategic and technical challenge for operators, who must implement a secure and actionable tracking system across the various access points to their subscriber databases: from internal customer relations and technical support teams, to outsourced customer care services, to malicious hackers.

At Qosmos, we like to say “Yes we can”. Qosmos Network Intelligence technology enables Data Protection and Data Theft Prevention solutions that monitor network traffic in real time, and extract complete and detailed access records for all intranet, web-based or legacy (e.g. TN3270) applications, whether standard or custom, and whatever network protocols are used. Which means fully traceable access, real-time reactivity, and rapid post-processing of information.

So that when government agencies and lawyers come knocking, you've not only got the answer - you know who else does...

Jerome