Network Intelligence Technology

Deep Packet Inspection: don’t mix-up content inspection and network analysis

2011-09-16T15:05:00.000+02:00

Deep Packet Inspection (DPI) is a term widely used in the cyber security area, but which has two different meanings depending on the context where the DPI is used, whether in the content inspection function or in the network analysis function.
Because I see a lot of confusion in the market between the two functions, I thought it would be useful to bring some clarification.

Deep Packet Inspection is a technology used to inspect packets circulating over the network by not only looking at the headers, but also in the packet payload. This being said, you can look in the packet payload to find different nature of information.

1) Content inspection: in this context DPI is used to look for virus or malware signatures that could be embedded in flows (packets, email or documents received by a user). The DPI will look for specific patterns and match it against a list of known malicious patterns. This is done using pattern matching algorithms and regular expression functions.
2) Network Analysis: in this context DPI is used to identify protocol and applications used on a network. This requires pattern matching, but also more complex protocol grammar analysis and statistical analysis. The advanced form of DPI will also extract metadata from flows, like sender an receiver of an email.

So we see that DPI is used to fulfill 2 different functions. They are complementary functions, and there is no sense comparing the features and performance of a Content Inspection engine and a Network Analysis engine even if they both use Deep Packet Inspection.
An advanced cyber security product design should embed both: Network Analysis to enable application aware firewall and normalized content extraction; content inspection to seach for virus signature in a given content normalized by Network Analysis.

The chart below shows the difference between the 2 categories of DPI implementation.

	Content inspection	Network Analysis
Method	DPI: Inspect the Content of the packets/flows and not only the headers
Objective / features	Detect 100k’s of virus/file signatures inside documents	Recognize & analyze protocols and Applications Fully decode a protocol to export metadata
How it works	Lexer: Detect patterns / regular expressions	Parser: Multiple algorithms used such as pattern matching, flow correlation, behavior analysis
Implementation	Can be software (PCRE, Sensory Networks) or hardware (Tarari, Netlogic NetL7)	Software only (e.g. Qosmos ixEngine)
Found in	IDS/IPS/AV	Next generation Firewall, NBAD, Forensics

NetWitness and ipoque Acquisitions Show That Network Intelligence Technology is Becoming Crucial

2011-05-26T16:52:00.002+02:00

Recently, there have been some interesting movements among vendors who leverage DPI and network intelligence technology.

Last month, EMC acquired NetWitness: http://netwitness.com/about/press-releases/2011-emc-acquires-netwitness-corporation. Netwitness is a leading network security analysis vendor who has been using DPI inside their solutions. According to the following article in Network World http://www.networkworld.com/news/2011/032411-emc-netwitness.html, EMC’s strategy is to strengthen RSA’s enVision SIEM offering with "an additional source of network activity" and "another angle on analysis.” This confirms the need for real-time network intelligence for cyber security applications. Will this lead to further M&A in this sector, involving companies like Solera or Niksun? We’ll see. In any case, I think SIEM vendors (Arcsight, Q1 Labs, LogLogic, CA, etc.) will take a closer look at DPI and Network Intelligence technology to enhance their products.

This week, Rohde & Schwarz announced the acquisition of ipoque:http://www.ipoque.de/news-and-events/news. ipoque sells mainly vertical solutions such as their PRX traffic management solution (with DPI inside), but also supply their Protocol and Application Decoding Engine (PADE) to vendors in an OEM model. It will be interesting to observe the strategic evolution of the ipoque product portfolio as part of Rohde & Schwarz’s offering: will they decide to increase focus on QoS solutions? Will they continue to sell PADE?

I think we will soon see more consolidation among vendors who leverage DPI and Network Intelligence technology. These are interesting developments – stay tuned.

Jerome

Is your DPI technology battle-proof?

2011-04-22T09:50:00.003+02:00

This is a question that becomes more frequent among product managers and CTOs that I speak to. And surprisingly this a topic on which there is very little information. Probably because it is a complex topic, that requires deep expertise. But you need to be aware that Deep Packet Inspection engines, like any systems, may be circumvented or blocked by malicious actions, or rendered inoperable by extreme traffic conditions.

The discipline of Deep Packet Inspection is not always an exact science. If you run the exact same traffic through 3 different brands of DPI equipment, you will get 3 different results. Why is this?

This result of a DPI analysis will depend on:

1)Deliberate actions to hide using the numerous opportunities given by non-standard, complex, decentralized network; for example people may use tunnels, or change the shape of their packets in order to by-pass a DPI system designed to handle only “normal” packet shapes. Some DPI systems may detect this behavior, some may not. Also, deliberate attacks on servers may alter the way a DPI engine performs even if the engine itself is not targeted; how would your DPI engine perform during a SYNFlood attack?

2)Accidental causes deriving from traffic conditions, configuration and bugs in network devices, mis-configured networks etc. For example a server configured in a “byte by byte mode”, would send the “GET” method used in the HTTP protocol in 3 different packets (one for G, one for E and one for T). But a traditional DPI engine would look for the “GET” pattern into a single packet, which means it is unable to detect the HTTP protocol. And this is just a very basic example of use case where basic DPI is ineffective. Here again, some DPI systems have been designed to cope with malformed traffic, some cannot.

The good news is that this not inevitable. Because reverse engineering protocols and applications means working in real-life traffic conditions, decoding both standard and malformed traffic, there is always a solution to accurately detect each networked event. But this requires considerable investment in building DPI software which is resilient, robust and reliable.

Many DPI engines not to pay sufficient attention to this topic, which could result potential performance and security issues. This is obviously a key concern for cyber security applications that could be weakened, but also for all applications that require accuracy and data quality such as charging or parental control.

Working on resiliency, robustness and reliability is an ongoing effort, and should be top of mind for Deep Packet Inspection developers and product managers.

Jerome

Security within an evolving Internet

2011-02-08T15:55:00.003+01:00

This time, I have invited a guest blogger: Pierre Françon , who is the president of Quaelys and a respected IP security expert. Pierre describes the new security challenges created by the coexistence on the Internet of both IPv4 and IPv6.

The Internet, as we know it, is based on IPv4, considered as homogeneous and open. Communications are established end to end. The only exception is the NAT (Network Address Translation) feature, used and controlled on the equipment of the subscriber/end customer (ADSL Box).

The depletion of public IPv4 addresses is accelerating. Therefore the Internet is going to evolve in the very short term (even if some Internet Providers may adopt slower migration then others). We will see a dual Internet based on two similar protocols but still incompatible (IPv4 & IPv6). Technically, the customer will be given two parallel communication channels being usable simultaneously on two independent networks of network at IP level.

The historical way of using IPv4 addresses end to end will cease. Instead Internet Providers will use NAT on their own network. We are talking about Carrier Grade NAT or CGN. This method will request new application gateways for protocols carrying IP addresses such as VoIP-SIP for example. More, the IPv4 traffic collected from the customers ADSL boxes to the CGN will be encapsulated on IPv6 using a method named Dual Stack Lite.
From the security point of view, besides the introduction of IPv6 and the CGN complexity, the real breakdown comes from this duality notion ... and how someone can use it for this own benefit.

This duality is twofold: (1) First, one has at his disposal two simultaneous and separate communications channels on the same technical environment (LAN and Desktop). (2) Second, from both channels one can join simultaneously the same servers or networks, infrastructures ... and targets (CGN, routers, cache, applications servers, desktops...)

In this new context, IPv4 and IPv6 cannot be treated separately. Risk analysis has to take this duality into account, as there is not a lot of IPv6 experience and because it has weakened the IPv4 world (when dual stack). Preventive security mechanisms must also be dual. For example, a spammer using one channel has to be black listed on both channels. It’s not easy as the protocols are different and because the user identification methods are not identical: prefix (full subnet) allocated to the subscriber in IPv6 versus the IP address and port numbers per protocol in IPv4 (the same IP address is shared on the CGN between many subscribers).

In parallel, legal requirements on this new Internet are far more complex. To detect illegal usage, the subscriber behaviour has to be analysed within the duality IPv4/IPv6. Similarly, filtering subscribers or dual web sites becomes more complex: allocated IP addresses based filter versus filter of the user identity authentified at the application level. Gathering evidences of illegal usage can become a big problem: Just imagine a P2P dual tool, where the contents research is made partially on IPv4 sessions and IPv6 sessions when the traffic is routed end to end without NAT.

Confronted to these new challenges, we have to rethink the security of data exchanges, communication infrastructures and end-equipment (servers or desktops). In parallel, putting in place traffic/flow and behaviour analysis on a dual basis requires new tools taking into account the diversity and sophistications of the Internet usage. To summarize, it is really urgent to think and act differently towards in the face of the evovling Internet.

Can Network Intelligence Technology Lower the Risk For Cyber War?

2011-01-25T17:41:00.005+01:00

I just read the latest article about Stuxnet: The Triumph of Hacker Culture - http://www.slate.com/id/2281938

Here is a quote: “The implications are vastly unsettling. If a Stuxnet-like worm can disable Iranian nuclear manufacturing controls, there is reason to be concerned that a similar or more highly evolved worm (devised by the much-feared Chinese military cyber corps, perhaps) could seize control of our nuclear missile launch-control capacity. Maybe not yet. But the potential can't be ruled out.”

Scary…

For those of you who haven’t followed all the details about Stuxnet, the common theory is the following:

Israel + US developed Stuxnet in order to delay Iran nuclear weapons program, since it was deemed less risky than bombing raids
Stuxnet is seen in cyber sec / SCADA circles as the first offensive, state-sponsored, weaponized malware of a new generation
The fear is that the Pandora box is now open, and that adversaries will retaliate in kind

See here for a Wired article: http://www.wired.com/dangerroom/2011/01/with-stuxnet-did-the-u-s-and-israel-create-a-new-cyberwar-era/

Some people believe that China could be behind Stuxnet: http://blogs.forbes.com/firewall/2010/12/14/stuxnets-finnish-chinese-connection/

In any case, I think we will see more focus on SCADA cyber defense.

What does this mean for Network Intelligence Technology?

Even the new generation weaponized malware uses IP networks to spread itself and communicate. In the case of Stuxnet, "Updates to this executable would be propagated throughout the facility through a peer-to-peer method established by Stuxnet." See http://www.zdnet.com/blog/security/stuxnet-a-possible-attack-scenario/7420?tag=rbxccnbzd1
At Qosmos, we are experts at decoding traffic. If we don’t recognize a protocol, it would be classified as “unknown”, which in itself is highly suspicious in a sensitive environment. A cyber defense solution can be configured to block all such traffic instantly.

Seems that Qosmos can provide the traffic visibility required for defense against new generation malware. It is our way of lowering the risk of cyber war.

JT

Cyber security artists need the right tools

2010-11-18T12:16:00.004+01:00

It has become evident to me that cyber security is an art, and like any other art, it has artists who need the right tools.
At Qosmos, we work with cyber security teams who protect very sensitive networks. These security analysts typically work in a Security Operations Center (SOC), monitoring traffic and checking for suspicious activity, such as:

Services or encrypted traffic on non-standard ports
Referring URI, which can be used to detect Phishing software loading partial content from a real site
Many (hundreds) of “IP gets” from black-listed countries
Specific malware file names (e.g. shell.exe)
Suspicious malformed traffic

Best practice cyber security filters out known threats with COTS cyber security products (AV, Firewalls, etc.) and focuses investigation and analyst time on 1% suspicious traffic only. So, what tools do the analysts need?

Information feeds in the form of logs and traffic metadata
Search and analysis capabilities

Logs are the obvious source of information to investigate potential security breaches. But a recent trend is to complement these logs with communications metadata, representing an additional source of real-time information.

Examples of communications metadata which are relevant for cyber security:

The advantages of metadata:

Not only do good metadata complement logs, they are also MORE valuable than full packet payloads to identify patterns! As someone said to me: “sometimes, you can’t see the forest (situational awareness) for the trees (packet payloads)”
In addition, metadata require less storage than full packet capture which means that historic info can be kept for longer time periods (months) than full packet capture: this means much stronger investigative capabilities.
Metadata also enables much faster forensic search, with the ability to search 2TB of data in less than 2 minutes!
Finally, metadata can be used to index flows and packet contents

Example of a best-of-breed cyber security tool case
A tool case can be built on Qosmos + Splunk. In this case, Qosmos does the protocol decoding up to Layer 7, providing complete visibility of all network traffic and applications, independently of ports. The extracted protocol metadata is indexed by Splunk in addition to log information. Splunk is then used for search, statistics and GUI.

Example of Searching for Suspicious Network Activity by using Qosmos + Splunk

Let’s give cyber security artists the tools they need to exercise their art!

Jerome

Network Intelligence: Coming Back to Qosmos

2010-09-08T17:11:00.001+02:00

It is interesting to see the increasing interest for embedding network intelligence software into solutions. At Qosmos, we have been speaking to network equipment suppliers, ISVs and systems integrators for years. Which means that many of them have known us for years. However, during the initial discussions, many of them tell us that they have DPI skills internally, and while Qosmos technology is really impressive, they don’t need to source externally. “No problem”, we say, “but don’t hesitate to contact Qosmos if you change your mind”.
We now see more and more companies coming back to Qosmos.
Why do they come back to us? The reasons are simple:

They find it increasing difficult to keep up with ever-changing protocols and applications
They face challenges in scaling existing solution to network speeds beyond Gbps
Resource constraints force them to focus all their energy on their core business (which is typically to build solutions, not enabling technology like DPI or Network Intelligence)

This is typical for new high-tech markets: initially, high-tech vendors will build everything in-house, because 1) it’s not too difficult and 2) there are no external suppliers. Think of databases: initially, all IT vendors built their own databases in-house (for example IBM DB2). Then vendors moved to source database technology from specialists like Sybase, Informix or Oracle. Same thing for micro-processors, which was initially developed internally by computer vendors, but is now sourced from specialists Intel and AMD.
There is now a similar trend with DPI and network intelligence technology: the market is shaping up for the benefit of everyone.
Welcome back : we are happy to work with you!

JT

The Dilemma of Government Cyber Security: Ensuring Strong Protection While Keeping Costs Down

2010-08-31T14:32:00.000+02:00

Imagine the following situation facing teams responsible for government cyber security: cyber threats are increasing in both numbers and complexity; and at the same time, government budgets are under pressure. Difficult equation…

Government IT teams in all countries are required to use more Commercial Off the Shelf (COTS) products in order to keep costs down. This may be OK for the majority of IT services and equipment, but when it comes to cyber security, it doesn’t always work. Relying only on COTS cyber security products can jeopardize national cyber security, since the features and capabilities are publicly known. This means that adversaries can devise attacks to circumvent COTS cyber security solutions.

Here is how Qosmos solves the dilemma:

1. We represent a “COTS traffic decoding component” which keeps costs down

2. We are NOT a specialized cyber security technology (only a traffic decoding technology) which means that government teams can keep their defense capabilities strong and confidential

Equation solved!

More info here: http://www.qosmos.com/sectors/government/cyber-defense

Network Intelligence Technology Experts Qosmos Comment on Intel’s Acquisition of McAfee

2010-08-20T16:05:00.002+02:00

Move Underscores the Need for Visibility into Data in Motion at ALL Levels

Intel today announced its acquisition of McAfee. Intel CEO Paul Otellini said on the conference call: "We believe security will be most effective when enabled in hardware."

Qosmos sees a broader picture for better secured systems across the technology value chain – enabled by better visibility into active data, regardless of where the data is at any moment. And the network is the converging point to access this intelligence.

Most companies lack the appetite and capital for such acquisitions, but will nonetheless require technologies that enable visibility into the path and content of data transiting networks. For such specialized expertise, there is Qosmos. Qosmos, the expert in so-called “network intelligence technologies,” provides software and hardware components that embed inside applications, equipment and networks to capture, extract and identify data in motion.

According to Qosmos CEO Thibaut Bechetoille, “In today’s network-dependent economy, this acquisition underscores the critical need for greater visibility of active data across the technology spectrum – whether in hardware and processors, in the networks themselves, in the systems that manage them or in the applications that run with real-time data – to enable more secure and better performing solutions.”

Technically speaking, Qosmos technology provides visibility and data extraction at unparalleled depth (up to and including layer 7), speed (with throughputs of up to 80 Gbps) and detail (recognizing 300+ network and application protocols and extracting more than 4,000 metadata elements).

In plain speak, technology providers including software vendors, systems integrators, developers and equipment manufacturers use Qosmos components inside their solutions to make them more secure, better performing and better monetized by having the detail to see patterns and aberrations that would otherwise be invisible.

Qosmos experts and executives are available to discuss why such visibility is critical and why network visibility – network intelligence – is THE keystone to improved security.

iPhone 4 = more network congestion

2010-06-30T14:51:00.000+02:00

Last week it was iPhone frenzy here in Paris, much like other places around the world. (maybe we are trying to forget about the French soccer team). See this photo showing customers queue up in front the Carrousel du Louvre shopping mall, waiting to buy the iPhone 4.

The new iPhone is work course much better and cooler than previous versions. One thing the iPhone 4 does better is video: it records HD video at 720p and 30-frames per second. Which means more need for mobile bandwidth.

Sounds like déjà vu all over again? Hum… let me think… reminds me of a blog post I wrote about the iPhone 3: http://networkintelligence.blogspot.com/search?q=iphone&x=0&y=0

Long live network congestion!

Jerome

QED: expert help to embed network intelligence into your solutions

2010-06-10T11:00:00.001+02:00

During discussions with prospective partners, we sometimes get comments like: “we love Qosmos, but right now we don’t have any engineers available for new developments based on your technology”.

This is why Qosmos just formed a network of developer partners, the Qosmos Expert Developers (QED).

The idea is to facilitate the development of applications based on Qosmos technology, with a network of partners who are experts on ixEngine. Qosmos expert developers serve as an extension of in-house resources and make it easier to embed Network Intelligence Technology into new solutions. These companies are experts in high-performance, multi-core network/security processing platforms and real-time architectures, and they have a track record of successful development based on Qosmos.

For our customers, this means on-demand access to best-in-class development expertise, faster time-to-market and quality assurance for new Qosmos-based solutions.

The first core group of QEDs include:
- Moore Performance Systems (USA)
- Mantaro (USA)
- MasterPeace Solutions (USA)
- DeCanio Engineering (USA)
- Philog (France)
- Bigsool (France)

We now have an easy answer for those of you who need a little extra help to use Qosmos technology!

Jerome

The Network Intelligence market is taking off

2010-05-27T16:46:00.002+02:00

Brian Partridge, VP at Yankee Group, just published a new Anchor Report called Network Intelligence Is Key to Profiting From Anywhere Demand.

It is an interesting document, which clearly explains the key role of network intelligence (NI) for billing, charging, revenue assurance and bandwidth management. In particular, Brian points out that NI is becoming a key enabling technology, as illustrated by the following excerpts from the report:

“In the past, network equipment and solution vendors have typically developed their DPI technology internally, but today we observe a major trend toward outsourcing. Providing effective NI technology is an expertise in itself, and network equipment vendors can no longer afford to do it in house.”

“By using proven tool kits from third-party specialists, vendors can improve development efficiency and time to market of new products and keep existing products updated in terms of feature enhancements such as new protocol support.”

At Qosmos, we see also beyond billing and bandwidth management: our daily activity shows that NI is also becoming crucial for other applications such as market research or cyber security.

Just more proof that the market for network intelligence technology is taking off!

Jerome

Network Intelligence across the telecom value chain

2010-05-12T18:15:00.000+02:00

Today, I just want to highlight a webcast sponsored by Qosmos and presented by Brian Partridge, Vice President of Enabling Technologies at Yankee Group.

This webcast describes how telecom solution vendors can leverage Network Intelligence Technology for three critical areas of the telecom value chain:
• Billing & Charging
• Revenue assurance
• Service assurance

It is interesting to get the Yankee views on network intelligence and the similarities with our own vision. And no, we didn’t censure any of Brian’s slides ;-)

Jerome

Staying under the radar

2010-04-29T14:23:00.002+02:00

The following article by the Register caught my attention: "NHS computers hit by voracious, data-stealing worm"

The interesting thing is that Qakbot is a known malware, well-documented by Symantec. Therefore, COTS Anti-Virus product should catch it, right?

As described on the Register, Qakbot spreads through Web pages that install malware by exploiting patched vulnerabilities in Microsoft’s Internet Explorer and Apple’s QuickTime software. It is able to self-propagate on local networks through file shares. It "moves slowly and with caution, trying not to bring attention to its presence" it is staying under the radar!

For some reason, the National Health Service (NHS) network was hit by a malware which has been known since May 2009… Could it be another sign that COTS cyber security products can be circumvented by advanced malware?

For me, this is just another confirmation that COTS security must be complemented by additional layers of custom-built cyber defense.

Jerome

Qosmos and Tilera: when 2 leaders join forces, developers win

2010-04-18T09:52:00.001+02:00

Qosmos has been working with processor specialist Tilera for some time. Our engineers had already optimized the way Qosmos DPI and Network Intelligence Technology is implemented on the Tilera TilePro.

Today we have gone a step further: we have designed a new DPI and network intelligence card (called ixBoard) based on the Tilera TILExpress-20G.

Why?

100% x86 CPU cycles are now available for the customer application, since all protocol decoding is offloaded on ixBoard
Customer can continue to develop applications under x86 while keeping the full benefits of the Tilera 10Gbps card

As an additional benefit, ixBoard also facilitates the work for product designers who are neither experts in multi-core architecture nor in DPI. They are free to do what they do best: develop complete solutions.

We use the 64 cores of the Tilera card to optimize performance and parse traffic in real time. Packets, content and metadata are extracted through the PCIe bus, and streamed as raw data to the buffer. This means that the host can use the data in any way and format. Application developers can remain in their familiar Intel/x86 environments, and the extraction and delivery of traffic data (at 10 Gbps) is transparent for them.

Today, most application developers come from the software world. They are not always familiar in network infrastructure (protocols and packets), they just need traffic metadata and events. Typically, it would take considerable time and energy for them to learn how to develop on multi-core processing, and also to develop the network analysis features needed.

Our initial customer feedback on the offload card is very positive, and the combination of two domain of expertise (Qosmos for DPI + Tilera for multi-core processing) save them a lot of time, and money, and headaches!

Jerome

Is Qosmos right for you?

2010-03-30T13:47:00.000+02:00

Sometimes people ask me: “how do I know if Qosmos products are right for me”?

So here are some simple questions to help you decide.

Question 1: Do you need detailed visibility of all network-based activity?

Beyond traffic classification > also traffic metadata?
Do you require absolutely accurate information?
At multi-Gbps speeds?

Question 2: Do you prefer to source DPI and network intelligence externally?

Want to focus internal developers on building complete solutions?
Looking for pre-developed building-blocks?
Need to shorten product development times and accelerate time-to-market?
Want somebody else to keep up with constantly evolving Web applications and protocols?

If you answered “YES” to these questions, you should probably check out our technology.

Jerome

Cap My iPhone? Try This Instead, Mobile QoS Vendors

2010-01-28T18:11:00.001+01:00

You have probably read about AT&T’s problems dealing with the perpetually clogged 3G networks in San Francisco and New York:
http://www.wired.com/epicenter/2009/12/iphone-caps/. To solve the problem, AT&T is considering one or all of the following: 1) convincing heavy iPhone users to stop using so much data (despite paying for unlimited plans), 2) introducing caps on data usage, 3) stop selling iPhones, 4) investing heavily in the network, 5) shut down streaming of live baseball games…

How can Qosmos Network Intelligence Technology help?

First, we can help mobile service assurance and QoS vendors. Not all solutions are not designed to support the huge throughputs generated by unlimited wireless data plans, which means that KPIs can no longer be computed on the entire traffic (as in the past). Instead, service assurance vendors must now select a panel of mobile users and analyze a representative sample to deduce QoS. They can use Qosmos probes to analyze only a portion of the traffic on mobile users IDs (IMSIs) and create a panel of representative users. For video, they don’t need to keep any of the content, but just identify that it is video traffic. The filtered traffic is then forwarded at bandwidths which are manageable for existing solutions. This means that mobile QoS vendors benefit from instant scalability and can remain operational even if traffic throughputs increase dramatically. In this case, AT&T can better optimize subscriber experience and iPhone users are happier!

Second, we can help suppliers of network optimization solutions. They can use Qosmos ixEngine to get full visibility of iPhone traffic and applications. This allows them to work with AT&T to optimize networks, prioritize applications and make many iPhone users happier. Bandwidth could be allocated in a more fair manner, so that heavy users don’t hog all the resources, and AT&T can optimize their investments in 3G network equipment.

Conclusion: you don’t need to cap my iPhone!

Jerome

Could you have used Qosmos to detect the Operation Aurora cyber attack?

2010-01-21T13:55:00.005+01:00

The short answer is: yes!

Let me explain.

A lot has been written about Operation Aurora, so as a reminder, let me just point you to the summary posted on Wikipedia: “Operation Aurora was a cyber attack, conducted in mid-December 2009 and originating in China, against Google and more than 20 other companies, including Adobe Systems, Juniper Networks, Rackspace, Yahoo, Symantec, Northrop Grumman and Dow Chemical”

How to protect sensitive assets against cyber threats

Governments and companies who have sensitive assets all use commercial off-the-shelf (COTS) solutions such as for anti-virus, anti-spyware, and intrusion detection systems. These systems provide effective protection against known vulnerabilities, but are not so good at protecting against new, unknown threats: so-called zero-day attacks. And Operation Aurora is a perfect illustration of this.

My experience shows that organizations who need advanced cyber protection must use two layers of defense:
- The first layer is built by COTS products and its main purpose is to filter out known threats
- The second layer of defense is a custom-built solution, developed by trusted cyber security teams to identify advanced, Aurora-type of threats. Qosmos technology plays a key role by feeding this solution with full visibility over network traffic.

You can read more about this layered cyber defense approach by downloading this solution brief.

How Qosmos technology could have been used to detect and mitigate Aurora

On the McAfee Labs Blog, I found a good description of the custom backdoor protocol used during Operation Aurora. Technically, the principle of the attack was simple: 1) a malware was installed on a PC by a Trojan exploiting a vulnerability in Internet Explorer, and 2) a covert connection was made on port 443 using a custom encrypted protocol, instead of the standard the HTTPS protocol encrypted with SSL.

In this case, a custom development based on Qosmos could have detected that abnormal traffic was flowing through port 443 and the system could have instructed to block the traffic, which would have stopped the attack.

Jerome

Similarities between Network Intelligence Technology and a relational database system

2009-12-17T12:48:00.000+01:00

Network Intelligence Technology aims at providing full visibility on traffic flows; it is in many aspects very similar to a relational database system which allows users to query large sets of data generated and computed by information systems. Similarities between these technologies are not limited to the technical aspect, they also follow the same business trends, but 30 years apart!

Back in the 60s, a database was specifically designed for each application because it was such a complex and cutting edge technology that only specific systems could afford it. This complexity limited the implementations to large systems including specific hardware to support the database. Databases were reserved for very large organizations.

In the 70s, the INGRES project introduced major technical enhancements such as the relational database, which made data more actionable. This project was the foundation of the first commercial products such as Sybase and Informix which enabled developers to build systems using standard database building blocks including a DBMS engine and a query language. With the availability of COTS database technology, software vendors could create applications for any type of business requirement and not only for very large organizations. SMBs could use financial systems to track their sales, the local library could use a database system to manage its book portfolio. Today, thousands of applications benefit from these database building blocks, and even very large systems such as ERPs are designed with standard COTS database products such as Oracle. Using a COTS database is more rational from a cost point of view and also reduces the time to market. Today any software vendor building an application will choose a COTS database and will not even think of redeveloping its own BD system like in the 70s.

I see the EXACT same trend with Network Intelligence Technology. The first implementations mainly focused around Deep Packet Inspections (DPI) took place in routers and in specific appliances using custom hardware. Applications where limited to very specific tasks such as P2P blocking and every vendor would develop its own “in house” network flow analysis.

But the complexity of IP networks with a growing number of applications and protocols makes it very complex to get a 360° visibility over network traffic, which confines applications to very high-end solutions and technically advanced systems usually managed by network administrators. On the other hand, solution vendors in the business of billing network usage, optimizing networks or marketing network services have a growing need to understand more in detail the behavior of network and do not always have the skills to develop in-house network intelligence. The emergence of specialized vendors of network intelligence technology, whose mission is to provide building blocks for solution vendors and network equipment providers, can be compared to the emergence of the COTS database companies. Providing ready-to-use network intelligence components which can feed applications allows any developer to use the gigantic amount of information computed by or travelling over an IP network.

Network Intelligence Technology is a fast moving market with new protocols appearing and being updated frequently. Many solution vendors and network equipment manufacturers realize that their core business is not to focus on protocol technology. Instead of developing DPI or network intelligence in house, they can now source it from a Network Intelligence Technology specialist, and benefit from deep expertise, in the same was as they would select a commercial database product :-).

Jerome

Lawful Intercept: Another Application which Requires Network Intelligence

2009-11-13T16:19:00.000+01:00

I have worked on IP networking technology for more than 15 years, and one thing is for sure: more and more applications require detailed network intelligence. Lawful Interception (LI) is one of them.

While standards have been well-conceived to ensure proper technical implementation and facilitate investigations by LEAs, implementing LI at a telco creates a couple of important challenges:

1. Today, most telcos implement only basic LI capabilities, based on router info (OSI layers 3-4). But this approach does not take into account all the latest trends with Internet access everywhere and new applications such as IM, social networking, MMOGs, etc. Today, LEAs must be able to make a clear connection between virtual Web identities (logins on Gmail, FaceBook, LinkedIn, msn, Amazon, Entropia, etc.) and physical locations (at home, at work, at a WiFi hotspot, at a friend’s, on the iPhone, etc.) in order to pinpoint suspects; in addition, LEAs must also intercept communication (such as IM) embedded in non-telco applications like WoW…

2. There is an incredible amount of data generated by apps like P2P, VoD, IPTV, etc. Here, the challenge is to reduce the storage requirements and speed up investigations by focusing only on person-to-person communication within the total traffic. Again, basic solutions are not able to extract the relevant information and therefore create unmanageable situations with huge amounts of data to store and lengthy post-processing.

For LI to be effective and serve its intended purpose, I believe we need a new approach based on passive LI probes which can be either a physical device or embedded software in routers. This is what we had in mind at Qosmos when developing our latest range of ixMachine probes specially designed for Lawful Interception. Hopefully, these probes will meet the challenges of LEAs and ensure that LI remains effective, even as technology evolves.

Jerome

Don’t procrastinate: process data on the fly!

2009-11-02T11:22:00.005+01:00

We all know that there is an explosive growth in both fixed and mobile bandwidth. But not everyone might be aware of the new challenges created by the huge volume of data flowing on the networks: storage costs and lengthy post-processing,

As with other human activity, it is much more efficient to process information in real-time than to first store it and then go back process it. A bit similar to people who make purchasing decisions on a daily basis and know how to keep within their budget limits. They avoid the hassle and time to balance their checkbooks on evenings and week-ends… and the bad surprises!

With network intelligence technology, there is no place for procrastination: it processes data on the fly and focuses on key data only (such as communications metadata), which makes it very efficient. This approach reduces the total amount information which needs to be stored (100 time less!), provides actionable information immediately AND speeds up any subsequent post-processing (since the data has been nicely indexed). This real-time, network-based approach is well-suited for applications such as VoIP fraud detection or cyber defense, which required immediate action.

So, what is the difference between using data in motion vs. stored data?

Data in motion:

Real-time processing of network traffic
Minimal storage and post-processing
Can be used to get a dynamic understanding of relationships between pieces of information (think “video”)

Stored data:

Post-processing of data from logs
Requires extensive storage and post-processing
Difficult to get a dynamic understanding of relationships between pieces of information (think “photo”)

Conclusion: use network intelligence technology to process data on the fly!

Jerome

Solving Unsolvable Problems: The Solution

2009-10-20T12:10:00.000+02:00

Solution vendors are realizing that network intelligence requires advanced expertise and that there are clear advantages of sourcing the technology from specialists. The economic downturn accelerates the movement; vendors that come out as winners are those who stay focused on their core business.

Qosmos is dedicated to network intelligence, identifying and managing the evolving and growing number of network communication protocols as well as capturing traffic metadata or content. This real-time visibility enables enhanced security features, optimized technical performance and more precise usage in third-party solutions. Delivered as software development kits and hardware that integrate seamlessly, systems integrators, ISVs and NEP’s rely on Qosmos for expert network intelligence technology while maintaining complete control over their solutions.

Qosmos users benefit in several ways:

1. Improve development efficiency

In high-tech, business success goes hand in hand with fast time to market, which in turn depends on development time. By using proven network intelligence toolkits, development teams can stay focused on bringing new solutions fast to market, able to adhere to more predictable development roadmaps in the face of the ever-changing network-based environment. Qosmos’ Software Development Kit includes fully documented developer tools, along with support and maintenance services designed to make solutions network-intelligent rapidly and efficiently. Qosmos efficiency is built-in to the technology itself: the same application building blocks can be integrated across different CPUs, NPUs and hardware platforms. In fact, the Qosmos network intelligence development kit can be ported to any type of modern hardware.

2. Focus on core competence

Sourcing an enabling technology externally allows solution vendors to concentrate all their efforts on the customer requirements of their solutions, instead of reinventing the wheel internally. In a perfect distribution of labor, network intelligence specialists such as Qosmos put all their resources on building the best network intelligence toolkits, providing unique expertise and pre-developed building blocks that may be integrated seamlessly and rapidly to empower third-party solutions. As an illustration of our technical know-how, we can extract network information by drilling down into 16 levels of protocol encapsulation.

3. Expand solution features

Finally, the use of Qosmos for specialized network intelligence provides more than efficient use of development resources, the depth and breadth of data and metadata extraction enables additional features that strengthen and expand the capabilities of the solutions. For example, cyber security solutions built on network intelligence technology can detect abnormal network behavior that is invisible to standard, commercial “COTS” products. In another example, network intelligence technology can be used as a front-end for Lawful Intercept to process raw traffic and efficiently dispatch only the relevant data to an existing solution, even at very high data volumes. Finally, sourcing network intelligence technology externally gives developers fast access to new protocols which can feed their solutions and expand their reach; Qosmos delivers a new batch of protocols each quarter.

Network Intelligence plays a key role in a network-based world; governments, operators and enterprises have a vital need to gather network intelligence from IP networks for protection, monetizing and optimizing purposes. There is an increasing need to understand network-based activity for a range of solutions such as lawful interception, cyber security, market research, network optimization, billing, and more. Without detailed network intelligence, these solutions will not continue to function adequately and their vendors will face serious business issues.

Many of the winners will be companies who chose to boost their solutions with expert Network Intelligence Technology – from Qosmos.

Jerome

Solving Unsolvable Problems (Part 4): Managing highly complex technology

2009-09-28T11:57:00.001+02:00

Since network intelligence goes beyond DPI in terms of the level of visibility, protocol management, attribute recognition, and information extraction, even companies who have incorporated DPI capabilities into their solutions will require a new level of expertise. For example, Qosmos engineers have developed a specific meta-programming language to build Webmail and HTTP protocol plug-ins. Specific techniques and tools must be developed for quality assurance and to make reverse engineering more efficient. This is an order of magnitude remote from the business of someone who sells complete solutions.

In most cases, a separate R&D organization must be created. But once committed, companies realize that development times are difficult to estimate, timelines are incompressible and that the skills are so specialized that it becomes nearly impossible to outsource parts of the development.

Jerome

Solving Unsolvable Problems (Part 3): Committing considerable resources, with uncertain returns

2009-09-17T15:02:00.003+02:00

If you choose to tackle a family of protocols (e.g. Webmails), you have to develop network intelligence capabilities for the most important protocols in this family, otherwise your solution will ineffective or incomplete (think of traffic optimization or cyber security applications). In addition, the total number of applications and protocols increase continuously (e.g. 50,000 applications are now available for the iPhone), and very few protocols ever disappear…

For a company whose core business is not network intelligence technology, this translates into high costs of entry and ever-rising investments. To make things worse, end customers of turn-key solutions may not appreciate the importance of continuous protocol updates and the amount of work required to keep the solutions current. This means that a solution vendor could end up investing considerable resources for which and end customer does not perceive high value and therefore may not be ready to pay…

Jerome

Solving Unsolvable Problems (Part 2)

2009-09-08T11:38:00.005+02:00

As I described in my previous post, I believe that solution vendors are facing a crucial decision point and are realizing that developing network intelligence capability internally would create a number of “unsolvable problems”.

Unsolvable problem number 2: Not being able to use traditional product development and management methods

The high-tech industry typically uses a structured approach for product development and management, with most key activities aligned around go/no go decision points and defined time lines. These processes are built to ensure that new products are delivered on time, according to specifications and with the adequate quality.

However, companies who chose to develop network intelligence technology internally quickly discover that the usual methods cannot be used. Web protocols such as Webmails change continuously without notice, which means that development roadmaps cannot be easily controlled. Development teams must be quick to react to new protocol evolutions and use reverse engineering techniques to update their network intelligence software.

This way of working is counter-cultural for many high-tech companies and can even be incompatible with the rest of the organization. From a business standpoint, it can even be unsustainable and unprofitable - unless you make network intelligence technology your core business, like Qosmos;-)

Jerome

Solving Unsolvable Problems (Part 1)

2009-09-03T17:23:00.006+02:00

For the past several years, Deep Packet Inspection (DPI) has been used by developers to get basic traffic visibility for applications such as traffic shaping and network security.

We are now at a key juncture of the market, where many solution vendors need much more detailed visibility into network-based activity. For example, for market research applications it is not enough to recognize generic http traffic; it is necessary to extract metadata such as name of visited Website, page content, time spent on visit, basket share, referent, etc. The same is true for applications such as lawful intercept and government cyber security, which demand extremely accurate and fine-grained information on communication flows in order to map exact communication flows and identify threat patterns.

So the key question becomes: - should solution vendors develop complex network intelligence technology internally or should they source it from a specialist?

Solution vendors realize that developing network intelligence capability internally necessitates a step-function in terms of investment and strategy. It requires considerable internal R&D resources, detracts from core business and poses a number of additional problems in term of organizational efficiency, culture, and return on investment.

During the next couple of weeks, I will describe the “unsolvable problems” facing solution vendors and highlight the advantages of sourcing complex network intelligence technology from a specialist like Qosmos.

The first “Unsolvable Problem”: Having to continuously redevelop software

Developing network intelligence software to handle a particular protocol is only the tip of the iceberg: you also need to invest continuously to update your software so it can handle new versions of protocols. Some basic protocols are easy to manage since they are relatively stable: IMAP, SMTP, POP, HTTP, etc. However, there is a vast number of other protocols which are proprietary and evolve at a rapid pace (Webmails, P2P, social networking, gaming, etc.).

For example, the Livemail protocol changed several times during 2008. And each time a new version is released, most of the network intelligence software has to be redeveloped, without much reuse of previous development. This requires a particular culture and special methods: reverse engineering, custom-made tools, fast reaction to protocols changes, etc.

Jerome

BBWF09 - DPI Workshop / Monday, 07 September 10:00 – 12:30

2009-09-01T17:36:00.002+02:00

Just a little heads-up to tell you that I will be a speaker at the upcoming Broadband World Forum which will be help in Paris 7-9 September. Il will participate in the workshop “Deep Packet Inspection: Technology, Promise & Controversy. What You Need to Know” on Monday, 07 September 10:00 – 12:30. It should be an interesting discussion, with views from solution suppliers, service providers and Network Intelligence Technology specialists (Qosmos).

For those of you who would like to see Qosmos Network Intelligence Technology in action, you are welcome to see a live demo on the exhibition floor: just come to our partner BreakingPoint’s Stand # 470.

I hope to see you there!

Jerome

Forbes article on Programming Innovation

2009-08-19T10:20:00.003+02:00

Dan Woods just published an interesting article on Forbes.com called “Programming Innovation”. He describes how Qosmos technology enables a new kind of market research for our partner GfK.

Market research is of course only one example of how Qosmos Network Intelligence can be used. I like Dan’s way of describing our toolkit as a catalyst for innovation across a wide range of applications: “Now, tens of thousands of companies potentially have access to a new form of information derived from the network”.

Based on my daily discussions with customers and prospects, I can confirm that the trend is real – I can’t wait to see tens of thousands of companies benefiting from Qosmos Network Intelligence ;-)

Jerome

COTS not enough for government cyber security

2009-07-22T13:52:00.002+02:00

I just read this interesting article by Nextgov, confirming that relying only on commercial products can increase vulnerabilities to cyber attacks: http://www.nextgov.com/nextgov/ng_20090618_3505.php. This is in line with my recent discussions with governments who use Qosmos technology to enhance their cyber security beyond COTS capabilities. Not only could COTS products contain hidden code; they often lack capabilities required by government security specialists for detecting certain threat patterns and/or they lack the flexibility to customize according to special government security policies.

Jerome

Scoble

2009-07-22T13:49:00.002+02:00

I came across an interesting post today: http://scobleizer.com/2009/05/29/kara-is-wrong-about-2010web/. Scoble’s point is there’s no “web3.0” but there are several key trends that are making the web different today.

Scoble:

The things that are happening are NOT just Twitter and search. Here, let me recount again what is making up the 2010 Web:

1. Real Time. Google caught the Wave of that trend today BIG TIME.

2. Mobile. Google, again, caught that wave big time Wednesday when it handed Android phones to everyone at its IO conference.

3. Decentralized. Does Microsoft or Twitter demonstrate that trend? Not really well.

4. Pre-made blocks. I call this “copy-and-paste” programming. Google nailed it with its Web Elements (I’ll add a few of those next week).

5. Social. Oh, have you noticed how much more social the web is? The next two days I’m hanging out on an aircraft carrier with a few people who do social media for the Navy.

6. Smart. Wolfram Alpha opened a lot of people’s eyes to what is possible in new smart displays of information.

7. Hybrid infrastructure. At the Twitter Conference this week lots of people were talking about how they were using both traditional servers along with cloud-based approaches from Amazon and Rackspace to store, study, and process the sizeable datasets that are coming through Twitter, Facebook, and friendfeed.

How is this relevant for Network Intelligence Technology?

Real Time: need to identify real-time data and metadata,
Decentralized: need to both manage infrastructure and maintain visibility and control over communication when you can’t “be there”
Pre-made blocks: this exactly what the Qosmos model - providing enabling technology to ISVs, SIs and developers. With ixEngine 5.0, we just announced broader protocol support and easier customization with the Protocol Plug-in API.
Smart: on the broadest level, Qosmos technology makes the network itself more intelligent and more of a “participant” in the information delivery process. In ixEngine 5.0, the Sessionizer organizes data flows in logical and flexible ways so that people can make more sense of the communication flying around the web.

It is nice to see that Qosmos not only captures the wave of the evolving web, but our technology is actually enabling this evolution!

Jerome

NI (Network Intelligence) – the next BI (Business Intelligence)?

2009-06-01T15:00:00.002+02:00

If you have been involved in data processing, this probably sounds familiar to you. Remember the 1980’s when the guys from the financial department had to ask database managers to program specific SQL queries into the DB2 accounting systems, and how difficult it was to get the right information, and how long it took? The result was that you had to wait for reports, or wait for program changes etc.

Why was it so difficult?

1) Programs and reports where hard-coded, build for only ONE type of use. Any other request was just impossible.

2) There was no way to access and use a database from outside the IT organization.

3) The financial controller could not program in SQL and the database manager did not know finance.

And this is why Business Intelligence has been so successful: it enabled non-programmers to access the information they need and create the report they want in a minute when it took weeks or months before… if even possible.

Now, people involved in the network world face the exactly same problem. Take the product manager from a Telco; he needs to gather detailed subscriber information to efficiently manage the portfolio of services and estimate future demand. The network can provide this information at a very high level of granularity, but how can our product manager get this information? He will go to the network administrator and ask him for the info. And the network administrator will go search for the answer inside the network equipment logs … and he will probably spend more time than he thought, ending up with partial results, because:

1) logs are complicated to use for anything else than troubleshooting, and they are not easily readable

2) logs only provide a partial view of events

3) logs have different formats when they come from different systems

4) how do you manage the huge amount of information???

And just like business intelligence opened up the world of databases to finance people, the network intelligence technology provides an actionable access to the enormous amount of information transiting through networks or computed by the network

1) Network intelligence probes are passive; they do not impact the production equipment in the network and can be access independently, without risk

2) Network Intelligence probes are not built for a specific use, and thus provide a simple access to all the data of the network, whatever the need

3) Network intelligence probes do not use logs, they scan the traffic in real time, providing the same data structure whatever the diversity of network equipments brand and models, making consolidation easy.

3) Network Intelligence probes allow users to select only the information they need, making data easily actionable

4) Network Intelligence probes feed reporting tools and enables non-programmers to access information, without going through the technical department.

The result: easy and fast access to the valuable information provided by traffic analysis. In the same way that business software solutions (e.g. ERP, CRM) today provide business intelligence (BI) capabilities, network solutions will provide Network Intelligence (NI) capabilities very soon!

Jerome

New Intelligence. A Smarter World

2009-05-14T15:07:00.005+02:00

IBM’s Smarter Planet website “New Intelligence. Smarter thinking about how we collect, analyze and use information.” has the merit of placing network intelligence at the heart of a multitude of activities we would not automatically associate with developments in the cybersphere.

Instrumented. Interconnected. Intelligent
We would all agree that the information explosion has permanently changed the way we experience the world: everything – and everyone – is leaving digital tracks. For IBM this revolution implies three-fold change: moving from the "World Wide Web" to a "Web Wide World" means becoming instrumented, interconnected and intelligent. Instrumented , because "sensors” of all sorts are embedded everywhere and in all areas of activity: cars, appliances, cameras, roads, pipelines, medical instruments, livestock,etc.) and interconnected via internet. … But as IBM recognizes, in the information age being instrumented and interconnected isn’t enough. It's got to be intelligent.

“With computational power in things we wouldn't recognise as computers, any person, object, process, service, or organisation can become digitally aware, connected and smart.”
(IBM's ”A Smarter Planet” web site: http://asmarterplanet.com/)

It is not only computational power, however, that makes for an intelligent world. But what makes aninstrumented and interconnected world an intelligent one?

For IBM: “People, machines, processes… are increasingly connected to systems that process data, and benefit from advanced analytics capable of turning data into real insight, in real time.”

This is the criticalpoint where information can be either unweildy volumes of unmanageable and unusable data, or truly valuable information that is instantly available and ready to use.

IBM's initiativesto promote a Web Wide World that is instrumented, interconnected and intelligent, are exactly in phase with Qosmos' vision for Network Intelligence. Put most simply, it's what connects connectedness to uses, possibilities and needs. Asthe IBM website underlines: “it can be a daunting task for any enterprise to sift through massive amounts of data, extract information and transform it into actionable knowledge.”

In my next blog, I'll be looking some more into IBM's vision for a Smarter Planet, and how Qosmos network intelligence can contribute to it.

Visit the qosmos web site at: www.qosmos.com
IBM's ”A Smarter Planet” web site: http://asmarterplanet.com/

Jerome

The Convergence of Network Intelligence and Business Intelligence

2009-04-28T17:06:00.007+02:00

Believe it or not, I am not the only one to blog on the topic of Network Intelligence (NI). Several analysts at Yankee Group have posted entries on this topic: see http://blogs.yankeegroup.com/category/network-intelligence. As a matter of fact, Yankee have been pioneering the concept of Network Intelligence in the telco space and I have had some good discussions with them. They just published an interesting report entitled “The Convergence of Network Intelligence and Business Intelligence”, with Jon Paisner as the main author, and co-authors David Vorhaus and Sheryl Kingstone. The document describes the similarities and synergies between Network Intelligence and Business Intelligence; I highly recommend it to all those interested in Network Intelligence technology as an enabler of smart pipe solutions. It also confirms the key role played by Network Intelligence-gathering technologies such as deep packet inspection and its various evolutions for protection, optimizing and monetizing network information.

Jerome

Breaking the high speed barrier

2009-04-17T15:54:00.018+02:00

I have recently had several interesting discussions with solution vendors who are faced with the serious challenge of making their systems work at multi-Gbps line rates. To break this “high-speed barrier”, they can either 1) spend considerable time and resources redeveloping their products to work at higher speeds, or 2) optimize their solutions with the help of Network Intelligence.

The challenge is especially acute for vendors in the mobile data environment, where bandwidth is growing at an exponential rate. Vendors who sell (for example) mobile service quality monitoring need to continue offering the same level of functionality at speeds in excess of 1 Gbps. However, it is not as easy to do well at high speeds what they used to do well at lower speeds… Of course, some vendors may be tempted to reduce functionality in order to handle the increased bandwidth, but this is not an option : it is clearly not an acceptable solution for mobile operators and their clients.

So how can Network Intelligence accelerate existing mobile solutions?

As we all know, mobile data traffic is literally exploding, due to the combination of new iPhone-type handsets, new broadband infrastructure (3G, HSDPA, HSUPA) and attractive pricing schemes. At the same time, video streaming, P2P, and social networking applications consume a large percentage of this traffic, pushing bandwidth at the core of mobile networks to multi-Gbps. This creates a need to generate useful data at workable bandwidths, and only retain essential information (e.g. IP metadata or IPDRs) for lower priority services (P2P, etc.).

Example 1: TroubleShooting

In order for customer service to respond to customer complaints, mobile operators often need to check service quality for certain subscribers. In this case, a Qosmos probe can filter selected IMSIs and forward only relevant traffic to an existing troubleshooting solution, at manageable speeds. No change is necessary to the existing solution.

Example 2: Subscriber Knowledge

Mobile operators carry massive amounts of data traffic from services outside their walled-gardens. This traffic can represent more than 90% of total traffic (see this Webcast for a concrete example). To optimize networks and service offerings, MNOs require better network intelligence to answer questions like: what mobile devices are being used (PC/handheld)? Which applications drive data growth? What are user the behavior patterns? All these questions can be answered with Qosmos Network Intelligence technology.

Example 3: Lawful Intercept

Beyond mobile solutions, similar challenges exist in the area of Lawful Intercept, where many intercept probes have not been designed to handle very high bandwidth. Here again, Network Intelligence technology saves the day by processing the high-bandwidth raw traffic and carrying out optimized dispatching of information to an existing lawful intercept system that can continue to process efficiently.

In summary, Network Intelligence technology can be used to enable existing solutions to continue performing efficiently even at very high speeds! Even though total bandwidth exceeds several Gbps, the traffic selected for forwarding can be handled easily by existing applications. The benefit for solution vendors: by adding a layer of Network Intelligence, they can avoid costly development and continue to sell existing solutions even as bandwidth grows exponentially!

I should also point out that our approach is different from “load balancing” provided by some suppliers, since Network Intelligence implements smart traffic filtering based on deeply embedded criteria (e.g. IMSI, MIME type, email sender, etc.), and can generate CDRs even for discarded traffic.

Jerome

The Perfect Storm

2009-03-11T18:37:00.001+01:00

We have all heard and read about the dramatic increase in mobile data usage. For me it became real when I talked to an employee at a mobile operator in Europe. He described as a "perfect storm": the combination of flat rate tariffs + the deployment of new infrastructure (3G, HSDPA, HSUPA) + new "iPhone like" phones have created an explosion in mobile data traffic in his country: + 50% over the past 3 months! He worries that his network would reach maximum capacity within 6 months...

Not only is he running out of bandwidth, but he is also blind: 98% of the subscriber traffic bypasses his own portal and goes directly to the (mobile) Web; he can't see the traffic. The key questions which are keeping him awake at night: - are people using mobile phones or PCs with 3G cards? Which services are used? What is the typical usage pattern?

For him, the solution was to install a Qosmos ixMachine behind the GGSN. The traffic data is extracted in real-time from the network, structured into .CSV files and fed into a BI software (QlickView in this case) which generates detailed reports. This creates complete visibility on devices (phones/PCs), services and usage patterns. Armed with this detailed network intelligence, he is now able to adjust pricing schemes, develop new services, optimize network traffic and even envisage partnerships with selected Web players.

I suspect most mobile operators face similar storms - luckily network intelligence technology is now available to help them ;-)

Jerome

The Power of Metadata

2009-03-09T23:32:00.001+01:00

Metadata is information about communication: who communicates with whom, how, when, and where. For example VoIP caller/called party, Email / Webmail sender / receiver / subject, IM contact list / status / sender / receiver, route update in routing protocol, etc.

With the exponential increase in IP communications, metadata has a great untapped potential for mapping communication patterns, especially for protection purposes as in the case of lawful intercept. It also solves the problem of ballooning storage requirements. In fact, the metadata approach may even be the ONLY way to handle lawful intercept in the future!

So the opportunity is to leverage metadata for intelligence gathering, for ex. to reconstruct links between people, to understand which virtual IDs the same person is using (starting from physical IP/IMSI) or to identify intentionally hidden information (“the Dark Web”).

However, IP communication metadata is not readily available on operator servers, or resides on third-party Web servers, outside the control of a given telco (think P2P,social networking, etc.). Therefore, metadata has to be extracted directly from the network.

So how can buyers of LI solutions leverage IP metadata?

Use the rich set of information available with IP metadata to build completely new types of intelligence solutions - as a complement to content-focused solutions. This gives a better understanding of potential threats (the macro-view)
Take advantage of more computer-based information processing 1.5 billion Internet users, thousands of Web applications… the number of analysts cannot increase at the same rate as the number of IP communications, the number of Web applications and the amount of content generated > need to think differently?!

The solution is to implement automatic detection of suspicious behavioral communication patterns and create real-time view of the threat situation based on continuous streaming of metadata and network information. There is also an opportunity to minimize storage and post-processing time by extracting significant information and structuring it as soon as each metadata is available

Long live metadata!

Jerome

Network Intelligence: a key trend in IT networking

2009-01-15T17:32:00.000+01:00

I took advantage of the Christmas break to catch up on some reading, like this article on Forbes.com that a friend sent me. It was interesting to see that the author Dan Woods cleverly identified a key trend in IT networking: networks aren't just highways!

As Dan puts it, "This week the JargonSpy looks at the gradual transformation of the network as transport, to the network as a source of information. As the network comes alive as a center of intelligence, the way that many applications do their jobs will change for the better."

It is great to see that Network Intelligence is becoming a discussion topic even in mainstream business media!

Wishing everyone a happy new year,

Jerome

A Network-based Approach to Enterprise Compliance

2008-11-25T19:45:00.001+01:00

This time, I'd like to talk about a new, network-based approach to Compliance. All kinds of enterprises need to comply with regulations such as SOX, PCI DSS, HIPAA; at the same time, they absolutely want to avoid costly upgrades and maintain service.

The challenges are many: What information should be stored? What upgrades arerequired? Where can the information be found? Is it accessible and meaningful? Are different information sources compatible? Is the information complete? reliable? tamper-proof ?...usable?

If you've ever had to sort out several jigsaw puzzles that have been mixed together and scattered about your home, you quickly realize the impracticality of log-based solutions to such challenges. And this is where a network-based approach comes in.

Qosmos network intelligence technology queries an IP network in real-time as though it were a database. This enables you to see in real-time what actually happens, wherever it happens, and to record the information in a single data format (e.g. SMTP for emails), and with a single database entry for each session. And you can do all this from one point on your network: it doesn't matter what your existing IT system is, and there's no impact on existing management policies, systems or services. Furthermore, Qosmos filters and structures relevant data to optimize storage and speed up post-processing, and extracts information directly from the network, independently of servers, and even in cases where they do not have access to (third party) logs or databases (such as for Webmail or public IM).

Judging from some of my recent discussions, it is clear that Qosmos has an interesting value proposition: we can work in partnership with suppliers of Compliance solutions, to provide them with information extraction components on which they can build complete systems – with all the advantages of a network-based approach!

Jerome

Time for Change

2008-11-24T18:31:00.001+01:00

You've certainly caught some of the buzz generated by the recent Obama - Verizon controversy.

It's clear that telecom operators’ databases contain sensitive customer information, ranging from subscriber phone numbers and addresses, to private bank details, call history and geo-localization data. And you don't need to be president-elect to understand that preventing the theft or misuse of this confidential information is of critical importance. Knowing who accesses what, where, when and why are the cardinal points of confidence in the information age.

Data protection represents, therefore, both a strategic and technical challenge for operators, who must implement a secure and actionable tracking system across the various access points to their subscriber databases: from internal customer relations and technical support teams, to outsourced customer care services, to malicious hackers.

At Qosmos, we like to say “Yes we can”. Qosmos Network Intelligence technology enables Data Protection and Data Theft Prevention solutions that monitor network traffic in real time, and extract complete and detailed access records for all intranet, web-based or legacy (e.g. TN3270) applications, whether standard or custom, and whatever network protocols are used. Which means fully traceable access, real-time reactivity, and rapid post-processing of information.

So that when government agencies and lawyers come knocking, you've not only got the answer - you know who else does...

Jerome

Network Intelligence for Audience Measurement

2008-11-06T19:31:00.003+01:00

Today I'd like to say a few words about information extraction for audience measurement. The diversity of IP-based media (Web, Mobile Web and IPTV, etc.) makes for fragmented audiences, for which traditional measurement approaches like user panels prove inadequate. Network Intelligence technology, on the other hand, can complement existing user-centric tools with network-centric data across a wide range of applications and devices, offering a unified vision of network usages and trends.

Network Intelligence technology does this by successfully extracting information like Website visited, Website itinerary, page content, duration, referent, mobile ID, IPTV viewer behavior, channel zapping, etc. which translates into faster reporting, more accurate information, and the ability to correlate user behavior across the Internet, IPTV, and mobile terminals. All while ensuring subscriber privacy through anonymous data collection. And this information generates new revenue possibilities. Market research providers can use this approach to gain competitive advantage, by providing valuable information to ISPs (who benefit from higher ARPU), while advertisers benefit from increased effectiveness by targeting ads to genuinely interested consumers.

In the end, users win out, through enhanced user experience, more adapted services, and the final choice to opt-in or not. If you'd like more information on how Network Intelligence can be used to leverage business intelligence, join our Business Intelligence Webinar:

http://www.telestrategies.com/ISS_WEBINARS/Deep_Packet_Inspection.htm

Jerome

The Challenges of Virtual Identity Management

2008-10-27T18:30:00.001+01:00

At ISS World Europe a couple of weeks ago, Qosmos presented at a seminar focused on lawful interception and network intelligence gathering.

What comes out clear from the ISS conference is the need for efficient Virtual Identity Management. Put simply: in the context of legal intercept, how do you track a user who can be anywhere, use anything, connect by any means, and who uses multiple virtual IDs ?

Successful and effective Virtual Identity Management, means being able to recognize and extract a target identifier over a wide variety of applications. And this is by no means easy. To take the case of http: the same protocol is used by different applications, which means you must understand the logic of the applications above http. In such cases, the successful recognition and extraction of target identifiers requires the implementation of complex mechanisms specifically developed for these applications, and a maintenance and upgrade policy that enables you both to keep tabs on frequent changes in protocol structures and take on board new applications. And from a global perspective, this means keeping abreast of regional apps like QQ (China), Mail.ru (Russia) and India Times webmail!

These are some of the interesting challenges we work on at Qosmos.

P.S.: On my next post, I'll speak of my upcoming TeleStrategies webinar, where the focus will be on information extraction in the field of Business Intelligence.

Jerome

DPI or Information Extraction? - Making the difference

2008-10-10T19:28:00.001+02:00

I just got back from the Broadband World Forum where I met a number of people to whom I had to describe the core competency of Qosmos. I realized that many people are not aware of the difference between DPI and Information eXtraction (IX for short).

Put simply, Deep Packet Inspection could be described in the following terms: it recognizes network traffic, it has the ability to act on the traffic, and vendors typically sell complete solutions for specific markets such as lawful intercept, traffic optimization, etc.

Information eXtraction – IX, on the other hand, is different: it actually extracts network information and makes complete sense out of it; ix doesn’t act on traffic flows. In the case of Qosmos and IX, the focus is on best-in-class network intelligence, for use in other people’s complete solutions. As a matter of fact, DPI specialists can even build their solutions based on an Information eXtraction engine like Qosmos.

This is Qosmos’s real differentiator, and if those we met at the BBWF were to take away one thing it should be this. Once potential users of information extraction understand this, the horizon is suddenly different. And a wealth of possibilities opens up – I’ll describe this in an upcoming post.

Jerome