Solving Unsolvable Problems: The Solution

Solution vendors are realizing that network intelligence requires advanced expertise and that there are clear advantages of sourcing the technology from specialists. The economic downturn accelerates the movement; vendors that come out as winners are those who stay focused on their core business.

Qosmos is dedicated to network intelligence, identifying and managing the evolving and growing number of network communication protocols as well as capturing traffic metadata or content. This real-time visibility enables enhanced security features, optimized technical performance and more precise usage in third-party solutions. Delivered as software development kits and hardware that integrate seamlessly, systems integrators, ISVs and NEP’s rely on Qosmos for expert network intelligence technology while maintaining complete control over their solutions. 

Qosmos users benefit in several ways:

1. Improve development efficiency

In high-tech, business success goes hand in hand with fast time to market, which in turn depends on development time. By using proven network intelligence toolkits, development teams can stay focused on bringing new solutions fast to market, able to adhere to more predictable development roadmaps in the face of the ever-changing network-based environment. Qosmos’ Software Development Kit includes fully documented developer tools, along with support and maintenance services designed to make solutions network-intelligent rapidly and efficiently. Qosmos efficiency is built-in to the technology itself: the same application building blocks can be integrated across different CPUs, NPUs and hardware platforms. In fact, the Qosmos network intelligence development kit can be ported to any type of modern hardware.

2. Focus on core competence

Sourcing an enabling technology externally allows solution vendors to concentrate all their efforts on the customer requirements of their solutions, instead of reinventing the wheel internally. In a perfect distribution of labor, network intelligence specialists such as Qosmos put all their resources on building the best network intelligence toolkits, providing unique expertise and pre-developed building blocks that may be integrated seamlessly and rapidly to empower third-party solutions. As an illustration of our technical know-how, we can extract network information by drilling down into 16 levels of protocol encapsulation.

3. Expand solution features

Finally, the use of Qosmos for specialized network intelligence provides more than efficient use of development resources, the depth and breadth of data and metadata extraction enables additional features that strengthen and expand the capabilities of the solutions.  For example, cyber security solutions built on network intelligence technology can detect abnormal network behavior that is invisible to standard, commercial “COTS” products. In another example, network intelligence technology can be used as a front-end for Lawful Intercept to process raw traffic and efficiently dispatch only the relevant data to an existing solution, even at very high data volumes.  Finally, sourcing network intelligence technology externally gives developers fast access to new protocols which can feed their solutions and expand their reach; Qosmos delivers a new batch of protocols each quarter.

Network Intelligence plays a key role in a network-based world; governments, operators and enterprises have a vital need to gather network intelligence from IP networks for protection, monetizing and optimizing purposes. There is an increasing need to understand network-based activity for a range of solutions such as lawful interception, cyber security, market research, network optimization, billing, and more.  Without detailed network intelligence, these solutions will not continue to function adequately and their vendors will face serious business issues.

Many of the winners will be companies who chose to boost their solutions with expert Network Intelligence Technology – from Qosmos.

Jerome

Solving Unsolvable Problems (Part 4): Managing highly complex technology

Since network intelligence goes beyond DPI in terms of the level of visibility, protocol management, attribute recognition, and information extraction, even companies who have incorporated DPI capabilities into their solutions will require a new level of expertise. For example, Qosmos engineers have developed a specific meta-programming language to build Webmail and HTTP protocol plug-ins. Specific techniques and tools must be developed for quality assurance and to make reverse engineering more efficient.  This is an order of magnitude remote from the business of someone who sells complete solutions. 

In most cases, a separate R&D organization must be created. But once committed, companies realize that development times are difficult to estimate, timelines are incompressible and that the skills are so specialized that it becomes nearly impossible to outsource parts of the development.

Jerome

Solving Unsolvable Problems (Part 3): Committing considerable resources, with uncertain returns

If you choose to tackle a family of protocols (e.g. Webmails), you have to develop network intelligence capabilities for the most important protocols in this family, otherwise your solution will ineffective or incomplete (think of traffic optimization or cyber security applications). In addition, the total number of applications and protocols increase continuously (e.g. 50,000 applications are now available for the iPhone), and very few protocols ever disappear…

For a company whose core business is not network intelligence technology, this translates into high costs of entry and ever-rising investments. To make things worse, end customers of turn-key solutions may not appreciate the importance of continuous protocol updates and the amount of work required to keep the solutions current. This means that a solution vendor could end up investing considerable resources for which and end customer does not perceive high value and therefore may not be ready to pay… 

Jerome

Solving Unsolvable Problems (Part 2)

As I described in my previous post, I believe that solution vendors are facing a crucial decision point and are realizing that developing network intelligence capability internally would create a number of “unsolvable problems”.

Unsolvable problem number 2: Not being able to use traditional product development and management methods

The high-tech industry typically uses a structured approach for product development and management, with most key activities aligned around go/no go decision points and defined time lines. These processes are built to ensure that new products are delivered on time, according to specifications and with the adequate quality.

However, companies who chose to develop network intelligence technology internally quickly discover that the usual methods cannot be used. Web protocols such as Webmails change continuously without notice, which means that development roadmaps cannot be easily controlled. Development teams must be quick to react to new protocol evolutions and use reverse engineering techniques to update their network intelligence software.

This way of working is counter-cultural for many high-tech companies and can even be incompatible with the rest of the organization. From a business standpoint, it can even be unsustainable and unprofitable - unless you make network intelligence technology your core business, like Qosmos;-)

Jerome

Solving Unsolvable Problems (Part 1)

For the past several years, Deep Packet Inspection (DPI) has been used by developers to get basic traffic visibility for applications such as traffic shaping and network security.

We are now at a key juncture of the market, where many solution vendors need much more detailed visibility into network-based activity. For example, for market research applications it is not enough to recognize generic http traffic; it is necessary to extract metadata such as name of visited Website, page content, time spent on visit, basket share, referent, etc. The same is true for applications such as lawful intercept and government cyber security, which demand extremely accurate and fine-grained information on communication flows in order to map exact communication flows and identify threat patterns.

So the key question becomes: - should solution vendors develop complex network intelligence technology internally or should they source it from a specialist?

Solution vendors realize that developing network intelligence capability internally necessitates a step-function in terms of investment and strategy. It requires considerable internal R&D resources, detracts from core business and poses a number of additional problems in term of organizational efficiency, culture, and return on investment.

During the next couple of weeks, I will describe the “unsolvable problems” facing solution vendors and highlight the advantages of sourcing complex network intelligence technology from a specialist like Qosmos.

The first “Unsolvable Problem”: Having to continuously redevelop software

Developing network intelligence software to handle a particular protocol is only the tip of the iceberg: you also need to invest continuously to update your software so it can handle new versions of protocols. Some basic protocols are easy to manage since they are relatively stable: IMAP, SMTP, POP, HTTP, etc. However, there is a vast number of other protocols which are proprietary and evolve at a rapid pace (Webmails, P2P, social networking, gaming, etc.).

For example, the Livemail protocol changed several times during 2008. And each time a new version is released, most of the network intelligence software has to be redeveloped, without much reuse of previous development. This requires a particular culture and special methods: reverse engineering, custom-made tools, fast reaction to protocols changes, etc.

Jerome

BBWF09 - DPI Workshop / Monday, 07 September 10:00 – 12:30

Just a little heads-up to tell you that I will be a speaker at the upcoming Broadband World Forum which will be help in Paris 7-9 September. Il will participate in the workshop “Deep Packet Inspection: Technology, Promise & Controversy. What You Need to Know” on Monday, 07 September 10:00 – 12:30. It should be an interesting discussion, with views from solution suppliers, service providers and Network Intelligence Technology specialists (Qosmos).

For those of you who would like to see Qosmos Network Intelligence Technology in action, you are welcome to see a live demo on the exhibition floor: just come to our partner BreakingPoint’s Stand # 470.

I hope to see you there!

Jerome

Forbes article on Programming Innovation

Dan Woods just published an interesting article on Forbes.com called “Programming Innovation”. He describes how Qosmos technology enables a new kind of market research for our partner GfK.

Market research is of course only one example of how Qosmos Network Intelligence can be used. I like Dan’s way of describing our toolkit as a catalyst for innovation across a wide range of applications: “Now, tens of thousands of companies potentially have access to a new form of information derived from the network”.

Based on my daily discussions with customers and prospects, I can confirm that the trend is real – I can’t wait to see tens of thousands of companies benefiting from Qosmos Network Intelligence ;-)

Jerome