Similarities between Network Intelligence Technology and a relational database system

Network Intelligence Technology aims at providing full visibility on traffic flows; it is in many aspects very similar to a relational database system which allows users to query large sets of data generated and computed by information systems. Similarities between these technologies are not limited to the technical aspect, they also follow the same business trends, but 30 years apart!

Back in the 60s, a database was specifically designed for each application because it was such a complex and cutting edge technology that only specific systems could afford it. This complexity limited the implementations to large systems including specific hardware to support the database. Databases were reserved for very large organizations.

In the 70s, the INGRES project introduced major technical enhancements such as the relational database, which made data more actionable. This project was the foundation of the first commercial products such as Sybase and Informix which enabled developers to build systems using standard database building blocks including a DBMS engine and a query language. With the availability of COTS database technology, software vendors could create applications for any type of business requirement and not only for very large organizations. SMBs could use financial systems to track their sales, the local library could use a database system to manage its book portfolio. Today, thousands of applications benefit from these database building blocks, and even very large systems such as ERPs are designed with standard COTS database products such as Oracle. Using a COTS database is more rational from a cost point of view and also reduces the time to market. Today any software vendor building an application will choose a COTS database and will not even think of redeveloping its own BD system like in the 70s.

I see the EXACT same trend with Network Intelligence Technology. The first implementations mainly focused around Deep Packet Inspections (DPI) took place in routers and in specific appliances using custom hardware. Applications where limited to very specific tasks such as P2P blocking and every vendor would develop its own “in house” network flow analysis.

But the complexity of IP networks with a growing number of applications and protocols makes it very complex to get a 360° visibility over network traffic, which confines applications to very high-end solutions and technically advanced systems usually managed by network administrators. On the other hand, solution vendors in the business of billing network usage, optimizing networks or marketing network services have a growing need to understand more in detail the behavior of network and do not always have the skills to develop in-house network intelligence. The emergence of specialized vendors of network intelligence technology, whose mission is to provide building blocks for solution vendors and network equipment providers, can be compared to the emergence of the COTS database companies. Providing ready-to-use network intelligence components which can feed applications allows any developer to use the gigantic amount of information computed by or travelling over an IP network.

Network Intelligence Technology is a fast moving market with new protocols appearing and being updated frequently. Many solution vendors and network equipment manufacturers realize that their core business is not to focus on protocol technology. Instead of developing DPI or network intelligence in house, they can now source it from a Network Intelligence Technology specialist, and benefit from deep expertise, in the same was as they would select a commercial database product :-).

Jerome