The critical importance of data normalization can also be seen while reviewing security failures and fundamental design flaws in many IPS devices that lack such normalization.
Data normalization explained
Data normalization is the process of intercepting and storing incoming data so it exists in one form only. This eliminates redundant data and protects the data’s integrity. The stored, normalized data is protected while any appearance of the data elsewhere is only making a reference to the data that is being stored and protected in the data normalizer.
The normalizer’s job is to patch up the incoming data stream to eliminate the risk of evasion as well as ambiguities. The monitor then views the data in its pure, protected and normalized form. Varying forms of normalization exist on levels of increasing complexity. The complexity is due to the set of requirements that must be met to achieve normalization. The most basic is known as First Normal Form, which is often abbreviated 1NF. It is followed by Second Normal Form, or 2NF, Third Normal Form, or 3NF and can continue increasing in forms and complexity as required or desired.
Normalization plays a key role in the security of a network, provided that normalization extends to every protocol layer. One of the major benefits is the forced integrity of the data as data normalization process tends to enhance the overall cleanliness and structure of the data. Normalization significantly contributes to the fortification of a network, especially in light of typical networks’ three main weak points: traffic handling, inspection and detection.
Where many IPS devices go wrong
When it comes to traffic handling, many IPS devices focus on throughput orientation for the most rapid and optimal inline performance. This process, while attractive for its rapidity, makes it impossible for full normalization to take place. The data traffic is then inspected without normalization, offering prime opportunities for infiltration to take place. One may agree that a rapid and optimal output performance is useless if the payload is riddled with malicious invaders.
When many IPS devices do employ normalization, they often rely on shortcuts that only implement partial normalization as well as partial inspection. This leaves gaps in the security and provides optimal opportunity for evasions. TCP segmentation handling is one example of such a process, as it is only executed in chosen protocols or ports and is drastically limited in its execution. Shortcut exploitation is a familiar evasion method and, with the proliferation of IPS devices that fail to perform full normalization, it is likely to remain that way due to its ease of execution.
Many IPS devices fall short in other areas, as well. They often perform only a single layer of analysis, execute traffic modifications and interpretations and rely on inspection of individual segments or pseudo-packets. Their detection methods are based on vulnerability and exploits, banner matching or shell detection. Their updates are generally delayed and their evasion coverage is extremely limited. Evasions can easily exploit the limited inspection scope by spreading attacks over segments or pseudo-packet boundaries.