Source Code Analysis and Integer Overflow
Some of the problems I learned to look for are well-known problems such as using an object after it has been deleted. Other problems are things that I knew about intellectually but was not in the habit of looking for in code, such as integer overflow.
Integer overflow can happen in a few different ways: Converting between unsigned and signed, converting 64-bit to 32-bit, adding, subtracting or multiplying.
One example is an application that receives a 4GB string or is asked to create one through a script language like Javascript. It then adds 1 before making the allocation to make room for the terminating zero byte of a C-style string. The size value overflows and becomes 0 which may cause all sorts of problems such as code which believes the string length is very large but the actual memory allocation is for zero bytes. That could allow the memory allocator to place a security sensitive object after the string. The program would allow that sensitive memory to be read or even written to because one part of the program believes the memory is owned by the string while another part of the program believes the memory is a function pointer or a password.
Another example is an application that gets a request to allocate an array of 70 million objects. Each object is 64 bytes long. The total number of bytes requested will be 70 million times 64 or 4.48 billion bytes. But wait! The maximum number that can be stored in a 32-bit integer value is approximately 4.3 billion. The multiplied value cannot fit and it will be truncated to about 185 million bytes. Which means that the application may believe it has allocated 70 million objects but the actual space is much less. That might let an attacker create a request that ends up calling a function pointer in an object array whose contents are under the attacker’s control.
One thought that I took away from this class is that these attacks rely on confusing the software so that parts of it treat the data in different ways. One function uses the string length before adding to it while another function uses the length after adding to it. One way to minimize this problem is to write code so that the same code and the same values are used everywhere. Another is to add consistent checks to all values coming in from outside the program. These checks should be designed in such a way that a new programmer in a hurry to add a new feature to unfamiliar code cannot easily bypass them. A third thought is to use extended compiler warnings and static analysis tools, if available, in order to catch every case where a signed integer may be confused with an unsigned one.
