.NET: NGEN versus JITting

For those not very familiar with the details on how the Common Language Runtime (CLR) for .NET functions, this will come as a surprise.

The basic premise of the CLR was to delay the interpretation of the .NET assemblies until the they actually get used. Actually, the level of granularity is more at a function level than at a module level. This is to say that every function gets "Just-in-time" compiled (called JITting) when it is first called, not when a module is loaded in memory. This late execution model was a decision made based on application execution heuristics. We are assuming that this study has been made over a large variety of applications making sure that the working set encompassed a wide range of scenarios to match every day program executions.

Windows executables for .NET are called Portable Executables (PEs) for reasons that is self-explanatory. The format what is fed into the JIT compiler is called the Intermediate Language (IL or sometime s MSIL).
So basically,
NGEN (for "Native Generation"): pre-compiled code
JITting (For "Just-in-time" compilation): compile code only when needed .

Each has it's own good points and shortcomings.
---
Your argument will seemingly be this: Why would I prefer JITting over NGEN anytime since I would need to JIT my function once per execution of the program? It's not as straightforward as that. NGENing your program will generate the compiled version of the IL for that specific machine. It will then become machine specific (more or less) but depending on how much of it is x86 compatible (and we assume it is 100%), should still allow you to run it on various machines. With the kind of machine heterogeneity out there, you definitely don't want these kind of dependencies. With machine-specific code, you always face the "hit" of falling back to use unoptimized code in case the specific assembly instructions are not available on the machine (e.g. MMX).

Note that this was one of the basic premise because of which the Internet is extremely successful. Java users will note the distinct similarity between IL and Java bytecode interpretation, the JITter and the JVM, etc. There are definitely parallels but the usage is dramatically different.

There are some things one should be aware of when deciding between NGEN and JITting:

• The resulting native image is, as a rule, generally larger than the corresponding IL code. As is the case with many applications, many methods are not executed during the course of an instance of the application. Thus, the in memory image of the code would be larger if the entire assembly is stored as a native image.

• The native image cannot reflect the way that the code is actually used. By dynamically compiling, the .NET infrastructure (CLR) can adjust the code image to reflect the current state of the application and machine. For example, some methods may end up occupying a different virtual memory page than the rest of the application, causing a fragmentation which has a negative impact on the working size of the application.

• By producing a native image, the image is now strongly bound to specific versions of other assemblies. Essentially, a contract has been established where the native image is dependent on specific versions of other assemblies being available, down to a level of a module version identifier (MVI) being assigned to each module when it is compiled. The MVI is a unique identifier that is stored with the native code to allow it to check other compiled dependencies to verify that the correct version of the module is being loaded. If it is not, the CLR must fall back to the IL version of the code and use JIT.

The above are not specific instances, but more to be uses as a defining guide for your decision making process.

Best scenarios for NGEN would be to make sure that there is not too much of code revamp in your application, i.e. frequent changes. Good choices would be code heavily dependent on mathematical or graphical operations that can be optimized largely with CPU-based optimizations. Examples of code that does not change can also serve as good candidates, e.g. the .NET framework itself.

In conclusion, JITting (the default behavior) should work for most of your cases, and NGEN should be seen as a performance enhancement for very specific scenarios.