This article explains how Data Execution Prevention (DEP) works and how to turn it off and on in Windows 7, Windows Vista, and Windows Server 2008 (R2).
Latest posts by Andreas Kroschel (see all)

Data Execution Prevention (DEP) is a security feature of the CPU that prevents an application from executing code from a non-executable memory region. This is supposed to prevent buffer overflow attacks from succeeding. Since Microsoft introduced support for Data Execution Prevention (DEP) on Windows XP Service Pack 2 and Windows Server 2003 Service Pack 1, it’s included in every version of Windows.

How DEP works: Hardware enforcement and the role of the OS ^

Data execution prevention works by marking certain memory pages being indented to hold only data and no executable code. This is achieved by setting a special bit in its page table entry called NX, for No eXecute, or XD, for eXecute Disabled, respectively. It’s the responsibility of the OS to set the NX bit for the stack and heap memory areas. If a malfunctioning program – or malware – should try to execute code from an NX-marked memory page, the CPU will refuse to do so and trigger an interrupt instead, which causes the OS to shut down the application accordingly.

Turn on and turn off DEP support in Control Panel ^

DEP can not only prevent the execution of malware or malfunctioning applications, but it may also highlight problems with legacy (not DEP-compliant) software, which can cause it to crash. Another potential problem is the support for third-party plugins such as those found in browsers or office applications: While the application itself may be DEP compliant, chances are that one or more of the plugins aren’t. Microsoft recommends updating your software if it’s experiencing crashes with DEP, but this is not always possible. For such situations, DEP support in Windows can be configured to meet the user’s needs, handling exceptions for certain software.

Some limitations exist when you turn off or turn on DEP support, however. Because DEP support is a kernel mode option, it must be configured as a boot option. Thus, it is not possible to manage and deploy DEP settings centrally by group policies; they have to be configured at the local machine in each case and need a reboot of Windows to take effect.

The settings GUI can be invoked this way: Open Control Panel, click on System and SecuritySystem → Advanced system settings. In the Advanced tab, click on the Settings button in the Performance section (the first one). In Performance Options, Data Execution Prevention has its own tab. Here you can turn on DEP support for Windows essential programs and services only (OptIn, default on Windows 7 workstation) or for all programs, with the possibility to define exceptions for non-compliant software (OptOut, default on Windows Server 2008/2008 R2). This can be achieved via the Add button, where a local administrator can add non-compliant executable files one by one.

Turn off  Data Execution Prevention - DEP Server default

Exceptions can also be configured as a DisableNX compatibility fix using the Application Compatibility Toolkit (ACT). The resulting Custom Compatibility Database can be deployed in the Active Directory. Note that those kinds of exceptions do not show up in the DEP support configuration GUI.

Turn off and turn on DEP support as a boot option ^

There are two more DEP settings for a Windows machine. These settings cannot be configured in the control panel but only as a boot option via the service program bcdedit in a command prompt with elevated rights.

Turn off Data Execution Prevention - DEP bcdedit

One possible choice is to turn on DEP support unconditionally:

bcdedit /set {current} nx AlwaysOn

In this mode, the DEP support options GUI is disabled and no exceptions can be defined. Any DisableNX compatibility options will also be ignored.

The opposite is to turn off DEP support completely:

bcdedit /set {current} nx AlwaysOff

With this setting in effect, the DEP support options GUI will be disabled as well as with the first option. To return to one of the GUI switchable modes, use:

bcdedit /set {current} nx OptIn

for the workstation default, which enables DEP support for Windows essential programs and services, or:

bcdedit /set {current} nx OptOut

for the server default, enabling DEP support for all executable files. The Windows machine must be rebooted each time for the bcdedit command to take effect. The output of the command:

bcdedit /enum

will tell the current status in each case.

This article has been translated from German language. You can find the original posting: Datenausführungsverhinderung (DEP) konfigurieren oder abschalten

2 Comments
  1. RecoveryTool 11 years ago

    We can also turn on/off DEP by changing the registry setting: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\NoExecuteState.

  2. Randy Mehling 10 years ago

    This page was a total lifesaver! And much easier than usign the command line referenced in Microsoft’s own bulletin.

Leave a reply

Your email address will not be published.

*

© 4sysops 2006 - 2022

CONTACT US

Please ask IT administration questions in the forums. Any other messages are welcome.

Sending

Log in with your credentials

or    

Forgot your details?

Create Account