IPSec Policies Overview

IPSec encrypts data information contained in IP datagrams through encapsulation to provide data integrity, data confidentiality, data origin authentication, and replay protection. The two main IPSec components that are installed when you install IPSec are the IPSec Policy Agent and the IPSec driver. The IPSec Policy Agent is a service running on a Windows Server 2003 computer that accesses IPSec policy information. The IPSec Policy Agent accesses the IPSec policy information in the local Windows registry or in Active Directory. The IPSec Policy Agent then passes this information to the IPSec driver. The IPSec driver performs a number of operations to enable secure network communications such as initiating IKE communication, creating IPSec packets, encrypts data, and calculates hashes.

IPSec policies are used to apply security in your network. The IPSec policies define when and how data should be secured. The IPSec policies also determine which security methods to use when securing data at the different levels in your network. You can configure IPSec policies so that different types of traffic are affected by each individual policy.

The different components of an IPSec policy are listed here:IPSec policies

  • IP filter; informs the IPSec driver on the type of inbound traffic and outbound traffic which should be secured.
  • IP filter list; used to group multiple IP filters into a single list in order to isolate a specific set of network traffic.
  • Filter action; used to define how the IPSec driver should secure traffic.
  • Security method; refers to security types and algorithms used for the key exchange process and for authentication.
  • Connection type: identifies the type of connection which the IPSec policy impacts.
  • Tunnel setting; the tunnel endpoint’s IP address/DNS name.
  • Rule; a grouping of components such as filters and filter actions to secure a specific subset of traffic in a particular manner:

IPSec policies can be applied at the following levels within a network:

  • Active Directory domain.
  • Active Directory site.
  • Active Directory organizational unit.
  • Computers.
  • Applications.

When you configure and manage IPSec, you would basically be configuring the following aspects of IPSec policies:

  • Assign the predefined default IPSec policies.
  • Create customized IPSec policies that include customized rules and filters.
  • Control how IPSec policies are applied.
  • Apply IPSec policies at different levels on the network.

To configure IPSec policies, you can use either of the following methods:

  • You can use the IP Security Policy Management snap-in to configure IP security policies on the local computer. To create a new IPSec policy, you have to right-click the IP Security Policies node in the IP Security Policy Management snap-in, and then click Create IP Security Policy.
  • You can use the Group Policy Object Editor snap-in to change local and domain GPOs. To create a new IPSec policy, you have to right-click the IP Security Policies node in the Group Policy Object Editor and then click Create IP Security Policy.

The IP Security Policy Management snap-in is used to manage IPSec with respect to:

  • Create IPSec policies.
  • Edit existing IPSec policies.
  • Assign IPSec policies.
  • Add and remove filters which are applied to IPSec policies.

When you install the IPSec IP Security Policy Management snap-in, you need to select which IPSec policy you want to manage, and on what network level you want to manage IPSec. You can select either of the following options:

  • Manage a local IPSec policy on the computer.
  • Manage the local IPSec policy a different computer.
  • Manage the default policy for the domain in which the computer resides.
  • Manage the default policy for a different domain.

Understanding Default IPSec Policies

Windows Server 2003 IPSec deployments include predefined IPSec rules, filter lists, filter actions, and three default IPSec policies. Each default IPSec policy contains a set of predefined rules, filter lists and filter actions.

Each IPSec policy is based on number of rules. An IPSec policy can contain a single rule, or a set of rules. It is these rules that enable secure connections, based on the following factors:

  • Source address.
  • Destination address.
  • Type of traffic.

An IPSec rule contains the following components:

  • A filter list.
  • A filter action.
  • An authentication method.
  • A connection type.
  • A tunnel configuration.

The three default IPSec policies and their predefined configuration are described below:

  • Client (Respond Only): The Client (Respond Only) default IPSec policy is the least secure default policy. With this default IPSec policy, the computer assigned the policy never initiates secure data communication. The computer only responds to IPSec requests from other computers who request it. The Client (Respond Only) default IPSec policy contains the default response rule that creates dynamic IPSec filters for inbound and outbound traffic based on the protocol and port which was requested. The predefined policy settings for the Client (Respond Only) default IPSec policy are listed here:
    • IP Filter List; All
    • Filter Action; None
    • Authentication; Kerberos
    • Tunnel Setting; None
    • Connection Type; All
  • Secure Server (Request Security): With the Secure Server (Request Security) default IPSec policy, the computer prefers and initiates secure data communication. If the other computer supports IPSec, secure data communication will take place. If the other computer does not support IPSec, the computer will allow unsecured communication with that computer. The Secure Server (Request Security) default IPSec policy contains three rules, and predefined policy settings:

    The predefined policy settings for Rule 1 are:

    • IP Filter List; All IP Traffic
    • Filter Action; Request Security (Optional)
    • Authentication; Kerberos
    • Tunnel Setting; None
    • Connection Type; All

    The predefined policy settings for Rule 2 are:

    • IP Filter List; All ICMP Traffic
    • Filter Action; Permit
    • Authentication; N/a
    • Tunnel Setting; None
    • Connection Type; All

    The predefined policy settings for Rule 3 are:

    • IP Filter List; Dynamic
    • Filter Action; Default Response
    • Authentication; Kerberos
    • Tunnel Setting; None
    • Connection Type; All
  • Secure Server (Require Security): With the Secure Server (Require Security) default IPSec policy only secure data communication is allowed. If the other computer does not support IPSec, the connection is not established. The Secure Server (Require Security) default IPSec policy contains three rules, and predefined policy settings:

    The predefined policy settings for Rule 1 are:

    • IP Filter List; All IP Traffic
    • Filter Action; Require Security
    • Authentication; N/a
    • Tunnel Setting; None
    • Connection Type; All

    The predefined policy settings for Rule 2 are:

    • IP Filter List; All ICMP Traffic
    • Filter Action; Permit
    • Authentication; Kerberos
    • Tunnel Setting; None
    • Connection Type; All

    The predefined policy settings for Rule 3 are:

    • IP Filter List; Dynamic
    • Filter Action; Default Response
    • Authentication; Kerberos
    • Tunnel Setting; None
    • Connection Type; All

You can also create customized IPSec policies that include customized rules and filters that suit specific security requirements of the organization. You can also create your own IPSec policy by using the IP Security Wizard which you can initiate from within the IP Security Policy Management MMC.

For filter actions, you can select between the filter actions listed below. Remember that the filter action which is defined determines how IPSec responds to computers matching a filter list, and it determines which security methods is used:

  • Permit action (pass through action); used to allow traffic to pass through without applying any security rules and modifying the traffic. The traffic is simply allowed. Typically used for data that is considered non-sensitive.
  • Block action; used to block all traffic.
  • Allow Unsecured Communication With Non-IPSec Aware Computers; when used unsecured connections will be accepted by your computers. Generally recommended that you do not utilize this option.
  • Accept Unsecured Communication, But Always Respond Using IPSec action; when used the computer will always request IPSec before allowing any connections, but it will allow unsecured connections. Secured connections will though always be requested. This option therefore allows for secured connections and unsecured connections.
  • Use These Security Settings action; used to specify custom security methods which should be applied for connections matching the filter.

How to view default IPSec policies:

  1. Click Start, click Run, type mmc in the Run dialog box, and then click OK.
  2. Click the File Menu item, and select Add/Remove Snap-in.
  3. The Add/Remove Snap-in dialog box opens. Click Add.
  4. The Add Standalone Snap-In dialog box opens.
  5. Select Group Policy Object Editor, and then click Add.
  6. Select the Local Computer default option.
  7. Click Finish.
  8. Click Close to close the Add Standalone Snap-in dialog box.
  9. Click OK to close the Add/Remove Snap-in dialog box.
  10. Expand Local Computer Policy, expand Computer Configuration, expand Windows Settings, expand Security Settings, expand IP Security Policies on Active Directory.
  11. The details pane displays the default IPSec policies.
  12. Right-click the initial default IPSec policy displayed, which should be the Server (Request Security) policy, and then click Properties to open the Server (Request Security) default policy’s Properties dialog box.
  13. Click the General tab. The configuration settings on the General tab are listed here:
    • The name of the policy is listed in the Name text box.
    • A description of the policy appears in the Description text box.
    • The Check For Policy Changes Every box contains the interval for which clients using this specific policy checks for policy updates.
  14. Clicking the Settings button on the General tab opens the Key Exchange Settings dialog box. On the Exchange Settings dialog box you can specify when new keys are generated for the policy.
  15. Clicking the Methods button opens the Key Exchange Security Methods dialog box. You change the IKE settings and security preference methods on this dialog box. This is where is you can change encryption, integrity, and Diffie-Hellman Group settings.
  16. To close the Key Exchange Security Methods dialog box, click Cancel.
  17. To close the Key Exchange Settings dialog box, click Cancel.
  18. The Server (Request Security) default IPSec policy’s Properties dialog box should be displayed once more.
  19. Click the Rules tab.
  20. The three IPSec rules described in this Article earlier are defined on the Rules tab.
  21. Each IPSec rule has IP filter list, Filter Action, Authentication, Tunnel Endpoint, and Connection Type settings.
  22. To view the settings of a rule, click the Edit button.
  23. The Edit Rule Propertie dialog box opens.
  24. The Edit Rule Properties dialog box contains the following tabs which you can use to set configuration settings for the IPSec rule:
    • IP Filter List tab; used to add, remove, and configure the filter lists for the rule. All currently configured filter lists are displayed in the IP Filter Lists list.
    • Filter Action tab; used to configure filter actions for the rule. The current filter actions defined for the rule are listed in the Filter Actions list. The Edit, Add, and Remove buttons can be used to change, add and remove filter actions for the rule. You can also specify whether the IP Security Filter Action Wizard should be initiated when a new filter action is added by enabling the Use Add Wizard checkbox.
    • Authentication Methods tab; used to set the authentication method(s) which should be used for the rule. Options include Kerberos, digital certificates, or preshared keys. If you define more than one authentication method, you can set the order of precedence for the authentication methods.
    • Tunnel Setting tab; used to configure whether the rule should establish an IPSec tunnel with another end system.
    • Connection Type tab; used to set the connection type for the rule:
      • All Network Connections option
      • Local Area Network option.
      • Remote Access option.
  25. To close the Edit Rule Properties dialog box dialog box, click Cancel.
  26. To close the Server (Request Security) Properties dialog box of the Default IPSec policy, click Cancel.

Understanding How IPSec Policy is Applied

Whenever a computer starts, the IPSec Policy Agent service starts automatically too. The IPSec Policy Agent service running on the computer accesses IPSec policy information in either the Windows registry or in Active Directory.

The main functions which the IPSec Policy Agent provides are listed below:

  • The IPSec Policy Agent accesses IPSec policy information from the local Windows registry when the computer does not belong to a domain.
  • The IPSec Policy Agent accesses IPSec policy information from the Active Directory when the computer is a member of a domain.
  • The IPSec Policy Agent scans IPSec policies for any configuration changes.
  • The IPSec Policy Agent passes information to the IPSec driver.

IPSec policies are accessed when the computer starts, and at the specific interval defined in the particular IPSec policy. For computers that belong to a domain in Active Directory but are however disconnected from the domain, then cached IPSec policy information is used.

As mentioned previously, the IPSec Policy Agent passes information to the IPSec driver. The IPSec driver performs a number of operations to enable secure network communication. The IPSec driver checks inbound and outbound packets to determine whether a packet matches criteria for secured communication. The IPSec driver checks the IP Filter List of the IPSec policy to determine this information. If a match is found, the IPSec driver uses the filter list and filter actions to determine how security should be applied.

A few functions performed by the IPSec driver are listed here:

  • Creates IPSec packets.
  • Generates checksums.
  • Initiates the IKE communication.
  • Adds the AH and ESP headers.
  • Encrypts data before it is transmitted.
  • Calculates hashes and checksums for incoming packets.

The IKE protocol is used by computers to create a security association (SA) and to exchange information to generate Diffie-Hellman keys. IKE manages and exchanges cryptographic keys so that computers can have a common set of security settings. Negotiation occurs on which authentication method, and encryption algorithm and hashing algorithm the computers will use. The computers negotiate and agree on a number of factors, including the following:

Determine whether the Authentication Header (AH) IPSec protocol should be used for the connection.

  • Determine whether the Encapsulating Security Payload (ESP) IPSec protocol should be used for the connection.
  • The connections that should be authenticated.
  • The encryption algorithm that should be used.
  • The algorithm that should be used to verify message integrity.

Understanding How the IPSec driver operates

The IPSec driver operates in the following three modes:

  • Computer startup mode: When the computer starts, the IPSec driver is loaded and the IPSec Policy Agent puts the IPSec driver in operational mode.

    In the Computer Startup mode, the IPSec driver can operate in any of the following modes:

    • Permit; the default mode if there are no IPSec policies defined for the computer. In Permit mode all traffic is allowed because no packets are filtered.
    • Stateful; the default mode if IPSec policy is applied for the computer. In this mode, outbound traffic is allowed. Unicast, multicast and broadcast inbound packets are dropped.
    • Block; only IP packets which match those filters defined to be used in this mode, and all DHCP-specific traffic is allowed.

    The configuration of the startup type of the IPSec service determines the mode in which the IPSec driver starts. The IPSec driver can start in one of the following modes:

    • Disabled; when the IPSec driver starts in Disabled mode, the following occurs:
      • The IPSec driver loads in Permit mode.
      • No packet filtering occurs.
      • No IPSec security occurs.
    • Manual; when the IPSec driver starts in Manual mode, the following occurs:
      • The IPSec driver loads in Permit mode.
      • No packet filtering occurs.
      • No IPSec security occurs.
    • Automatic; when the IPSec driver starts in Automatic mode, the following occurs:
      • The IPSec driver loads in the mode which was defined by the IPSec policy agent.
      • The IPSec driver loads in Stateful mode if there is IPSec policy applied.
      • The IPSec driver loads in Permit mode if there is no IPSec policy applied.
  • Operational mode: After the IPSec service has started, the IPSec driver moves to either of the following operational modes:
    • Secure; when the IPSec driver runs in Secure mode, the following occurs:
      • If no IPSec policy is assigned, then no IPSec security is applied.
      • The IPSec policy filters are applied for normal IPSec operations if IPSec policy is assigned.
      • IPSec security is applied after persistent policies are applied but before local policies and Active Directory policies are applied.
      • If there are no persistent policies, then IPSec security is applied after local policies and Active Directory policies are applied.
    • Permit; the IPSec driver runs in the Permit mode if the IPSec service was manually stopped on the computer. In Permit mode, the following occurs:
      • No packet filtering occurs.
      • No IPSec security occurs.
    • Block; when the IPSec driver runs in the Block mode, the following occurs:
      • No inbound traffic is allowed.
      • No outbound traffic is allowed.
  • Diagnostic mode: used to log inbound and outbound packet drop events when the IPSec driver runs in Startup mode and Operational mode. You first though have to enable logging because it is disabled by default. It is strongly recommended that you do not enable logging for a log time frame because the System log file can become full in a short period of time.

Configuring and Assigning IPSec Policy

You can use the IP Security Policy Management snap-in to manage IPSec policy, create IPSec policies edit existing IPSec policies, and assign IPSec policies. You can use the tool to addand remove filters which are applied to IPSec policies. If you are planning a Windows Server 2003 IPSec implementation, then you have to use the Windows Server 2003 IPSec Policy Management MMC snap-in if you want to use the latest IPSec features.

You can also configure IPSec using the Netsh command-line utility. The netsh command-line utility replaces the previously used Ipsecpol.exe command-line utility. The netsh command-line utility can be used to view information on IPSec policies, configure startup security for computers, and enable IPSec driver