IPSEC Failing (Secure Server)

Server A has local policy configured as Secure Server(Require Security).
Client B has local policy configured as Client(Respond Only). Both A and B
are members of the same W2K3 AD domain. Event log error on Server A: IKE
security ssociation failed: Key Exchange Mode (Main Mode). Further down it
says, Failure Point: Me, Failure Reason: Failed to authenticate using
kerberos.

Doing some trouble shooting, I found that if I changed the policy on Server
A to Server(Request Security) the communication did occur and was
encapsulated (verified using NetMon). I also could get this to work if,
leaving the policy on Server A on Secure Server, I changed the policy on
Client B to Server(Request Security).

this may help. quoted from http://www.ChicagoTech.net
Troubleshooting IPSec
1. Audit Policy: To troubleshoot IPSec when it does not behave the way that
you expect it to, first check the results of the Phase One and Phase Two
exchanges by enabling Audit Policy, which causes security events to be
logged in the security log of the Event Viewer.
2. Netdiag: netdiag /test:ipsec /debug. If both Phases are Outbound or
Inbound, check Tunnel Settings.
3. If the logged events indicate that Phase One Main Mode exchange is
failing, do both of the following: 1) Check the IKE settings in your IPSec
policy properties: Click the General tab, click the Advanced tab, and then
click the Methods tab. 2) Check the configured IKE authentication methods in
your IPSec policy properties: Select the IP Security rule that you want to
check, click Edit, and then click the Authentication Methods tab.
4. If the logged events indicate that Phase Two Quick Mode is failing, check
the IPSec security methods configured on your IPSec rules in your IPSec
policy properties: Select the IP Security rule that you want to check, click
Edit, select the Filter Action tab, select the filter action that is
enabled, and then click Edit.
5. IP Security Monitor: The IP Security Monitor can be used to monitor SAs,
IPSec, and IKE statistics. To start IP Security Monitor, click Start, click
Run, and then type ipsecmon.
6. Checking Oakley Log: To enable Oakley Log, use Registry Editor to locate
the following key in the registry, and if it does not exist, create it:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Servic es\PolicyAgent\Oakley
Add a REG_DWORD value named EnableLogging with a value of 1 to this key. The
Oakley.log file is created in the %SystemRoot%\debug folder. NOTE: A value
of 0 for EnableLogging disables logging.
7. Check VPN server log.


--
For more and other information, go to http://www.ChicagoTech.net

Don't send e-mail or reply to me except you need consulting services.
Posting on MS newsgroup will benefit all readers and you may get more help.

Bob Lin, MS-MVP, MCSE & CNE
Networking, Internet, Routing, VPN, Anti-Virus, Tips & Troubleshooting on
http://www.ChicagoTech.net
Networking Solutions, http://www.chicagotech.net/networksolutions.htm
VPN Solutions, http://www.chicagotech.net/vpnsolutions.htm
VPN Process and Error Analysis, http://www.chicagotech.net/VPN%20process.htm
VPN Troubleshooting, http://www.chicagotech.net/vpn.htm
This posting is provided "AS IS" with no warranties.
"Aaron" <> wrote in message
news:4DEDBBBE-DA95-4CBB-9803-...
> Server A has local policy configured as Secure Server(Require Security).
> Client B has local policy configured as Client(Respond Only). Both A and
> B
> are members of the same W2K3 AD domain. Event log error on Server A: IKE
> security ssociation failed: Key Exchange Mode (Main Mode). Further down
> it
> says, Failure Point: Me, Failure Reason: Failed to authenticate using
> kerberos.
>
> Doing some trouble shooting, I found that if I changed the policy on
> Server
> A to Server(Request Security) the communication did occur and was
> encapsulated (verified using NetMon). I also could get this to work if,
> leaving the policy on Server A on Secure Server, I changed the policy on
> Client B to Server(Request Security).
>
>

You must exempt domain controller from your ipsec policy as domain
controllers can not use ipsec to communicate with domain members because
they are the kerberos distribution centers. Modify your ipsec policy by
adding a new rule to it with a permit filter action and a filter with a
mirrored entry for all traffic for domain controllers listed by their IP
addresses . Reboot your server after configuring the ipsec policy and use
the ipsecmon mmc snapin to verify that the new policy is in effect that
exempts domain controllers. --- Steve



"Aaron" <> wrote in message
news:4DEDBBBE-DA95-4CBB-9803-...
> Server A has local policy configured as Secure Server(Require Security).
> Client B has local policy configured as Client(Respond Only). Both A and
> B
> are members of the same W2K3 AD domain. Event log error on Server A: IKE
> security ssociation failed: Key Exchange Mode (Main Mode). Further down
> it
> says, Failure Point: Me, Failure Reason: Failed to authenticate using
> kerberos.
>
> Doing some trouble shooting, I found that if I changed the policy on
> Server
> A to Server(Request Security) the communication did occur and was
> encapsulated (verified using NetMon). I also could get this to work if,
> leaving the policy on Server A on Secure Server, I changed the policy on
> Client B to Server(Request Security).
>
>

I can see why having a 'Secure Server' policy would prevent Server A from
being able to communicate with the DC. Can you tell me why it works when I
configure the client to use the Server(Request Security) Setting?


"Steven L Umbach" <> wrote in message
news:uh$...
> You must exempt domain controller from your ipsec policy as domain
> controllers can not use ipsec to communicate with domain members because
> they are the kerberos distribution centers. Modify your ipsec policy by
> adding a new rule to it with a permit filter action and a filter with a
> mirrored entry for all traffic for domain controllers listed by their IP
> addresses . Reboot your server after configuring the ipsec policy and use
> the ipsecmon mmc snapin to verify that the new policy is in effect that
> exempts domain controllers. --- Steve
>
>
>
> "Aaron" <> wrote in message
> news:4DEDBBBE-DA95-4CBB-9803-...
> > Server A has local policy configured as Secure Server(Require Security).
> > Client B has local policy configured as Client(Respond Only). Both A
and
> > B
> > are members of the same W2K3 AD domain. Event log error on Server A:
IKE
> > security ssociation failed: Key Exchange Mode (Main Mode). Further
down
> > it
> > says, Failure Point: Me, Failure Reason: Failed to authenticate using
> > kerberos.
> >
> > Doing some trouble shooting, I found that if I changed the policy on
> > Server
> > A to Server(Request Security) the communication did occur and was
> > encapsulated (verified using NetMon). I also could get this to work if,
> > leaving the policy on Server A on Secure Server, I changed the policy on
> > Client B to Server(Request Security).
> >
> >
>
>

This message was posted by me. Sorry for the 'generic' display name.

"microsoft" wrote:

> I can see why having a 'Secure Server' policy would prevent Server A from
> being able to communicate with the DC. Can you tell me why it works when I
> configure the client to use the Server(Request Security) Setting?
>
>
> "Steven L Umbach" <> wrote in message
> news:uh$...
> > You must exempt domain controller from your ipsec policy as domain
> > controllers can not use ipsec to communicate with domain members because
> > they are the kerberos distribution centers. Modify your ipsec policy by
> > adding a new rule to it with a permit filter action and a filter with a
> > mirrored entry for all traffic for domain controllers listed by their IP
> > addresses . Reboot your server after configuring the ipsec policy and use
> > the ipsecmon mmc snapin to verify that the new policy is in effect that
> > exempts domain controllers. --- Steve
> >
> >
> >
> > "Aaron" <> wrote in message
> > news:4DEDBBBE-DA95-4CBB-9803-...
> > > Server A has local policy configured as Secure Server(Require Security).
> > > Client B has local policy configured as Client(Respond Only). Both A
> and
> > > B
> > > are members of the same W2K3 AD domain. Event log error on Server A:
> IKE
> > > security ssociation failed: Key Exchange Mode (Main Mode). Further
> down
> > > it
> > > says, Failure Point: Me, Failure Reason: Failed to authenticate using
> > > kerberos.
> > >
> > > Doing some trouble shooting, I found that if I changed the policy on
> > > Server
> > > A to Server(Request Security) the communication did occur and was
> > > encapsulated (verified using NetMon). I also could get this to work if,
> > > leaving the policy on Server A on Secure Server, I changed the policy on
> > > Client B to Server(Request Security).
> > >
> > >
> >
> >
>
>
>

When you have the "request" policy does that apply to the domain controller
also or just to the server?? I have found that even with the request ipsec
policy that problems can arise if the domain controllers try to engage in
ipsec negotiation with a domain member. After you configure the request
policy be sure to reboot the computer to see if the user can logon to the
domain and make sure that cached logons are disabled via Local Security
Policy security settings/local policies/security options - number of
previous logons to cache and set it to zero. If they are not it may appear
that you are logging onto the domain while you actually are not. One would
think that request would work with domain controllers but it can cause
problems. I am not sure of the exact technical details but it has to do with
the kerberos authentication process used for machine authentication. The
ipsecmon and ipsecpolicy mmc snapins can help determine what ipsec policy is
applied to a computer. The Windows 2003 Deployment Kit has a great article
on deploying ipsec and discusses the need for exempting domain controllers.
You can download the full chapter or read it online at the link below. ---
Steve


http://www.microsoft.com/downloads/d...displaylang=en
-- download chapter six from this link.
http://tinyurl.com/49pn9 -- same link as above, shorter.

"microsoft" <> wrote in message
news:...
>I can see why having a 'Secure Server' policy would prevent Server A from
> being able to communicate with the DC. Can you tell me why it works when
> I
> configure the client to use the Server(Request Security) Setting?
>
>
> "Steven L Umbach" <> wrote in message
> news:uh$...
>> You must exempt domain controller from your ipsec policy as domain
>> controllers can not use ipsec to communicate with domain members because
>> they are the kerberos distribution centers. Modify your ipsec policy by
>> adding a new rule to it with a permit filter action and a filter with a
>> mirrored entry for all traffic for domain controllers listed by their IP
>> addresses . Reboot your server after configuring the ipsec policy and use
>> the ipsecmon mmc snapin to verify that the new policy is in effect that
>> exempts domain controllers. --- Steve
>>
>>
>>
>> "Aaron" <> wrote in message
>> news:4DEDBBBE-DA95-4CBB-9803-...
>> > Server A has local policy configured as Secure Server(Require
>> > Security).
>> > Client B has local policy configured as Client(Respond Only). Both A
> and
>> > B
>> > are members of the same W2K3 AD domain. Event log error on Server A:
> IKE
>> > security ssociation failed: Key Exchange Mode (Main Mode). Further
> down
>> > it
>> > says, Failure Point: Me, Failure Reason: Failed to authenticate using
>> > kerberos.
>> >
>> > Doing some trouble shooting, I found that if I changed the policy on
>> > Server
>> > A to Server(Request Security) the communication did occur and was
>> > encapsulated (verified using NetMon). I also could get this to work
>> > if,
>> > leaving the policy on Server A on Secure Server, I changed the policy
>> > on
>> > Client B to Server(Request Security).
>> >
>> >
>>
>>
>
>