Server 2008 Cluster Issues

There was an identical incident posted at:
http://www.microsoft.com/communities...a-a0ff03f55183

However, I thought I would pose it as a seperate question in the hope of
bringing additional attention to this problem. I am basically copying and
editing the previous posters' information as it is so close to our issue.

We have a problem with a Windows 2008 SP1 cluster. It has Exchange 2007 SP1
with Cumulative Rollup 4 installed. The second node is continually going
down. Here is what is going on:

• Cluster servers are IBM xSeries 3650s
• Using a IBM DS4800 SAN for shared storage
• The NIC configuration on the nodes is as follows:
o Onboard Broadcom adapter - v4.4.16.0
o 2 Intel PCI-X adapters
o 3 network connections setup: public - 10.2.105.x Intel
switch VLAN 1, private - 10.2.109.x Intel switch VLAN 2, private -
192.168.1.x Broadcom crossover cable
o We setup the 3 network connections to help eliminate the
network as the issue.
o IPv4 Connectivity only, no teaming
o Windows cluster validation does not report any issues.

The issue that we are seeing is that intermittently Node 2 gets kicked out
of the cluster and shuts down the cluster service generating an 1177 error in
the event log. Basically, this means that it lost quorum due to losing
connectivity with the cluster nodes. This sometimes happens 3 times an hour,
but might not happen for a few hours. The cluster service will always
automatically restart and everything is fine again for a period of time.

The problem is NOT isolated to Node 2 however. If we make Node 2 the cluster
owner, then Node 1 exhibits the problem, if Node 1 is the owner, then Node 2
exhibits the problem. Using Node and Disk Majority for quorum setting.

It looks like the nodes are losing network connectivity to each other based
on the cluster logs indicating the routes as down, but we now have 3 network
connections between the 2 nodes using 3 different adapters from 2 different
vendors. So I doubt this is the issue.

MS believes the issue to be storage related due to "error 170" appearances
in the cluster logs and indicates these are related to persistent reservation
problems. We have installed the latest MPIO from IBM which supposedly
resolves some of these types of issues. However, the problem continues. IBM
is also looking into this, but we await a solution.

Has anyone else ran into this problem? Suggestions? Any help is greatly
appreciated.

Dale,

This is not an issue with your Quorum disk. Your cluster node SHOULD fail
with a 170 error on the disk (The requested resource is in use) because the
resource is in use by the other cluster node. This is called quorum
arbitration and is supposed to happen when network connectivity fails
between the nodes.

If it were a disk issue, depending on your quorum model, either both nodes
would fail or just the disk resource would fail. Quorum disk failure
wouldn't cause just one node to fail in any quorum model. I'm a little
surprised that Microsoft PSS would even suggest this...you mustn't be
dealing with a cluster person. I'd request escalation to speak with someone
that is familiar with clustering so you have a chance at working towards a
resolution.

I would say that this is some sort of network issue. Have you disabled any
NIC offloading that your netword adapters might be doing? Disable this on
all 6 of your NIC cards.

Are all of your network connections going thru a common switch/router/hub?
If so, you might want to consider spreading these connections across
switches/routers to see if you can isolate a specific piece of hardware.

Make sure your NIC drivers are running the latest versions.

I'd also recommend dropping down to a single (Intel) NIC and see if the
issue persists. If your issue goes away, this is definitely a NIC driver
issue. You might then try adding back NICs until the issue reoccurs. I've
seen instances, specifically with Broadcom, where having a second network
connection caused the links to constantly bounce...that issue was corrected
in an update NIC driver.

Hope this helps.

Regards,
John

Visit my blog: http://msmvps.com/blogs/jtoner

"Dale Kiefer" <> wrote in message
news:F354C29E-05C4-4DF7-94C8-...
> There was an identical incident posted at:
>
http://www.microsoft.com/communities...ng&p=1&tid=c08
6d541-b2f7-4db1-899a-a0ff03f55183
>
> However, I thought I would pose it as a seperate question in the hope of
> bringing additional attention to this problem. I am basically copying and
> editing the previous posters' information as it is so close to our issue.
>
> We have a problem with a Windows 2008 SP1 cluster. It has Exchange 2007
SP1
> with Cumulative Rollup 4 installed. The second node is continually going
> down. Here is what is going on:
>
> . Cluster servers are IBM xSeries 3650s
> . Using a IBM DS4800 SAN for shared storage
> . The NIC configuration on the nodes is as follows:
> o Onboard Broadcom adapter - v4.4.16.0
> o 2 Intel PCI-X adapters
> o 3 network connections setup: public - 10.2.105.x Intel
> switch VLAN 1, private - 10.2.109.x Intel switch VLAN 2, private -
> 192.168.1.x Broadcom crossover cable
> o We setup the 3 network connections to help eliminate the
> network as the issue.
> o IPv4 Connectivity only, no teaming
> o Windows cluster validation does not report any issues.
>
> The issue that we are seeing is that intermittently Node 2 gets kicked out
> of the cluster and shuts down the cluster service generating an 1177 error
in
> the event log. Basically, this means that it lost quorum due to losing
> connectivity with the cluster nodes. This sometimes happens 3 times an
hour,
> but might not happen for a few hours. The cluster service will always
> automatically restart and everything is fine again for a period of time.
>
> The problem is NOT isolated to Node 2 however. If we make Node 2 the
cluster
> owner, then Node 1 exhibits the problem, if Node 1 is the owner, then Node
2
> exhibits the problem. Using Node and Disk Majority for quorum setting.
>
> It looks like the nodes are losing network connectivity to each other
based
> on the cluster logs indicating the routes as down, but we now have 3
network
> connections between the 2 nodes using 3 different adapters from 2
different
> vendors. So I doubt this is the issue.
>
> MS believes the issue to be storage related due to "error 170" appearances
> in the cluster logs and indicates these are related to persistent
reservation
> problems. We have installed the latest MPIO from IBM which supposedly
> resolves some of these types of issues. However, the problem continues.
IBM
> is also looking into this, but we await a solution.
>
> Has anyone else ran into this problem? Suggestions? Any help is greatly
> appreciated.

Also look for any applications which may be temporarily blocking all network
traffic... such as a Firewall... or IPSec policies getting refreshed.

If you have a group policy which is causing the IPSec policy to get
refreshed, when IPSec updates it's policies it closes all network traffic...
and since clustering is watching the health of the network, when it can't
communicate with the node it considers it a failure.

Thanks for the replies. I will let you know where we get to in the next few
days.

"Elden" wrote:

> Also look for any applications which may be temporarily blocking all network
> traffic... such as a Firewall... or IPSec policies getting refreshed.
>
> If you have a group policy which is causing the IPSec policy to get
> refreshed, when IPSec updates it's policies it closes all network traffic...
> and since clustering is watching the health of the network, when it can't
> communicate with the node it considers it a failure.

It appears we have fixed the problem. IBM noticed in the support logs that
the server wasn't seeing 52 LUNs (26 being presented, 2 HBAs). This led us
to the HBA BIOS/drivers. We were running the drivers/BIOS from the IBM
xSeries site and not the IBM storage subsystem site for our DS4800.
Apparently this is an important distinction. We could not update the HBA
drivers because our DS4800 is not running the latest firmware (this is a
major upgrade and only recommended if required). We did update our HBA BIOS
version and it has now been 24 hours without any problems. The problem
occurred every 1-2 hours and sometimes multiple times in an hour prior to
this update so we are very hopeful that a fix has been found.

Thank you for your responses. I will post any additional relevant
information if available.

"Dale Kiefer" wrote:

> Thanks for the replies. I will let you know where we get to in the next few
> days.
>
> "Elden" wrote:
>
> > Also look for any applications which may be temporarily blocking all network
> > traffic... such as a Firewall... or IPSec policies getting refreshed.
> >
> > If you have a group policy which is causing the IPSec policy to get
> > refreshed, when IPSec updates it's policies it closes all network traffic...
> > and since clustering is watching the health of the network, when it can't
> > communicate with the node it considers it a failure.

Disregard my previous post, the issues have returned. I will provide further
information later.

"Dale Kiefer" wrote:

> It appears we have fixed the problem. IBM noticed in the support logs that
> the server wasn't seeing 52 LUNs (26 being presented, 2 HBAs). This led us
> to the HBA BIOS/drivers. We were running the drivers/BIOS from the IBM
> xSeries site and not the IBM storage subsystem site for our DS4800.
> Apparently this is an important distinction. We could not update the HBA
> drivers because our DS4800 is not running the latest firmware (this is a
> major upgrade and only recommended if required). We did update our HBA BIOS
> version and it has now been 24 hours without any problems. The problem
> occurred every 1-2 hours and sometimes multiple times in an hour prior to
> this update so we are very hopeful that a fix has been found.
>
> Thank you for your responses. I will post any additional relevant
> information if available.
>
> "Dale Kiefer" wrote:
>
> > Thanks for the replies. I will let you know where we get to in the next few
> > days.
> >
> > "Elden" wrote:
> >
> > > Also look for any applications which may be temporarily blocking all network
> > > traffic... such as a Firewall... or IPSec policies getting refreshed.
> > >
> > > If you have a group policy which is causing the IPSec policy to get
> > > refreshed, when IPSec updates it's policies it closes all network traffic...
> > > and since clustering is watching the health of the network, when it can't
> > > communicate with the node it considers it a failure.

I seem to have come across similar problems:

We have 2 datacenters both running identical hardware and we are testing
upgrade for exchange to exchange 2007.
I built 2 x Windows 2008 SP1 clusters one in each DC and put exchange 2007
clustered mailbox role on them and standby continuous replication between
them.

Hardware is running on IBM BladeCenter HS21 blades
Storage is on IBM System Storage DS4700 storage boxes.

Have latest drivers and patches applied and also Exchange SP1 Rollup Update 5.
The system that is on the receiving end of the standby relication does not
seem to suffer from the issues however the Exchange box that is running live
gets after couple of days the following errors:

Source: Microsoft-Windows-FailoverClustering
Event ID: 1069
Level: Error
Description: Cluster resource 'FileServer-(E2K7CLUS02)(Cluster Disk 9)' in
clustered service or application 'E2K7CLUS02' failed.

Source: Microsoft-Windows-FailoverClustering
Event ID: 1230
Level: Error
User: SYSTEM
Computer: E2K7NODE2.domain.local
Description: Cluster resource 'FileServer-(E2K7CLUS02)(Cluster Disk 10)'
(resource type '', DLL 'clusres.dll') either crashed or deadlocked. The
Resource Hosting Subsystem (RHS) process will now attempt to terminate, and
the resource will be marked to run in a separate monitor.

After these errors come, the Cluster administrator will hang and Windows
Explorer will also hang and not ba able to browse the drives.
Exchange will however continue to work and mail will keep flowing and
clients can still access mailboxes. To restore the functionality of admin
tools so far I have found nothing short of reboot that would do the trick.

When I move the cluster over to the standby cluster in the other DC the same
problems start there.
I will try the firmware upgrade of the storage in the next couple of days in
one of the DC-s to se it it has any impact. It does look like it is some sort
of issue between IBM storage or storage drivers and the Windows 2008 SP1
cluster. We have multiple Windows 2003 clusters running off the same storage
boxes and so far we have had no errors on any of them.



"Dale Kiefer" wrote:

> There was an identical incident posted at:
> http://www.microsoft.com/communities...a-a0ff03f55183
>
> However, I thought I would pose it as a seperate question in the hope of
> bringing additional attention to this problem. I am basically copying and
> editing the previous posters' information as it is so close to our issue.
>
> We have a problem with a Windows 2008 SP1 cluster. It has Exchange 2007 SP1
> with Cumulative Rollup 4 installed. The second node is continually going
> down. Here is what is going on:
>
> • Cluster servers are IBM xSeries 3650s
> • Using a IBM DS4800 SAN for shared storage
> • The NIC configuration on the nodes is as follows:
> o Onboard Broadcom adapter - v4.4.16.0
> o 2 Intel PCI-X adapters
> o 3 network connections setup: public - 10.2.105.x Intel
> switch VLAN 1, private - 10.2.109.x Intel switch VLAN 2, private -
> 192.168.1.x Broadcom crossover cable
> o We setup the 3 network connections to help eliminate the
> network as the issue.
> o IPv4 Connectivity only, no teaming
> o Windows cluster validation does not report any issues.
>
> The issue that we are seeing is that intermittently Node 2 gets kicked out
> of the cluster and shuts down the cluster service generating an 1177 error in
> the event log. Basically, this means that it lost quorum due to losing
> connectivity with the cluster nodes. This sometimes happens 3 times an hour,
> but might not happen for a few hours. The cluster service will always
> automatically restart and everything is fine again for a period of time.
>
> The problem is NOT isolated to Node 2 however. If we make Node 2 the cluster
> owner, then Node 1 exhibits the problem, if Node 1 is the owner, then Node 2
> exhibits the problem. Using Node and Disk Majority for quorum setting.
>
> It looks like the nodes are losing network connectivity to each other based
> on the cluster logs indicating the routes as down, but we now have 3 network
> connections between the 2 nodes using 3 different adapters from 2 different
> vendors. So I doubt this is the issue.
>
> MS believes the issue to be storage related due to "error 170" appearances
> in the cluster logs and indicates these are related to persistent reservation
> problems. We have installed the latest MPIO from IBM which supposedly
> resolves some of these types of issues. However, the problem continues. IBM
> is also looking into this, but we await a solution.
>
> Has anyone else ran into this problem? Suggestions? Any help is greatly
> appreciated.

SCC. This sounds a lot like http://support.microsoft.com/kb/953652


"Dale Kiefer" <> wrote in message
newsBE0F834-92CF-4517-B755-...
> Disregard my previous post, the issues have returned. I will provide
> further
> information later.
>
> "Dale Kiefer" wrote:
>
>> It appears we have fixed the problem. IBM noticed in the support logs
>> that
>> the server wasn't seeing 52 LUNs (26 being presented, 2 HBAs). This led
>> us
>> to the HBA BIOS/drivers. We were running the drivers/BIOS from the IBM
>> xSeries site and not the IBM storage subsystem site for our DS4800.
>> Apparently this is an important distinction. We could not update the HBA
>> drivers because our DS4800 is not running the latest firmware (this is a
>> major upgrade and only recommended if required). We did update our HBA
>> BIOS
>> version and it has now been 24 hours without any problems. The problem
>> occurred every 1-2 hours and sometimes multiple times in an hour prior to
>> this update so we are very hopeful that a fix has been found.
>>
>> Thank you for your responses. I will post any additional relevant
>> information if available.
>>
>> "Dale Kiefer" wrote:
>>
>> > Thanks for the replies. I will let you know where we get to in the
>> > next few
>> > days.
>> >
>> > "Elden" wrote:
>> >
>> > > Also look for any applications which may be temporarily blocking all
>> > > network
>> > > traffic... such as a Firewall... or IPSec policies getting refreshed.
>> > >
>> > > If you have a group policy which is causing the IPSec policy to get
>> > > refreshed, when IPSec updates it's policies it closes all network
>> > > traffic...
>> > > and since clustering is watching the health of the network, when it
>> > > can't
>> > > communicate with the node it considers it a failure.

We have the exact same issues you are experiencing with the cluster becoming
non-responsive. I have also posted about this in the Exchange forums at:

http://www.microsoft.com/communities...2-f8f2de77d68f

Please let me know how you make out with the storage firmware upgrade or any
other changes. I am not running in a test environment so I don't have the
ability to make changes as easily.

What version of firmware are you upgrading from/to?

I'm glad to see a few more people are bumping into these same problems.
Hopefully we can find a resolution soon.

"Priit Vosu" wrote:

> I seem to have come across similar problems:
>
> We have 2 datacenters both running identical hardware and we are testing
> upgrade for exchange to exchange 2007.
> I built 2 x Windows 2008 SP1 clusters one in each DC and put exchange 2007
> clustered mailbox role on them and standby continuous replication between
> them.
>
> Hardware is running on IBM BladeCenter HS21 blades
> Storage is on IBM System Storage DS4700 storage boxes.
>
> Have latest drivers and patches applied and also Exchange SP1 Rollup Update 5.
> The system that is on the receiving end of the standby relication does not
> seem to suffer from the issues however the Exchange box that is running live
> gets after couple of days the following errors:
>
> Source: Microsoft-Windows-FailoverClustering
> Event ID: 1069
> Level: Error
> Description: Cluster resource 'FileServer-(E2K7CLUS02)(Cluster Disk 9)' in
> clustered service or application 'E2K7CLUS02' failed.
>
> Source: Microsoft-Windows-FailoverClustering
> Event ID: 1230
> Level: Error
> User: SYSTEM
> Computer: E2K7NODE2.domain.local
> Description: Cluster resource 'FileServer-(E2K7CLUS02)(Cluster Disk 10)'
> (resource type '', DLL 'clusres.dll') either crashed or deadlocked. The
> Resource Hosting Subsystem (RHS) process will now attempt to terminate, and
> the resource will be marked to run in a separate monitor.
>
> After these errors come, the Cluster administrator will hang and Windows
> Explorer will also hang and not ba able to browse the drives.
> Exchange will however continue to work and mail will keep flowing and
> clients can still access mailboxes. To restore the functionality of admin
> tools so far I have found nothing short of reboot that would do the trick.
>
> When I move the cluster over to the standby cluster in the other DC the same
> problems start there.
> I will try the firmware upgrade of the storage in the next couple of days in
> one of the DC-s to se it it has any impact. It does look like it is some sort
> of issue between IBM storage or storage drivers and the Windows 2008 SP1
> cluster. We have multiple Windows 2003 clusters running off the same storage
> boxes and so far we have had no errors on any of them.
>
>
>
> "Dale Kiefer" wrote:
>
> > There was an identical incident posted at:
> > http://www.microsoft.com/communities...a-a0ff03f55183
> >
> > However, I thought I would pose it as a seperate question in the hope of
> > bringing additional attention to this problem. I am basically copying and
> > editing the previous posters' information as it is so close to our issue.
> >
> > We have a problem with a Windows 2008 SP1 cluster. It has Exchange 2007 SP1
> > with Cumulative Rollup 4 installed. The second node is continually going
> > down. Here is what is going on:
> >
> > • Cluster servers are IBM xSeries 3650s
> > • Using a IBM DS4800 SAN for shared storage
> > • The NIC configuration on the nodes is as follows:
> > o Onboard Broadcom adapter - v4.4.16.0
> > o 2 Intel PCI-X adapters
> > o 3 network connections setup: public - 10.2.105.x Intel
> > switch VLAN 1, private - 10.2.109.x Intel switch VLAN 2, private -
> > 192.168.1.x Broadcom crossover cable
> > o We setup the 3 network connections to help eliminate the
> > network as the issue.
> > o IPv4 Connectivity only, no teaming
> > o Windows cluster validation does not report any issues.
> >
> > The issue that we are seeing is that intermittently Node 2 gets kicked out
> > of the cluster and shuts down the cluster service generating an 1177 error in
> > the event log. Basically, this means that it lost quorum due to losing
> > connectivity with the cluster nodes. This sometimes happens 3 times an hour,
> > but might not happen for a few hours. The cluster service will always
> > automatically restart and everything is fine again for a period of time.
> >
> > The problem is NOT isolated to Node 2 however. If we make Node 2 the cluster
> > owner, then Node 1 exhibits the problem, if Node 1 is the owner, then Node 2
> > exhibits the problem. Using Node and Disk Majority for quorum setting.
> >
> > It looks like the nodes are losing network connectivity to each other based
> > on the cluster logs indicating the routes as down, but we now have 3 network
> > connections between the 2 nodes using 3 different adapters from 2 different
> > vendors. So I doubt this is the issue.
> >
> > MS believes the issue to be storage related due to "error 170" appearances
> > in the cluster logs and indicates these are related to persistent reservation
> > problems. We have installed the latest MPIO from IBM which supposedly
> > resolves some of these types of issues. However, the problem continues. IBM
> > is also looking into this, but we await a solution.
> >
> > Has anyone else ran into this problem? Suggestions? Any help is greatly
> > appreciated.

I upgraded the IBM DS4700 firmware from 6.23.05.02 to 7.36.08.00
I also upgraded the storage MPIO drivers on the Windows 2008 server to 10.36
version and since then the system has been stabile for a week which is not
very long time, but previously the error occured way sooner then that.

I am also running standby replication off that server and I still get
occasional Event ID: 2082 errors, but the replication itself is not breaking
and the servers are working correctly so far.

We are planning to make the same upgrade in couple of days in the production
environment if all will be still ok at that time. So far it does seam that
the firmware and MPIO driver update have made the system more stabile and
maybe even fixed it.


"Dale Kiefer" wrote:

> We have the exact same issues you are experiencing with the cluster becoming
> non-responsive. I have also posted about this in the Exchange forums at:
>
> http://www.microsoft.com/communities...2-f8f2de77d68f
>
> Please let me know how you make out with the storage firmware upgrade or any
> other changes. I am not running in a test environment so I don't have the
> ability to make changes as easily.
>
> What version of firmware are you upgrading from/to?
>
> I'm glad to see a few more people are bumping into these same problems.
> Hopefully we can find a resolution soon.
>
> "Priit Vosu" wrote:
>
> > I seem to have come across similar problems:
> >
> > We have 2 datacenters both running identical hardware and we are testing
> > upgrade for exchange to exchange 2007.
> > I built 2 x Windows 2008 SP1 clusters one in each DC and put exchange 2007
> > clustered mailbox role on them and standby continuous replication between
> > them.
> >
> > Hardware is running on IBM BladeCenter HS21 blades
> > Storage is on IBM System Storage DS4700 storage boxes.
> >
> > Have latest drivers and patches applied and also Exchange SP1 Rollup Update 5.
> > The system that is on the receiving end of the standby relication does not
> > seem to suffer from the issues however the Exchange box that is running live
> > gets after couple of days the following errors:
> >
> > Source: Microsoft-Windows-FailoverClustering
> > Event ID: 1069
> > Level: Error
> > Description: Cluster resource 'FileServer-(E2K7CLUS02)(Cluster Disk 9)' in
> > clustered service or application 'E2K7CLUS02' failed.
> >
> > Source: Microsoft-Windows-FailoverClustering
> > Event ID: 1230
> > Level: Error
> > User: SYSTEM
> > Computer: E2K7NODE2.domain.local
> > Description: Cluster resource 'FileServer-(E2K7CLUS02)(Cluster Disk 10)'
> > (resource type '', DLL 'clusres.dll') either crashed or deadlocked. The
> > Resource Hosting Subsystem (RHS) process will now attempt to terminate, and
> > the resource will be marked to run in a separate monitor.
> >
> > After these errors come, the Cluster administrator will hang and Windows
> > Explorer will also hang and not ba able to browse the drives.
> > Exchange will however continue to work and mail will keep flowing and
> > clients can still access mailboxes. To restore the functionality of admin
> > tools so far I have found nothing short of reboot that would do the trick.
> >
> > When I move the cluster over to the standby cluster in the other DC the same
> > problems start there.
> > I will try the firmware upgrade of the storage in the next couple of days in
> > one of the DC-s to se it it has any impact. It does look like it is some sort
> > of issue between IBM storage or storage drivers and the Windows 2008 SP1
> > cluster. We have multiple Windows 2003 clusters running off the same storage
> > boxes and so far we have had no errors on any of them.
> >
> >
> >
> > "Dale Kiefer" wrote:
> >
> > > There was an identical incident posted at:
> > > http://www.microsoft.com/communities...a-a0ff03f55183
> > >
> > > However, I thought I would pose it as a seperate question in the hope of
> > > bringing additional attention to this problem. I am basically copying and
> > > editing the previous posters' information as it is so close to our issue.
> > >
> > > We have a problem with a Windows 2008 SP1 cluster. It has Exchange 2007 SP1
> > > with Cumulative Rollup 4 installed. The second node is continually going
> > > down. Here is what is going on:
> > >
> > > • Cluster servers are IBM xSeries 3650s
> > > • Using a IBM DS4800 SAN for shared storage
> > > • The NIC configuration on the nodes is as follows:
> > > o Onboard Broadcom adapter - v4.4.16.0
> > > o 2 Intel PCI-X adapters
> > > o 3 network connections setup: public - 10.2.105.x Intel
> > > switch VLAN 1, private - 10.2.109.x Intel switch VLAN 2, private -
> > > 192.168.1.x Broadcom crossover cable
> > > o We setup the 3 network connections to help eliminate the
> > > network as the issue.
> > > o IPv4 Connectivity only, no teaming
> > > o Windows cluster validation does not report any issues.
> > >
> > > The issue that we are seeing is that intermittently Node 2 gets kicked out
> > > of the cluster and shuts down the cluster service generating an 1177 error in
> > > the event log. Basically, this means that it lost quorum due to losing
> > > connectivity with the cluster nodes. This sometimes happens 3 times an hour,
> > > but might not happen for a few hours. The cluster service will always
> > > automatically restart and everything is fine again for a period of time.
> > >
> > > The problem is NOT isolated to Node 2 however. If we make Node 2 the cluster
> > > owner, then Node 1 exhibits the problem, if Node 1 is the owner, then Node 2
> > > exhibits the problem. Using Node and Disk Majority for quorum setting.
> > >
> > > It looks like the nodes are losing network connectivity to each other based
> > > on the cluster logs indicating the routes as down, but we now have 3 network
> > > connections between the 2 nodes using 3 different adapters from 2 different
> > > vendors. So I doubt this is the issue.
> > >
> > > MS believes the issue to be storage related due to "error 170" appearances
> > > in the cluster logs and indicates these are related to persistent reservation
> > > problems. We have installed the latest MPIO from IBM which supposedly
> > > resolves some of these types of issues. However, the problem continues. IBM
> > > is also looking into this, but we await a solution.
> > >
> > > Has anyone else ran into this problem? Suggestions? Any help is greatly
> > > appreciated.