iscsi multipath fails when cluster service is started

hi,

i' ve configured a cluster with two nodes. each node has the following
network interfaces:

public network (1)
cluster-heartbeat (1)
iscsi network (2)

the iscsi infaces are in the same subnet and are disabled in the cluster
configuration.
we use a equallogic ps 5000 iscsi storage and two dedicated iscsi network
switches.
i've also installed and configured the equallogic mpio software.

when i pull one of the two iscsi interfaces and the cluster service is
stopped, everything works fine. but when the cluster service is started i got
timeouts (ping and disk) and the host is completely isolated from the iscsi
network.

we use broadcom nics for the iscsi connection, but i also tried intel nics.

any ideas why the cluster service affects the iscsi interfaces?

i think that it can't be an iscsi or storage problem, because the ping fails
too.

note:
i can reproduce this behavior on each cluster node and i also have
completely reinstalled each node with windows server 2003 enterprise x64.

Based on your description, it sounds like when you are pulling the iSCSI
cable, this is causing access failures to the disk which is why your cluster
service would be affected by pulling this cable.

Assuming that the above is correct, it sounds like your iSCSI initiator is
not configured properly. If you've got the iscsicli command available, check
out the output of the following command:

iscsicli.exe SessionList

For each connection, check the "Target Portal" value. If this is the same
for all sessions or if is set to 0.0.0.0, then your initiator is not setup
correctly. When adding your iSCSI targets, you need to click on the
"Advanced" button and make sure that you specify the Source IP for each
connection.

Hope this helps.

Regards,
John

Visit my blog: http://msmvps.com/blogs/jtoner

"MarkusN" <> wrote in message
news:BF95395B-5255-4DB2-B3EB-...
>i think that it can't be an iscsi or storage problem, because the ping
>fails
> too.
>
> note:
> i can reproduce this behavior on each cluster node and i also have
> completely reinstalled each node with windows server 2003 enterprise x64.
>
>

the initiator is configured fine and i can see the connections on the host
and on the storage site.
the problem is that the whole ip stack colapses, when i pull one interface.
(but only the interface that is not listed in the cluster admin network.
there is listed only one interface, because of the same subnet.)
the host is completely isolated from the iscsi network.

when i disable the interface in the windows network connections, the
failover works fine.



"John Toner [MVP]" wrote:

> Based on your description, it sounds like when you are pulling the iSCSI
> cable, this is causing access failures to the disk which is why your cluster
> service would be affected by pulling this cable.
>
> Assuming that the above is correct, it sounds like your iSCSI initiator is
> not configured properly. If you've got the iscsicli command available, check
> out the output of the following command:
>
> iscsicli.exe SessionList
>
> For each connection, check the "Target Portal" value. If this is the same
> for all sessions or if is set to 0.0.0.0, then your initiator is not setup
> correctly. When adding your iSCSI targets, you need to click on the
> "Advanced" button and make sure that you specify the Source IP for each
> connection.
>
> Hope this helps.
>
> Regards,
> John
>
> Visit my blog: http://msmvps.com/blogs/jtoner
>
> "MarkusN" <> wrote in message
> news:BF95395B-5255-4DB2-B3EB-...
> >i think that it can't be an iscsi or storage problem, because the ping
> >fails
> > too.
> >
> > note:
> > i can reproduce this behavior on each cluster node and i also have
> > completely reinstalled each node with windows server 2003 enterprise x64.
> >
> >
>
>
>

I'd recommend dropping down to a single iSCSI connection and removing any
multipathing software and see if the same behaviour occurs. If so, you will
probably need to open a ticket with MSFT to troubleshoot this further.

If dropping down to a single path resolves this issue, you'll want to work
with your network/iSCSI vendor to resolve this issue.

Regards,
John

Visit my blog: http://msmvps.com/blogs/jtoner

"MarkusN" <> wrote in message
news:1E5BB2EC-6F1C-457C-B4FF-...
> the initiator is configured fine and i can see the connections on the host
> and on the storage site.
> the problem is that the whole ip stack colapses, when i pull one
> interface.
> (but only the interface that is not listed in the cluster admin network.
> there is listed only one interface, because of the same subnet.)
> the host is completely isolated from the iscsi network.
>
> when i disable the interface in the windows network connections, the
> failover works fine.

i'm already in contact with the storage/server/network/iscsi vendor. it's dell.
but it seems to be a simple windows problem.

i've configured a virutal machine with windows server 2003 under esx4 to
test the behavior of the network inferfaces.
both interfaces are in the same subnet with no gateway.
i've started a ping to a reachable host. when i disable the network
connection of the vm, i only got 2 timeouts and then it goes on.

when i do the same procedure on my physical server with installed cluster
services, the ping stops and the host is isolated even though the second
interface still has a connection.
is this a common behavior under windows server 2003 enterprise x64 with
cluster service?
i think the problem is that the interfaces are in the same subnet.
there is only one of two interfaces in the cluster administrator under
"networks" and the missing interfaces causes the problem.


"John Toner [MVP]" wrote:

> I'd recommend dropping down to a single iSCSI connection and removing any
> multipathing software and see if the same behaviour occurs. If so, you will
> probably need to open a ticket with MSFT to troubleshoot this further.
>
> If dropping down to a single path resolves this issue, you'll want to work
> with your network/iSCSI vendor to resolve this issue.
>
> Regards,
> John
>
> Visit my blog: http://msmvps.com/blogs/jtoner
>
> "MarkusN" <> wrote in message
> news:1E5BB2EC-6F1C-457C-B4FF-...
> > the initiator is configured fine and i can see the connections on the host
> > and on the storage site.
> > the problem is that the whole ip stack colapses, when i pull one
> > interface.
> > (but only the interface that is not listed in the cluster admin network.
> > there is listed only one interface, because of the same subnet.)
> > the host is completely isolated from the iscsi network.
> >
> > when i disable the interface in the windows network connections, the
> > failover works fine.
>
>
>

Sorry if I missed this previously, are your iSCSI network connections on the
same subnet as your public network interface? If so, then I could see the
behavior you're describing ocurring. This would be described in
http://support.microsoft.com/kb/175767

If the iSCSI connections are not on the same subnet as your other cluster
networks, they shouldn't affect these connections when pulled.

Regards,
John

Visit my blog: http://msmvps.com/blogs/jtoner

"MarkusN" <> wrote in message
news:CE8968AF-1AF0-4780-84B4-...
> i'm already in contact with the storage/server/network/iscsi vendor. it's
> dell.
> but it seems to be a simple windows problem.
>
> i've configured a virutal machine with windows server 2003 under esx4 to
> test the behavior of the network inferfaces.
> both interfaces are in the same subnet with no gateway.
> i've started a ping to a reachable host. when i disable the network
> connection of the vm, i only got 2 timeouts and then it goes on.
>
> when i do the same procedure on my physical server with installed cluster
> services, the ping stops and the host is isolated even though the second
> interface still has a connection.
> is this a common behavior under windows server 2003 enterprise x64 with
> cluster service?
> i think the problem is that the interfaces are in the same subnet.
> there is only one of two interfaces in the cluster administrator under
> "networks" and the missing interfaces causes the problem.

No, I use dedicated subnets for the heartbeat, public, and iscsi connections.
The addresses are:

10.10.5.11 / 255.255.255.0 heartbeat
10.10.1.31 / 255.255.255.0 iscsi 1
10.10.1.32 / 255.255.255.0 iscsi 2
192.168.0.81 / 255.255.0.0 public


"John Toner [MVP]" wrote:

> Sorry if I missed this previously, are your iSCSI network connections on the
> same subnet as your public network interface? If so, then I could see the
> behavior you're describing ocurring. This would be described in
> http://support.microsoft.com/kb/175767
>
> If the iSCSI connections are not on the same subnet as your other cluster
> networks, they shouldn't affect these connections when pulled.
>
> Regards,
> John
>
> Visit my blog: http://msmvps.com/blogs/jtoner
>
> "MarkusN" <> wrote in message
> news:CE8968AF-1AF0-4780-84B4-...
> > i'm already in contact with the storage/server/network/iscsi vendor. it's
> > dell.
> > but it seems to be a simple windows problem.
> >
> > i've configured a virutal machine with windows server 2003 under esx4 to
> > test the behavior of the network inferfaces.
> > both interfaces are in the same subnet with no gateway.
> > i've started a ping to a reachable host. when i disable the network
> > connection of the vm, i only got 2 timeouts and then it goes on.
> >
> > when i do the same procedure on my physical server with installed cluster
> > services, the ping stops and the host is isolated even though the second
> > interface still has a connection.
> > is this a common behavior under windows server 2003 enterprise x64 with
> > cluster service?
> > i think the problem is that the interfaces are in the same subnet.
> > there is only one of two interfaces in the cluster administrator under
> > "networks" and the missing interfaces causes the problem.
>
>
>