Win2K3E & Multiple NICs

Discussion in 'Server Networking' started by Noctaire, Feb 23, 2008.

  1. Noctaire

    Noctaire Guest

    I have a Win2K3 Enterprise box with 2 NICs. I have each NIC on a
    separate LAN segment, each with a different network (call it 10.23.1.x
    and 10.23.2.x). I've set a metric of 1 and 2 respectively. Every so
    often, the server just seems to vanish from the first segment. It also
    sometimes just starts routing over the second segment for no readily
    apparent reason.

    This is my first attempt at have two different segments on the same
    machine like this; the goal is redundancy of course, and ideally
    transactions will come in across either to the system's various services
    (call it load balancing, although it's more a matter of telling some
    clients to use one, others to use the secondary).

    Are there any particulars to setting such a design up? Clearly I'm
    missing something, I'm just not sure what. Suggestions, white papers...?

    James
     
    Noctaire, Feb 23, 2008
    #1
    1. Advertisements

  2. Robert L. \(MS-MVP\), Feb 24, 2008
    #2
    1. Advertisements

  3. Noctaire

    James Guest

    Ok...not sure how that will help, but I can do that. Primary has been
    given a metric of 1, secondary a metric of 2. The two networks are not
    bridged.

    Windows IP Configuration

    Host Name . . . . . . . . . . . . : myhost
    Primary Dns Suffix . . . . . . . :
    Node Type . . . . . . . . . . . . : Unknown
    IP Routing Enabled. . . . . . . . : No
    WINS Proxy Enabled. . . . . . . . : No

    Ethernet adapter Primary:

    Connection-specific DNS Suffix . :
    Description . . . . . . . . . . . : Broadcom NetXtreme Gigabit
    Ethernet #2
    Physical Address. . . . . . . . . : XX-XX-XX-XX-XX-XX
    DHCP Enabled. . . . . . . . . . . : No
    IP Address. . . . . . . . . . . . : 10.23.1.3
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    IP Address. . . . . . . . . . . . : 10.23.1.2
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    IP Address. . . . . . . . . . . . : 10.23.1.1
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    Default Gateway . . . . . . . . . : 10.23.1.1
    DNS Servers . . . . . . . . . . . : 127.0.0.1
    216.1.2.1

    Ethernet adapter Secondary:

    Connection-specific DNS Suffix . :
    Description . . . . . . . . . . . : Broadcom NetXtreme Gigabit Ethernet
    Physical Address. . . . . . . . . : XX-XX-XX-XX-XX-XX
    DHCP Enabled. . . . . . . . . . . : No
    IP Address. . . . . . . . . . . . : 10.23.2.2
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    Default Gateway . . . . . . . . . : 10.23.2.1
    DNS Servers . . . . . . . . . . . : 192.168.1.1
    192.168.1.2
     
    James, Feb 24, 2008
    #3
  4. Noctaire

    Noctaire Guest

    Ok...not sure how that will help, but I can do that. Primary has been
    given a metric of 1, secondary a metric of 2. The two networks are not
    bridged.

    Windows IP Configuration

    Host Name . . . . . . . . . . . . : myhost
    Primary Dns Suffix . . . . . . . :
    Node Type . . . . . . . . . . . . : Unknown
    IP Routing Enabled. . . . . . . . : No
    WINS Proxy Enabled. . . . . . . . : No

    Ethernet adapter Primary:

    Connection-specific DNS Suffix . :
    Description . . . . . . . . . . . : Broadcom NetXtreme Gigabit
    Ethernet #2
    Physical Address. . . . . . . . . : XX-XX-XX-XX-XX-XX
    DHCP Enabled. . . . . . . . . . . : No
    IP Address. . . . . . . . . . . . : 10.23.1.3
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    IP Address. . . . . . . . . . . . : 10.23.1.2
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    IP Address. . . . . . . . . . . . : 10.23.1.1
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    Default Gateway . . . . . . . . . : 10.23.1.1
    DNS Servers . . . . . . . . . . . : 127.0.0.1
    216.1.2.1

    Ethernet adapter Secondary:

    Connection-specific DNS Suffix . :
    Description . . . . . . . . . . . : Broadcom NetXtreme Gigabit Ethernet
    Physical Address. . . . . . . . . : XX-XX-XX-XX-XX-XX
    DHCP Enabled. . . . . . . . . . . : No
    IP Address. . . . . . . . . . . . : 10.23.2.2
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    Default Gateway . . . . . . . . . : 10.23.2.1
    DNS Servers . . . . . . . . . . . : 192.168.1.1
    192.168.1.2
     
    Noctaire, Feb 24, 2008
    #4
  5. Hello Noctaire,

    With this kind of ip configuration you must have problems. You have mixed
    internal and external addresses on the NIC's. First remove the loopback address
    127.0.0.1 and use the real one from the server. Also you have to configure
    a forwarder to your ISP's DNS server on the DNS management console properties
    under the forwarders tab. This is for NIC one.
    On the second you have mixed to different ip ranges 10.x.x.x and 192.168.x.x
    for the DNS, will also not work this way.

    Best regards

    Meinolf Weber
     
    Meinolf Weber, Feb 24, 2008
    #5
  6. Noctaire

    Noctaire Guest

    Thanks for your suggestion, but DNS isn't really a concern here; I can
    remove the DNS entries and simply point them directly at the box's IP
    address (since DNS is run locally) such that we have....

    ***********************************************************************

    Windows IP Configuration

    Host Name . . . . . . . . . . . . : myhost
    Primary Dns Suffix . . . . . . . :
    Node Type . . . . . . . . . . . . : Unknown
    IP Routing Enabled. . . . . . . . : No
    WINS Proxy Enabled. . . . . . . . : No

    Ethernet adapter Primary:

    Connection-specific DNS Suffix . :
    Description . . . . . . . . . . . : Broadcom NetXtreme Gigabit
    Ethernet #2
    Physical Address. . . . . . . . . : XX-XX-XX-XX-XX-XX
    DHCP Enabled. . . . . . . . . . . : No
    IP Address. . . . . . . . . . . . : 10.23.1.3
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    IP Address. . . . . . . . . . . . : 10.23.1.2
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    IP Address. . . . . . . . . . . . : 10.23.1.1
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    Default Gateway . . . . . . . . . : 10.23.1.1
    DNS Servers . . . . . . . . . . . : 10.23.1.2


    Ethernet adapter Secondary:

    Connection-specific DNS Suffix . :
    Description . . . . . . . . . . . : Broadcom NetXtreme Gigabit Ethernet
    Physical Address. . . . . . . . . : XX-XX-XX-XX-XX-XX
    DHCP Enabled. . . . . . . . . . . : No
    IP Address. . . . . . . . . . . . : 10.23.2.2
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    Default Gateway . . . . . . . . . : 10.23.2.1
    DNS Servers . . . . . . . . . . . : 10.23.2.2

    ***********************************************************************

    Those were simply defaults that I haven't gotten around to changing (and
    FWIW, they work fine) but as I mentioned, DNS isn't the issue -- routing
    traffic in general is where things aren't working properly and that's IP
    based, not hostname based.

    With metrics established for each NIC, traffic should route accordingly.
    Likewise, one segment shouldn't just seemingly vanish for no apparent
    reason. This is why I'm wondering if there isn't something else that
    needs to be done to make this sort of arrangement work properly.

    James
     
    Noctaire, Feb 25, 2008
    #6
  7. Noctaire

    Bill Grant Guest

    No, that in incorrect. You can only have one default gateway per machine,
    not one per interface. If you specify more than one gateway, the machine
    will only use one of them. The only redundancy you get is that, if that
    gateway fails, the machine will switch to the other gateway. This depends on
    dead gateway detection. It is very limited, because it will not switch back
    when the original gateway comes back to like. You would have to do that
    manually.

    The only way to ensure traffic for a particular destination uses a
    gateway other than the default is to use static routes. Otherwise the only
    traffic going out the non-default gateway will be to traffic in the directly
    connected subnet.

    Having two NICs in a server is a great idea in principle but it has
    major problems in practice. These involve name resolution and browsing. The
    cause of this is that there are two IP addresses associated with the
    server's name.
     
    Bill Grant, Feb 25, 2008
    #7
  8. Noctaire

    Noctaire Guest

    So the addition of static routes should correct the issue then, right?

    Without specifying routes, I expected traffic destined for the
    non-default gateway to go over the directly connected subnet. Problem
    is, windows ignored the metrics and went the other way -- using the
    secondary connection (with metric 2) as the default. That behavior
    doesn't SOUND correct...?

    This box will also be used to host virtual servers so having the ability
    to route across multiple networks is required. Surely Microsoft
    realized that...?
     
    Noctaire, Feb 25, 2008
    #8
  9. It isn't Microsoft. It is the way TCP/IP works

    Virtual machines have nothing to do with the Nic's TCP/IP config on the Host
    machine. The Nics just have to physically be there and the drivers must be
    loaded and functional. The actual TCP/IP Config can be totally bogus or even
    left blank. The Virtual Machines function based on the TCP/IP specs you
    give *them* on their Virtual Nic and which Physical Nic you associate the
    Virtual Nic to,.. which represents the Physical "wire" that the Virtual Nic
    functions with.

    Disclaimer:
    I do no have any of the previous posts,...so I may not have the full context
    of the discussion.

    --
    Phillip Windell
    www.wandtv.com

    The views expressed, are my own and not those of my employer, or Microsoft,
    or anyone else associated with me, including my cats.
    -----------------------------------------------------
     
    Phillip Windell, Feb 25, 2008
    #9
  10. Noctaire

    Bill Grant Guest

    I agree with Philip. The virtual machine issue is a red herring. If you
    want the virtual machines to communicate with the physical networks you will
    need to link them to one physical NIC or the other. Or do you plan to have
    multiple NICs in the virtual machines as well? If, on the other hand, you
    want one vm to connect to one NIC and another to connect to a different NIC,
    that is no trouble. You can do that without the host being in both networks.
    The virtual machine's NICs will have their own IP addresses and their own
    MAC addresses. As far as TCP/IP is concerned they are separate machines with
    no link to the host.

    Stick to one NIC per server and let your routers do the routing. Trying to
    out-think TCP/IP is a recipe for disaster.
     
    Bill Grant, Feb 25, 2008
    #10
  11. Noctaire

    Noctaire Guest

    Phil/Bill,

    This is going a bit further in a different direction but since it has....

    Bearing in mind I'm only just starting to get a handle on Microsoft's
    virtual machine app, my impression of the virtual environment is that I
    can set up actual, physical NICs on the server that go to entirely
    different network segments then bind a virtual machine to those separate
    NICs. Is that correct or am I missing something?

    James
     
    Noctaire, Feb 25, 2008
    #11
  12. Noctaire

    Noctaire Guest

    I agree with Philip. The virtual machine issue is a red herring. If
    As I mentioned in an earlier message -- bear in mind I'm a bit new to
    the virtual machine environmenet so go easy on me. :)

    I was under the impression that the virtual machines routed over the
    physical NICs but apparently that's not the case...? If I put, say, 4
    virtual machines on this server and I have 2 NICs how else would they
    share them? I understand they each get their own configuration but
    doesn't the virtual machine app have to route traffic from their virtual
    NICs across the real NIC?
    I don't get this kind of thinking -- what's the point of having multiple
    NICs in a server (built-in) if not to put them on separate network
    segments? Just failover? I've worked in other environments that have
    as many as 4 or 5 NICs in a single box, all working different network
    segments -- administrative, transactional, backups, and so forth -- so
    I'm a little confused.

    James
     
    Noctaire, Feb 25, 2008
    #12
  13. Noctaire

    Bill Grant Guest

    Yes, you can do that. But the host machine does not have to see those
    networks.

    When you install VPC or Virtual Server, extra code is added to the
    device drivers for the NICs in the host. This allows traffic for virtual
    machines to use the physical NIC. Ther extra code reroutes packets addressed
    to a vm to the the OS running in the vm, not the network stack in the host.
    In the network properties of the NIC on the host, this is represented by the
    Virtual Machine Network Services.

    You can control the way the NIC works using this setting. If you want
    the NIC to be used by the host only, you clear the checkbox for VMNS. This
    NIC will not be available to virtual machines. On the other hand, if you
    clear all of the boxes except VMNS, the vms can use the NIC but not the
    host.
     
    Bill Grant, Feb 26, 2008
    #13
  14. That's correct. But the Physical nics do not need any TCP/IP Config at all.
    They can have a completely non-fuctional TCP/IP Config. The only thing that
    needs to function is the Physical Nic itself, the Driver for the Nic and the
    Physical Cabling. All they do is provide the physical "path" for the
    Virtual machine to follow.

    So if two VMs are using the same physcial "path" they can communicate,...if
    they are not using the same physical "path" then they cannot.

    You can also create many more "fake" Physical paths by installing multiple
    instances of the Microsoft Loopback Adapter. Give them uniques "names", and
    leave the TCP/IP unconfigured. Any virtual machines using the same
    "instance" of a Microsoft Loopback Adapter will be on the same "network"
    with each other.

    --
    Phillip Windell
    www.wandtv.com

    The views expressed, are my own and not those of my employer, or Microsoft,
    or anyone else associated with me, including my cats.
    -----------------------------------------------------
     
    Phillip Windell, Feb 26, 2008
    #14
  15. That depends on what you mean by "routed over the physical NICs".
    See my other post.
    That depends on what you mean by "sharing".
    If the 4 VMs are supposed to be on the same "network" together,...then there
    needs to be only one Nic on the physical machine.
    See my other post.
    No,...almost never failover,...unless Nic Teaming is being performed,...and
    that is a fairly recent development in technology. There are a *lot* of
    things done in the industry that are just plain *bad* ideas., or in many
    cases based on a misunderstanding, or outdated understanding, or just
    "superstition" of how TCP/IP networking functions.

    The only Server that should even run multiple Nics (except for Nic Teaming)
    is a Server that is built specifically to be a Firewall/Proxy or a LAN
    Router,...and it is probably cheaper to just buy a real LAN Router,...so
    that just leaves the Firewall/Proxy functioanlity (like MS ISA Server).
    Using that for tape backups is a valid thing back when everyone still ran
    10mbps networks over *hubs*. It often caused identification issues between
    machines if "naming" wasn't handled properly. With "fully switched" 100mbps,
    and better yet 1000mbps, it isn't needed. I run all my backups right over
    the LAN and I do it in a "big way",....and it goes totally unoticed. Proper
    scheduling helps with that too. I'm not saying that you should never have a
    dedicated network for doing backups (if done correctly), but I am saying
    that it is not so much of a "must have" as it used to be.

    Using it for administrative purposes is (IMO) one of those things done based
    on "superstition" and the misguided idea that security only comes from
    Layers 3&4. I've been in this stuff for nearly a decade and I can say that
    the industry is full is network designs and decisions based on
    "superstition" and misunderstanding. Some of it may go back to when "hubed"
    networks ran by other protocols (IPX/SPX, Netbeui, etc) before TCP/IP and
    Layer2 Switching took over, and people never managed to "unlearn" some of
    the old ideas.

    TCP/IP packets do not flow over multiple cabling & pathes like water through
    pipes. It does not take the "path of least resistance" the way water does.
    It follows pathes based on routing decisions and protocol behavor and so
    multiple pathes does not translate into more efficiency unless everything
    falls into place to justify the network design.

    --
    Phillip Windell
    www.wandtv.com

    The views expressed, are my own and not those of my employer, or Microsoft,
    or anyone else associated with me, including my cats.
    -----------------------------------------------------
     
    Phillip Windell, Feb 26, 2008
    #15
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.