Available Memory to 32-Bit Applications

We are running Windows XP64 and we are using 4 CPU machines that have 8GB
total. We have a program that was written in 32-Bit code that runs on these
computers. It does some complex mathematics and data crunching. Since we
have 4 CPUs on this computer, we create 4 separate processes. Our assumption
was that each process can have 2 GB worth of addressable space. Since the
code is written in 32-Bit, do these 4 CPUs each share the same 4GB of lower
memory?

What we are seeing is that our code is running out of memory when the total
system memory usage is around 1.8GB. This leads me to believe that the
64-Bit version of Windows cannot remap memory in the 4GB+ range and make that
exposed to these processes.

What I believe is happening is that:
CPU 1 -> CPU 4 all share the same lower 4GB of memory
Instead of
CPU 1 -> gets to use 2GB (say 0GB -> 2GB)
CPU 2 -> gets to use 2GB (say 2GB -> 4GB)
CPU 3 -> gets to use 2GB (say 4GB -> 6GB)
CPU 4 -> gets to use 2GB (say 6GB -> 8GB)

Can someone definitively answer how multiple 32-Bit processes use memory in
the 64-Bit OS? What I am seeing sort of goes against everything that people
have been telling me.

The 32-bit system environment is the limiting factor, it can only deal
with a maximum of 4GB while the actual limit is usually in the 3.2GB -
3.5GB range.

See the info on _*Large Memory Support*_ at:
http://msdn.microsoft.com/en-us/libr...18(VS.85).aspx


Mike Sharpe wrote:
> We are running Windows XP64 and we are using 4 CPU machines that have 8GB
> total. We have a program that was written in 32-Bit code that runs on these
> computers. It does some complex mathematics and data crunching. Since we
> have 4 CPUs on this computer, we create 4 separate processes. Our assumption
> was that each process can have 2 GB worth of addressable space. Since the
> code is written in 32-Bit, do these 4 CPUs each share the same 4GB of lower
> memory?
>
> What we are seeing is that our code is running out of memory when the total
> system memory usage is around 1.8GB. This leads me to believe that the
> 64-Bit version of Windows cannot remap memory in the 4GB+ range and make that
> exposed to these processes.
>
> What I believe is happening is that:
> CPU 1 -> CPU 4 all share the same lower 4GB of memory
> Instead of
> CPU 1 -> gets to use 2GB (say 0GB -> 2GB)
> CPU 2 -> gets to use 2GB (say 2GB -> 4GB)
> CPU 3 -> gets to use 2GB (say 4GB -> 6GB)
> CPU 4 -> gets to use 2GB (say 6GB -> 8GB)
>
> Can someone definitively answer how multiple 32-Bit processes use memory in
> the 64-Bit OS? What I am seeing sort of goes against everything that people
> have been telling me.
>

Thank you for the reply but that seems to only make sense on a 32-Bit OS. I
am running a 64-Bit OS so I would expect that the OS should handle where the
2GB of RAM comes from and remap it to the 32-Bit process. For example:

32-Bit Process A requests 8 bytes of memory from the 64 Bit OS. The 64-Bit
OS gets that memory from lets say the 6GB block in its own memory space. It
should then provide this to 32-Bit process in a way that the 32-Bit process
thinks it is coming from a 32-Bit space.

I would think that the 64-bit OS would handle memory in this fashion.
Otherwise, moving to 64-Bit doesn't gain you anything if you want to run
multiple 32-Bit process. I could have 16GB of RAM and it wouldn't make a
difference. Let's say I have a dual quad core machine for a total of 8
CPUs/Cores. Are you suggesting that all 8 of these will use the exact same
32-Bit address space? So ~3GB will be used and the remaining 13GB would not
be used by these?

Hi Mike,

You are quite correct, each app has its own virtual memory space and the OS
will duly map this to real RAM or page file as necessary. In fact the amount
of physical RAM is totally irrelevent - you are most likely running out of
virtual memory (assuming its not something obvious like disk space for the
page file). However it is not sufficient just to tot up the amount of memory
(virtual) you are using. The most likely barrier you are running into is
virtual memory fragmentation whereby the memory allocator cannot find
sufficient *contiguous* memory. This type of issue needs to be dealt with at
a programming level and you are not clear whether you have control over
this? If you do then a programming group might be a more appropriate place
to continue the discussion, but commonly memory fragmentation can be caused
by using auto expanding data structures that are repeatedly resized.

Cheers
Doug Forster

"Mike Sharpe" <> wrote in message
news:07654363-8AF1-43FB-ABB3-...
> Thank you for the reply but that seems to only make sense on a 32-Bit OS.
> I
> am running a 64-Bit OS so I would expect that the OS should handle where
> the
> 2GB of RAM comes from and remap it to the 32-Bit process. For example:
>
> 32-Bit Process A requests 8 bytes of memory from the 64 Bit OS. The
> 64-Bit
> OS gets that memory from lets say the 6GB block in its own memory space.
> It
> should then provide this to 32-Bit process in a way that the 32-Bit
> process
> thinks it is coming from a 32-Bit space.
>
> I would think that the 64-bit OS would handle memory in this fashion.
> Otherwise, moving to 64-Bit doesn't gain you anything if you want to run
> multiple 32-Bit process. I could have 16GB of RAM and it wouldn't make a
> difference. Let's say I have a dual quad core machine for a total of 8
> CPUs/Cores. Are you suggesting that all 8 of these will use the exact
> same
> 32-Bit address space? So ~3GB will be used and the remaining 13GB would
> not
> be used by these?

But, the 32-bit processes are running in a 32-bit environment within the
64-bit OS - the 32-bit environment is in its own little 32-bit world,
not in the OS's 64-bit world.

See info on _*WOW64 Implementation Details*_ at:
http://msdn.microsoft.com/en-us/library/aa384274.aspx

Your 32-bit processes are running in *WOW64* which is the x86 emulator
that allows 32-bit Windows-based applications to run seamlessly on
64-bit Windows.

Se info at: http://msdn.microsoft.com/en-us/libr...49(VS.85).aspx


Mike Sharpe wrote:
> Thank you for the reply but that seems to only make sense on a 32-Bit OS. I
> am running a 64-Bit OS so I would expect that the OS should handle where the
> 2GB of RAM comes from and remap it to the 32-Bit process. For example:
>
> 32-Bit Process A requests 8 bytes of memory from the 64 Bit OS. The 64-Bit
> OS gets that memory from lets say the 6GB block in its own memory space. It
> should then provide this to 32-Bit process in a way that the 32-Bit process
> thinks it is coming from a 32-Bit space.
>
> I would think that the 64-bit OS would handle memory in this fashion.
> Otherwise, moving to 64-Bit doesn't gain you anything if you want to run
> multiple 32-Bit process. I could have 16GB of RAM and it wouldn't make a
> difference. Let's say I have a dual quad core machine for a total of 8
> CPUs/Cores. Are you suggesting that all 8 of these will use the exact same
> 32-Bit address space? So ~3GB will be used and the remaining 13GB would not
> be used by these?
>

Just noticed your original post said you are running out at 1.8GB. This is
actually about par for the course for a 32 bit app that does not have the
LARGEADDRESSAWARE flag set. If you want the full 4GB to be available then
you can set this flag with the EditBin utility (or your dev tool if you are
in control of this) in which case you would run out of virtual memory
somwhere between 3 and 4 GB

http://www.microsoft.com/whdc/system...AE/PAEmem.mspx has
more information about _*Memory Support and Windows Operating Systems*_

Mike Sharpe wrote:
> Thank you for the reply but that seems to only make sense on a 32-Bit OS. I
> am running a 64-Bit OS so I would expect that the OS should handle where the
> 2GB of RAM comes from and remap it to the 32-Bit process. For example:
>
> 32-Bit Process A requests 8 bytes of memory from the 64 Bit OS. The 64-Bit
> OS gets that memory from lets say the 6GB block in its own memory space. It
> should then provide this to 32-Bit process in a way that the 32-Bit process
> thinks it is coming from a 32-Bit space.
>
> I would think that the 64-bit OS would handle memory in this fashion.
> Otherwise, moving to 64-Bit doesn't gain you anything if you want to run
> multiple 32-Bit process. I could have 16GB of RAM and it wouldn't make a
> difference. Let's say I have a dual quad core machine for a total of 8
> CPUs/Cores. Are you suggesting that all 8 of these will use the exact same
> 32-Bit address space? So ~3GB will be used and the remaining 13GB would not
> be used by these?
>

The documents were helpful but did not answer my question. I understand
64-Bit addresses. I understand 32-Bit addresses. What I am not
understanding is whether or not the 64-Bit Windows Virtual Memory Manager can
provide unique blocks of 2GB (or 3GB with the switch) for each 32-bit
process or if they all share the same. I have an 8 core computer with 16GB
of RAM running 64-Bit OS. I have 8 32-Bit processes running. My page file
is setup for 10GB which should leave me with a total of approximately 26GB of
Virtual Memory.

I start up an application 8 times. Each one runs in its own process and
assigned to its own CPU. This application is 32-Bit and let's say each one
uses about 1GB of RAM. What I am seeing is that when the Physical Memory of
the box (not per process, the whole box) reaches about 1.8GB, my processes
start experiencing problems and I get Out of Memory errors. If I do some
quick math, 8 CPU x 1GB = 8GB of memory total. Since my system has 16GB
total, I should be consuming only about half. And each 32-Bit process is
under the 2 GB limit.

What I suspect is happening is that each process is sharing the SAME 32-Bit
space instead of being granted their own space in the Virtual memory and the
64-Bit memory manager controlling where each process recognizes its space.
Even if some data is in the 64-Bit range for a single process, it should be
able to map that down to the process specific 32-Bit range.

Each 32-bit program has it own block of memory which will be shared by
the processes running within that program just as if it were actually
running that program on a Windows XP Pro (32-bit) system. I thought
that was pointed out in the info on WOW64.


Mike Sharpe wrote:
> The documents were helpful but did not answer my question. I understand
> 64-Bit addresses. I understand 32-Bit addresses. What I am not
> understanding is whether or not the 64-Bit Windows Virtual Memory Manager can
> provide unique blocks of 2GB (or 3GB with the switch) for each 32-bit
> process or if they all share the same. I have an 8 core computer with 16GB
> of RAM running 64-Bit OS. I have 8 32-Bit processes running. My page file
> is setup for 10GB which should leave me with a total of approximately 26GB of
> Virtual Memory.
>
> I start up an application 8 times. Each one runs in its own process and
> assigned to its own CPU. This application is 32-Bit and let's say each one
> uses about 1GB of RAM. What I am seeing is that when the Physical Memory of
> the box (not per process, the whole box) reaches about 1.8GB, my processes
> start experiencing problems and I get Out of Memory errors. If I do some
> quick math, 8 CPU x 1GB = 8GB of memory total. Since my system has 16GB
> total, I should be consuming only about half. And each 32-Bit process is
> under the 2 GB limit.
>
> What I suspect is happening is that each process is sharing the SAME 32-Bit
> space instead of being granted their own space in the Virtual memory and the
> 64-Bit memory manager controlling where each process recognizes its space.
> Even if some data is in the 64-Bit range for a single process, it should be
> able to map that down to the process specific 32-Bit range.
>
>
>

A 32-bit process can only adddress up to 2 GB of virtual address space. A
32-bit Large Address Aware process can get up to 4 GB of virtual address
space when run on Windows x64.

Each process has it's own unique address space. Every THREAD in that process
shares the same virtual address space.

Physical memory is managed by the operating system and will be mapped to
each process on demand. The mapping is completely up to the OS and is
dynamic, so there's no hard and firm rule here. If you are runnning a x64 OS
with 8 GB of physical RAM available, then the system will allocate it based
on the runtime demand. Multiple 32-bit processes running at the same time
can utilize more than 2 GB of physical memory.

To determine what is really happening you should look at your runtime
environment and determine how much viritual address space is allocated by
each process, and how large each working set is at runtime.

--
-Chuck Walbourn
SDE, XNA Developer Connection

This posting is provided "AS IS" with no warrenties, and confers no rights.