maxinum of memory can i allocate for fast read of 2GByate/s

i have question and looking for help.

the situation: the appliction needs to read from the driver/Board (PCIe)
data rate of about ~2GByte/s to the pc first save it to memory then to buffer
it to loacal drive for later process.

question: who will allocate the memory the appliction from the user mode and
pass it to the driver? can the driver allocate that memory from the kernel?
(OS x64)
what can of design patten should i used for example? any advice?

> question: who will allocate the memory the appliction from the user mode and
> pass it to the driver?

Better to do this, so that there will be no extra memcpy().

--
Maxim S. Shatskih
Windows DDK MVP

http://www.storagecraft.com

pascal <> wrote:
>
>i have question and looking for help.
>
>the situation: the appliction needs to read from the driver/Board (PCIe)
>data rate of about ~2GByte/s to the pc first save it to memory then to buffer
>it to loacal drive for later process.

For how long? Hot drives can save about 40MB/s, so you will need to store
it all in memory.
--
Tim Roberts,
Providenza & Boekelheide, Inc.

if i need to store in all data in memory for example 6 GByte all in memory
(64bit OS), the question is what memory? Virtual /physical VirtualAlloc can
reserve 6 GByte in the Virtual memory page file but will that will be too
slow? i mean that windows memory manger may could not stand up to the rate of
data needed to writen to the page file.
or use physical memory, what is the best way/practice ?

Thanks

"Tim Roberts" wrote:

> pascal <> wrote:
> >
> >i have question and looking for help.
> >
> >the situation: the appliction needs to read from the driver/Board (PCIe)
> >data rate of about ~2GByte/s to the pc first save it to memory then to buffer
> >it to loacal drive for later process.
>
> For how long? Hot drives can save about 40MB/s, so you will need to store
> it all in memory.
> --
> Tim Roberts,
> Providenza & Boekelheide, Inc.
> .
>

> For how long? Hot drives can save about 40MB/s

More. 80 is real, but... after ~30 sec of such writing, some drives do the internal recalibration, which means that all next writes are stopped for 10s or such.

--
Maxim S. Shatskih
Windows DDK MVP

http://www.storagecraft.com

pascal <> wrote:
>
>if i need to store in all data in memory for example 6 GByte all in memory
>(64bit OS), the question is what memory? Virtual /physical VirtualAlloc can
>reserve 6 GByte in the Virtual memory page file but will that will be too
>slow? i mean that windows memory manger may could not stand up to the rate of
>data needed to writen to the page file.

I already told you, disks can't handle it. Doesn't matter whether you're
writing it directly or letting the page manager write it. It's still the
same disk. You will have to allocate virtual memory to hold the whole
thing, grab 2 or 3 seconds worth, then stop gathering for 3 minutes while
you flush it out to the disk.

>> >the situation: the appliction needs to read from the driver/Board (PCIe)
>> >data rate of about ~2GByte/s to the pc first save it to memory then to buffer
>> >it to loacal drive for later process.

You do realize that it's going to require a 16-lane PCIExpress board to
transfer 2GB/s, right? You are designing it as 16-lane from the beginning?
And are you quite sure your memory subsystem can sustain 2GB/s?
--
Tim Roberts,
Providenza & Boekelheide, Inc.

Hi thanks for the help:

from the information i gathered my plan is:

1. try to alloc virutal memory about 4GByte for 2sec properly using
VirtualAlloc and VirutalLock API function to lock all memory in to the
physical memory to avoid paging out of memory to the disk. (are VirtualAlloc
and VirutalLock do work? i saw alot poeple had problem with those function
when trying to lock alot of memory?)

2. pass the memory pointer to the driver using IOCTL command or when using
the ReadFile() IOREAD routine.

3. when the transfer and IO is complete, save data to the Hard drive.

4. the platform used is winXP x64 with 6GByte memory and the target board is
the xilinx PCIe core that current support up to 10Gbit/s (1.25GByte) using x8
lanes, but the plan is to support 2GByte later on.

what to you think ? is that plan practicable/feasible?

pascal <> wrote:
>
>3. when the transfer and IO is complete, save data to the Hard drive.
>
>4. the platform used is winXP x64 with 6GByte memory and the target board is
>the xilinx PCIe core that current support up to 10Gbit/s (1.25GByte) using x8
>lanes, but the plan is to support 2GByte later on.

1.25GB/s is a stretch for an 8-lane device

>what to you think ? is that plan practicable/feasible?

As long as you never need to capture more than you can hold in memory,
sure.
--
Tim Roberts,
Providenza & Boekelheide, Inc.