Kernel malloc/realloc?

Discussion in 'Windows Vista Drivers' started by Bogdan Bejan, Feb 4, 2004.

  1. Bogdan Bejan

    Tim Roberts Guest

    char * newBuffer = ExAllocatePool( newBufferSize, NONPAGED_POOL );
    memcpy( newBuffer, oldBuffer, oldBufferSize );
    ExFreePool( oldBuffer );
    Yes, because it isn't that common an operation in kernel mode, and it
    SHOULDN'T be common.

    Memory is a precious resource in kernel mode; non-paged pool is fixed and
    cannot be increased. In many cases a memory allocation failure is a
    blue-screen-worthy event. As a result, well-designed drivers tend to
    allocate memory at device initialization and keep it around forever.

    Further, many kernel buffers are shared with user-mode apps. If a buffer
    is going to change its address, the driver writer needs to make a conscious
    decision to do that.
     
    Tim Roberts, Feb 8, 2004
    #21
    1. Advertisements

  2. Bogdan Bejan

    G. Guest

    That is verry smart, I never thought of that.
    But, how you do that **without the copy** ?
    You do know you can do that, don't you ?

    Besides the real question is this, so answer this if you can (malloc/realloc
    was more of a metafore see you guys didn't get it):
     
    G., Feb 8, 2004
    #22
    1. Advertisements

  3. Bogdan Bejan

    G. Guest

    To spam...not to spam....oh well, to spam

    So, can you acctualy answer the question?

     
    G., Feb 8, 2004
    #23
  4. I think he has decided that realloc without copying is his solution and
    nothing else matters. Obstinacy doesn't make writing drivers easy and
    just wastes everyone's time when they ask a question to which they don't
    want a valid answer. Of course, the question could be about how do I
    grab an executing image, enlarge it, and add some virus code.
     
    David J. Craig, Feb 8, 2004
    #24
  5. Bogdan Bejan

    G. Guest

    Try this instead

    How do I implement some streams in a driver.

    And no, I don't care is the driver is not the place for streams.
     
    G., Feb 8, 2004
    #25
  6. Bogdan Bejan

    G. Guest

    If i have a "virus" driver running in a system, do I reallly care about
    infecting a file IN memory?

    Do infecting a IN memory image really needs resizing a section? can you
    really resize a section without unmapping it first(etc...etc..)?

    "Should you be wrinting drivers?" :p

    And sice after you the misteries of kernel memory management hide in the
    realloc source code from your Visual C++ (how stupid is that?), i suppose
    if I look in the fopen source code I will learn how to make file system
    driver?

    The fact is you don't know how to *really* answer the original question so
    you spam,flame,etc..

    Why do you even bother to post if your not going to add any meaningfull data
    to the discussion?
     
    G., Feb 8, 2004
    #26
  7. Why do you even bother to post if your not going to add any meaningfull data
    There is no discussion.

    NT kernel has NO routine which will reallocate the memory block with pointer
    value preservation and without data copy. Period.

    All of the rest - "why", "is it not bad?" - are unnecessary phylosophy, which
    has nothing to do with professional development.
     
    Maxim S. Shatskih, Feb 8, 2004
    #27
  8. Bogdan Bejan

    G. Guest

    First thing, I **did not asked that**, al least read the thread then post to
    it? That is a reall challenge?
    Second, You can never, ever, guarantee to grow a buffer and keep the old VA.
    Third ****I DON'T CARE ABOUT THE MALLOC/REALLO/ETC... or.... PRESERVING
    THE OLD VA****

    The acctual question is how you do this:

    In kernel mode, a dirver, how can do this steps?
    1. Allocate a bunch of pf phisical pages
    2. Map them to some VA, continguosly
    3. Clear the VA mapping, but not free the phisical pages
    4. Alocate more phisical pages
    5. Continguosly in VA map the pages allocated from step 1 & 4
    6. Destroy the VM mapping.
    7. Free the phisical pages.

    I'm curious how many more dirver gurus will spam this thread without giving
    a decent answer
    Or if any of you acctualy knows the answer


    I guess spaming with pointless posts has something to do with the
    professional development?
     
    G., Feb 8, 2004
    #28
  9. Congratulations you have become the only person in my blocked senders
    list. Guess I will have to add what may be your other name, Bogdan
    Bejan, since that was the name used on the first post. Both had the
    same email address of .

    I will have no tutorials on writing virus code, so no comment there.
    Security software requires activities very similar, well OK, identical,
    to virus code.

    From your other message:
    I care and won't provide an answer to a question that should have not
    been asked.

    Has anyone provided an answer other than some subtle and not so subtle
    hints that your design is stupid or deficient in some way?
     
    David J. Craig, Feb 8, 2004
    #29
  10. 1. Allocate a bunch of pf phisical pages

    MmAllocatePagesForMdl
    MmFreePagesFromMdl/ExFreePool for MDL itself.
     
    Maxim S. Shatskih, Feb 8, 2004
    #30
  11. Bogdan Bejan

    G. Guest

    I really thank you

    It's a bit frustrating to post a simple and clear(sorry if i was not clear,
    that is my fault, i though i was) question and to get aything but an answer
     
    G., Feb 8, 2004
    #31
  12. Bogdan Bejan

    Tim Roberts Guest

    Look, you jumped in on the middle of a thread entitled "Kernel
    malloc/realloc" that absolutely DID start out with the exact question Maxim
    referred to. Further, your first contribution to this thread, according to
    my newsreader, was:
    You did describe your algorithm a few days later, without changing the
    subject line, but I think you are being a bit hard on those of us who might
    have assumed the question was still about realloc.
    Maxim did give you the answer, but it is a pyhrric victory. No one would
    ever really do this. Much of non-paged pool comes out of the 1:1
    physical/virtual mapping that lives at 80000000. You can't change the
    underlying page mapping of those pages.

    Further, I will wager that this would not be faster than a new alloc and a
    copy. Dinking with the page tables requires flushing the TLB, which turns
    your performance to mush.
    Ask a tacky question, get a tacky answer.
     
    Tim Roberts, Feb 10, 2004
    #32
  13. Bogdan Bejan

    Alex Guest

    Knolidge is power :)
    even if it sems ussles at first....

    Isn't the same allgorithm behing the user mode reallocation (not necesarly
    realloc, i'm shure this thread has nothing to do with the acctual realloc
    function :p) which we all use in user mode apps?
    How can you map 1:1 the VA 80M, let's sey, if you don't have 80M of ram in
    the system?
    Or, isn't VA 80M in the first 2G of VA, which are mapped per process
    context, and have mothing to do with
    kernel memory?
     
    Alex, Feb 10, 2004
    #33
  14. Bogdan Bejan

    Ray Trent Guest

    As far as I know, all the realloc functions (in user mode) that I've
    ever seen have just done an alloc/copy/free unless they happened to have
    extra space coincidentally in the heap right after the original allocation.
    I think the only thing that one can say about this entire discussion is
    that it proves the old maxim that 10 people that know nothing about a
    topic know 10 times less than 1 person that knows nothing about the topic...
     
    Ray Trent, Feb 10, 2004
    #34
  15. Bogdan Bejan

    Alex Guest

    That is teribly suboptimal, the only reason a C/C++ runtime would do that is
    because it needs to be OS portable...
    ....but, the OS specific win32 function GlobalRealloc dosn't need to be
    portable, i really don't think it works that way (with copy alghoritm).

    Meybe someone from MS can acctualy shed some light on this issue.
     
    Alex, Feb 10, 2004
    #35
  16. Bogdan Bejan

    Ray Trent Guest

    Any allocator (like GlobalAlloc) that normally allocates sub-page blocks
    of memory doesn't have any other choice. What makes you think that the
    address immediately after the end of your allocation, but still in the
    same page, belongs to you necessarily? Among other things, remember: the
    kernel memory physical/virtual mapping is the same in all processes.

    Allocating 4k of memory for a "new char" wouldn't exactly be efficient
    either...
     
    Ray Trent, Feb 10, 2004
    #36
  17. Bogdan Bejan

    Tim Roberts Guest

    No. Both relloc in the C run-time library and LocalRealloc/GlobalRealloc
    use the new/copy/free model.
    What I mean is that there is a direct mapping of the first gigabyte of
    physical memory at virtual address 80000000 hex. It's done using 4MB pages
    in order to save page space. So, 80000000 has the DOS interrupt vectors
    from physical address 0, 800C0000 has the VGA BIOS at physical address
    C0000, and so on.
     
    Tim Roberts, Feb 12, 2004
    #37
  18. Bogdan Bejan

    Alex Guest

    Is that a fact? Or a beleif?
    pages

    A little 'h' changes a lot of things :)
    This mapping is documented behaviour? For what OS-es is it used?
     
    Alex, Feb 12, 2004
    #38
  19. Bogdan Bejan

    Tim Roberts Guest

    The source code for Microsoft's C run-time library is included with Visual
    C++. You can look it up. If there's empty space following the block, it
    just expands the block. Otherwise, it's new/copy/free.

    Think about it for a bit. Your page table magic will ONLY work if all heap
    allocations are done in units of whole pages. As soon as you have two
    objects in the same page, you can't alter the mapping without screwing up
    the other objects in the page. The C run-time malloc and the Win32 heap
    alloc work in units of 32 bytes.
    I've never seen it documented by Microsoft, but it's fact, and operating
    system routines rely on it. I know it's true for the NT-based systems
    (NT/2K/XP). It used to be true on Windows 3.1. I do not remember about
    95/98.
     
    Tim Roberts, Feb 14, 2004
    #39
  20. I don't think it's true. A debugger doesn't show accessible memory there.

    This used to be true for Linux, but I don't know if it still is.
     
    Alexander Grigoriev, Feb 14, 2004
    #40
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.