Kernel malloc/realloc?

Discussion in 'Windows Vista Drivers' started by Bogdan Bejan, Feb 4, 2004.

  1. Bogdan Bejan

    Bogdan Bejan Guest

    How can you alocate then realocate a memory buffer in kernel mode (W2k+) ?
     
    Bogdan Bejan, Feb 4, 2004
    #1
    1. Advertisements

  2. I never heard about realloc() in kernel, but ExAllocatePoolWithTag is used
    instead of malloc().
     
    Maxim S. Shatskih, Feb 4, 2004
    #2
    1. Advertisements

  3. Should you be writing drivers? Look at the source code for realloc that
    comes with Visual Studio and adapt it to kernel mode.
     
    David J. Craig, Feb 4, 2004
    #3
  4. no such API exists. you must be able to update the pointer value atomically
    if you need to have a bigger allocation

    d
     
    Doron Holan [MS], Feb 4, 2004
    #4
  5. Bogdan Bejan

    Bogdan Bejan Guest

    Let me guess, realloc is written using system calls like GlobalRealloc, to
    which i don't have the souce code, do I?
    Why I should not write drivers? When i was learning about virtual vs.
    hardware memory the top of the lince CPU was 386...soo..
    Besides, I know how realloc works on x86 CPUs, what i don't know is the
    windows kernel API to get the same thing.
     
    Bogdan Bejan, Feb 4, 2004
    #5
  6. Bogdan Bejan

    Bogdan Bejan Guest

    I wasn't sure is shuch API exists in kernel, thanx for clearing it.

    Don't you have to update the pointer with realloc to? Isn't realloc based on
    GlobalRealloc?Which i suppose works by allocating more hardware pages and
    mapping them in a new vitual location, where you have enough virtual space
    for the new buffer?
     
    Bogdan Bejan, Feb 4, 2004
    #6
  7. Bogdan Bejan

    Bogdan Bejan Guest

    Do you think I should attend a seminar to find out if there is or not a
    kernel mode API function to do what GlobalRealloc/realloc function does?

    The fact is that you and David did not understood my question, even wors you
    started to assume what I know and what I don't know and gave generic
    unhelpfull answers, advertised some book etc..etc..

    As i sed, I do know what is behind GlobalRealloc/realloc/etc.. from the x86
    point of view (do you?), what I didn't know was if there was a kernel mode
    function to facilitate it.

    Aniway, my question was answered, thanx again Dorono..
     
    Bogdan Bejan, Feb 4, 2004
    #7
  8. Let me guess, realloc is written using system calls like GlobalRealloc, to

    This is not a syscall. This is hooked to user-mode ntdll!RtlxxxHeap, which do
    all heap management.

    VirtualAlloc and other Virtualxxx are syscalls, hooked to
    ZwAllocateVirtualMemort.
    Just - no such API. Allocate the second block and copy.
     
    Maxim S. Shatskih, Feb 5, 2004
    #8
  9. As i sed, I do know what is behind GlobalRealloc/realloc/etc.. from the x86
    There is nothing under realloc() "from x86 point of view" (except the virtual
    memory, but this is too low a level - for instance, it only knows pages and not
    small memory blocks).

    realloc() is a function of the heap manager package, and, if the heap manager
    has no such function - then sorry.

    ExxxxPool heap manager has no such function.
     
    Maxim S. Shatskih, Feb 5, 2004
    #9
  10. Bogdan Bejan

    Bogdan Bejan Guest

    I didn't mean syscall (as in a call to kernel) i meant a system call or API,
    imlemented in a user mode dll, like you sed :)

    All the runtime function from C/C++ have to go trough windows user mode API
    to do the real-work, so if I look at the realloc code I won't learn much, do
    I?

    So the real newbe here was David J.
    :p

    When i sed malloc/realloc some people understood it literaly when I only
    ment them as the end result I was looking for, that is being able to resize
    a kernel mode memory buffer.
    For startes I will do just that, but, you should be able to realocate the
    number of fizical pages behind a buffer and map them in a diffrent location
    (yes, i still have to finish looking at the zilions of memory functions from
    the DDK) where there is enough virtual space, right?
    That way you don't do the costly copy operation.
     
    Bogdan Bejan, Feb 5, 2004
    #10
  11. Bogdan Bejan

    Bogdan Bejan Guest

    virtual

    In realloc there isn't a single bit of x86 dependent code (ok , meybe one or
    two :)
    and it can never be.Why? Because it is a C/C++ runtime library that is ment
    to be portable and has to realy on the OS to acctually do the reallocation.

    "from x86 point of view" = read, I know how the x86 memory model works,I
    know it at the lowest possible level, I don't need to buy some books for
    that. Don't we have the Architecture/OS manual from Intel? Which is a free
    download?

    I don't want the want the damn C/C++ runtime in a driver, I only need to
    resize a memory buffer. Simple as that
     
    Bogdan Bejan, Feb 5, 2004
    #11
  12. I don't want the want the damn C/C++ runtime in a driver, I only need to
    The ExxxxPool memory allocator package does not support this operation. Maybe
    this is because MS just forgotten to do this. It is rarely needed anyway.
     
    Maxim S. Shatskih, Feb 5, 2004
    #12
  13. one of the semantics of realloc is that the same pointer as before can be
    returned (i haven't used realloc in such a long time, i can't remember if
    this true 100% of the time or not). Based on this assumption if you were to
    resize a buffer, you would need to
    1) make sure any threads that are using the previous buffer have stopped
    using it
    2) update the field which contains the pointer value so that no one picks
    up the old value and starts using the new one

    d
     
    Doron Holan [MS], Feb 5, 2004
    #13
  14. Bogdan Bejan

    Bogdan Bejan Guest

    Any for of realocation of a memory buffer can't possible guarantee to return
    the same adress, what if you don't have enought linear virtual memory there
    to fulfill the request?

    So this applayes to maloc or any other function, i think.
    I really know this.
    I just don't know how to acctualy resize a memory buffer in kernel mode.
     
    Bogdan Bejan, Feb 5, 2004
    #14
  15. In most designs, realloc is so rarely used, that any implementation is left
    up to you. You won't have an optimized realloc (which can resize a memory
    block), though.
     
    Alexander Grigoriev, Feb 5, 2004
    #15
  16. maloc/ExAllocatePoolWithTag has a much higher likelyhood of failure in the
    kernel then in user mode. so, yes, as such you need to guard against NULL
    being returned as well w/out a doubt. But there is a difference between a
    NULL check and making sure all current users of valid pool are not touching
    memory when you decide to grow the allocation.

    d
     
    Doron Holan [MS], Feb 6, 2004
    #16
  17. Bogdan Bejan

    G. Guest

    Still, how do you perform the acctual resize?
    Any reasons as to why there isn't a direct function in kernel to do this?
     
    G., Feb 6, 2004
    #17
  18. depends on the application for how to do the resize, but i would either keep
    a reference count of all the threads who want access or perform all access
    under a lock (like a spinlock) which guarantees only one user of the buffer
    at once. one reason realloc does not exist is that VA space is limited and
    more often then not, the same pointer cannot be returned so this is
    functionally the same as free/alloc anyways.

    d
     
    Doron Holan [MS], Feb 6, 2004
    #18
  19. Bogdan Bejan

    G. Guest

    Let's try to be as technical as possible.
    In kernel mode (driver) how you do this:

    1. Allocate a bunch of pf phisical pages
    2. Map them to some VA, continguosly
    3. Clear the VA mapping, but not free the phisical pages
    4. Alocate more phisical pages
    5. Continguosly in VA map the pages allocated from step 1 & 4
    6. Destroy the VM mapping.
    7. Free the phisical pages.
     
    G., Feb 6, 2004
    #19
  20. Bogdan Bejan

    G. Guest

    In user space, I can only think of only one way. don't use realloc, malloc
    some other buffer then copy.

    I kernel mode you should be able to do this without the copy operation, but
    i don't know how..

    location?
     
    G., Feb 6, 2004
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.