Intercepting memory accesses

I want to get some idea of how to approach the following problem.

I'm very interested in being able to intercept read/writes from/to mapped
memory pages, be they resident in physical memory or paged out to disk. I'd
like to detect these operations when they arise from either a local process
or one running on a networked machine.

I'd also like to be able to detect when an app on some remote machine opens
and maps a file from the local machine and unmaps it.

I've not written a device driver before and know it is far from trivial,
although I am a seasoned developer and very familiar with most of the
internals of the OS.

I would suspect that it is possible to craft a driver that can trap these
operations and then do something (or queue some work to be done later).

This is just a vague exploration just now, not a real need, I'm just trying
to ascertain from experts the main issues that this raises.

Thanks

Hugo

You suspect wrong, you cannot trap memory accesses. Been there looked at
ways to do this from NT 3.5 to present.


--
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply



"Hugo >" <hugh<underbar> wrote in message
news:0C44A04B-A6A7-46FE-8055-...
>I want to get some idea of how to approach the following problem.
>
> I'm very interested in being able to intercept read/writes from/to mapped
> memory pages, be they resident in physical memory or paged out to disk.
> I'd
> like to detect these operations when they arise from either a local
> process
> or one running on a networked machine.
>
> I'd also like to be able to detect when an app on some remote machine
> opens
> and maps a file from the local machine and unmaps it.
>
> I've not written a device driver before and know it is far from trivial,
> although I am a seasoned developer and very familiar with most of the
> internals of the OS.
>
> I would suspect that it is possible to craft a driver that can trap these
> operations and then do something (or queue some work to be done later).
>
> This is just a vague exploration just now, not a real need, I'm just
> trying
> to ascertain from experts the main issues that this raises.
>
> Thanks
>
> Hugo
>
>
>
>
>
>
> __________ Information from ESET NOD32 Antivirus, version of virus
> signature database 4603 (20091113) __________
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>
>



__________ Information from ESET NOD32 Antivirus, version of virus signature database 4603 (20091113) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

Hi Don

Well thanks for that direct and unhesitating reply, I appreciate it.

OK, so hardware can trap this; so far as I am aware (and my internals
knowledge is pretty good but pretty rusty) once the CPU is running with VM
support enabled, all accesses to memory go through the address translation
stage.

During this, if the address (or rather page) is found to be absent, then an
interrupt is raised, yes?

So there must be a page fault interrupt handler, but I guess even if we did
intercept this we are only intercepting page faults, not accesses.

Can that handler be replaced, (like a driver can) or is it too fundamental a
part of the kernel?

Can we thread another handler (so to speak) so that we can "do something"
then forward the interrupt to the real handler?

I guess deep down, every read/write to a page MUST involve the address
translation system, does this (to your knowledge) always involve some degree
of software or (in the case of a page that is present) are there times when
it is handled with only hardware doing the work?

This last question amounts to this:

If there were a way or Microsoft wanted to introduce a way to intercept
user-space read/writes, what could they do? or would they too be unable due
to the nature of how the hardware handles this?

Can you suggest any good books or web articles on this that go pretty low
level?

I have several Windows Internals books, these are pretty good but I always
have questions after reading them that I can't get answers to.

Thanks for your time.

Hugo



"Don Burn" wrote:

> You suspect wrong, you cannot trap memory accesses. Been there looked at
> ways to do this from NT 3.5 to present.
>
>
> --
> Don Burn (MVP, Windows DKD)
> Windows Filesystem and Driver Consulting
> Website: http://www.windrvr.com
> Blog: http://msmvps.com/blogs/WinDrvr
> Remove StopSpam to reply
>
>
>
> "Hugo >" <hugh<underbar> wrote in message
> news:0C44A04B-A6A7-46FE-8055-...
> >I want to get some idea of how to approach the following problem.
> >
> > I'm very interested in being able to intercept read/writes from/to mapped
> > memory pages, be they resident in physical memory or paged out to disk.
> > I'd
> > like to detect these operations when they arise from either a local
> > process
> > or one running on a networked machine.
> >
> > I'd also like to be able to detect when an app on some remote machine
> > opens
> > and maps a file from the local machine and unmaps it.
> >
> > I've not written a device driver before and know it is far from trivial,
> > although I am a seasoned developer and very familiar with most of the
> > internals of the OS.
> >
> > I would suspect that it is possible to craft a driver that can trap these
> > operations and then do something (or queue some work to be done later).
> >
> > This is just a vague exploration just now, not a real need, I'm just
> > trying
> > to ascertain from experts the main issues that this raises.
> >
> > Thanks
> >
> > Hugo
> >
> >
> >
> >
> >
> >
> > __________ Information from ESET NOD32 Antivirus, version of virus
> > signature database 4603 (20091113) __________
> >
> > The message was checked by ESET NOD32 Antivirus.
> >
> > http://www.eset.com
> >
> >
> >
>
>
>
> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4603 (20091113) __________
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>
>
>
> .
>

Hugo,

Yes the system sees page faults, but there is no safe blessed way to
intercept that handler. No most reads and writes do not involve software
(other than the software issuing the operation). Note: also that DMA does
not go through the address translation. Microsoft could introduce a way to
see page faults, and if they provided management you could do things to
determine to some granularity (of time and address) the actions. Also,
with the hardware virtualization technology, one could in theory do the same
in a virtualization layer including seeing DMA.


--
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply



"Hugo >" <hugh<underbar> wrote in message
news:75CF3E75-1D09-4896-977C-...
> Hi Don
>
> Well thanks for that direct and unhesitating reply, I appreciate it.
>
> OK, so hardware can trap this; so far as I am aware (and my internals
> knowledge is pretty good but pretty rusty) once the CPU is running with VM
> support enabled, all accesses to memory go through the address translation
> stage.
>
> During this, if the address (or rather page) is found to be absent, then
> an
> interrupt is raised, yes?
>
> So there must be a page fault interrupt handler, but I guess even if we
> did
> intercept this we are only intercepting page faults, not accesses.
>
> Can that handler be replaced, (like a driver can) or is it too fundamental
> a
> part of the kernel?
>
> Can we thread another handler (so to speak) so that we can "do something"
> then forward the interrupt to the real handler?
>
> I guess deep down, every read/write to a page MUST involve the address
> translation system, does this (to your knowledge) always involve some
> degree
> of software or (in the case of a page that is present) are there times
> when
> it is handled with only hardware doing the work?
>
> This last question amounts to this:
>
> If there were a way or Microsoft wanted to introduce a way to intercept
> user-space read/writes, what could they do? or would they too be unable
> due
> to the nature of how the hardware handles this?
>
> Can you suggest any good books or web articles on this that go pretty low
> level?
>
> I have several Windows Internals books, these are pretty good but I always
> have questions after reading them that I can't get answers to.
>
> Thanks for your time.
>
> Hugo
>
>
>
> "Don Burn" wrote:
>
>> You suspect wrong, you cannot trap memory accesses. Been there looked at
>> ways to do this from NT 3.5 to present.
>>
>>
>> --
>> Don Burn (MVP, Windows DKD)
>> Windows Filesystem and Driver Consulting
>> Website: http://www.windrvr.com
>> Blog: http://msmvps.com/blogs/WinDrvr
>> Remove StopSpam to reply
>>
>>
>>
>> "Hugo >" <hugh<underbar> wrote in message
>> news:0C44A04B-A6A7-46FE-8055-...
>> >I want to get some idea of how to approach the following problem.
>> >
>> > I'm very interested in being able to intercept read/writes from/to
>> > mapped
>> > memory pages, be they resident in physical memory or paged out to disk.
>> > I'd
>> > like to detect these operations when they arise from either a local
>> > process
>> > or one running on a networked machine.
>> >
>> > I'd also like to be able to detect when an app on some remote machine
>> > opens
>> > and maps a file from the local machine and unmaps it.
>> >
>> > I've not written a device driver before and know it is far from
>> > trivial,
>> > although I am a seasoned developer and very familiar with most of the
>> > internals of the OS.
>> >
>> > I would suspect that it is possible to craft a driver that can trap
>> > these
>> > operations and then do something (or queue some work to be done later).
>> >
>> > This is just a vague exploration just now, not a real need, I'm just
>> > trying
>> > to ascertain from experts the main issues that this raises.
>> >
>> > Thanks
>> >
>> > Hugo
>> >
>> >
>> >
>> >
>> >
>> >
>> > __________ Information from ESET NOD32 Antivirus, version of virus
>> > signature database 4603 (20091113) __________
>> >
>> > The message was checked by ESET NOD32 Antivirus.
>> >
>> > http://www.eset.com
>> >
>> >
>> >
>>
>>
>>
>> __________ Information from ESET NOD32 Antivirus, version of virus
>> signature database 4603 (20091113) __________
>>
>> The message was checked by ESET NOD32 Antivirus.
>>
>> http://www.eset.com
>>
>>
>>
>>
>> .
>>
>
> __________ Information from ESET NOD32 Antivirus, version of virus
> signature database 4604 (20091113) __________
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>
>



__________ Information from ESET NOD32 Antivirus, version of virus signature database 4604 (20091113) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

Why can't you write your own virtualization code and run the OS inside with
monitoring of all page accesses. I guess you could get 8088 performance out
of a new Core i7 doing this. You might need a driver in the target OS that
could play with the page tables or just monitor them and inform the VM
manager to watch specific pages.

"Don Burn" <> wrote in message
news:...
> You suspect wrong, you cannot trap memory accesses. Been there looked at
> ways to do this from NT 3.5 to present.
>
>
> --
> Don Burn (MVP, Windows DKD)
> Windows Filesystem and Driver Consulting
> Website: http://www.windrvr.com
> Blog: http://msmvps.com/blogs/WinDrvr
> Remove StopSpam to reply
>
>
>
> "Hugo >" <hugh<underbar> wrote in message
> news:0C44A04B-A6A7-46FE-8055-...
>>I want to get some idea of how to approach the following problem.
>>
>> I'm very interested in being able to intercept read/writes from/to mapped
>> memory pages, be they resident in physical memory or paged out to disk.
>> I'd
>> like to detect these operations when they arise from either a local
>> process
>> or one running on a networked machine.
>>
>> I'd also like to be able to detect when an app on some remote machine
>> opens
>> and maps a file from the local machine and unmaps it.
>>
>> I've not written a device driver before and know it is far from trivial,
>> although I am a seasoned developer and very familiar with most of the
>> internals of the OS.
>>
>> I would suspect that it is possible to craft a driver that can trap these
>> operations and then do something (or queue some work to be done later).
>>
>> This is just a vague exploration just now, not a real need, I'm just
>> trying
>> to ascertain from experts the main issues that this raises.
>>
>> Thanks
>>
>> Hugo
>>
>>
>>
>>
>>
>>
>> __________ Information from ESET NOD32 Antivirus, version of virus
>> signature database 4603 (20091113) __________
>>
>> The message was checked by ESET NOD32 Antivirus.
>>
>> http://www.eset.com
>>
>>
>>
>
>
>
> __________ Information from ESET NOD32 Antivirus, version of virus
> signature database 4603 (20091113) __________
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>
>
>

">" <hugh<underbar> wrote in message
news:0C44A04B-A6A7-46FE-8055-...
> I want to get some idea of how to approach the following problem.
>
> I'm very interested in being able to intercept read/writes from/to mapped
> memory pages, be they resident in physical memory or paged out to disk.
> I'd
> like to detect these operations when they arise from either a local
> process
> or one running on a networked machine.
>
> I'd also like to be able to detect when an app on some remote machine
> opens
> and maps a file from the local machine and unmaps it.
>
> I've not written a device driver before and know it is far from trivial,
> although I am a seasoned developer and very familiar with most of the
> internals of the OS.

Then, are you familiar with auditng access to objects in NT,
or some virtual machines?

--pa

Hugo,

You may want to check out Tsukasa Ooi's AVTokyo 2009 and PacSec 2009
presentations. http://a4lg.com/.

Regards

Rossetoecioccolato.

It is possible to monitor network operations on several levels (For network
operations, both the network stack and the redirector will be involved). As
others have noted, local memory access can only be observed by hardware (or
virtualization software) because by design the kernel stays out of the way
as much as possible.

">" <hugh<underbar> wrote in message
news:0C44A04B-A6A7-46FE-8055-...
> I want to get some idea of how to approach the following problem.
>
> I'm very interested in being able to intercept read/writes from/to mapped
> memory pages, be they resident in physical memory or paged out to disk.
> I'd
> like to detect these operations when they arise from either a local
> process
> or one running on a networked machine.
>
> I'd also like to be able to detect when an app on some remote machine
> opens
> and maps a file from the local machine and unmaps it.
>
> I've not written a device driver before and know it is far from trivial,
> although I am a seasoned developer and very familiar with most of the
> internals of the OS.
>
> I would suspect that it is possible to craft a driver that can trap these
> operations and then do something (or queue some work to be done later).
>
> This is just a vague exploration just now, not a real need, I'm just
> trying
> to ascertain from experts the main issues that this raises.
>
> Thanks
>
> Hugo
>
>
>
>
>

They do look interesting, I will study these later.

Thanks

Hugo


"RossettoeCioccolato" wrote:

> Hugo,
>
> You may want to check out Tsukasa Ooi's AVTokyo 2009 and PacSec 2009
> presentations. http://a4lg.com/.
>
> Regards
>
> Rossetoecioccolato.
>
>
> .
>

OK

Well thanks for the various replies, I appreciate it.

Now let's not forget that I CAN TRAP memory accesses now.

If one uses the Win32 function VirtualProtect or VirtualProtectEx, it is
possible to set access to selected pages to be PAGE_NOACCESS (or even exploit
Guard Page support).

When code in the affected process tries to read or write such a page, an
exception is raised in that thread and the handler can ascertain if the
operation was a read or write, take some action and perhaps optionally enable
the required access and issue an instruction retry (I have done this and it
works, once debugged it is actually pretty neat).

The core problem though, is that this must be handled by user code (the
exception handler) and any tracking data won't be shared (unless that is
resident in shared memory, which of course can be done).

However perhaps this offers the only hope of designing a mechanism to do
what I want to do.

Does anyone have a view on this approach?

Hugo







"m" wrote:

> It is possible to monitor network operations on several levels (For network
> operations, both the network stack and the redirector will be involved). As
> others have noted, local memory access can only be observed by hardware (or
> virtualization software) because by design the kernel stays out of the way
> as much as possible.
>
> ">" <hugh<underbar> wrote in message
> news:0C44A04B-A6A7-46FE-8055-...
> > I want to get some idea of how to approach the following problem.
> >
> > I'm very interested in being able to intercept read/writes from/to mapped
> > memory pages, be they resident in physical memory or paged out to disk.
> > I'd
> > like to detect these operations when they arise from either a local
> > process
> > or one running on a networked machine.
> >
> > I'd also like to be able to detect when an app on some remote machine
> > opens
> > and maps a file from the local machine and unmaps it.
> >
> > I've not written a device driver before and know it is far from trivial,
> > although I am a seasoned developer and very familiar with most of the
> > internals of the OS.
> >
> > I would suspect that it is possible to craft a driver that can trap these
> > operations and then do something (or queue some work to be done later).
> >
> > This is just a vague exploration just now, not a real need, I'm just
> > trying
> > to ascertain from experts the main issues that this raises.
> >
> > Thanks
> >
> > Hugo
> >
> >
> >
> >
> >
> .
>