Capture, modify and redisplay of graphic user interface

Hello

I am try to capture graphical user interface of any program running on
Windows (or for whole desktop) then to do something with it and display it
back on the screen directed by my "program". My "program" will capture the
user interface (like mouse and keyboard) and will redirect it to the
appropriate running program. I have worries about first part of that idea –
How I can capture all graphical information. After researching in last days I
come to conclusion that I have to develop my custom video driver (with
DDK/WDK) where all drawings from other programs will go and my "program" will
redraw on the "real" video driver? - Is it posible that - to have one video
driver (mine, virtual) for "all" programs and other video driver (the real
one) for some (mine) programs? Or I have to find better solution?

I am long time Win32 developer, have basic knowledge of DDK, sure I have to
learn it very well, but is my idea possible for developing? I am not asking
for step by step "how to" solve that problem - just asking for "show-stopper"
problems in my idea.

The very basic question - that is the right way - custom display driver? If
it is not - it will be pity all time spend of learning WDK, because I will
not use it for something else.

Thanks in advance.

Miroslav Penchev <> wrote:
>
>I am try to capture graphical user interface of any program running on
>Windows (or for whole desktop) then to do something with it and display it
>back on the screen directed by my "program".

Do you mean something like Remote Desktop, or one of the many versions of
VNC? Or do you mean something like CamStudio?

Part of what I'm saying is that there are many, many, many programs doing
this kind of thing. You might make better use of your time to see if
there's something already out there.

>After researching in last days I
>come to conclusion that I have to develop my custom video driver (with
>DDK/WDK) where all drawings from other programs will go and my "program" will
>redraw on the "real" video driver? - Is it posible that - to have one video
>driver (mine, virtual) for "all" programs and other video driver (the real
>one) for some (mine) programs? Or I have to find better solution?

You can READ the screen from a simple application, with no driver work at
all. Before embarking on the long and difficult road to a mirror driver,
make ABSOLUTELY SURE you can't do this from a simple application.
--
Tim Roberts,
Providenza & Boekelheide, Inc.

anshul makkar <> wrote:
>
>No need to develop virtual video driver. All you need is upper filter
>graphics driver.

You make it sound so simple. GDI drivers are not plug-and-play, and they
don't communicate via IRPs. You can't just slap in an upper filter like
other drivers. It IS possible to filter a GDI driver, basically by making
yourself the primary display driver and loading the previous primary
yourself, but it's error prone, and for the most part it's more work than a
mirror driver.

>Filter graphics driver will capture all the drawing calls and in this
>graphics driver you can again blit the drawings on the screen using
>GDI calls.

It's not that easy.
--
Tim Roberts,
Providenza & Boekelheide, Inc.

Thank you guys for your replies.

After I refined my idea I come to conclusion that it is must for me to
capture drawings per process basis (not for whole desktop), so that exclude
the idea of mirror display driver. What I found till now reading DDK
documentation and examples makes me think that this cant be done with low
level driver, because there is no identification from which process comes
current drawing. I can be wrong, but digging on internet and forums I come to
conclusion that I have to use “hooking� in user processes of some of GDI
functions (functions related to creating and releasing of DCs and some other
functions like CreateProcess) and to redirect drawings to in-memory DC with a
lot of other tweaks I suppose.

Will check in documentation what I can done with upper filter graphics
driver, but the most important requirement for now is capture per process
basis of all drawings.


"Miroslav Penchev" wrote:

> Hello
>
> I am try to capture graphical user interface of any program running on
> Windows (or for whole desktop) then to do something with it and display it
> back on the screen directed by my "program". My "program" will capture the
> user interface (like mouse and keyboard) and will redirect it to the
> appropriate running program. I have worries about first part of that idea –
> How I can capture all graphical information. After researching in last days I
> come to conclusion that I have to develop my custom video driver (with
> DDK/WDK) where all drawings from other programs will go and my "program" will
> redraw on the "real" video driver? - Is it posible that - to have one video
> driver (mine, virtual) for "all" programs and other video driver (the real
> one) for some (mine) programs? Or I have to find better solution?
>
> I am long time Win32 developer, have basic knowledge of DDK, sure I have to
> learn it very well, but is my idea possible for developing? I am not asking
> for step by step "how to" solve that problem - just asking for "show-stopper"
> problems in my idea.
>
> The very basic question - that is the right way - custom display driver? If
> it is not - it will be pity all time spend of learning WDK, because I will
> not use it for something else.
>
> Thanks in advance.

> No need to develop virtual video driver. All you need is upper filter
> graphics driver.
>
> Filter graphics driver will capture all the drawing calls and in this
> graphics driver you can again blit the drawings on the screen using
> GDI calls.

Hi Anshul,

What "filter graphics driver" are you referring to?