How do I get smooth handwriting on a Tablet PC using Java2/Swing?

Thread view:

Christian Stapfer - 17 Mar 2007 07:08 GMT

I have written a distributed presentation/whiteboarding application
(Java2/Swing) for online-tutoring. Currently I use a graphics tablet to draw
and write on the whiteboard.
Now I would like to switch from using a separate graphics tablet to
using the stylus of a Tablet PC. However, my Java application seems to drop
a great many mouse drag events, which results in hardly legible handwriting
(not so with native Tablet PC applications like Windows Journal).

http://www.freesoft.org/software/tablet-java/ suggests patching the Java
runtime to solve this problem - an idea from which I instinctively recoil in
horror. (I'd rather rewrite the whole thing in C#, which I even might
consider a good occasion to finally learn a reasonable amount of C# -
although, sadly, Java's level of platform independence would be lost in the
process.) Does anyone of you know another (less hackish) way to get more
mouse-path information in a Java2/Swing application?

Regards,
Christian

Reply to this Message

Boudewijn Dijkstra - 18 Mar 2007 14:37 GMT

Op Sat, 17 Mar 2007 07:08:31 +0100 schreef Christian Stapfer <nil@dev.nul>:

> I have written a distributed presentation/whiteboarding application
> (Java2/Swing) for online-tutoring. Currently I use a graphics tablet to
[quoted text clipped - 16 lines]
> process.) Does anyone of you know another (less hackish) way to get more
> mouse-path information in a Java2/Swing application?

java.awt.MouseInfo.getPointerInfo().getLocation()

Signature

Gemaakt met Opera's revolutionaire e-mailprogramma:
http://www.opera.com/mail/

Reply to this Message

Christian Stapfer - 18 Mar 2007 15:37 GMT

> Op Sat, 17 Mar 2007 07:08:31 +0100 schreef Christian Stapfer
> <nil@dev.nul>:
[quoted text clipped - 20 lines]
>
> java.awt.MouseInfo.getPointerInfo().getLocation()

Thank you very much for your reply. - I'm not sure whether
this helps to solve my problem, however.
There are actually two problems involved:

1. How to get high-resolution mouse-drag events
2. How to get high-resolution stylus positioning information

As I understand the situation, neither of these two problems
is really solvable with java.awt.MouseInfo.getPointerInfo().getLocation()
in itself: the resolution is not as high as it should be on
a tablet PC (the resolution of the digitizer is not the same
- or at the very least need not be the same - as the resolution
of the screen: but java.awt.MouseInfo.getPointerInfo().getLocation()
is). And continuous polling of the mouse position information
is completely out of the question: I need suitable events.

Actually I'm now slowly drifting in the direction of reimplementing
the whole program in C#, since being able to produce high-quality
handwriting during online-tutoring sessions with a tablet PC would
be great.

Regards,
Christian

Reply to this Message

Chris Uppal - 18 Mar 2007 18:09 GMT

> I have written a distributed presentation/whiteboarding application
> (Java2/Swing) for online-tutoring. Currently I use a graphics tablet to
[quoted text clipped - 7 lines]
> http://www.freesoft.org/software/tablet-java/ suggests patching the Java
> runtime to solve this problem [...]

A couple of thoughts.

If the referenced article is correct then the system is only coalescing mouse
events that you haven't consumed -- which means there should only be a problem
if you are not keeping up with mouse movements. In such a case, you have a
problem /anyway/. The normal system behaviour attempts to compensate for that,
but if the strategy it uses -- chucking stuff away instead of buffering it
up -- is unsuitable (which sounds likely for any kind of mouse drawing), then
the obvious solution is to do the buffering yourself, instead of putting the
slow processing inline with each event as it happens.

Another thought is that the tablet's Java implementation may have the AWT
method Component.coalesceEvents() available for override in your custom
Component. I have never tried it myself, but it looks as if you can override
that to control what events are coalesced.

-- chris

Reply to this Message

Tom Hawtin - 18 Mar 2007 21:34 GMT

> Another thought is that the tablet's Java implementation may have the AWT
> method Component.coalesceEvents() available for override in your custom
> Component. I have never tried it myself, but it looks as if you can override
> that to control what events are coalesced.

Looking at the 1.6 EventQueue source (which is quite different from
1.5), you might need to use coalesceEvents as if it were your event
listener. Also note, rather unusually coalesceEvents is not necessarily
invoked in the EDT.

Tom Hawtin

Reply to this Message

Christian Stapfer - 19 Mar 2007 07:10 GMT

>> Another thought is that the tablet's Java implementation may have the AWT
>> method Component.coalesceEvents() available for override in your custom
[quoted text clipped - 8 lines]
>
> Tom Hawtin

Thank you both for your replies: I will look into this.
As to my application possibly not being able to keep
up with mouse movements: although my application has
hardly anything other to do than to process those mouse
movements (i.e. storing them but also drawing a small
line segment connecting the last point with the current
point of the stroke being drawn), this seems quite
possible to me.
If I am not mistaken, the C# low-level Ink handling
system actually takes in an entire stroke before
handing the complete stroke to the application...
Also I vaguely remember having read a comment
stating that in order to have really high-quality
digitizer data for a stroke one needs to handle
the incoming digitizer data with a "real-time"
priority thread. - But maybe there is a medium-quality
way, instead of just a low- or a high-quality
way of capturing an ink stroke? - And medium-quality
just might be good enough for me, and my pupils...

Regards, and thanks again for your replies,
Christian

Reply to this Message

Boudewijn Dijkstra - 28 Mar 2007 22:11 GMT

Op Mon, 19 Mar 2007 07:10:40 +0100 schreef Christian Stapfer <nil@dev.nul>:

> As to my application possibly not being able to keep
> up with mouse movements: although my application has
[quoted text clipped - 3 lines]
> point of the stroke being drawn), this seems quite
> possible to me.

Nobody should paint in an event dispatch thread. Just store it and issue
a repaint.

Signature

Gemaakt met Opera's revolutionaire e-mailprogramma:
http://www.opera.com/mail/

Reply to this Message

Christian Stapfer - 29 Mar 2007 06:36 GMT

> Op Mon, 19 Mar 2007 07:10:40 +0100 schreef Christian Stapfer
> <nil@dev.nul>:
[quoted text clipped - 8 lines]
> Nobody should paint in an event dispatch thread.
> Just store it and issue a repaint.

I can understand your position, kind of. However,
the line drawing has to follow the dragging of
the mouse *immediately*: because, for the user,
it represents the movement of the pen.

How, for example, would I handle rubberbanding,
if I were to follow your precept ("don't ever
draw anything in an event handler")? As compared
to rubberbanding of a large rectangle, I have
to draw a really only very tiny line segment when
incrementally drawing the pen-path from within
an event handler...

Regards,
Christian

P.S: No amount of clever coding seems to help
compensate for the fact that mouse-coordinates
are coarser than pen-coordinates on a Tablet PC
(because they are screen coordinates rather
than pen-digitizer coordinates) .

Reply to this Message

Boudewijn Dijkstra - 02 Apr 2007 21:30 GMT

Op Thu, 29 Mar 2007 07:36:02 +0200 schreef Christian Stapfer <nil@dev.nul>:

>> Op Mon, 19 Mar 2007 07:10:40 +0100 schreef Christian Stapfer
>> <nil@dev.nul>:
[quoted text clipped - 13 lines]
> the mouse *immediately*: because, for the user,
> it represents the movement of the pen.

Drawing involves at least an order of magnitude more work than registering
and storing a mouse event. It does not matter much if drawing is
interrupted shortly when an event comes in. Note that "immediately" is
for the user something different than to the machine.

> How, for example, would I handle rubberbanding,
> if I were to follow your precept ("don't ever
> draw anything in an event handler")?

What is the problem with rubberbanding?

> As compared
> to rubberbanding of a large rectangle, I have
[quoted text clipped - 10 lines]
> (because they are screen coordinates rather
> than pen-digitizer coordinates) .

Does that include Newtonian physics calculations on speed, drag and
momentum? If you do not paint on the event dispatch thread, you can
collect more useful timing information from the events.

Signature

Gemaakt met Opera's revolutionaire e-mailprogramma:
http://www.opera.com/mail/

Reply to this Message

Christian Stapfer - 03 Apr 2007 05:43 GMT

> Op Thu, 29 Mar 2007 07:36:02 +0200 schreef Christian Stapfer
> <nil@dev.nul>:
[quoted text clipped - 26 lines]
>
> What is the problem with rubberbanding?

Well, maybe I am just not being professional. I took
the basic idea of how to implement rubberbanding from
the book "Graphic Java: Mastering the JFC" by David
M. Geary (published by Sun Microsystems).
Geary does the drawing of rubberband lines from ...
... well, from within mouse event handlers.
If I were to queue the handling of every tiny piece
of a freehand line being drawn for later processing,
I might end up queuing about two hundred line draws
in the process. The whole queuing process would require
more work than the drawing of those many, usually
very small line segments.

>> As compared
>> to rubberbanding of a large rectangle, I have
[quoted text clipped - 14 lines]
> momentum? If you do not paint on the event dispatch thread, you can
> collect more useful timing information from the events.

Unfortunately, I don't get your point. I don't necessarily
need more timining information: what I need is sufficient
resolution of pen coordinates. On a tablet PC, the pen
has higher resolution than the screen. Why did they provide
higher resolution pen coordinates, if, according to you,
higher resolution can be had some other way than by suppording
it with appropriate hardware?
But maybe you have a very clever idea of a workaround
for the limitation to mouse (screen) coordinates. As for me:
I'm not a great friend of cleverness, and avoid it whenever
I can. Getting good pen stroke information is very easy
from within a C# application. Not so from Java2, and that
was my problem. For that reason I am already trying to port
my application to C#, and have made substantial progress in
that direction.

Regards,
Christian

Reply to this Message

Boudewijn Dijkstra - 03 Apr 2007 17:27 GMT

Op Tue, 03 Apr 2007 06:43:17 +0200 schreef Christian Stapfer <nil@dev.nul>:

>> Op Thu, 29 Mar 2007 07:36:02 +0200 schreef Christian Stapfer
>> <nil@dev.nul>:
>>>> Op Mon, 19 Mar 2007 07:10:40 +0100 schreef Christian Stapfer
>>>> <nil@dev.nul>:

>>>> Nobody should paint in an event dispatch thread.
>>>> Just store it and issue a repaint.
[quoted text clipped - 21 lines]
> Geary does the drawing of rubberband lines from ...
> ... well, from within mouse event handlers.

Examples often take shortcuts in order to not confuse the layman.

> If I were to queue the handling of every tiny piece
> of a freehand line being drawn for later processing,
> I might end up queuing about two hundred line draws
> in the process. The whole queuing process would require
> more work than the drawing of those many, usually
> very small line segments.

Don't underestimate the time a drawing operation takes, and the time it
takes until the next screen refresh.

>>> P.S: No amount of clever coding seems to help
>>> compensate for the fact that mouse-coordinates
[quoted text clipped - 13 lines]
> higher resolution can be had some other way than by suppording
> it with appropriate hardware?

Clearly, fancy physics calculations are a bit over-the-top and of course
the hardware provides a relatively high resolution. You can notice the
relatively low resolution of very cheap optical mice, which will once in a
while start to 'walk' towards one end of the screen.

> But maybe you have a very clever idea of a workaround
> for the limitation to mouse (screen) coordinates. As for me:
[quoted text clipped - 4 lines]
> my application to C#, and have made substantial progress in
> that direction.

For one, you could consider using some JNI functionality to pump custom
mouse events into the system. You will have to manually perform the
coordinate conversions, though.

Signature

Gemaakt met Opera's revolutionaire e-mailprogramma:
http://www.opera.com/mail/

Reply to this Message

How do I get smooth handwriting on a Tablet PC using Java2/Swing?

Free Magazines