Data Dropouts with Communications API

Data Dropouts with Communications API - Help!

Thread view:

Thomas M. Alldread - 19 Dec 2005 06:29 GMT

Greetings:

    I am developing a java 1.5 application using the NetBeans 5
beta 2 IDE on the Windows XP platform. My application does data
acquisition from an external microcontroller based device via a
conventional RS232 serial (Com) port. The data rate is set at 115,200
Baud although the actual throughput is quite a bit lower than that.
The problem does not seem to be baudrate sensitive but does seem to be
associated with unrelated system activities such as accessing the XP
desktop taskbar or activity in other applications. The task manager
indicates a CPU loading of around 18% maximum by my program during
data transfers.

The configuration is as follows:

    -There is a 256 byte FIFO buffer in the microcontroller data
output.
    -RTS/CTS hardware flow control is utilized.

    -I have set the com port buffer size to 16K using the
appropriate javax.comm method

    - I use a 16K byte "read" buffer within the "data available"
ISR to capture the data from the input stream

    I encounter occasional data dropouts of a few characters. The
lost characters do not get to the input stream read buffer. I
determined when the program is running that usually the ISR is
serviced quickly as there are usually only about 6 bytes picked up
from the input stream per read cycle which coincidentally is about
equal to the number of characters lost when a dropout occurs. It
appears as if occasionally the communications API fails to add data
bytes to the input stream. I wonder if anyone in this group has
encountered similar problems, can shed any light on this dilemma or
can offer any suggestions that could lead to a fix?

    I am using version 2 of the javax.comm communications API. I
notice that Sun MicroSystems have released a version 3 of the
communications API for Linux and other operating systems but not for
Windows. I think I read that they corrected some bugs in version 2 and
wonder if I have run across one? I wonder if anyone here knows where I
might find a version 3 communications API for Windows?

    I wonder if anyone here can suggest any other java language
newsgroups or mailing lists where I might also post this cry for help?

    Many thanks in advance for any help!

Tom Alldread

Reply to this Message

JScoobyCed - 19 Dec 2005 07:12 GMT

> Greetings:
>
[quoted text clipped - 3 lines]
> conventional RS232 serial (Com) port. The data rate is set at 115,200
> Baud

[...]

> I encounter occasional data dropouts of a few characters. The
> lost characters do not get to the input stream read buffer.

[...]

I had similar trouble when reading serial data from a magnetic card
reader. It was quite a while ago, and I fixed the problem by encoding
the data I put on the magnetic card in a way that I can correct lost
bits (CRC style encoding)

> I am using version 2 of the javax.comm communications API. I
> notice that Sun MicroSystems have released a version 3 of the
> communications API for Linux and other operating systems but not for
> Windows. I think I read that they corrected some bugs in version 2 and
> wonder if I have run across one?

[...]
I have triied the RXTX port of the JavaComm. I needed it because of the
OS: I was on linux ( a different project than the card reader which was
on Win2000). I've heard a lot of good from this RXTX port:
http://users.frii.com/jarvi/rxtx/

> I wonder if anyone here knows where I
> might find a version 3 communications API for Windows?

I hope it'll come soon :)

Signature

JSC

Reply to this Message

Roedy Green - 19 Dec 2005 08:43 GMT

>I encounter occasional data dropouts of a few characters. The
>lost characters do not get to the input stream read buffer. I
[quoted text clipped - 6 lines]
>encountered similar problems, can shed any light on this dilemma or
>can offer any suggestions that could lead to a fix?

This takes me back. One trick you can use is a timer that
periodically probes the device, the idea being you presume some
interrupts will get lost, and you poke it from time to time to pick up
any stray data before it gets lost.

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Reply to this Message

Thomas M. Alldread - 19 Dec 2005 17:15 GMT

Hi Roedy:

    This is interesting the hear about as it sounds like others
may have run into a similar problem.

    In this case I am capturing a continuous data acquisition
stream of about 2K bytes per second and the dropouts are pretty
random. Sometimes it will run for as long as a minute without a
dropout. The interrupts occur about 3mS apart. Your recommendation of
a timer has got me wondering if I might be able to call the ISR more
frequently by scanning the ISR after each block of data is processed
without loading the CPU too much.

    What puzzles me is that as I understand it even if an
interrupt is missed by my application the data should still be added
to the input stream by the underlying communications API. I think the
USART has a 15 byte buffer in it and the port buffer (which I think
might be an alias for the input stream which I assume is a FIFO buffer
in disguise?) is set at 16K bytes. The only explanation that makes
sense to me is a possible bug related to the communications API with
Windows XP.

    Many thanks for helping!

>>I encounter occasional data dropouts of a few characters. The
>>lost characters do not get to the input stream read buffer. I
[quoted text clipped - 11 lines]
>interrupts will get lost, and you poke it from time to time to pick up
>any stray data before it gets lost.

Reply to this Message

Roedy Green - 19 Dec 2005 22:06 GMT

> This is interesting the hear about as it sounds like others
>may have run into a similar problem.

My equivalent was building home brew hardware and writing an home brew
device driver to let an PDP11 talk to an Apple II back in the days
before there were LANS.

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Reply to this Message

stixwix - 19 Dec 2005 09:38 GMT

Have you tried different hardware?
I once had a similar problem when testing RS232 connectivity with a
Dell laptop. I seem to recall something about non-standard UARTS?

Reply to this Message

Thomas M. Alldread - 19 Dec 2005 16:42 GMT

Hi Andy:

Yes I forgot to mention that I installed it on a second
computer and have the same problem. The second computer is a 2.6GHz
AMD Athlon where the peak CPU usage from my application was only about
10%. My next plan is to re-compile it under Linux with the version 3
communications API to see if the problem persists.

Many thanks for your suggestion!

Tom

>Have you tried different hardware?
>I once had a similar problem when testing RS232 connectivity with a
>Dell laptop. I seem to recall something about non-standard UARTS?

Reply to this Message

Roedy Green - 19 Dec 2005 22:14 GMT

>I am developing a java 1.5 application using the NetBeans 5
>beta 2 IDE on the Windows XP platform. My application does data
>acquisition from an external microcontroller based device via a
>conventional RS232 serial (Com) port.

I handled this in a department store process control system by using
a not-that-expensive smart serial i/o card in the PC. IIRC the card
had a buffer of about 64K and fielded the interrupts itself. My
Abundance app then had plenty of time to process the messages. The
card could handle multiple serial ports. Output too was simple and
required no threading. The app send the message which the card
accepted and buffered.

To do the same thing in Java today, you would have JNI-C interface to
the card.

If you are using the Java COMM stuff, people have complained it was
badly written only suitable for driving a printer.

See http://mindprod.com/jgloss/serialport.html

for other serial port software.

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Reply to this Message

Thomas M. Alldread - 20 Dec 2005 23:01 GMT

Hi again Roedy:

    I have downloaded the RXTX 2.1 package which I am going to try
and run under Windows XP.

    Since there are not any dropouts with HyperTerminal it would
seem that the computer hardware is able to handle the data stream OK.
I use an external 56K modem for Internet access all the time. The com
port interface to it runs at 115K and it seems absolutely solid. I can
only conclude that there must be something wrong with my code or
underlying comm API. It will be interesting to see how RXTX performs
(assuming I can get it to compile).

    Many thanks again for your interest and help!

Tom Alldread

>I handled this in a department store process control system by using
>a not-that-expensive smart serial i/o card in the PC. IIRC the card
[quoted text clipped - 13 lines]
>
>for other serial port software.

Reply to this Message

Roedy Green - 21 Dec 2005 01:18 GMT

> Since there are not any dropouts with HyperTerminal it would
>seem that the computer hardware is able to handle the data stream OK

Heck, we used to read 9600 baud on a Z80. I should hope so. There are
still ways to screw up the software to lose interrupts. My gut
reaction is to get the OS out the way and let me handle the interrupts
myself so that I know what the heck is going on and I can fix
problems. But today you can't get away with that. You have to find a
solid driver.

Now that the cost of Ethernet chips have dropped so low, you almost
expect every device now to be lan-pluggable, and say to hell with
RS232C.

Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Reply to this Message

Data Dropouts with Communications API - Help!

Free Magazines