Switching from Non-Blocking to Blocking IO

Thread view:

Christian - 29 Nov 2007 21:56 GMT

Hello

I have just tried using non-Blocking IO in a JavaSE program.
My problem is that at one point I have to switch during the the running
protocol to blocking-io as some compressed data arrives and my gzip
implementation only allows decompressing from a blocking stream.

so what I tried was this

@Override
public ByteChannel retrieveChannel() throws IOException {

SocketChannel sc = (SocketChannel)key.channel();
key.cancel();

sc.configureBlocking(true);
sc.socket().setSoTimeout(20000);
return sc;
}

the problem with this code is that after calling retrieveChannel() some
data arrvies on the channel and is immediately read by the thread that
uses the selector to get all non-blocking io..

Now what have I done wrong?
It smells like a synchronization problem ..

Any hints for me? Or better solutions to the problem?

Christian

Reply to this Message

Mike Schilling - 30 Nov 2007 17:44 GMT

> Hello
>
[quoted text clipped - 25 lines]
>
> Any hints for me? Or better solutions to the problem?

If you can buffer the entire gzip file in memory, do that, and point a
ByteArrayInputStream at the result. If not, create an InputStream subclass
that returns data from the socket and blocks when none is available. The
second is more complicated, of course, since it requires multiple threads
and some synchronization between them.

Reply to this Message

Christian - 30 Nov 2007 18:54 GMT

Mike Schilling schrieb:

>> Hello
>>
[quoted text clipped - 31 lines]
> second is more complicated, of course, since it requires multiple threads
> and some synchronization between them.

I can't buffer it in memory..
as size could be anything from 100KiB to 16 GiB
I was just hoping that switching would be possible to spare me the work
of implementing this workaround.

Reply to this Message

Esmond Pitt - 01 Dec 2007 00:59 GMT

> It smells like a synchronization problem ..

Yup. Try waking up the selector immediately you deregister the channel.
Make sure the selector loop copes correctly with zero ready channels.
*Then* put the channel into blocking mode.

> Any hints for me? Or better solutions to the problem?

On second thoughts, how did you get here in the first place? It must
have been in response to an OP_READ originally, which must have been
processed in the selector thread, so the deregister and switch to
blocking mode should have been done there, then you wouldn't have a problem.

Reply to this Message

Christian - 01 Dec 2007 13:24 GMT

Esmond Pitt schrieb:

>> It smells like a synchronization problem ..
>
> Yup. Try waking up the selector immediately you deregister the channel.
> Make sure the selector loop copes correctly with zero ready channels.
> *Then* put the channel into blocking mode.

My selector-thread sleeps for a few milliseconds when it is awakened and
no keys are selected. Is this the correct handling?
It works now but not allways.
Would it be better to submit a Runnable to the selector-thread to do
this unregistering and set to Blocking?

>> Any hints for me? Or better solutions to the problem?
>
[quoted text clipped - 3 lines]
> blocking mode should have been done there, then you wouldn't have a
> problem.

IO thread reads protocol commands from the stream. Then an Executor is
used to handle this command. So the reading selector-thread doesn't know
the protocol and can therefore not do this.

Thank you for your help
Christian

Reply to this Message

Esmond Pitt - 02 Dec 2007 23:24 GMT

> My selector-thread sleeps for a few milliseconds when it is awakened and
> no keys are selected. Is this the correct handling?

Seems OK. As long as it sleeps longer than it takes the other thread to
get from wakeup() to cancel().

> It works now but not allways.
> Would it be better to submit a Runnable to the selector-thread to do
> this unregistering and set to Blocking?

I've seen this technique used, sometimes quite extensively, but I've
never had to use it myself. It's always seemed like overkill to me, but
if it works for you ...

Reply to this Message

Switching from Non-Blocking to Blocking IO

Free Magazines