This is a bit of a distributed systems design issue question. This is
my current set-up:

Server A (Win): SQL database
Server B (Linux): computation engine (C++ that reads proprietary
format)

Currently, we have a JDBC client on server A which extracts data
from the SQL database and writes it to a proprietary data format
and zipped, all in the Java client. The data is then scp'ed over to
Server B, where we unzip the file by hand so it can be read by
the C++ application.

I would like to bypass the file I/O on server A. I'm thinking of
creating a client-server pair on Server A and B. Our Java process
on A still uses JDBC to get SQL data. However, instead
of writing to disk, it serializes the data into objects, which are
then shipped over secure sockets to Server B,  where a
complementary process deserializes the data and writes it to file.

Many files need to be transferred between A and B, on the order
of thousands of files, each about 50K to 100K each.

What do you make of this?