
Signature
Ian Shef 805/F6 * These are my personal opinions
Raytheon Company * and not those of my employer.
PO Box 11337 *
Tucson, AZ 85734-1337 *
> > while(1){ //do forever until terminated
>
> OK, what is your real code? This is not Java, it won't compile.
>
> I want to help, but if this is wrong, then I cannot trust the rest of what
> you posted.
OK, while this isn;t the exact code, its the relevant section....
static final int numRatings = 100456599;
static final int epoch = 100000;
FileChannel fc = new FileInputStream (filepath).getChannel();
MappedByteBuffer mappedBuff = fc.map(FileChannel.MapMode.READ_ONLY, 0,
fc.size());
for (int epoch = 0; epoch < numEpochs; epoch++) {
mappedBuff.rewind();
for (int k = 0; k < numRatings; k++) {
cUser = mappedBuff.getInt();
cMovie = (int)mappedBuff.getShort();
cRating = (int)mappedBuff.get();
//doStuff;
}
}
thebad1 - 08 Jan 2007 10:38 GMT
> FileChannel fc = new FileInputStream (filepath).getChannel();
> MappedByteBuffer mappedBuff = fc.map(FileChannel.MapMode.READ_ONLY, 0,
> fc.size());
Where filepath is a binary file packed with over 100,000,000 sets of
int,short,bytes concated together like so;
$ od -x -N 30 ratings.bin
0000000 1600 ccb7 0100 0003 8b0c 005d 0501 0d00
0000020 1581 0100 0004 7800 009e 0401 0c00
I'm now down to about 13 seconds for a complete loop over that file, I
was wondering whether there were any further improvements to such a
sweep that can be done in Java.
Ian Shef - 08 Jan 2007 20:05 GMT
"thebad1" <thomas.hodder@gmail.com> wrote in news:1168251545.671290.310410@
51g2000cwl.googlegroups.com:
> OK, while this isn;t the exact code, its the relevant section....
>
[quoted text clipped - 17 lines]
> }
> }
Quite different from what was originally posted! Perhaps you have been
experimenting and learning along the way.
At this point I will shut up on this topic, as I have no experience with
FileChannel and MappedByteBuffer.
Anyone with experience have suggestions for speeding this up?

Signature
Ian Shef 805/F6 * These are my personal opinions
Raytheon Company * and not those of my employer.
PO Box 11337 *
Tucson, AZ 85734-1337 *
thebad1 - 08 Jan 2007 21:53 GMT
> Quite different from what was originally posted! Perhaps you have been
> experimenting and learning along the way.
I've changed the inner loop to avoid checking the variable every time;
for (;;) {
cUser = mappedBuff.getInt();
cMovie = mappedBuff.getShort();
cRating = mappedBuff.get();
}
}catch ( BufferUnderflowException ex ){
// System.out.println("");
}
Ian Shef - 09 Jan 2007 19:18 GMT
"thebad1" <thomas.hodder@gmail.com> wrote in news:1168293201.398727.188740
@v33g2000cwv.googlegroups.com:
>> Quite different from what was originally posted! Perhaps you have been
>> experimenting and learning along the way.
[quoted text clipped - 9 lines]
> // System.out.println("");
> }
Did this provide a measurable speedup ? How much?

Signature
Ian Shef 805/F6 * These are my personal opinions
Raytheon Company * and not those of my employer.
PO Box 11337 *
Tucson, AZ 85734-1337 *