Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / December 2005

Tip: Looking for answers? Try searching our database.

Java and SHA - with big files...

Thread view: 
Benoît - 30 Nov 2005 23:17 GMT
Hi
I want to compare two big files (like films), so I create a fingerprint
and I look...
It works very well with little files, but a java.lang.outOfMemory
appears with big files...
I think the array data in readFile(String filename) function is too
big...so how do I do?

try {
  byte[] data = readFile(myfile.getAbsolutePath());
  MessageDigest md = MessageDigest.getInstance("SHA");
  md.update(data);
  byte[] digest = md.digest();
  System.out.println("Hexadecimal: " + myfile.getAbsolutePath()+new
BigInteger(digest).toString(16));
}catch (Exception e) {
  System.out.println(e);
}

...

private static byte[] readFile(String filename) {
       try {
         RandomAccessFile raf = new RandomAccessFile(filename, "r");
         byte[] data = new byte[(int)raf.length()];
         raf.readFully(data);
         raf.close();
         System.out.println("He");
         return data;
       }
       catch (FileNotFoundException e) {}
       catch (IOException e) {}
       return null;
     }

Thanks
Stefan Schulz - 01 Dec 2005 00:56 GMT
> Hi
> I want to compare two big files (like films), so I create a fingerprint
[quoted text clipped - 3 lines]
> I think the array data in readFile(String filename) function is too
> big...so how do I do?

Take a look at MessageDigest.update() and friends. These methods are
explicitly meant to deal with chunks of data, instead of the whole file.
You can just fill up a fixed-size array until the file has been traversed,
adding one "block" at a time to the digest.

Signature

You can't run away forever,
But there's nothing wrong with getting a good head start.
          --- Jim Steinman, "Rock and Roll Dreams Come Through"
         

Roedy Green - 01 Dec 2005 03:33 GMT
>Take a look at MessageDigest.update() and friends. These methods are
>explicitly meant to deal with chunks of data, instead of the whole file.
>You can just fill up a fixed-size array until the file has been traversed,
>adding one "block" at a time to the digest.

see http://mindprod.com/jgloss/digest.html

for what you are doing an size/date/adlerian checksum might suffice.
It is much faster to compute.

See http://mindprod.com/jgloss/adler.html
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.



Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.