Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / February 2006

Tip: Looking for answers? Try searching our database.

Finding Duplicate File Objects

Thread view: 
Luc The Perverse - 25 Feb 2006 06:33 GMT
I'm rewriting my duplicate file checker program so it doesn't require
recompilation every time I want to use it.   (I'm basically starting over
from scratch.)

The problem is, I have no way to prevent a user from adding a directory
twice, or adding a directory inside of a directory.

What is the best way to get rid of duplicate files (right now they are going
into a linked list, but I can change that.)

Here is the code - it doesn't really do much yet (but if you know I'm
looking for duplicate File objects in a linked list I don't think you should
need it.)

import java.util.*;
import java.io.*;
import java.security.*;

class FileFilter{
String[] extInclude = {};
String[] extExclude = {};
long minFileSize = 0;          // 0
long maxFileSize = 2147483647; // 2 Gb
boolean matches(File cur){
 long curL= cur.length();
 if(curL<minFileSize || curL>maxFileSize)
  return false;
 int i;
 String v = cur.getAbsolutePath();
 if(extExclude != null)
  for(i = 0;i<extExclude.length;i++)
   if(extExclude[i].compareToIgnoreCase(v.substring(v.length()-extExclude[i].length()))==0)
    return false;
 if(extInclude == null)
  return true;
 for(i = 0;i<extInclude.length;i++)
  if(extInclude[i].compareToIgnoreCase(v.substring(v.length()-extInclude[i].length()))==0)
   return true;
 return false||(extInclude.length==0);
}
}

public class FileUtil{
public static boolean DoDir(LinkedList<File> fileList, String dirName,
FileFilter filter){
 File theDir = new File(dirName);
 if(!theDir.exists())
  return false;
 LinkedList<File> directoryList = new LinkedList<File>();

 directoryList.add(theDir);
 while(!directoryList.isEmpty()){
  File cur = directoryList.removeFirst();
  String[] list = cur.list();
  if(list==null)
   continue;
  for (int i=0; i<list.length; i++){
   File looking = new File(cur, list[i]);
   if(looking.isDirectory())
    directoryList.add(looking);
   else{
    if(looking.canRead()&&(filter==null || filter.matches(looking)))
     fileList.add(looking);
   }
  }
 }
 return true;
}

public static void main(String[] h){
 int i;
 FileFilter myFilter = new FileFilter();
 LinkedList<File> FileList = new LinkedList<File>();
 //--
 if(h.length==0){
  System.out.println("\"p=PATH\"");  //param 0
  System.out.println("e=.EXT (include)"); //param 1
  System.out.println("E=.EXT (exclude)"); //param 2
  System.out.println("f=MINFILESIZE");    //param 3
  System.out.println("F=MAXFILESIZE");    //param 4
  return;
 }
       int[] parameterCount = new int[20];
 for(i=0;i<h.length;i++){
  String curString = h[i];
  if(curString.length()<3||curString.charAt(1)!='='||!(curString.charAt(0)>='a'&&curString.charAt(0)<='z'||curString.charAt(0)>='A'&&curString.charAt(0)<='Z')){
   System.out.println("Improperly formatted parameter - " + curString);
  }
  switch(curString.charAt(0)){
        case 'p': parameterCount[0]++; break;
                 case 'e': parameterCount[1]++; break;
                 case 'E': parameterCount[2]++; break;
                 case 'f': parameterCount[3]++; break;
                 case 'F': parameterCount[4]++; break;
                 default: System.out.println("Unrecognized parameter - " +
curString);
  }
 }
 if(parameterCount[1]>0&&parameterCount[2]>0){
      System.out.println("I'm sorry - until wildcards are implemented,
include and exclude extensions are mutually exclusive.");
      return;
 }
 String[] extensions = null;
 if(parameterCount[1]>0||parameterCount[2]>0){
      extensions = new String[parameterCount[1]+parameterCount[2]];
      int extCount = 0;
      for(i=0;i<h.length;i++){
              String curString2 = h[i];
        if(curString2.charAt(0)=='e'||curString2.charAt(0)=='E')
             extensions[extCount++] = curString2.substring(2);
   }
   if(parameterCount[1]>0)
                 myFilter.extInclude = extensions;
            else
                 myFilter.extExclude = extensions;
       }
       if(parameterCount[3]>1){
            System.out.println("Warning: Redundant min file size
declarations.   First one will be used.");
       }
       if(parameterCount[3]>0)
           for(i=0;i<h.length;i++){
               String curString2 = h[i];
               if(curString2.charAt(0)=='f'){
                   myFilter.minFileSize=Integer.decode(curString2.substring(2))
;
                   break;
               }
           }
       if(parameterCount[4]>1){
            System.out.println("Warning: Redundant max file size
declarations.   First one will be used.");
       }
       if(parameterCount[4]>0)
           for(i=0;i<h.length;i++){
               String curString2 = h[i];
               if(curString2.charAt(0)=='F'){
                   myFilter.maxFileSize=Integer.decode(curString2.substring(2))
;
                   break;
               }
           }
       if(myFilter.maxFileSize < myFilter.minFileSize){
           System.out.println("Max file size is set lower than min file
size!");

       }

 for(i=0;i<h.length;i++){
  String curString = h[i];
  if(curString.length()<3)
   continue;
  if(curString.charAt(0)=='p'&&curString.charAt(1)=='=')
  DoDir(FileList, curString.substring(2), myFilter);
 }
 System.out.println(FileList.size());  //display report of how many files
for debug purposes
}
}

Signature

ALEX TAYLOR

Roedy Green - 25 Feb 2006 09:40 GMT
On Fri, 24 Feb 2006 23:33:11 -0700, "Luc The Perverse"
<sll_noSpamlicious_z_XXX_m@cc.usu.edu> wrote, quoted or indirectly
quoted someone who said :

>The problem is, I have no way to prevent a user from adding a directory
>twice, or adding a directory inside of a directory.
you could look at the source code for the Replicator to see how I
handle that.  Basically you have a HashMap of filenames in compact
canonical form.
Signature

Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.



Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.