I am facing an inconvenience which I believe should have been faced
before by other Java developers but I am finding it difficult to
articulate it in keywords so that google will give me the right
answers..so here goes

1. I am using eclipse ide with mulitple java projects, each one sourced
from a CVS repository on an external server in the local LAN.
2. Almost all of these projects basically handle big data sets (read
100mbs - 500mbs of xml and text files) which is basically data crawled
from the web, act and transform it in some way and then pass it along
for other projects to act on it. Some of hte data is in single big
files and some of it is in 100's of small files inside a single
directory.

Basically what I am looking for is a better way to handle this data.
Currently if I put the data in CVS then it is not that efficient , plus
there needs to be some central lookup for all the data.I guess this is
partly a java design question and partly ignorance on my part to use
the right tools to do this job.

Thanks for any help.