 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
Java Forum / General / March 2008Java DB, is it for small databases only?
I'm looking into Java DB, and although Sun claims that it can be a full- scale DBMS I'm wondering what the views on this are? Can Java DB be used for large databases? Thanks Lionel. Lionel van den Berg, 03.03.2008 11:19: > I'm looking into Java DB, and although Sun claims that it can be a full- > scale DBMS I'm wondering what the views on this are? Can Java DB be used > for large databases? Define "large" > Lionel van den Berg, 03.03.2008 11:19: >> I'm looking into Java DB, and although Sun claims that it can be a >> full- scale DBMS I'm wondering what the views on this are? Can Java DB >> be used for large databases? > > Define "large" Sigh, I knew that was coming. I'm not sure, I'm not a database person, I do enough to get by. Let's say 10 tables with an average of 4 fields with each table having up to 10 million entries. Not sure if that constitutes large, I'm sure it's not to google. Lionel van den Berg, 03.03.2008 12:47: >>> I'm looking into Java DB, and although Sun claims that it can be a >>> full- scale DBMS I'm wondering what the views on this are? Can Java DB [quoted text clipped - 8 lines] > > Not sure if that constitutes large, I'm sure it's not to google. OK, I would think that qualifies as "large" (at least not as small :) ) I have no experience with Derby using such a setup. I would be a bit hesitating about using it for such a database. But I guess this can only be answered by setting up a testdatabase of that size. You might want to post that question to the Derby mailing list, support over there is quite good. Thomas > You might want to post that question to the Derby mailing list, support > over there is quite good. Thanks. Sort of trying to get the unbiased opinion, but it's still worth a try :). >> Lionel van den Berg, 03.03.2008 11:19: >>> I'm looking into Java DB, and although Sun claims that it can be a [quoted text clipped - 7 lines] > Let's say 10 tables with an average of 4 fields with each table having up > to 10 million entries. If it is integer fields then 10*4*10M = 400 MB which is a small database. Even though I have not tested Derby with a 400 MB database, then I am sure it can handle it. Arne >>> Lionel van den Berg, 03.03.2008 11:19: >>>> I'm looking into Java DB, and although Sun claims that it can be a [quoted text clipped - 13 lines] > Even though I have not tested Derby with a 400 MB database, then I > am sure it can handle it. Translation: MB = small GB = medium TB = large PB = arghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh Arne >> Even though I have not tested Derby with a 400 MB database, then I >> am sure it can handle it. [quoted text clipped - 5 lines] > TB = large > PB = arghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh You mean: PB = Google  SignatureDaniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/> Daniel Pitts schrieb: >>> Even though I have not tested Derby with a 400 MB database, then I >>> am sure it can handle it. [quoted text clipped - 6 lines] >> PB = arghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh > You mean: PB = Google nope not Really.. I googled a and found some blog entry from 2006 estimating Googles data in compressed form to 220TB. I doubt it grew to a PB since then. http://googlesystem.blogspot.com/2006/09/how-much-data-does-google-store.html > Daniel Pitts schrieb: >>>> Even though I have not tested Derby with a 400 MB database, then I [quoted text clipped - 13 lines] > > http://googlesystem.blogspot.com/2006/09/how-much-data-does-google-store.html Compressed form... SignatureDaniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/> >> Daniel Pitts schrieb: >>>>> Even though I have not tested Derby with a 400 MB database, then I [quoted text clipped - 15 lines] >> > Compressed form... I am a bit surprised that it is no more data. According to: http://research.google.com/archive/disk_failures.pdf Google has >100000 disks 80-400 GB. 100000*100 GB is 10 PB. Arne Arne Vajhøj wrote on 03.03.2008 22:46: >>> Lionel van den Berg, 03.03.2008 11:19: >>>> I'm looking into Java DB, and although Sun claims that it can be a [quoted text clipped - 10 lines] > If it is integer fields then 10*4*10M = 400 MB which is a small > database. The actual data size is not necessarily an indicator whether a database is "large". 10 Million rows (per table) could be a problem, if e.g. the access path to retrieve a subset is not well chosen by the engine and it has to scan all rows in order to retrieve e.g. 5 of them. If the number of updates and parallel reads is very high, but the design does not allow for concurrent updates and reads ("writers blocking readers") then the performance of the application will suffer as. Btw: writers blocking readers is a problem even for some of the "big" names Thomas > Arne Vajhøj wrote on 03.03.2008 22:46: >>>> Lionel van den Berg, 03.03.2008 11:19: [quoted text clipped - 16 lines] > access path to retrieve a subset is not well chosen by the engine and it > has to scan all rows in order to retrieve e.g. 5 of them. I would phrase it as: large/small depends on size, but whether a database is suited for a given problem may depend on many other factors than the size of the database. Arne Hi Lionel, Java DB (Apache Derby) can definitely handle such a dataset size. We have run some performance tests in multi-user mode with databases > 30+GB. You can also check about feedback on using large datasets at: http://www.ohloh.net/projects/3816 Derby is suitable for medium to high-level departmental type of databases. It was built originally (back at Cloudscape 10+ years ago) with no particular limitation in mind. The engine has always been multi-threaded, with a cost-based optimizer and a storage engine that can handle large datasets. Of course and as some people have pointed here, large datasets can mean different things to a lot of people. If you still need to, I would recommend you to post on the derby-user mailing list, which is very active by the way: http://db.apache.org/derby/derby_comm.html You can also browse the list via nabble at http://www.nabble.com/Derby-f356.html Regards, Francois Orsini > > Lionel van den Berg, 03.03.2008 11:19: > >> I'm looking into Java DB, and although Sun claims that it can be a [quoted text clipped - 10 lines] > > Not sure if that constitutes large, I'm sure it's not to google. >I'm looking into Java DB, and although Sun claims that it can be a full- >scale DBMS I'm wondering what the views on this are? Can Java DB be used >for large databases? Java DB (aka Derby, aka Cloudscape) is for small databases. MySQL which Sun now owns is for intermediate sized ones. For alternatives running the gamut, see http://mindprod.com/jgloss/sqlvendors.html -- Roedy Green Canadian Mind Products The Java Glossary http://mindprod.com >> I'm looking into Java DB, and although Sun claims that it can be a full- >> scale DBMS I'm wondering what the views on this are? Can Java DB be used >> for large databases? > > Java DB (aka Derby, aka Cloudscape) is for small databases. MySQL > which Sun now owns is for intermediate sized ones. Los Alamos apparently runs an intermediate sized database: http://www.mysql.com/why-mysql/case-studies/mysql-losalamos-casestudy.pdf Arne >Los Alamos apparently runs an intermediate sized database: > http://www.mysql.com/why-mysql/case-studies/mysql-losalamos-casestudy.pdf Even some Mickey Mouse databases can manage terabyte plus datasets. The catch is what is your transaction volume, and how much data do you comb out at a pop. The good news is RAM is getting cheaper and disks faster. This means databases get more capable by the day even if the software does not change. Exxon sometimes buys enough RAM to float the entire database into cache to get the performance they want. -- Roedy Green Canadian Mind Products The Java Glossary http://mindprod.com > I'm looking into Java DB, and although Sun claims that it can be a full- > scale DBMS I'm wondering what the views on this are? Can Java DB be used > for large databases? It's easier to say in one post, thanks to all the responses, I have enough food for thought :). Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.