 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
Java Forum / General / April 2007any datatype which can handle GB's of Data.
Hi All, Is there any variable in java like Stringbuffer which can handle GB's of char data. As far as i know every variable has a limit is there any variable which can handle unlimited data. it sounds little bit stupid but which data type or design pattern can solve this problem. Thanks Taurn Garg > Hi All, > [quoted text clipped - 3 lines] > As far as i know every variable has a limit is there any variable > which can handle unlimited data. No *computer* can handle unlimitted data, so how could any type in Java do so? In general, you are limitted in any programming language by how much memory (virtual or otherwise) you have. Sometimes there are other limits, but most people don't have enough memory to hit those other limits. Its generally better to accept this, and use algorithms that support some sort of streaming or subset processing. > it sounds little bit stupid but which data type or design pattern can > solve this problem. Its not a data type or design pattern. In this case, its an algorithm or data structure. Hope this helps, Daniel. >> Hi All, >> [quoted text clipped - 3 lines] > No *computer* can handle unlimitted data, so how could any type in > Java do so? Some objects such as java.net.Socket or java.io.OutputStream can handle unlimited data... But that's probably not what the OP meant > Is there any variable in java like Stringbuffer which can handle GB's > of char data. In memory? On a 32 bit Windows system a JRE can use approx. 1.5GB heap. > As far as i know every variable has a limit is there any variable > which can handle unlimited data. > > it sounds little bit stupid but which data type or design pattern can > solve this problem. The question is: what do you want to do with that data? Chances are that you can only deal with this data on disk or in a database. robert >Is there any variable in java like Stringbuffer which can handle GB's >of char data. StringBuffer is a class, not a variable. I don't mean this just as a nitpick; the way you describe a problem has a very large affect on the types of answers you'll get. If you're imprecise in describing what you're doing, and unclear in your description of the Java language, you'll get general answers that assume you're a fairly inexperienced developer. Which may be what you're looking for :) >As far as i know every variable has a limit is there any variable >which can handle unlimited data. Every computer has a limit. AFAIK, Java's limits for this kind of class are based on available memory and the fact that arrays must be indexable by a 32-bit signed integer. I'll be surprised if someone says there's no JVM on any hardware that can handle 1.8 billion characters in a StringBuilder. But then you get into WHY you want to do such a thing. >it sounds little bit stupid but which data type or design pattern can >solve this problem. Generally, look for ways to avoid having it all in memory at the same time. If possible, a streaming approach is good: read from a file/socket a little at a time, process it, and write to a file/socket. Alternately, java.io.RandomAccessFile lets you seek() to the section you want to work on. In both cases, you'll need to be careful about the distinction between bytes and characters. -- Mark Rafn dagon@dagon.net <http://www.dagon.net/> > Hi All, > [quoted text clipped - 9 lines] > Thanks > Taurn Garg I can create such a variable for you - for a slight fee! :-) Seriously, you can use a File (or even memory mapped file) and just put a delegate of some sort. E.g. public class BigBuf { // herein are the public external methods that look just like a StringBuffer or what have you // herein is the underlieing implementation that actually write/reads to/from a file. } thanks for reply, What i really need to do is that that i have to take the data from database and then send it to one JSP page where javascript will handle that data to show to the user. the reading from database and sending to jsp or gui is been done ny serlvet. so what i am doing is like i am reading the data from database one by one record and then creating a table and puting that in the StringBuffer. but as data can be of any size. String buffer will not be able to handle that. so in this case what should i do. thanks tarun > thanks for reply, > [quoted text clipped - 8 lines] > StringBuffer. but as data can be of any size. String buffer will not > be able to handle that. so in this case what should i do. Don't. Separate String (or StringBuilder or other type such as java.sql.Date) for each column in each row (not record). Don't show all gazillion rows (not records) at once - use some paging. The datatype to hold everything at once is the "RDBMS" type. Everything else should only hold a view into that. Certainly even if your server could suck the whole data store into memory at once, there's no way someone's browser can.  SignatureLew > thanks for reply, > [quoted text clipped - 11 lines] > thanks > tarun First, you should use StringBuilder over StringBuffer in most cases. Second, What makes you think StringBuilder can't handle it? It can handle a surprising amount of data. You don't want to load all of your database at once though. Try displaying a row at a time. > Hi All, > [quoted text clipped - 9 lines] > Thanks > Taurn Garg Research the java.nio.ByteBuffer* classes such - specifically the MemoryMapped (google is your friend). Basically disk-backed buffers that you can perform many convenient slicing on which is more efficient than sequentially navigating a file. If you've got the RAM, then the OS should properly cache the contents of the file. Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.