Once again... wow to the knowledge about random programming stuff.
Especially the bit about C programmers using long out of habit to
ensure that they get a 32 bit number on older machines. This little
tid bit of course dates back to the ACTUAL DEFINITON of what C is. C
defines the MINIMUM number of bits that you use for each datatype. int
has a definition of minimum 16 bits. LOL.
One more thing. Okay so you cant create arrays that are larger than
INT_MAX. Well then lets say I'm doing some sort of database work that
requires arrays larger than INT_MAX. What then???!
I suppose the only solution would be to wrap some class in something
that creates a new array whenever it goes past INT_MAX... and then wrap
the indexing function so that it always finds the right index or
something?
TrevorBoydSmith@gmail.com wrote On 08/31/06 17:40,:
> Once again... wow to the knowledge about random programming stuff.
> Especially the bit about C programmers using long out of habit to
> ensure that they get a 32 bit number on older machines. This little
> tid bit of course dates back to the ACTUAL DEFINITON of what C is. C
> defines the MINIMUM number of bits that you use for each datatype. int
> has a definition of minimum 16 bits. LOL.
Why laugh? 16-bit machines were common when C was new,
32-bit machines were mostly mainframes and expensive (one
prominent company wouldn't even sell their 32-bit systems;
you had to lease them and pay by the CPU-minute). A language
that insisted on 32-bit integers would not have survived;
people didn't want to be forced to do double-precision integer
arithmetic in software emulation when they didn't need it.
Here's a proposal: Get rid of long and int and short
and byte, and just use BigNum in place of all of them.
Perhaps that would silence your laughter, but I have to
believe it would provoke laughter in other quarters.
> One more thing. Okay so you cant create arrays that are larger than
> INT_MAX. Well then lets say I'm doing some sort of database work that
> requires arrays larger than INT_MAX. What then???!
Do you really need the entire contents of your database
to be in memory at one time, and in the form of an array?
Do you know how to find the average of a million numbers
without creating a million-place array to hold them all?
> I suppose the only solution would be to wrap some class in something
> that creates a new array whenever it goes past INT_MAX... and then wrap
> the indexing function so that it always finds the right index or
> something?
Um, er, I think you've just described the database
itself.
Trevor, I'm not trying to make fun of you, but I *am*
asking you to step back for a moment and ask yourself
whether you actually need an array of more than two billion
elements, or whether you're simply chafing at the imposition
of any limit, however great. Don't just say "databases can
be big," but sit down and think up an actual, plausible
situation where you need more than two billion "things"
*and* where dumping them all in a single flat array is a good
idea. I'm not saying such situations do not or cannot exist,
just that if they do they are rare, and corner cases don't
drive the design of any widely-applicable language.

Signature
Eric.Sosman@sun.com
Patricia Shanahan - 31 Aug 2006 23:46 GMT
...
> Trevor, I'm not trying to make fun of you, but I *am*
> asking you to step back for a moment and ask yourself
> whether you actually need an array of more than two billion
> elements, or whether you're simply chafing at the imposition
> of any limit, however great.
...
I seem to have heard that story a couple of times before.
I've also met people who needed to solve sets of tens of thousands of
linear equations in complex variables.
Patricia
Eric Sosman - 01 Sep 2006 17:41 GMT
Patricia Shanahan wrote On 08/31/06 18:46,:
> ...
>
[quoted text clipped - 10 lines]
> I've also met people who needed to solve sets of tens of thousands of
> linear equations in complex variables.
The array indices for "tens of thousands" will fit
quite comfortably in `int's. If you need a 50Kx50Kx50K
array you're going to have plenty of problems, but the
range of the array indices won't be among them.
From Trevor's posts I have the impression that he
actually doesn't have enormous arrays to deal with. He's
got some C code that he's converting to Java, and the C
occasionally (or maybe systematically) uses a `long' as
an array index. When he transliterates this to Java,
the compiler slaps his wrist.
The problem isn't that he needs big arrays, but that
he needs to do a more sensitive translation. Both Java
and C use the word `long' to denote a kind of integer,
but there the similarity ends: the word means different
things in the two languages, and copying the word blindly
from one language to the other changes the meaning. C
and Java both use `char', and it means different things.
C and Java both recognize `goto', and it means different
things. Transliteration is not translation, and cognates
are not synonyms.

Signature
Eric.Sosman@sun.com
Patricia Shanahan - 04 Sep 2006 21:57 GMT
> Patricia Shanahan wrote On 08/31/06 18:46,:
>> ...
[quoted text clipped - 15 lines]
> array you're going to have plenty of problems, but the
> range of the array indices won't be among them.
That depends in part on whether the array-of-array-of-array approach
works, or whether you want to control the memory layout a bit more by
using a flat array.
> From Trevor's posts I have the impression that he
> actually doesn't have enormous arrays to deal with. He's
[quoted text clipped - 13 lines]
> things. Transliteration is not translation, and cognates
> are not synonyms.
Indeed. I agree that when translating an ordinary C program to Java, I
would expect many uses of C "long" to translate to Java "int".
Patricia
Chris Uppal - 01 Sep 2006 08:45 GMT
> Here's a proposal: Get rid of long and int and short
> and byte, and just use BigNum in place of all of them.
> Perhaps that would silence your laughter, but I have to
> believe it would provoke laughter in other quarters.
Done right, that is the only way to fly...
(Needless to say, "right" does not include autoboxing.)
-- chris
TrevorBoydSmith@gmail.com - 01 Sep 2006 14:30 GMT
I think the matter that bothers me is having any limit at all. But to
be truthful you don't need to have 2 billion length arrays. Only a
fool would do something like that right now. (in the future ... i.e.
couple of years it might be another story.)
The largest continuous arrays are images. And even then those are only
in the millions. 2048x2048x3 for example. When you go to video
processing it is not continuous flat arrays but rather it is images
stored in an array. or a multidimensional array.
-----
So ya the integer indexing is an annoyance. but point taken.
Patricia Shanahan - 01 Sep 2006 15:14 GMT
> I think the matter that bothers me is having any limit at all. But to
> be truthful you don't need to have 2 billion length arrays. Only a
[quoted text clipped - 5 lines]
> processing it is not continuous flat arrays but rather it is images
> stored in an array. or a multidimensional array.
Do you really mean "The largest continuous arrays" or just "The largest
continuous array in home data processing"?
The largest continuous arrays I've met are far bigger than that, in
scientific and engineering programs, and had nothing at all to do with
images. People were solving systems of equations with more than two
billion coefficients back when they had to do it with out-of-core
solvers, shuffling blocks of data between memory and disk, because the
whole array did not fit in memory.
And my experience with large scale linear algebra was a couple of
employers ago, when I was working for FPS and then Cray Research, back
when Java was a type of coffee. Given the general trends, array sizes
have probably increased in the last ten years.
It bothers me to have a limit that prevents application of simple array
techniques to the largest arrays that fit in physical memory on a large
server. People whose arrays are bigger than that are stuck anyway.
Patricia
Chris Uppal - 01 Sep 2006 15:34 GMT
> It bothers me to have a limit that prevents application of simple array
> techniques to the largest arrays that fit in physical memory on a large
> server. People whose arrays are bigger than that are stuck anyway.
I think that the problem here is not the unavailability of contiguous data
beyond a certain size -- I'm pretty sure that if you are dealing with data of
that order that you would want to handle memory layout issues with far greater
finesse than just "it's all laid end-to-end". I think the real problem is the
lack of a decent notation which would allow us to create large array-like
structures (with whatever physical layout we needed) and treat them as
transparently and (nearly) as efficiently as we can "real" arrays.
I have nothing /against/ extending indexing to 64-bit indexing[*], you
understand, it's just that I think that would be solving only a relatively
small part of the real problem.
-- chris
[*] beyond purely practical issues like backward compatibility.
John W. Kennedy - 03 Sep 2006 20:52 GMT
> The largest continuous arrays I've met are far bigger than that, in
> scientific and engineering programs, and had nothing at all to do with
> images. People were solving systems of equations with more than two
> billion coefficients back when they had to do it with out-of-core
> solvers, shuffling blocks of data between memory and disk, because the
> whole array did not fit in memory.
Do you literally mean systems of >2Gi equations with >2Gi coefficients
apiece? >4Ei coefficients in all, plus a >2Gi RHS? Because that's what's
in play here.
(Not to mention that even a system with 64-bit pointers could not hold
such an array in memory.)

Signature
John W. Kennedy
"The blind rulers of Logres
Nourished the land on a fallacy of rational virtue."
-- Charles Williams. "Taliessin through Logres: Prelude"
Patricia Shanahan - 04 Sep 2006 21:58 GMT
>> The largest continuous arrays I've met are far bigger than that, in
>> scientific and engineering programs, and had nothing at all to do with
[quoted text clipped - 9 lines]
> (Not to mention that even a system with 64-bit pointers could not hold
> such an array in memory.)
No, I meant a total of over 2 billion coefficients in the system of
equations.
Patricia
> One more thing. Okay so you cant create arrays that are larger than
> INT_MAX. Well then lets say I'm doing some sort of database work that
> requires arrays larger than INT_MAX. What then???!
Then you loose...
;-)
BTW, I'm having difficulty imagining a reasonable application/library design
which would require /contiguous/ integer-indexed storage for more than 2**32
elements -- did you have anything specific in mind ?
> I suppose the only solution would be to wrap some class in something
> that creates a new array whenever it goes past INT_MAX... and then wrap
> the indexing function so that it always finds the right index or
> something?
Yes, you can create container objects (whether subclassing java.util.Collection
or not) which are indexed however you like -- by negative BigIntegers if that
makes you happy ;-)
Unfortunately you can't use array access syntax to read/write such containers
in Java, which is a definite nuisance -- making it more difficult both to read
and write the code.
-- chris