That is simply not true on either account, I don't need to "fix" the start
of anything. There is no memory overhead for each class. The "allocation"
overhead time is nothing more than a simple integer addition.

Yes.

Well, not seen in the example would be that the string is read from a file
into a local buffer which gets reused.

How is this possible?

char buffer[MAX_SIZE]

while(!feof(f))
{
  if(fgets(buffer, sizeof(buffer), f))
  {
      new(buffer) foo();
  }
}

Where is the allocation overhead that you are referring too?

Like I said, I was showing a technique that was simplified for example,
don't nit-pick the example because it is obviously much less sophisticated
then the actual application that uses the technique. In the real code, the
new operator is much more complex and reads the data itself.

Java would allocate a new string for each string. My code does not issue any
memory allocation for the string, it comes from a fixed length block and it
is combined with the object proper.

Try this:

allocate 20,000,000 objects with each object containing at least one string.
The memory footprint of the application has AT LEAST 160 megabytes of
overhead that can be virtually eliminated by pre-allocating 4 megabyte
blocks and sub-allocating out of that.

Java allocations take 10-20 machine cycles? Not on your life. In JIT
compiled code in which objects lose scope on exit of the function, yes,
that may be the overall time (it could even be much less), but objects that
live beyond a function scope have the overhead of a memory allocation.

Java:

class foo
{
       String m_test;

       foo(String value)
       {
               m_test = value;
       }
       void print()
       {
               System.out.println(m_test+"\n");
       }
};
public class test
{
       public static void main(java.lang.String[] args)
       {
               int i=0;
               try
               {
                       foo arr[] = new foo[2000000];
                       for(i=0; i < 2000000; i++)
                               arr[i] = new foo(Integer.toHexString(i));
               }
               catch(OutOfMemoryError e)
               {
                       System.out.println(e.getMessage());
                       System.out.println("Total object:" + i);
               }
       }
};

C++
#include <stdlib.h>
#include <unistd.h>
#include <assert.h>
#include <string.h>
#include <stdio.h>

class foo
{
       char *m_test;

       public:
       foo();
       void * operator new(size_t size, void *p, size_t cb);
};

#define BLKSIZE 1024*1024

unsigned char *block = NULL;
size_t blk_size=0;
size_t blk_offset=0;

foo::foo()
{
}
void *foo::operator new(size_t size, void *p, size_t cb)
{
       size_t totalcb = size+cb;

       if(!block || totalcb > (blk_size - blk_offset))
       {
               block = (unsigned char *) malloc(BLKSIZE);
               blk_size = BLKSIZE;
               blk_offset = 0;
       }
       assert(block);

       foo *fooT = (foo *) &block[blk_offset];
       blk_offset += size;
       fooT->m_test = (char *) &block[blk_offset];
       blk_offset += cb;
       memcpy(fooT->m_test, p, cb);
       return (void *) fooT;
}

int main()
{
       int i=0;
       foo ** ar = (foo **) malloc(sizeof(foo*) * 2000000);

       for(i=0; i < 2000000; i++)
       {
               char buffer[64];
               size_t cb = snprintf(buffer,sizeof(buffer), "%X", i)+1;
               ar[i] = new((void *)buffer,cb) foo();
       }
       printf("%d\n", i);
}

The results:
test@localhost:~/scat$ time java test
Java heap space
Total object:914828

real    0m23.519s
user    0m21.009s
sys     0m2.004s
test@localhost:~/scat$ time ./test
2000000

real    0m0.838s
user    0m0.724s
sys     0m0.064s