Implement Join in Java: looking for ideas

Thread view:

David Baer - 17 Dec 2003 10:20 GMT

Hi, for a proto type i'm implementing a join-operation (as in relational
databases) on potentialy huge tables in java.

if implemented naively, this takes at least size(Table1) * size(Table2).
my plan was to put the keys of the smaller table into a hashtable and than
iterate over the larger table.

is there any comment on this, or you have ideas for better data structures?

thank you!

david

Reply to this Message

Ruud de Koter - 18 Dec 2003 09:23 GMT

Hi David,

You wrote:

> Hi, for a proto type i'm implementing a join-operation (as in relational
> databases) on potentialy huge tables in java.
[quoted text clipped - 4 lines]
>
> is there any comment on this, or you have ideas for better data structures?

Given the fact that most RDBMS manufacturers have (tens of) years of experience
with joins, why would you want to implement that yourself? Or, more positively:
are you sure you have good reason to assume you can do the joining better than
the RDBMS will do it?

Regards,

Ruud.

> thank you!
>
> david

Signature

--------------------------------------------------------------------------------------
Ruud de Koter HP OpenView Software Business Unit
Senior Software Engineer IT Service Management Operation
Telephone: +31 (20) 514 15 89 Van Diemenstraat 200
Telefax : +31 (20) 514 15 90 PO Box 831
Telnet : 547 - 1589 1000 AV Amsterdam, the Netherlands
Email : ruud_dekoter@hp.com

internet: http://www.openview.hp.com/products/servicedesk
intranet: http://ovweb.bbn.hp.com/itservicemanager
--------------------------------------------------------------------------------------

Reply to this Message

David Baer - 21 Dec 2003 18:06 GMT

Hi Ruud,
The data is coming from different data sources (even allowing others types
than databases).

So I could put the data back into a database, just for the purpose of
joining it, where i could not benefit much from indices in the database.

I think it is better to do the joins myself, don't you think?

david

> Hi David,
>
[quoted text clipped - 22 lines]
>>
>> david

Reply to this Message

Ruud de Koter - 29 Dec 2003 09:54 GMT

Hi David,

You wrote:

> The data is coming from different data sources (even allowing others types
> than databases).
[quoted text clipped - 3 lines]
>
> I think it is better to do the joins myself, don't you think?

I would reason this is open for investigation. There might be several options:
- implement yourself,
- read in and join in the RDBMS,
- some RDBMSs have gateway products for a 'seamless' integration with other
products.

Depending on your exact requirements either option might offer best performance.
Once you have identified the likely candidates, experimentation will have to
tell you what 's the best approach.

Regards,

Ruud.

> david
>
[quoted text clipped - 24 lines]
> >>
> >> david

Signature

Reply to this Message

Implement Join in Java: looking for ideas

Free Magazines