Chase. The basic is the words. Then you correlate the words into
clusters. Historically these are called Information Retrieval Systems
("IRS" oh my). The simplest idea is pages with words in common must be
like one another and like your topic. Imagine what would happen if you
had one list with all URLs. One list with all words. And one list
connecting the two. Then you could lookup all matching URL's for each
word. These lists might be large though! Then you could find the set
matching the search phrase by intersecting each set for each word.

Second thing to know is words have forms so maybe you'd work off of
all lower case and reduce all words to a base form. Well, what about
"Farenheit 451"? Do you also store numbers? What about "(...)"? Can
you also search on computerese? So, it starts to get complicated. One
idea is the "edit distance" or number of changes to get from the word
entered to a base word. That might tell if it might be the same word.
What about synonyms (I haven't seen a search engine do this). What
about bigrams and n-grams? That is, multi-word combinations. If one
types super computer then maybe any occurrences of "super computer"
should be matched higher than a page with just the word super or
computer.

OK, so a real search engine uses ranking and bases this on many
things. Things like how long the site has been up. How many other
sites link to them. How stuffed full of links their pages are. Maybe
if they buy ads from teh search engine? Nah, that wouldn't b
right. :-) Etc. Also, by clustering a person's past searches or areas
of interest then you can greatly increase your precision.

In 2001 I took an IR course and we studied MSN, Google, and Yahoo.
Everyone found Google to have about the same recall (document
universe) but superior precision (accuracy). Now if I'd had the common
sense to buy stock!!!!