I know what I like and I like what I know

[deflatermouse]

I actually wrote this about 2 years ago and, for some reason, never posted it but I was having a conversation about it recently and figured I might as well publish. And be damned.

Searching isn't a one step malarkey. Ignoring tokenising and stemming, thesauruses and normalisation you have to select a working set of documents and then rank them.

How

Comments

[deflatermouse]

Tokenise using the same thing that your search engine is then either have each token have its own id, or hash the token into a numerical id. There's really nothing to stop you using the tokens as the input though.