In the original one: he is killing all of the children, of people who burned him alive in his boiler room hideout; after he was let out of jail on a technicality. I don't remember if the other movies have a similar back story. As they go on, they get FAR more complex, and confusing, from what I remember (and I have seen all of them, except for the new remake; even though I really should, because I LOVE Jackie Earl Haley) one of them has something to do with his daughter (who is revealed to exsit in on of the later movies) and another has something to do with the only girl who survives from the first movie. :D

In the original Nightmare on Elm Street films (there are seven movies in the series, not including Freddy vs Jason because that's not party of the saga or even the first sequel because, well, that's not considered correct by hardcore fans anyway -- so really the true sequel starts at the third film) Freddy was a child murderer who was arrested but, after being acquitted on a technicality, the parents all went after Krueger and killed him in a fire. Years later, Freddy's spirit torments and kills the children of those adults who originally killed him in their dreams.

(they originally thought about having Freddy be a child molester in the original film, but that was discarded which probably was why they took that idea into the newer film, to add more ~darkness~ to the plot, I suppose)

There is a continuity with each film (not with the second film, but with the third one onwards) with characters. I think the only one that didn't have a recurring character was Freddy's Dead: The Final Nightmare. There is also Wes Craven's New Nightmare ( ... )