A script writer named Gerry Davis was the guy who (so to speak) came up with the idea of regenerations. Here's a toast to you, Gerry wherever you are, without the idea of regeneration Dr Who might have been out of business in 1966. If a TV series loses a secondary actor/actress, perhaps because they found a better opportunity elsewhere, or they're tired of their character, or maybe the writers/directors simply decide that they're wrong for the part, or someone else is righter, it's easy to just "write them out" of future episodes. Several female Dr Who companions have been written out by having them fall in love with men from other times and stay with them when the TARDIS leaves, but the Doctor is such a central part of the series that "writing him out" simply isn't an option.
Toward the end of 1966 William Hartnell was in increasingly ill health, and was unhappy with the current crop of directors/producers, etc of Dr Who. He decided it was time to go, leaving the show in a big pile of you-know-what, but Gerry and regeneration came to the rescue. Patrick Troughton, a popular character actor, was chosen as the second Doctor and a basic regeneration scene written for the 4th episode of "The Tenth Planet." No fireworks and laser beams like some of the modern regenerations, a shot of William Hartnell's face fades out to white and then fades back in to Patrick Troughton's face. Voila, a new doctor.
The first regeneration was different than subsequent ones. The first big difference is that Dr Who didn't want to be regenerated! He had grown weaker throughout the episodes in "The Tenth Planet" and finally collapsed on the floor of the TARDIS and began to regenerate. "No, no, you can't do this to me," the Doctor shouted, but who or what was he shouting to, and why didn't he want to be regenerated? The only other Time Lord that refused regeneration was The Master at the end of "Last of the Time Lords" but he supposedly wanted to die as opposed to the first doctor who wanted to go on living as himself.
The process is unknown as well. Is it an entirely biological ability of the Time Lords, or does it have something to do with the TARDIS? Most of the regenerations have been inside the TARDIS. Tom Baker to Peter Davidson wasn't, but there were some strange, possibly Time Lord instigated, sock puppets following Tom around for the last episode that helped with the regen. The really non-TARDIS regeneration was Sylvester McCoy to Paul McGan which occurred in a London hospital, but who knows how long an arm the TARDIS has.