In the days before click tracks, tempos varied because musicians are humans, not crystal-controlled clocks. However, these changes were far from random. While researching an article for Sweetwater's inSync web publication, I analyzed the tempo changes for several hits from the past that didn't use a click track and noticed a common element of most songs: the tempo would accelerate up to a crucial point in the song, then decelerate during a verse or chorus. This type of change was repeated so often, in so many songs I analyzed, that it seems to be an important musical element that's almost inherent in music played without a click track. It makes sense this would add an emotional component that could not be obtained with a constant tempo.
As one example, here's what the tempo looks like for the Beatles "Love Me Do." Their tempo variations are quite premeditated.
Note the dramatic pause at "so please, love me do" around measure 16 and again at 49, and the natural increase in tempo when it went into the "Love, love me do" verse. They also sped up a bit over the course of the track, which happens a lot in songs recorded without a click track.
If you start a song with MIDI tracks, it's easy to experiment with tempo variations because the sound of the instruments won't change. Once you've nailed a good feel for the tempo, then you can start adding audio tracks that follow the tempo changes.