The contrast between the blues-style lyrical content ("my woman don't treat me right, but I just can't leave her") and the Sonny Bono sweetness-and-light ("who needs food or money when we have each other?") is, to me, what makes the joke at the end. It ain't a particularly "funny" joke, but I think it's a twist that fits in with the lyrical contrast/irony that is present throughout the song, with its "you treat me so bad, but why don't I leave? You got me." The whole song is basically filled with contrasting twists of meaning and plays on words ("I'd enjoy a long vacation --- in hell!" "You got me tending to your comes -- and your gos"), so to me the switch to the "sweet" kind of Sonny & Cher waltz-tempo music, contrasting with the already-established bluesy beat and lyrical harshness of "you got me" as a double-meaning ("I don't know why I stay" and "you have this hold on me") seems natural. In fact, when I wrote the song, the first time I played it, I fell into that ending without really thinking about it, just as a switch on "I got you" to "you got me".
And THAT is a whole boatload of analysis about the silly song that I hadn't really thought out before, and no doubt WAY more than you were interested in reading. But thanks for making me consider my songwriting choices and think things through on a deeper level than just my gut feeling. Having thought it through, I still think the ending works. I don't mean for that windy paragraph above to imply that the song is "deep" in any way; it isn't. It's just a collection of blues cliches, wordplay and quips, meant to amuse an audience in taverns. (Incidentally, my wife was not amused when I wrote it and played it for her for the first time. She was afraid people would think it was actually about her. I had to assure her that people would understand that it was a product of imagination, not real life.)
Also, thanks for the compliment about the vocal delivery. I ain't much of a singer, but I try to use my voice in a way that works for the material. Your comment made my day.