I haven't done it often - but it has happened.... where I start a song idea with a drum part, a groove. I love killer rhythm. So why not? It's unusual - but the rules are: there ain't no rules.
I do that a lot with SD. I will get the notes/chords (mainly chords, I'm a chords guy), rhythm, tap out the BPMs to the rhythm, then dumpster dive SD to find a beat/groove I can work with. Build everything off that. Sometimes I start out with the bass (but chords are still driving it) and work with SD.
Yeah I used to do it that way...exactly like you said...dumpster dive for midi/ beat/ groove that fits. But I found it way too frustrating. I did it A LOT like that. Many, many songs. But was rarely happy with the feel. Coz though you find something that matches...it rarely matches well enough. That's why I changed my approach to just finding a groove/ midi clip I like then writing the song/ tune FOR that.