It's a really thin line.
I've taken riffs or melodies from pieces, made them the bass part and changed the form and time signature, put a new melody and harmonies there and basically it's unrecognizable from what was originally there.
Similarly, I've had records where the writer of the piece explain that they took a line from a sax solo but then worked something totally different from it.
Then there's the kind of blatant theft like the Deep Purple classic "Child in time" which lifts wholesale the riff from It's a beautiful day's "Bombay calling". Now, even though Purple take their song in a different direction, the riff in both songs is central to both songs. If you were humming "Child in time" or "Bombay calling" to yourself, chances are you'd hum the riff. It's not a part either song can exist without.
There are probably thousands of examples like this. And therefore, to me, it depends. Legally I don't know, but realistically, much is dependent on how central to the song you're "borrowing"/'stealing' from the bit you're appropriating is. And how much does it 'make' the song you're writing.