Ok, so you 100% have to solve the issue of authentication for that. Who is allowed to upload audio files to your server?
And once you have a user authentication process in place, this wordpress plugin might work. You'd configure your site to allow your users to make audio posts, which would be...