I find that a hot signal from the instrument is usually a good start, so turn that guitar volume up!
Gain from a pedal can be good - if it's a good pedal. If it's a cheapo pedal it'll probably introduce more hiss than you might want. I guess it depends on what pedal(s) your using and what you want from them.
Gain on the amp can affect the sound in all sorts of ways and, as your discovering getting a good balance with levels across the board is crucial to a good sound.
I would say start without any pedals (or effects from the amp - yes, including reverb!), the amp EQ flat and the guitar on full volume.
Then slowly dial in the amp gain until it's just a bit too loud and then maybe take some of the guitar volume away. From this point try balancing the two things by making very small changes and noting the effect on tone until your happy.
I'm going to assume a very simple EQ for the guitar amp. If you have an amp like my old solid state one you will have a mid control in addition to treble and bass along with several others for hi-mid, low mid etc. Go through the steps below for bass and treble with all the amp controls in this case.
Then mess with the bass and treble controls on the amp - one at a time. Start by boosting the treble to full, check the sound and then and then return it to the center. Now cut all the bass, check the sound and then return that to the center. Note the difference in sound between boosting treble and cutting bass.
Now do the same with the bass control. Boost it to full, check the sound, return to the center. Now cut the treble etc.
You should now have a good idea of the differences in boosting and cutting bass and treble on your amp. Now experiment with both bass and treble on full and on minimum at the same time to get an idea of where the amp's mid range lies and how it's affected by these controls.
Now try and imagine the sound you want to achieve and try and visualise how you might set your amp and guitar controls (including your guitar's tone controls - which you should test in the same way as the amp controls).
Don't make any noise while your imagining the sound you want. Just think about the sounds you've heard while you've been going through the above process over a cuppa or something.
Now you've given your ears a rest come back to the set-up and dial in a tone. The difference this time is that your going to be dialing in a tone for *you* that you know your gear can do - rather than trying to dial in something without knowing really what your aiming for, or some famous guitarist's signature sound.
Good luck - and have fun with it
Although it helps to have nice gear, you can get a usable tone out of *almost* anything if you know its limitations first.
BTW - now that you have a good basic sound it's time to mess with your pedal(s) and amp's effects
