The iPod Touch doesn't have a "stop" button for music, which can be confusing when you are first getting used to the device. Instead, there's a pause button -- the parallel vertical lines -- that pauses the music and remembers where it stops when you begin playing again. Once you get used to it, you'll be using the iPod Touch controls like a pro. Once you've stopped the music, when you go to another app on your iPod Touch, the music stays off.
Tap the screen while the music is playing if the iPod Touch has gone to a black screen, or tap the "sleep" button on the top to wake up the screen.
Press the "pause" button, or double vertical lines, located under the image for the currently playing music to stop the music.
Press the home button, which is the circle at the bottom of the device, to return to the main screen and avoid accidentally turning the music on again.