Most of us write because we love the act of putting characters through hell. But let's face the fact that we also want to publish what we write.
Keeping careful track of submissions is crucial. At the start of your fledgling writing career, you might want to believe that you only have a story or two and naturally you can keep track of who has rejected your precious new babies. Trust me, five years from now, when you have many stories in circulation and some of them have been rejected by over a dozen markets you need to remember which stories have been where.
I use a spreadsheet for this task.
The first section of my spreadsheet lists each short story, in the order they were written (to give me a sense of history) but you could go with alphabetical by title if that works for you. After the title, I show the number of words for the story (important when you want to send one out in a hurry and you're checking guidelines to determine what fits where.) Then I have a calculation that determines how many acceptances (#Y), rejections (#N), and submissions (#S) have occurred for each story.
To the right, I list each submission, the date the story was submitted and the expected date of reply. Once the reply comes in, I update that space with a yes or a no. So an entry would look something like:
Title       #Y #N #S     Words    Sub 1                     Response       Sub 2                     Response
Slimed    0   1     2      3500     Asimov's Dec 3/11   No Jan 18/12   IGMS Jan 19/12     Due Feb 28/12
Most markets do NOT accept re-submissions of a story (even if you've made some improvements,) so it's best to keep track of where a story has been so you don't accidently send it twice.
When a story is accepted, I highlight the cell in a bright color so I can easily see which stories have sold. When a story is currently not submitted anywhere, I change the text for the title to bright red so at a glance I can see which stories need to go back into circulation ASAP.
In another part of the spreadsheet, I list all of the markets and write the title of the currently submitted story beside the market. That way, I don't accidently send two stories to the same place at the same time. (Many markets don't accept multiple submissions.)
I believe there are several "submission trackers" available online, if you don't want to create your own. The most important concept is to use one that works for you.
I must admit, there's a certain "bragging right" in knowing how many rejections you've accumulated over the years. As of this morning, I'm at 284 rejections out of 302 submissions for my short fiction.
And yes, I have a different spreadsheet for tracking novels, mostly to keep a record of agent submissions as well as publisher submissions for each novel.
If you're still deciding whether or not to bother with a submission tracker, keep in mind that setting up a spreadsheet is the perfect procrastination on a day when you should be writing but you can't quite face the blank page.
But eventually, writer, you must face that blank page!