Grabbed verbatim from Neil Gaiman‘s blog:
I’m not sure. There’s definitely something that tells you whether an idea has legs or not — but for me, it’s still, often as not, trial and error. My two favourite stories of the last few years, “A Study In Emerald” and “How to Talk to Girls at Parties” both started as conceits that didn’t work, that I put aside, ready to abandon, and then thought, days or weeks later “Hang on, what if I…?” and they both wrote themselves extremely fast after that. Both of them would have been abandoned though if they hadn’t had good, insistent editors (Michael Reaves and Jonathan Strahan respectively) and the need to get a story out of unpromising fragments.
It’s not a science. It’s an art and a sometimes it’s a craft. The most important thing (and I know I say this a lot but it’s true, or at least it’s true for me) is finishing things, because that’s when you find out if they worked or not. The rest of the time it’s just hoping. And if you stop writing when a promising beginning runs out of steam, maybe you need something more in the planning stages. Or maybe you just need to soldier madly onward and see what Chance and Necessity (the mother, it must be remembered, of invention) provide.
Makes me feel like I’m on the right track.