Zuska found an interesting article in the Chronicle of Higher Education discussing how scholars generate new ideas, and a nice conversation about it is going on at her place. I have some ideas of my own that have been expressed to some extent in the course of that conversation, but I thought I would collect them and organize them here.
The Chronicle article lists the following four key sources of inspiration for generating novel ideas for future research:
(1) Future research arises from current research.
(2) Future research can be autobiographical.
(3) Future research often arises from conversations.
(4) Future research can derive from what others want and might pay for.
These sources are certainly reasonable, but subject to a fatal flaw. They are each guaranteed to only lead to future research directions that are already visible from where one currently stands. Different approaches are required to generate truly novel--that is, unexpected and unpredictable--ideas.
A route to the truly novel that Comrade PhysioProf frequently relies upon in his laboratory is to invent new methodological of technological approaches. As part of the generation of a novel technique, one needs of course to validate the technique. And I have found that taking a new approach and applying it to an already answered question as an initial exercise in validation--you know what answer to expect so you know if the new approach is working--almost always leads to unexpected results and opens up large tracts of previously unimaginable fertile ground.
A cool historical example of this principle in practice is the first experiment to demonstrate that electrons have spin. The experimenters thought they were measuring the interaction of quantized angular momentum of an entire atom with an electromgnetic field--which is theoretically predicted to be either -1, 0, or +1--and thus strongly expected that the incoming beam of atoms would segregate into more than one beam after interaction with the field, with the predicted number of beams being three.
It turned out that the beam did segregate--thus partially supporting their expectation--but into only two. It turned out that they were measuring the interaction not of atomic angular momentum with the field, but rather electron spin. At this point in time, there was no theoretical prediction that electron spin even existed, so it would have been impossible to intentionally design an experiment to detect electron spin.
Regardless of the particular method used to generate new ideas, the key is that one needs to figure out ways of exposing oneself to the completely unexpected and unpredictable. Studying the literature to look for "unanswered questions" is guaranteed to only lead you down roads that are already visible. Genuinely new ideas are, by definition, not visible from where a field currently stands. Inventing novel tools, techniques, methods is one way of heading down roads whose existence one couldn't have predicted ahead of time.
There are, of course, others. Please share with us your own strategies for generating truly novel scientific ideas.