Fortunately, while attending CSMR-WCRE, enjoying some great code search talks from both academia and industry, it hit me. Code search is extremely well-positioned to make a real impact in the next few years. Here's why:
Researchers are focusing on human aspects
Researchers have the infrastructure to make rapid progress
First, we have TraceLab, a framework for creating and carrying out search algorithm evaluations in a reproducible manner. I've seen TraceLab used, for instance, to show the effect of including and excluding commonly used components like splitters and stemmers into code search algorithms. Thus, for researchers who develop any improvement on top of existing approaches, evaluating their work against the state-of-the-art is as simple as implementing a TraceLab component.
Second, once researchers determine a new top algorithm in the lab using TraceLab, we also have a framework for evaluating that new algorithm in the field, Sando. Assuming that the algorithm can be plugged into Sando, we've recently shown that anonymous activity stream metrics correlate with user satisfaction, and thus by watching these metric we can determine whether the new algorithm improves performance in the field too.
I believe that this infrastructure will dramatically reduce the time-to-market for new ideas coming out of research. Each new idea, quickly tested in TraceLab, validated in the field by Sando, can impact users in a matter of months instead of years.