Software engineering researchers and practitioners have traditionally had limited interaction, contributing to a fruitless technology cycle where researchers often work on irrelevant problems and practitioners often ignore research results. To address this issue, many conferences have long since introduced industrial tracks. Unfortunately, these conferences presented barriers to entry that discourage practitioner participation and thus many “industry” papers are written or presented entirely by academics! To address these issues the Journal of Software and Systems is introducing a new track called “Stories from the Front”. Alongside existing initiatives, this track is intended to contribute in shrinking the gap between research and practice.
In many computer science fields the impact of research is instantaneous; the latest Pixar movie dazzles audiences with recent advances in rendering hair and water. In software engineering the impact seems glacial; the most advanced development shops have adopted few research-inspired techniques. While it is difficult to pinpoint the genesis of the divide between software engineering research and practice, it undeniably exists. As a community we have been studying it for years, for instance in the Impact Project and have written passionate editorials for relevance, yet our best evangelization efforts have had limited effect.
Over the past years, one effort has shown great promise in bridging this gap:the industrial tracks at (otherwise academic) conferences. Such tracks are currently experiencing a renaissance, not just in broad conferences (e.g., ICSE, FSE) but also in field-specific conferences (e.g. ICSA, RE, ICSME). However, these tracks present a few key challenges for practitioners that limit their reach. First, they necessarily operate on a deadline system. Most practitioners cannot afford to neglect their main duties during the weeks leading up to a deadline. Second, they require management authorization and travel funding, including conference fees that may be hard to justify given the limited industrial content. Practitioner training budgets often cannot stretch to meet these costs. Finally, practitioners submitting to industrial tracks are often shocked by the inappropriate reviews they receive, over-emphasizing on novelty or rigor or comparison to related work and disregarding the value of realism.
Perhaps because of these challenges, conference-based industry tracks will likely always face a related-problem: papers that are not (co-)authored by practitioners. While academic authors can, in theory, conduct industrial research without an industrial co-author, more often this ‘paper smell’ is indicative of a repurposed research paper, a paper that does not reflect accurately and in-depth genuine industrial problems and experiences. Worse, it creates presentations, sometimes entire sessions, where both presenters and audience are academics, eliminating the desired practitioner and researcher dialog and the exposure of researchers to industrial realities.
Thus, to alleviate these shortcomings and increase the flow from practice to research, the Journal of Systems and Software is introducing its “Stories from the Front”. This track will (1) accept only experience reports and problem descriptions, (2) use specific review criteria for such submissions, (3) require at least one industrial author on every paper, and (4) remove practical barriers to entry for would-be industrial authors. We elaborate below on these characteristics.
First, “Stories from the Front” is exclusively focused on work that will increase knowledge transfer from industry to academia. It will accept (1) experience reports, showing what actually happens in practice (contrasted to theory), illustrating the challenges (and pain) that practitioners face, detailing how research results fair in realistic settings, and presenting lessons learned. It will also accept (2) problem descriptions with significant details on the context, underlying causes and explicit symptoms of the problem, the technical and organizational impact of the problem, as well as relevant research questions for researchers to investigate.
Second, submissions will be evaluated for their merit in reporting useful industrial experience; not in terms of academic novelty of research results. A set of review criteria will be used for that purpose by both reviewers and handling editors, e.g. does the manuscript provide enough context to understand the problem or case; is it generalizable enough to be useful beyond the original organization; does it lead to meaningful research questions? Consistently providing industry-appropriate reviews will differentiate this track from industrial conference tracks that are often chaired by academics.
Another unique feature of this track is that it will require at least one author from industry (co-authors from academia can be zero or more). This has the straight-forward benefit of eliminating academic-only papers, which would not address this track’s goals, while at the same time encouraging collaboration between practitioners and researchers. Additionally, in our experience with industrial tracks, papers with both industrial and academic authors were often the most valuable, as the industrial authors provided input on context and realism while the academic authors ensured a clearly presented paper.
Finally, this track will focus on eliminating smaller, practical issues that discourage industrial authors from submitting to conference industry tracks. JSS’s inherent features--lack of deadlines, lack of travel requirements, lack of required formatting on submissions--reduce the barriers to entry for busy practitioners. Yet the track will offer all of the normal perks and archival nature of a top journal. For those practitioners that do wish to present their work in person we will be working, in conjunction with relevant conference industry tracks, to secure journal-first, industry track presentations.
Recently, steady progress towards decreasing the gap between research and practice has been made. To continue this momentum we are opening up our “Stories from the Front” track to submissions immediately. We hope that this track can significantly increase the information flow from practice to research, grounding research to the realities of software engineering and pointing out new problems. To the practitioners who may submit and the researchers who may collaborate on such submissions, we are looking forward to your submissions and your feedback on this track!
David Shepherd leverages software engineering research to create useful additions to the IDE.