David C. Shepherd
  • Blog
  • Colleagues
  • Contact
  • About

Transferring Tech

Stories from the Front

5/29/2018

 
Picture
Bridging the gap between research and practice. Photo Credit: Jon Matthias on Flickr
Software engineering researchers and practitioners have traditionally had limited interaction, contributing to a fruitless technology cycle where researchers often work on irrelevant problems and practitioners often ignore research results. To address this issue, many conferences have long since introduced industrial tracks. Unfortunately, these conferences presented barriers to entry that discourage practitioner participation and thus many “industry” papers are written or presented entirely by academics! To address these issues the Journal of Software and Systems is introducing a new track called “Stories from the Front”. Alongside existing initiatives, this track is intended to contribute in shrinking the gap between research and practice.

Details

In many computer science fields the impact of research is instantaneous; the latest Pixar movie dazzles audiences with recent advances in rendering hair and water. In software engineering the impact seems glacial; the most advanced development shops have adopted few research-inspired techniques. While it is difficult to pinpoint the genesis of the divide between software engineering research and practice, it undeniably exists. As a community we have been studying it for years, for instance in the Impact Project and have written passionate editorials for relevance, yet our best evangelization efforts have had limited effect.  

Over the past years, one effort has shown great promise in bridging this gap:the industrial tracks at (otherwise academic) conferences. Such tracks are currently experiencing a renaissance, not just in broad conferences (e.g., ICSE, FSE) but also in field-specific conferences (e.g. ICSA, RE, ICSME). However, these tracks present a few key challenges for practitioners that limit their reach. First, they necessarily operate on a deadline system. Most practitioners cannot afford to neglect their main duties during the weeks leading up to a deadline. Second, they require management authorization and travel funding, including conference fees that may be hard to justify given the limited industrial content. Practitioner training budgets often cannot stretch to meet these costs. Finally, practitioners submitting to industrial tracks are often shocked by the inappropriate reviews they receive, over-emphasizing  on novelty or rigor or comparison to related work and disregarding the value of realism.

Perhaps because of these challenges, conference-based industry tracks will likely always face a related-problem: papers that are not (co-)authored by practitioners. While academic authors can, in theory, conduct industrial research without an industrial co-author, more often this ‘paper smell’ is indicative of a repurposed research paper, a paper that does not reflect accurately and in-depth genuine industrial problems and experiences. Worse, it creates presentations, sometimes entire sessions, where both presenters and audience are academics, eliminating the desired practitioner and researcher dialog and the exposure of researchers to industrial realities.

Thus, to alleviate these shortcomings and increase the flow from practice to research,  the Journal of Systems and Software is introducing its “Stories from the Front”. This track will (1) accept only experience reports and problem descriptions,  (2) use specific review criteria for such submissions, (3) require at least one industrial author on every paper, and (4) remove practical barriers to entry for would-be industrial authors. We elaborate below on these characteristics.

First, “Stories from the Front” is exclusively focused on work that will increase knowledge transfer from industry to academia. It will accept (1) experience reports, showing what actually happens in practice (contrasted to theory), illustrating the challenges (and pain) that practitioners face, detailing how research results fair in realistic settings, and presenting lessons learned. It will also accept (2) problem descriptions with significant details on the context, underlying causes and explicit symptoms of the problem, the technical and organizational impact of the problem, as well as relevant research questions for researchers to investigate.

Second, submissions will be evaluated for their merit in reporting useful industrial experience; not in terms of academic novelty of research results. A set of review criteria will be used for that purpose by both reviewers and handling editors, e.g. does the manuscript provide enough context to understand the problem or case; is it generalizable enough to be useful beyond the original organization; does it lead to meaningful research questions? Consistently providing industry-appropriate reviews will differentiate this track from industrial conference tracks that are often chaired by academics.

Another unique feature of this track is that it will require at least one author from industry (co-authors from academia can be zero or more). This has the straight-forward benefit of eliminating academic-only papers, which would not address this track’s goals, while at the same time encouraging collaboration between practitioners and researchers. Additionally, in our experience with industrial tracks, papers with both industrial and academic authors were often the most valuable, as the industrial authors provided input on context and realism while the academic authors ensured a clearly presented paper.

Finally, this track will focus on eliminating smaller, practical issues that discourage industrial authors from submitting to conference industry tracks. JSS’s inherent features--lack of deadlines, lack of travel requirements, lack of required formatting on submissions--reduce the barriers to entry for busy practitioners. Yet the track will offer all of the normal perks and archival nature of a top journal. For those practitioners that do wish to present their work in person we will be working, in conjunction with relevant conference industry tracks, to secure journal-first, industry track presentations.

Conclusion

Recently, steady progress towards decreasing the gap between research and practice has been made. To continue this momentum we are opening up our “Stories from the Front” track to submissions immediately. We hope that this track can significantly increase the information flow from practice to research, grounding research to the realities of software engineering and pointing out new problems. To the practitioners who may submit and the researchers who may collaborate on such submissions, we are looking forward to  your submissions and your feedback on this track!
Submit: http://journals.elsevier.com/journal-of-systems-and-software/

You Are What You Eat. Why You Should Choose the SEIP Track at ICSE '17

5/9/2017

 
​“I cannot remember the books I've read any more than the meals I have eaten; even so, they have made me.” 
― Ralph Waldo Emerson
​While you might not think it makes a big difference which talks you attend at ICSE 2017, I can assure you it does. How do I know? From experience, let me explain...

To obtain my PhD I moved directly from undergraduate studies, obtaining a bachelor's, to early graduate studies, obtaining a master's, and finally doctoral studies, obtaining a PhD. It is safe to say that my intellectual diet was primarily academic. However, shortly after completing my PhD I spent three years in a software tools startup. In this setting my diet changed significantly, and after three years of intake from some of the brightest developers I've ever known, I changed.

How do I know I changed? When I returned to research in 2011, accepting a job at ABB Corporate Research, I began performing a literature search for my first project. What I read horrified me. Why were researchers studying toy projects? Why weren't they collaborating with industry? Why were they studying such esoteric topics? Why didn't I notice this when I was a graduate student? 

By spending years working with industrial-sized projects--where scalability matters--and by ranking sprint items according to impact, I had shed the academic values of "every idea is worth exploring" and "scalability is an implementation problem". While I have since mellowed, finding a happy medium somewhere between the academic and the industrial outlook, I would argue that most researchers in our field fail to take into account the industrial point-of-view and thus are at high risk of producing low-impact research.

At this point of the post, academic readers may have lost hope. Perhaps you don't have a sabbatical coming up soon, where you could get industrial experience, and consulting on the side is infeasible time-wise. Fortunately, there is a much lower impact way to glean industrial experience. Attend the Software Engineering in Practice (SEIP) track at ICSE. Every talk. Because... let's face it, if your research is in fault localization you're going to read every sentence of the latest paper on "Spectrum Search-Based Deep Learning Analytics Fault Localization" at home. But on your home campus you're unlikely to run across a group of practitioners sharing their experiences and challenges... and that's what you really need to make your research great.
Picture
ICSME '15's Industry Track Attendence, Credit: @gbavota

Industrial Density

Assuming you're convinced that it's important to hear a practitioner's point-of-view, there's one more point to discuss: which track offers the best opportunity. First, let's examine each track's calls for submissions. The research track calls for ".. the most recent and significant innovations in the field." The SEIP track calls for papers on similar topics, yet what sets it apart "..is that it values impact and realism over novelty." Judging by the calls alone SEIP seems more likely to include practitioners' insights.

Yet, because software engineering is a (relatively) applied field, the calls for submissions do not tell the complete story. In software engineering reseach we are fortunate to have many authors from non-academic institutions. In the research track we have contributions from 27 different companies or institutes (shown below, right). To create this list I analyzed the proceedings, adding an entry in the list for each company, for each paper. Thus, the list contains multiple entries for some companies (e.g., Microsoft, IBM, etc.), as they contributed to multiple papers. I created the same list for the SEIP track, and in this track we have contributions from 25 different companies or institutes. Thus, in terms of raw numbers, the research track has more contributions from industry!

Raw numbers are misleading in this case. The research track contains 96 papers (68 conference and 28 J1C2 papers), giving it an "Industrial Density" of 0.28. The SEIP track contains 31 papers, giving it an "Industrial Density" of 0.81, nearly three times as high as the research track. As expected, attending the SEIP track is an easy way to optimize your chance of learning from practitioners. 
Non-academics, SEIP Track

​Orion Health
Ericsson
Viktoria Swedish ICT
Microsoft
Deloitte Consulting
Ericsson
Accenture
TopCoder
Google
Microsoft
Ericsson
Magnet.me
IBM
Cisco
Brazilian National Development Bank
Fraunhofer
Google
Blackberry
Tencent
Westerdals ACT
SEB Life & Pension Holding
Riga Branch
Microsoft
BMW
Cisco
Non-academics, Research Track 

​Fraunhofer IEM
Zikko
SICS Swedish ICT
Pivotal Software
Microsoft
Siemens
IBM
Blackberry
Samsung
Fraunhofer SIT
Kakao Corporation
Fireeye
Inc.
Microsoft
Microsoft
SAP
ABB
CWI
ISTI-CNR
SICS Swedish ICT
Simula
Microsoft
IBM
Microsoft
Microsoft
Microsoft
Microsoft
Total: 25
Total papers in track: 31
Total: 27
Total papers in track: 96 (68 + 28 J1C2)

Industrial Density: 0.81

Industrial Density: 0.28

Picture
Credit: jonathan_hamner @flickr

Eat Well at ICSE '17: 

ICSE '17 is upon us. I hope you enjoy it. I hope you get a chance to eat meat, tango, and re-connect with friends and colleagues. During this week I encourage you to make a decision. Attend every talk in the SEIP track; it will change you. If enough of you make this decision it will change the field. 
SEIP program: http://icse2017.gatech.edu/?q=seip-track

Fostering Flow at Work

5/3/2017

 
Research by: Manuela Züger, Christopher Corley, André Meyer, Boyang Li, Thomas Fritz, David Shepherd, Vinay Augustine, Patrick Francis, Nicholas Kraft, Will Snipes
Picture
Picture
Modern offices are dominated by creative workers. Programmers, designers, analysts, and engineers. These creative workers need long blocks of time to tackle the cognitively demanding activities that their jobs require. Unfortunately, they are constantly interrupted via chat messages, drop-in visitors and notifications, causing stress and sapping their productivity. According to UC Irvine professor Gloria Mark each of these interruptions require an average of 23 minutes and 15 seconds of recovery time.
To eliminate these distractions, we introduce FlowLight. FlowLight is an application that runs in the background on a user’s computer, automatically detecting when they are most focused and then changing their status accordingly. While the user is most active, FlowLight discourages remote interruptions by dynamically updating the user’s digital availability as well as discouraging local interruptions by changing the color status of the USB powered FlowLight Bulb
​To evaluate the FlowLight's effectiveness we partnered with the University of Zurich to perform an 18 month field study with over 400 participants from 12 different countries. This study showed that the FlowLight eliminated 46% of interruptions, that 85% of participants continued using it after the two month trial, and that users appreciated its impact, claiming that "[the FlowLight caused] less interruptions both in person and via Skype. [resulting] in more focus and ability to finish work.

​​“It brings more awareness to what people are doing. Sometimes people take it for granted that people are always interruptible. But there is actually a cost or a penalty when you interrupt someone. So, I think just the concept is good because it reminds people that there is sometimes a good time and a bad time to interrupt people. So, I think just from an awareness campaign, it’s valuable as well.” -Interview #20
Picture
Interruptions per day before and after FlowLight deployment
Picture
Partial results from a survey deployed after five weeks of FlowLight usage

Reducing interruptions with the FlowLight is just the first step in our work to redefine the modern workplace. Our ultimate mission is this: to foster flow at work.

This work has been featured in New Scientist and Le Matin.
It will be presented on Monday, May 8th, 2017 at 11:30 at CHI'17.
​
​For questions, please contact David Shepherd at david.shepherd@us.abb.com or Thomas Fritz at fritz@cs.ubc.ca.
CHI17 Reducing Interruptions at Work--A Large-Scale Field Study of FlowLight.pdf
File Size: 9903 kb
File Type: pdf
Download File

How to Review an Industry Track Paper

11/1/2016

 
Recently there's been an increasing amount of excitement around industrial tracks. They are becoming well attended, competitive, and discussion-worthy. Alas, this recent enthusiasm could be quickly extinguished if we, the program committees of these industrial tracks, provide inappropriate or low quality reviews. Imagine the frustration of a practitioner if their real world case study gets rejected while an experiment with ten freshman undergraduate students is accepted. To avoid this, we need to provide clear, distinct guidelines for evaluating industry track papers.

Publish at an industry track: SANER Industrial Track, ICPC Industry Track, FSE Industry Track

Values

An industry track focuses on the same topics and expects the same rigor as the corresponding research track, yet papers featured in the industry track are distinct. What sets the industry track apart is that it values impact and realism over novelty. How should this affect paper reviews? Industry track reviews should be the same in terms of quality, length, objectiveness, etc. *except* that they should emphasize industrial values instead of academic values. For instance, the ICSME Research track in 2015 (credit: Robillard & Krinke) used the following list of criteria to rate their papers (note: ordering is my interpretation of importance):
  1. soundness 
  2. originality 
  3. importance
  4. presentation
  5. related work
In contrast, an industrial track should use the following list of criteria:
  1. soundness
  2. potential impact
  3. real world focus
  4. presentation
  5. related work
Of this list, the potential impact and real world focus differ from items on the research track list, and a few words about related work must be said.

Potential impact refers to the potential impact of the findings, tool, or technique on industry. The potential for impact manifests itself in two ways.  First, it can be measured by the magnitude of the result. An evaluation of different regression test selection techniques that shows that on one industrial system, one technique is 5% more effective than another would garner a low score in terms of potential impact. That same study showing that one technique is 50% more effective across several industrial systems would rate higher. Second, it can be measured by the applicability of the technique; an improved solution for the overly specific task of model-based, aspect-oriented, real-time fuzz testing is less widely applicable than an improved solution for the more general task of unit testing.

Real world focus differs depending on the type or section of the paper. For a tool-focused paper it refers to the maturity of the tool (an open source tool with 100s of users is better than a closed source tool with 100s of users is better than an open source tool with a few users..). For an evaluation section, it refers to the evaluation environment (an industrial environment with real developers working on their own code is better than industrial developers working on toy tasks is better than students..). For a case study it refers to the type of industrial environment (a large software company is better than a small startup is better than an undergraduate class) or the type of software systems studied (both proprietary and open source systems is better than just open source systems is better than student/research projects..).
​
Related work is extremely important in research papers because it is used to establish novelty. In industry track papers novelty is much less important, and thus related work serves a diminished role. In industry track papers related work should give context to the contribution, cite original sources of ideas, and cite original studies. Yet reviewers should be lenient with authors who miss related work, as practitioners may not always keep up with late-breaking results. Reviewers should provide feedback to the related work section in an industrial track with a collaborative mindset.

Format

While the underlying values of a review are important, appropriate organization can dramatically affect a review's interpretation. To avoid unnecessary complexity, we recommend that you include the following in your reviews (from B. Adams and D. Poshyvanyk). Note that this advice is also applicable for research track reviews.

1. Paper summary
2. Points for the paper
3. Points against the paper
4. Supporting argumentation for your points
5. Suggested paper improvements

Although the Program Committee Co-Chairs determine the available ratings for each track, we recommend using the following ratings and guidelines whenever possible. This rating system is preferred over the five point rating system because it makes it easier to identify the champion (+3) and the opponent (-3) for each paper.

 3: strong accept - I will object if rejected
 2: accept - I support acceptance
 1: weak accept - accept, but could reject
 0: borderline paper - I have no opinion
-1: weak reject - reject, but could accept
-2: reject - I support rejection
-3: strong reject - I will object if accepted

Examples

While it's important to have clear values and formatting, I always find it easiest to learn through examples. Here's an anonymized example of a well done review:

Summary: The authors introduce an interesting problem, the performance of…, and provides a case study on their approach to dramatically reducing the runtime of... They also include some discussion on both the challenges and future opportunities.

Summary of Opinion: This paper introduces an interesting problem and offers a solution that improves performance by 50X (in some cases). While I wish the authors had included an example walkthrough of the porting process and that the takeaways were more interesting, I feel that the introduction of the problem and the improvement of performance is valuable enough to warrant publication.

Positives:
  • Presents a real world problem and a real world solution while at the same time introducing a new problem for academics to focus on, which makes it ideally suited for an industrial track at a research conference.
  • The approach shows that they are able to achieve dramatic speedup.
  • The paper is well-written.


Issues:
  • The Case Study sections Design - Implementation are either too generic (e.g., we parallelized things) or too specific (e.g., we had to serialize and deserialize user-defined types). I found myself wanting you to walk me through a small example of how a small part of … had been parallelized, visiting all phases. Perhaps adding an "example" section before IV.A?
  • The Opportunities section was of limited value. I would remove it. The Challenges section was not especially valuable either, I would consider reducing/removing it.
  • I would have liked to hear more about the total cost, in terms of man-hours, of the porting process. This is an important part of determining whether the port was worth it. I would have also liked to hear more about the maintainability of the ported model. Are NN workers able to modify it directly? Or do they have to port it every time it is changed?


Supporting Arguments/Details:
  • There's a typo in..
  • …

In addition to the above (nearly) full review I'd like to include a few snippets from industry track reviews which you may not have ever seen in research track reviews:
  • "I appreciated the paper in that this topic is underrepresented in the software engineering community. Although one might expect this type of topic to presented in the industrial systems communities, the paper presents a good argument that much of industrial control is in fact software, and that software engineers are the ones who need to be tackling these challenges."
  • "Tackles a hairy problem, one that has a larger scope than many 'typical' research papers."
  • "The biggest problem with this paper is that the motivation is extremely weak. Why is it problematic that… Why would anyone really care enough to run a tool of this kind?"

It's also important to know what NOT to include in this type of review. Here are a few examples of phrases I hope to NOT see in an industry track review:
  • "My main concern with this paper is that the <findings> are not new as many of them are well discussed …"
  • "The paper lacks concrete scientific contributions. As a reader, I did not learn anything new and interesting."
  • This work is not novel.
While novelty is a major concern for research track papers it is NOT a large factor for the industry track. The bar that academia has for findings being "well discussed" or "widely accepted" is dramatically different than in industry, and so study replications or real world trials of proposed approaches are welcomed.

Conclusion

While it's great that the software engineering community is rallying around the importance of bridging the gap between academia and industry we face a major threat to this progress: our own biases. If we do not consciously shift our values when reviewing industry papers we will end up alienating practitioners further.  Now that we have positive momentum let's put in the work to create professional, fair, and useful reviews.

SEIP's Corollary to the #ICSEnumber Policy

5/16/2016

 
As the PC Co-Chair for ICSE's Software Engineering in Practice (SEIP) track in 2017, I'm jealous. While the submission policy for the research track of ICSE 2017 has recently been the center of hot debate, SEIP's submission policies have NOT been slandered. To remedy this imbalance, I propose adding the following to SEIP's submission policy:
All researchers who have submitted to a "Visions" track in the past year *must* submit at least one paper to SEIP to help restore balance to the field.
One of the main reasons for this policy is that every year researchers submit more papers to "Visions" tracks, but the pool of qualified researchers doing technology transfer work does not grow in proportion. Allowing the imbalance between the amount of "Visions" being created and the technology being transferred erodes our community's already tenuous relationship with working software engineers. These engineers have recently made comments on research in our field such as: "the tool that would result would not be something I would use or can imagine anyone else using."  Perhaps our "Visions" would be better if we spent more time in the field.
Vision without execution is hallucination. -Thomas Edison
A key to breaking this cycle is to recognize that technology transfer work is valuable, and on this point the community has begun to respond. In the past few years SEIP has had an acceptance rate hovering around the low 20s and, more importantly, it has become a legitimate venue for professors interested in impact. For instance, Dr. Shane McIntosh, a tech-transfer-curious assistant professor at McGill, has published at SEIP '14 & '16 with no negative effect on his career. Technology transfer has become tolerated.
Yet tolerance is not enough. We look forward to the day when technology transfer work is seen as a first class member of the ICSE community. We are working hard to bring this vision to fruition, and thus, while this proposed policy won't be popular, we feel these steps must be undertaken to push forward tech transfer in an efficient, fair, and sustainable way. 
Comments? Let's discuss on Twitter! @davidcshepherd

Sparking Interest in Computer Science

10/19/2015

 
Picture
My Care Bear riding in one of my Construx creations, a precursor to the automated cars I'd make in Lego Logo camp
Picture
Another Construx creation from my childhood
In the summer between my fourth and fifth grade years my parents sent me to a Lego Logo camp (i.e., now called Lego Mindstorms).  As an elementary student already trying to build useful, interesting contraptions like an automated NES game dispenser with my Construx, this class changed my perspective. Using software to dynamically control the various motors and sensors opened up a new class of possibilities that I hadn't considered feasible before, and my fascination with the subject subsequently bloomed into an undergraduate degree, internships, and eventually a PhD in Computer Science.
Picture
Brainstorming with my colleagues at ABB Corporate Research
My story ends well, as I'm now happily employed at ABB Corporate Research, but I've often wondered what would have happened if my mom hadn't pushed me to attend that camp. Would I have started college as "undeclared", floating around a few years until I found CS, or would I have simply done something else? I'm honestly not sure how life would have worked out if that camp hadn't sparked my interest, and I'm hugely thankful that it did.
Picture
How likely are students in inner city schools to have their interest in computer science sparked by an expensive summer camp?
As I've thought through what might have happened to me, a middle class child, I wonder what happens to kids in more challenging situations. If no one ever pulls back the curtain of technology and shows them how things work will they ever get excited about STEM careers? My guess is "no". Without that spark, the same one that hit me during a summer camp way back in elementary school, it's hard to sustain the drive and determination to get through what is certainly a challenging major.  
Picture
One of our participants from this year's camp with his HP Stream 7 Tablet, which students get to keep to ensure they can continue to program long after class ends.
And so, to spark interest in computer science for those unwilling or unable to pay hefty summer camp fees my friends (Nicholas Kraft and Christopher Corley) and I have, for the past two summers, been running a (very cheap) one week camp to teach 10-12 year olds how to program. In this post I'll share all about the camp, from the tablets the children used to the language we used to the speakers we brought in.

However, this post is not about how great a camp we put on. This post is about giving you, the working software developer, the corporate sponsor, or the volunteer enough information to decide whether you want to join us next year. In 2016 we hope to reach twice as many of Raleigh's youth, but we need three additional dedicated teachers, about $3000 in corporate sponsorship, and 60 volunteer hours if we're going to reach our goal of 40 campers.  

We'd like to thank our corporate sponsors for 2015, Deutsche Bank and ABB, Inc. Without these sponsors the camp simply would not have happened. Thank you!
Picture
Picture

Setting

Picture
Christopher Corley, one of the instructors, demonstrates looping in TouchDevelop, the language we used to program in our camp.
We hold our camp each year at Roberts Park Community Center. In addition to having a variety of sized rooms to accommodate our growing camp Roberts Park has an excellent and supportive staff. Ms. Sherri Hartsfield, along with her colleagues, not only take care of all facilities booking and administrative necessities, but they are instrumental in recruiting students that may otherwise not be exposed to programming. Pulling from the Roberts Park after school and summer childcare program, neighborhood contacts, and personal contacts Ms. Hartsfield has no trouble filling our classroom each summer.

Schedule

Picture
The printed agenda for Thursday of the Summer 2015 camp.
As you can see from the above copy of Thursday's schedule we tried to have a full agenda each day, finding that keeping the participants busy was key to a productive camp. We utilized a variety of activity types to fill out our schedule. In between programming tutorials, the meat of our code camp, we had outside speakers from working software engineers, videos on the importance of programming, computer science unplugged activities, and demos. Breaking up tutorial/programming sessions with these other activities helped motivate students and had the nice side effect of making programming time seem scarce, which meant that students were very focused when they finally did have programming sessions.

Access

Picture
Students programming on HP Stream 7s generously donated by our corporate sponsors.
In my computer science journey the Lego Logo class I attended sparked my interest, but this spark was kindled in a nurturing environment. I had a friend whose parents bought a Lego Logo system for home, an educational system where computers were common place, and a brand new TI-82 which I could program to my heart's content. However, students with less affluent friends and school systems do not necessarily go home to such a nurturing environment. Thus, a key part of our code camp is that, along with the knowledge of how to program, we deliver the means for students to continue programming after the class ends; students get to keep the tablet they use in class. Due to generous corporate sponsors in the first two years of our camp students have taken home either a Chromebook or a tablet.   
Picture
Student programming using Microsoft's TouchDevelop, a Scratch-inspired language that is specifically designed to be programmed using a phone or tablet.

Language

In the first iteration of our camp (2014) we utilized Scratch, MIT's visual programming language specifically designed for teaching programming. We enjoyed using Scratch and found it effective for teaching programming concepts. However, Scratch is best used on a laptop or Chromebook, which are about $250 at a minimum. Fortunately, the language we used, TouchDevelop by Microsoft Research, was designed from the beginning to be programmed via touch interface, and so we were able to purchase more cost effective tablets (e.g., HP Stream 7). TouchDevelop, which is similar to Scratch, provided us with the same great teaching experience. In addition, TouchDevelop also had built-in, interactive tutorials, which reinforced lecture lessons and gave students clear guidance on projects. We found that the interactive tutorials led to much higher completion rates for individual exercises.

Speakers

Picture
Robert Robinson keeps the kids mesmerized as he explains CS concepts using football analogies. Robert was one of several excellent speakers provided by Deutsche Bank.
Part of sparking an interest in computer science is setting a vision for a desirable future. At least part of my interest in computer science was because of the countless adults that told me "You'll make a great living in computer science!"  That, combined with the minimum wage jobs that I had in high school helped show me that education really did lead to a better future.

To help foster the same realization in our camp's participants we had a speaker, who had to actively be earning a living in the computer field. You would be amazed by how effective these 10-15 minute sessions were in growing the children's visions of their future. I'm not sure what it was--perhaps these students had never met a working computer scientist before--but the kids grew absolutely silent, listening intently during many of these talks. And they had a strong effect, by the end of the week I'd hear some saying "I want to be like Mr. Mark when I grow up!"

Join Us Next Year!

As you can tell, we had a lot of fun this past summer. More importantly, we helped level the playing field for twenty students, increasing the chances that they'll benefit from the tech boom that is happening all around them in the Triangle. Yet this year we only reached 20 kids when there are hundreds growing up within walking distance of Roberts Park who are falling prey to the digital divide. We want to reach more.

If you are a working computer scientist, a corporation that wants to make a difference in Raleigh, or someone who is willing to volunteer their time, we need your help. In 2016 we hope to reach twice as many of Raleigh's youth, but we need three additional dedicated teachers (anyone with a CS background), about $3000 in corporate sponsorship, and 60 volunteer hours if we're going to reach our goal of 40 campers.  

Please contact david.shepherd@us.abb.com if you are interested in joining us!

SCAM '16 in the Land of BBQ, Beer, & Bluegrass

9/28/2015

 
Picture
Just a short walk from the conference venue you can get your fill of real NC BBQ at The Pit Restaurant.
The International Conference on Source Code Analysis and Manipulation (SCAM) has always been a small conference, but it consistently delivers high value for attendees due to its interactive format, engineering roots, and strong community. SCAM's 2016 edition will be held in tech-savvy Raleigh, a place that offers visitors a taste of all that's great about the American South without having to leave the comforts of a city. In this post I'll introduce our organizing committee, explain updates to SCAM's format, and give a sneak peak into potential Raleigh diversions. 

Format

For those of you unfamiliar with SCAM, SCAM is what I would call an interactive venue. Speakers still present during each session, but a significant portion of each session is devoted to discussing the papers that were presented as well as provocative questions that each author prepares. To me this is what sets SCAM apart from other venues, as it provides a safe open forum to quickly get feedback on your work from lots of experts. Additionally, it forces you to become better at public discussion, an invaluable skill for an aspiring researcher. For those seeking deeper feedback and looking to improve their discussion skills in a friendly environment I'd encourage you to try SCAM out. 

Format Tweak

Before I introduce our tracks' co-chairs I first need to explain one change in SCAM's format that will occur in 2016. As always, SCAM will have only two tracks. However, in 2016 SCAM will have a Research Track and an Engineering Track, the Engineering Track replacing the Tools Track. This is not to discourage tool paper submissions--they will now fall into the Engineering Track--but to broaden the scope of the tools track. I do not want to say too much about this track's scope, as we have two excellent co-chairs working on the concrete definition, but for those of you that invest blood, sweat, and tears into tooling, infrastructure, or realistic field studies SCAM recognizes the value of this work, which is not always pure research, and we are designing this track to attract that type of work. 

The Committee

Thus far we have been able to assemble a great set of co-chairs for SCAM. Our strategy at every level was to select one co-chair from academia and one from industry. Without further ado, I present our General Co-Chairs:
Picture
Oscar Nierstrasz, University of Bern
Picture
David Shepherd, ABB Corporate Research
Next, I'd like to present our Research Track Co-Chairs:
Picture
Gabriele Bavota, Free University of Bolzano
Picture
Michaela Greiler, Microsoft
And finally, I'd like to introduce SCAM's very first Engineering Track Co-Chairs:
Picture
Jurgen Vinju, CWI
Picture
Neil Ernst, SEI

Diversions

While SCAM itself is worth the trip it's always nice when the venue has a cultural experience to offer. On this front Raleigh shines. Raleigh's just Southern enough to offer great BBQ and sweet tea but with enough outside influence to still be comfortable to visitors.  Here I'll point out just a few of the places you'll want to visit while you're here.

North Carolina has great BBQ, as everyone knows, but our deep-seated Southern hospitality forces us to ensure we also have great offerings for vegetarians too. We'd hate for anyone to feel left out! Below are pictures from two restaurants that are just a short walk from the venue, The Pit for BBQ lovers, and the Fiction Kitchen for vegetarians.  
One of America's greatest contributions of late is the craft brewery scene, which has been burgeoning North Carolina. As of last count, there were 135+ craft breweries in NC alone! While the diversity of brews available has been great I must say it's a bit overwhelming for the uninitiated to choose from the aisles and aisles of craft suds. That's why I'm recommending you visit Tasty Beverage, a bottle shop and bar just a short walk from the conference venue. The staff at Tasty Beverage are true craft beer connoisseurs and they are always ready with helpful recommendations. Treat them as your tour guide through America's craft brew scene and you'll soon see why this movement has become so popular. 

Picture
Tasty Beverage is a bottle shop and a bar with staff that act as tour guides on your craft brew experience.
For the music lovers there's a genre of music you must hear live when you're in Raleigh, bluegrass. There's been a recent resurgence of interest in this art form led by traditionalists like Balsam Range and the more pop-influenced Mipso. As luck would have it, the Wide Open Bluegrass festival has coincided with SCAM's timing for the past two years, and likely as well in 2016, so there will be plenty of chances to experience bluegrass the only way it was meant to be heard, live.
Picture
Mipso, one of the many bluegrass bands based in and around Raleigh that are best when heard live.

Conclusion

As you can see, SCAM '16 has a lot to offer, but I haven't given it all away here. You see, SCAM, much like its participants, is a wonderfully quirky venue. Having ridden on a duck boat on the way to the banquet in '15 and each year paid my tribute to Monty Python, SCAM has always surprised and amused me. I hope you'll join us for next year's ride.

How to Double the Submissions to Your Industry Track

8/31/2015

 
Picture
An illustration of how our PC members felt after we exceeded our expected number of submissions (via klauseonline).
Jochen Quante and myself recently had the pleasure of co-chairing the Industry Track for ICSME '15. Due to the strong work on the ground from our program committee, aided by a few tweaks from us, the track received about twice as many submissions as in recent years. For those of you chairing industry tracks in the future, we'd like to share our strategy.

1. Choose an Industry-Leaning PC

While an earlier post addresses this topic at length, its main point bears reiterating: choose an industry-heavy PC. We intentionally selected a program committee with a much higher percentage of those in industry, raising this percentage from a respectful 46% (11/24) in 2014 to a notable 78% (18/23) in 2015. This choice not only led to more appropriate reviews but ultimately was key in soliciting quality industry submissions. Here's the email we used to invite PC-members.

ICSME Industry PC Invitation Email 2015

Dear [*FIRST-NAME*],

Each year the "Industry Track" at ICSME is created to encourage industry participants, yet the program committee is often composed of 70-90% academics. Not only does this discourage industry participation, as they rightly worry about academics judging their work without a proper perspective on the challenges in industry, it misses a key opportunity to engage industrialist as equals by inviting them onto the PC. It is a microcosm of our field's tech transfer problems; we do not want to listen to practitioners, we want them to listen to us.

To take baby steps towards addressing this issue we have intentionally selected a PC that is more industrial. We invited you to be on this PC either because you are an applied research scientist, an industrialist, or an academic with industry credibility. As we found out when trying to create this PC, those who work somewhere between academia and industry are a rare breed, and so **we ask that you seriously consider accepting this position**

If you accept, you'll be asked to review between 4 and 6 papers starting June 29th with reviews due July 26th.

Best regards,
David Shepherd <david.shepherd@us.abb.com>
Jochen Quante <jochen.quante@de.bosch.com>

2. Leverage Your PC's Professional Network

Between the time that PC members sign on and the papers start rolling in there's usually a long silent period. Use this period to build momentum for your track. We did so by leveraging the network of our PC. We encouraged PC members to personally reach out to their contacts in industry, guessing that the success rate via personal contact is likely much higher than spamming mailing lists or tweeting ad nauseum. Here's the email that we sent to PC members (a few times), which includes a draft email they could use as a starting point when contacting friends.
Dear [*FIRST-NAME*],

It's exactly X months until the ICSME Industry deadline. Consider reminding your industry friends to submit to the ICSME Industry Track, as the academic ones will remember on their own :). In case it helps, feel free to use the email below to invite them. IMO, the viability of industry tracks like this one depends on getting quality submissions from real practitioners, and so reaching our to our industry friends is an important step.

All the best,
David and Jochen

--

Subject: Share Your Industry Know-How with Academia!

Hi,

I'm on the PC of the ICSME Industry Track this year, a relatively industry-leaning research conference on software maintenance. The deadline is coming up in late June and I thought your work on <something they work on> would be a good fit for this track. Depending on the scope of the work, you can consider submitting either a 4 or 10 page paper.

Hope you consider submitting, here's the link:
http://www.icsme.uni-bremen.de/cfp_industry.php

Cheers,
<You>

3. Blog About it

To help generate buzz around your track it's helpful to blog about it, but what do you say about a track that's existed for years? We focused our single publicity blog post on changing the percentage of industrialists on the PC. While hardly an earth-shattering change, it represented the industry-focus that we were bringing to the track this year, and perhaps struck a chord with potential authors. Thus, I'd encourage you too to find your unique take on a track, implement it via concrete changes (no matter how small), and communicate it in a post. Well-timed scope or theme updates to a track can be especially fruitful.

Summary: Invite, Encourage, Blog

While industry tracks were once second class citizens they are now gaining credibility with both academics and industrialists alike. If you're an upcoming chair for an industry track I hope these tips will help you run an even stronger track, as ultimately I see industry tracks as a promising vehicles to help solve our field's impact problem... but that's a post for another day!

To close, I'd like to list the program committee members. Even though their day jobs are demanding, they gave time that made this track much stronger. Thank you PC!

    Benjamin Klatt,    inovex GmbH
    Carl Worms,    Credit Suisse AG
    David Hovemeyer,     York College of Pennsylvania
    Davy Landman,    CWI/Univ. Amsterdam
    Elmar Jürgens,    CQSE GmbH
    Eric Bouwers,    Software Improvement Group
    Felienne Hermans,    Delft University of Technology
    Jacek Czerwonka,    Microsoft
    Jan Wloka,    Quatico Solutions Inc.
    Jens Knode,    Fraunhofer IESE
    Jeroen van den Bos,    Netherlands Forensic Institute
    John Penix,    Google
    Joost Visser,    Software Improvement Group
    Magiel Bruntink,    CWI/Univ. Amsterdam
    Marc Rambert,    Coverity
    Marin Litoiu,    York University
    Marius Marin,    Microsoft
    Paul Anderson,    GrammaTech Inc.
    Ray Buse,    Google
    Tiago Alves,    Microsoft
    Tom Tourwe,    Sirris
    Vinay Augustine,    ABB Corporate Research
    Zachary Fry,    GrammaTech Inc.

On Selecting an Industrial Track Program Committee

1/23/2015

 
Recently I was asked to be the co-chair of the ICSME Industry Track. Of course, one of the responsibilities of the chairs is to chose the program committee, those who review papers and ultimately decide what's in and what's out. While there's some good advice on choosing PCs for academic tracks there's much less written on choosing PCs for industrial tracks. So, below I'm sharing a couple of tips we learned as we chose an industrial PC.
Picture
Prem Devanbu and Sebastián Uchitel listening to arguments at the ICSE 2010 PC meeting. Credit: Kevin Sullivan

Tip #1: Don't Invite Pure Academics

Don't get me wrong--I have a ton of respect for academics who are pushing the boundaries of knowledge--but their focus on novelty is not necessarily an asset during industry track reviews. Let's consider the following review, which my group received for an ICSE Industry Track (SEIP) submission:
The paper presents a tool (and process) called Prodet, to assists developer in navigating the code, in MSFT Visual studio. They validate their assumptions about the benefit of Prodet through an experiment.

The tool is nicely presented, the experiment rigorously  documented and executed; the results are convincingly presented and supported by the statistical data. The results and conclusion are contrasted with existing literature.

A good paper indeed, well executed and documented. I just find it not very exciting, sorry.
Obviously, this is an unusually kind review (thanks reviewer!), but let me bring your attention to the bolded ending sentence: "I just find it not very exciting, sorry." This is exactly the reaction that I expect, and regularly get, when presenting work to academics. I don't fault them for it. If your focus is on pushing the boundaries and novelty as the primary metric industrial work can seem quite boring. However, for industrialists this type of work is quite exciting. Taking an idea in its infancy, finding all the ways that it fails miserably when applied at scale, innovating around these issues, and convincing busy developers that the resulting tool will save them time is what we live for. Put simply, it's disheartening to receive reviews from those who don't value that work.

Fortunately, there's a simple fix for this issue: invite those from industry. As you'll see from a quick glance at our PC, this is what we've tried to do. We have people like:
  • Paul Anderson, the VP of Engineering at Grammatech, a company that was founded based on software engineering research,
  • Marc Lambert, former CEO of Kalistick, a company that regularly interfaced with researchers and was was recently acquired by Coverity,
  • Jacek Czerwonka, a member of the groundbreaking Tools for Software Engineers group at Microsoft, that is working at the boundary of research and practice,
  • John Penix, who's innovating Google's testing tools.
People like this have the academic background to read and evaluate papers, but also have experience in industry that gives them appreciation for the less glamorous work of industrial research.

Tip #2: Invite Some  Industry-Leaning Academics

Picture
Hamid Bagheri giving a cutting edge research talk at ICSE. Credit: Kevin Sullivan
While I've just told you to avoid inviting pure academics, it's actually important to invite some academics. Why? When an industrialist is knee deep in his/her latest tool-building effort, it's easy to miss a few relevant papers from the latest conferences, and so academics are sometimes more up-to-date on advances in the academic field. Thus, a few academics on the PC can ensure an appropriate framing of the work and help inject new ideas into the conversation. 

However, when inviting academics I'd still recommend inviting a certain type of academic, one with an industrial leaning or background. For instance, we have invited people like:
  • David Hovemeyer, one of the original authors of FindBugs, a tool which has over 200K downloads,
  • Magiel Bruntink, who spent four years in industry as a consultant prior to returning to academia,
  • Joost Visser, who, in addition to being a professor, is also head of research at the Software Improvement Group, a consultancy.
These industry-leaning academics help ensure proper attention is given to the state-of-the-art.

Spread the Word: ICSME Industry Track

Hopefully these tips help others as they go about choosing an industry track PC. More immediately, I hope any industrial researchers or practitioners reading this are encouraged by these improvements, as I think they are part of a larger trend. As ICSE's SEIP continues to get better (22.5% acceptance rate this year), ICSME's Industry track continues to grow and become more competitive, and more attention is paid to the engineering side of software engineering than perhaps ever before, I think that those living on the boundary between research and practice can expect an exciting future.

Submit to the ICSME Industry Track. Abstracts due Jun 19th.

Searching the Linux Source Tree in 0.5 Seconds

9/3/2014

 
Picture
Our recent work on the Sando Code Search extension, a tool which leverages Lucene to search code, has been focused on making it more scalable and robust. To demonstrate our progress I'll provide demos of both Sando and FindInFiles (i.e., a grep-like feature in Visual Studio) searching the entire Linux kernel. As you'll see, there's a fundamental difference between Lucene-based search tools and regular expression based search tools.

Before we begin, let's first briefly examine the Linux source tree. At the time of our demo it contained 47,528 files which occupied 1.71 GB on disk. Most of these files were C code, yet there was also a fair amount of documentation and configuration files. Sando and FindInFiles both search all text files.


Searching the Linux Source Tree with FindInFiles

To use FindInFiles I configured it to search the directory containing the Linux code, entered my search, and selected Find All. In this running example the user is searching for encryption algorithms, specifically those related to AES, and thus they use the regular expression query "encrypt*aes". Executing this search caused FindInFiles to run its regular expression matching algorithm against every line of every file in that directory, recursively. As you can see in "Starting the Search", this utilized about 50% of the CPU on an eight core machine for a considerable amount of time.
PictureStarting the Search: Notice when the FindInFiles search begins the CPU utilization becomes 50% on a 8-core machine.
After about one minute and forty seconds the search completed, having searched 47,407 files. Unfortunately, no lines matched this particular search (see "Finishing the Search"). As often happens with a regular expression based search, the word ordering in the query did not match the word ordering in the code. In this situation the user would likely have to run another search with re-ordered search terms (e.g., "aes*encrypt") to find relevant code.

Picture
Finishing the Search: After about 1m 40s the search completes; no results were found after searching 47,407 files.

Searching the Linux Source Tree with Sando

Next we searched the same Linux source tree using Sando. Unlike FindInFiles, which is based on regular expression matching, Sando is built upon information retrieval technology (think Google). It leverages Lucene.NET to pre-index source code and provide ranked results almost instantly. Typing in the same query as before minus the regular expression syntax (i.e., "encrypt aes") you can see below that results are returned almost instantly. Just as importantly, the most relevant results are returned first with less relevant results toward the bottom. Additionally, in Sando's UI, selecting a result in the list provides a preview of the program element with matching terms in bold.
Picture
Searching with Lucene: The same search returns almost instantly when using Lucene-based searchers.
Of course, there is a cost to pre-indexing. For the Linux source tree that cost is about 50 minutes of low CPU background processing. Fortunately, this only happens once  after which incremental updates and switching branches trigger at most a few seconds of indexing. Additionally, for most medium-sized projects initial indexing completes in a matter of seconds. For instance, Sando can index its own source code in less than ten seconds.

Try It For Yourself: Online, in Eclipse, or in Visual Studio

I hope this quick post has piqued your interest in Lucene-based code search tools. If you'd like to try an advanced code search tool there are several options, both online (e.g., GitHub's search) and offline (e.g., Sando for VS and Instasearch for Eclipse). Follow the links below to try them out.

Eclipse
  • Instasearch plugin - Uses Lucene to index and search text files (including source code).  Built on Lucene, returns results as a ranked list of files.
Visual Studio
  • Entrian Source Search - An inexpensive ($30) code search extension with fast recommendations developed by my friend Richie Hindle. Popular within the gaming community, built using Lucene.NET, returns results as a list of files.
  • Sando Code Search - A free, open source extension that leverages srcML, Lucene.NET, and research on extracting phrases from identifiers. Built on Lucene.NET, returns results as a ranked list of program elements (e.g., methods and classes).
  • Future Visual Studio Feature? - While full-text search is not part of Visual Studio itself yet, there is a feature request to include it which has already received 80+ votes. If you too want to see this feature in a future version of VS, vote it up!
Web/Server-based
  • Github - Because GitHub uses ElasticSearch, built on Solr, built on Lucene, it provides a similar project-level search to the above tools. It does a much better job than many web searches because Solr uses the WordDelimiterFilter class, which is much more effective than standard splitters at splitting camel-cased identifiers. Provides search results at the file level.
  • OpenGrok - Perhaps the first open source search tool for source code, OpenGrok has been around since 2004 and supports many languages. Built on Lucene as an Apache web service it provides file level results.
<<Previous

    Author

    David Shepherd leverages software engineering research to create useful additions to the IDE. 

    Archives

    May 2018
    May 2017
    November 2016
    May 2016
    October 2015
    September 2015
    August 2015
    January 2015
    September 2014
    May 2014
    March 2014
    February 2014
    January 2014
    October 2013
    September 2013
    August 2013
    July 2013
    May 2013
    April 2013
    March 2013
    February 2013
    December 2012
    November 2012
    October 2012
    July 2012
    June 2012
    May 2012
    January 2012

    Categories

    All
    Abb
    Extensions
    Grants
    Ide
    Software Engineering
    Software Engineering Research
    Software Tools
    Visual Studio
    Visual Studio Extensions

    RSS Feed

Powered by Create your own unique website with customizable templates.
  • Blog
  • Colleagues
  • Contact
  • About