Research in times of Corona
Corona influences pretty much all areas of our lives by now – both privately and professionally. This year, Corona also had a significant impact on my doctoral studies, about which I would like to report in the following blog post.
In research, almost everything is about conferences. They are the best way to achieve visibility for your research and build a name in the community. But more importantly, they are a great opportunity to get in contact with other researchers, for example to get feedback on your current research activities or to develop ideas for further papers. However, this is exactly what Corona has made increasingly difficult, if not impossible, this year. Forced by the increasing number of COVID-19 cases, a number of conferences had to be held virtually: either fully virtual or in a hybrid mode (i.e. people who are allowed to travel participate physically, all others attend remotely). These decisions are absolutely correct and understandable, but it was a pity, especially for a newcomer to the „research business“ like me, that the direct exchange with other researchers was almost totally absent.
I presented three papers this year: at the Requirements Engineering Conference (RE), the International Symposium on Empirical Software Engineering and Measurement (ESEM) and the International Conference on Software Testing, Verification and Validation (ICST). In order to give a short overview of what all three papers were about, I add the abstract and the link to the publication below this blog post.
The RE took place in a hybrid fashion in Zurich. The program committee did a great job in finding the right balance for the attendance of physical and remote participants. Unfortunately, there were only very few people in Zurich, which reduced the direct exchange on-site significantly. The Q&A rounds on the papers were well organized, but due to the hybrid setting there was little exchange beyond these sessions. A similar picture emerged at the other two conferences (ESEM and ICST). Similarly, all participants tried hard to have a smooth virtual conference (thanks again to all organizers, especially the YouTube / Facebook combination worked excellently at the ESEM conference). Nevertheless, the usual coffee and lunch breaks between the paper sessions were missing , which proved to be the best medium for exchanging information or making new contacts in the past.
In general, I found that the special Corona situation was well handled during all conferences. Anyway, I hope that physical conferences will take place again in the near future, as it makes the research experience more vibrant and productive.
Overview of my papers published in 2020:
Specmate: Automated creation of test cases from acceptance criteria
In the agile domain, test cases are derived from acceptance criteria to verify the expected system behavior. However, the design of test cases is laborious and has to be done manually due to missing tool support. Existing approaches for automatically deriving tests require semi-formal or even formal notations of acceptance criteria, though informal descriptions are mostly employed in practice. In this paper, we make three contributions: (1) a case study of 961 user stories providing an insight into how user stories are formulated and used in practice, (2) an approach for the automatic extraction of test cases from informal acceptance criteria and (3) a study demonstrating the feasibility of our approach in cooperation with our industry partner. In our study, out of 604 manually created test cases, 56 % can be generated automatically and missing negative test cases are added.
Towards Causality Extraction from Requirements
System behavior is often based on causal relations between certain events (e.g. If event1, then event2). Consequently, those causal relations are also textually embedded in requirements. We want to extract this causal knowledge and utilize it to derive test cases automatically and to reason about dependencies between requirements. Existing NLP approaches fail to extract causality from natural language (NL) with reasonable performance. In this paper, we describe first steps towards building a new approach for causality extraction and contribute: (1) an NLP architecture based on Tree Recursive Neural Networks (TRNN) that we will train to identify causal relations in NL requirements and (2) an annotation scheme and a dataset that is suitable for training TRNNs. Our dataset contains 212,186 sentences from 463 publicly available requirement documents and is a first step towards a gold standard corpus for causality extraction. We encourage fellow researchers to contribute to our dataset and help us in finalizing the causality annotation process. Additionally, the dataset can also be annotated further to serve as a benchmark for other RE-relevant NLP tasks such as requirements classification.
What Makes Agile Test Artifacts Useful? An Activity-Based Quality Model from a Practitioners’ Perspective
Background: The artifacts used in Agile software testing and the reasons why these artifacts are used are fairly well-understood. However, empirical research on how Agile test artifacts are eventually designed in practice and which quality factors make them useful for software testing remains sparse. Aims: Our objective is two-fold. First, we identify current challenges in using test artifacts to understand why certain quality factors are considered good or bad. Second, we build an Activity-Based Artifact Quality Model that describes what Agile test artifacts should look like. Method: We conduct an industrial survey with 18 practitioners from 12 companies operating in seven different domains. Results: Our analysis reveals nine challenges and 16 factors describing the quality of six test artifacts from the perspective of Agile testers. Interestingly, we observed mostly challenges regarding language and traceability, which are well-known to occur in non-Agile projects. Conclusions: Although Agile software testing is becoming the norm, we still have little confidence about general do’s and don’ts going beyond conventional wisdom. This study is the first to distill a list of quality factors deemed important to what can be considered as useful test artifacts.
