2 | Annibale Panichella

What Challenges Do Developers Face in AI Agent Systems? An Empirical Study on Stack Overflow & GitHub Issues

Fri, 04 Sep 2026 00:00:00 +0200

Metamorphic Testing of Deep Code Models: A Systematic Literature Review

Tue, 09 Sep 2025 11:21:45 +0200

Abstract: Large language models and deep learning models designed for code intelligence have revolutionized the software engineering field due to their ability to perform various code-related tasks. These models can process source code and software artifacts with high accuracy in tasks such as code completion, defect detection, and code summarization; therefore, they can potentially become an integral part of modern software engineering practices. Despite these capabilities, robustness remains a critical quality attribute for deep-code models as they may produce different results under varied and adversarial conditions (e.g., variable renaming). Metamorphic testing has become a widely used approach to evaluate models’ robustness by applying semantic-preserving transformations to input programs and analyzing the stability of model outputs. While prior research has explored testing deep learning models, this systematic literature review focuses specifically on metamorphic testing for deep code models. By studying 45 primary papers, we analyze the transformations, techniques, and evaluation methods used to assess robustness. Our review summarizes the current landscape, identifying frequently evaluated models, programming tasks, datasets, target languages, and evaluation metrics, and highlights key challenges and future directions for advancing the field.

Multi-objective differential evolution in the generation of adversarial examples

Thu, 06 Jun 2024 17:56:40 +0200

The Slow and The Furious? Performance Antipattern Detection in Cyber-Physical Systems

Mon, 01 Jan 2024 17:32:54 +0200

JUGE: An Infrastructure for Benchmarking Java Unit Test Generators

Mon, 12 Dec 2022 11:11:22 +0200

Continuous Integration and Delivery practices for Cyber- Physical systems: An interview-based study

Fri, 11 Nov 2022 11:21:45 +0200

Generating Class-Level Integration Tests Using Call Site Information

Tue, 13 Sep 2022 09:29:14 +0200

Abstract: Search-based approaches have been used in the literature to automate the process of creating unit test cases. However, related work has shown that generated tests with high code coverage could be ineffective, i.e., they may not detect all faults or kill all injected mutants. In this paper, we propose CLING, an integration-level test case generation approach that exploits how a pair of classes, the caller and the callee, interact with each other through method calls. In particular, CLING generates integration-level test cases that maximize the Coupled Branches Criterion (CBC). Coupled branches are pairs of branches containing a branch of the caller and a branch of the callee such that an integration test that exercises the former also exercises the latter. CBC is a novel integration-level coverage criterion, measuring the degree to which a test suite exercises the interactions between a caller and its callee classes. We implemented CLING and evaluated the approach on 140 pairs of classes from five different open-source Java projects. Our results show that (1) CLING generates test suites with high CBC coverage, thanks to the definition of the test suite generation as a many-objectives problem where each couple of branches is an independent objective; (2) such generated suites trigger different class interactions and can kill on average 7.7% (with a maximum of 50%) of mutants that are not detected by tests generated randomly or at the unit level; (3) CLING can detect integration faults coming from wrong assumptions about the usage of the callee class (25 for our subject systems) that remain undetected when using automatically generated random and unit-level test suites.

Test Smells 20 Years Later: Detectability, Validity, and Reliability

Thu, 21 Jul 2022 11:34:14 +0200

Single and Multi-objective Test Cases Prioritization for Self-driving Cars in Virtual Environments

Mon, 04 Apr 2022 11:21:45 +0200

Large scale inverse design of planar on-chip mode sorter

Sat, 01 Jan 2022 09:57:31 +0100

How to Kill Them All: An Exploratory Study on the Impact of Code Observability on Mutation Testing

Tue, 01 Dec 2020 17:32:54 +0200

Serverless Testing: Tool Vendors' and Experts' Point of View

Wed, 07 Oct 2020 10:14:08 +0200

A Systematic Comparison of Search-Based Approaches for LDA Hyperparameter Tuning

Wed, 09 Sep 2020 21:50:29 +0200

Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test Case Generation

Wed, 09 Oct 2019 09:51:33 +0200

Abstract: Automated test case generation is an effective technique to yield high-coverage test suites. While the majority of research effort has been devoted to satisfying coverage criteria, a recent trend emerged towards optimizing other non-coverage aspects. In this regard, runtime and memory usage are two essential dimensions: less expensive tests reduce the resource demands for the generation process and later regression testing phases. This study shows that performance-aware test case generation requires solving two main challenges: providing a good approximation of resource usage with minimal overhead and
avoiding detrimental effects on both final coverage and fault detection effectiveness. To tackle these challenges, we conceived a set of performance proxies -inspired by previous work on performance testing- that provide a reasonable estimation of the test execution costs (i.e., runtime and memory usage). Thus, we propose an adaptive strategy, called aDynaMOSA, which leverages these proxies by extending DynaMOSA, a state-of-the-art evolutionary algorithm in unit testing. Our empirical study -involving 110 non-trivial Java classes- reveals that our adaptive approach generates test suite with statistically significant improvements in runtime (-25%) and heap memory consumption (-15%) compared to DynaMOSA. Additionally, aDynaMOSA has comparable results to DynaMOSA over seven different coverage criteria and similar fault detection effectiveness. Our empirical investigation also highlights that the usage of performance proxies (i.e., without the adaptiveness) is not sufficient to generate more performant test cases without compromising the overall coverage.

Search-based Multi-Vulnerability Testing of XML Injections in Web Applications

Mon, 08 Apr 2019 10:11:19 +0200

A Test Case Prioritization Genetic Algorithm guided by the Hypervolume Indicator

Sat, 08 Sep 2018 11:51:02 +0200

A Large Scale Empirical Comparison of State-of-the-art Search-based Test Case Generators

Sat, 18 Aug 2018 07:39:53 +0200

On the Impact of Code Smells on the Energy Consumption of Mobile Applications

Mon, 13 Aug 2018 19:14:00 +0200

Search-Based Crash Reproduction and Its Impact on Debugging

Mon, 13 Aug 2018 18:55:36 +0200

The Scent of a Smell: An Extensive Comparison between Textual and Structural Smells

Tue, 10 Jul 2018 11:25:49 +0200

Developer Testing in The IDE: Patterns, Beliefs, And Behavior

Tue, 10 Jul 2018 11:24:46 +0200

Automated Test Case Generation as a Many-Objective Optimisation Problem with Dynamic Selection of the Targets

Tue, 10 Jul 2018 11:23:34 +0200

Automatic Generation of Tests to Exploit XML Injection Vulnerabilities in Web Applications

Tue, 10 Jul 2018 11:21:45 +0200

A Machine Learning- Driven Evolutionary Approach for Testing Web Application Firewalls

Tue, 10 Jul 2018 11:11:22 +0200

A Systematic Literature Review of How Mutation Testing Supports Quality Assurance Processes

Tue, 10 Jul 2018 11:11:22 +0200

Defect Prediction as a Multi-Objective Optimization Problem

Sun, 19 Jul 2015 23:50:41 +0200

Improving Multi-Objective Search Based Test Suite Optimization through Diversity Injection

Sun, 19 Jul 2015 23:46:36 +0200

Abstract: A way to reduce the cost of regression testing consists of selecting or prioritizing subsets of test cases from a test suite according to some criteria. Besides greedy algorithms, cost cognizant additional greedy algorithms, multi-objective optimization algorithms, and multi-objective genetic algorithms (MOGAs), have also been proposed to tackle this problem. However, previous studies have shown that there is no clear winner between greedy and MOGAs, and that their combination does not necessarily produce better results. In this paper we show that the optimality of MOGAs can be significantly improved by diversifying the solutions (sub-sets of the test suite) generated during the search process. Specifically, we introduce a new MOGA, coined as DIversity based Genetic Algorithm (DIV-GA), based on the mechanisms of orthogonal design and orthogonal evolution that increase diversity by injecting new orthogonal individuals during the search process. Results of an empirical study conducted on eleven programs show that DIV-GA outperforms both greedy algorithms and the traditional MOGAs from the optimality point of view. Moreover, the solutions (sub-sets of the test suite) provided by DIV-GA are able to detect more faults than the other algorithms, while keeping the same test execution cost.

Labeling Source Code with Information Retrieval Methods: An Empirical Study.

Sat, 19 Jul 2014 23:52:54 +0200

Improving IR-based traceability recovery via noun-based indexing of software artifacts

Thu, 19 Jul 2012 23:55:49 +0200

Applying a Smoothing Filter to Improve IR-based Traceability Recovery Processes: An Empirical Investigation.

Thu, 19 Jul 2012 23:54:19 +0200