A Systematic Comparison of Search-Based Approaches for LDA Hyperparameter Tuning

Context: Latent Dirichlet Allocation (LDA) has been successfully used in the literature to extract topics from software documents and support developers in various software engineering tasks. While LDA has been mostly used with default settings, previous studies showed that default hyperparameter values generate sub-optimal topics from software documents. Objective: Recent studies applied meta-heuristic search (mostly evolutionary algorithms) to configure LDA in an unsupervised and automated fashion. However, previous work advocated for different meta-heuristics and surrogate metrics to optimize. The objective of this paper is to shed light on the influence of these two factors when tuning LDA for SE tasks. Method: We empirically evaluated and compared seven state-of-the-art meta-heuristics and three alternative surrogate metrics (i.e., fitness functions) to solve the problem of identifying duplicate bug reports with LDA. The benchmark consists of ten real-world and open-source projects from the Bench4BL dataset. Results: Our results indicate that (1) meta-heuristics are mostly comparable to one another (except for random search and CMA-ES), and (2) the choice of the surrogate metric impacts the quality of the generated topics and the tuning overhead. Furthermore, calibrating LDA helps identify twice as many duplicates than untuned LDA when inspecting the top five past similar reports. Conclusion: No meta-heuristic and/or fitness function outperforms all the others, as advocated in prior studies. However, we can make recommendations for some combinations of meta-heuristics and fitness functions over others for practical use. Future work should focus on improving the surrogate metrics used to calibrate/tune LDA in an unsupervised fashion.

JCOMIX: A Search-Based Tool to Detect XML Injection Vulnerabilities in Web Applications


R Scripts to configure LDA using meta-heuristics

A Test Case Prioritization Genetic Algorithm guided by the Hypervolume Indicator

Search-Based Crash Reproduction and Its Impact on Debugging

A Search-based Approach for Accurate Identification of Log Message Formats

Automatic Generation of Tests to Exploit XML Injection Vulnerabilities in Web Applications

A Machine Learning- Driven Evolutionary Approach for Testing Web Application Firewalls

Parameterizing and Assembling IR-based Solutions for Software Engineering Tasks using Genetic Algorithms

A Search-based Training Algorithm for Cost-aware Prediction