Call for papers, Less is More

Submissions to this track must be situated in the SE literature and come with a cover letter stating that they wish to be reviewed as “less is more” paper.

Throughout the years, there have been numerous accounts of very basic models that have achieved remarkable results 1 2 3 4 5 6 7 8 9, and that such simpler models are essential for people to understand a subject 10. So where are the researchers inquiring ``could this imply that we can make software analytics simpler and easier to comprehend?’’

We are proposing the creation of a platform where researchers can openly discuss the idea that some of our methods are overly-complex and that simpler approaches can be just as effective. This call is ongoing and we are looking for papers that; e.g.

  • Re-examine existing results and demonstrate how task A can be better performed by a simpler method (e.g. smaller, faster, etc.).
  • We are also interested in ablation studies that remove parts of an implementation until performance drops;
  • Instance or feature selection methods to reduce the training set;
  • Distillation methods to reduce the size of a learned model;
  • Variance studies that show that the improvement of a complex method over a simpler one is insignificant;
  • Studies that show that a 10% system can perform as well as a 100% system.
  • Semi-supervised learning methods that let us do much more with much less data;

or any other kind of study that illustrates when less can be more in automated SE.

Deadline

None. This is an ongoing track.

Editor

Tim Menzies (timm@ieee.org)

References

  1. Amritanshu Agrawal, Wei Fu, Di Chen, Xipeng Shen, and Tim Menzies. 2019. How to “dodge” complex software analytics. IEEE Transactions on Software Engineering 47, 10 (2019), 2182–2194. ↩

  2. Robert C. Holte. 1993. Very Simple Classification Rules Perform Well on Most Commonly Used Datasets. Machine Learning 11 (1993), 63–90 ↩

  3. Ron Kohavi and George H. John. 1997. Wrappers for Feature Subset Selection. Artificial Intelligence 97, 1-2 (1997), 273–324. ↩

  4. Tim Menzies, Burak Turhan, Ayse Bener, Gregory Gay, Bojan Cukic, and Yue Jiang. 2008. Implications of ceiling effects in defect predictors. In Proceedings of the 4th international workshop on Predictor models in software engineering. 47–54. ↩

  5. Vali Tawosi, Rebecca Moussa, and Federica Sarro. 2023. Agile Effort Estimation: Have We Solved the Problem Yet? Insights From a Replication Study. IEEE Transactions on Software Engineering 49, 4 (2023), 2677–2697. https://doi.org/10.1109/TSE.2022.3228739 ↩

  6. Zhou Xu, Li Li, Meng Yan, Jin Liu, Xiapu Luo, John Grundy, Yifeng Zhang, Xiaohong Zhang, A comprehensive comparative study of clustering-based unsupervised defect prediction models, Journal of Systems and Software, Volume 172, 2021. ↩

  7. Di Chen, Wei Fu, Rahul Krishna, and Tim Menzies. 2018. Applications of psychological science for actionable analytics. ESEC/FSE 2018. 456–467. https://doi.org/10.1145/3236024.3236050 ↩

  8. Zhongxin Liu, Xin Xia, Ahmed E. Hassan, David Lo, Zhenchang Xing, Xinyu Wang: Neural-machine-translation-based commit message generation: how far are we? ASE 2018: 373-384 ↩

  9. Hong Jin Kang, Tegawendé F. Bissyandé, David Lo: Assessing the Generalizability of Code2vec Token Embeddings. ASE 2019: 1-12 ↩

  10. Nathaniel D Phillips, Hansjoerg Neth, Jan K Woike, and Wolfgang Gaissmaier. 2017. FFTrees: A toolbox to create, visualize, and evaluate fast-and-frugal decision trees. Judgment and Decision Making 12, 4 (2017), 344–368. ↩