Unit Test Case Generation with Transformers (Research Paper Walkthrough)

#unittest #transformers #software
Unit test is a way of testing the smallest piece of code that can be logically isolated in a system. This paper from Microsoft introduces a technique to automatically generate unit test cases using transformers model. They also release large scale unit test-focal method training pair for public use. They extensively evaluate their system and found it to be surpassing the performance of existing systems.

⏩ Abstract: Automated Unit Test Case generation has been the focus of extensive literature within the research community. Existing approaches are usually guided by the test coverage criteria, generating synthetic test cases that are often difficult to read or understand for developers. In this paper we propose AthenaTest, an approach that aims at generating unit test cases by learning from real-world, developer-written test cases. Our approach relies on a state-of-the-art sequence-to-sequence transformer model which is able to write useful test cases for a given method under test (i.e., focal method). We also introduce methods2test - the largest publicly available supervised parallel corpus of unit test case methods and corresponding focal methods in Java, which comprises 630k test cases mined from 70k open-source repositories hosted on GitHub. We use this dataset to train a transformer model to translate focal methods into the corresponding test cases. We evaluate the ability of our model in generating test cases using natural language processing as well as code-specific criteria. First, we assess the quality of the translation compared to the target test case, then we analyze properties of the test case such as syntactic correctness and number and variety of testing APIs (e.g., asserts). We execute the test cases, collect test coverage information, and compare them with test cases generated by EvoSuite and GPT-3. Finally, we survey professional developers on their preference in terms of readability, understandability, and testing effectiveness of the generated test cases.

Please feel free to share out the content and subscribe to my channel :)
⏩ Subscribe -    / @techvizthedatascienceguy  

⏩ OUTLINE:
0:00 - Overview
01:03 - Abstract
02:11 - Data Collection approach
03:30 - AthenaTest Pictorial Overview
04:41 - BART Transformers in a Nutshell
06:42 - Pre-training
07:48 - Fine-Tuning
08:39 - Pre-training model variants
10:03 - Can our models learn to generate test cases?
11:04 - What is the quality of generated test cases?
13:18 - How does our approach compare to EvoSuite and GPT-3?
14:16 - Do developers prefer AthenaTest’s test over EvoSuite?

⏩ Paper Title: Unit Test Case Generation with Transformers
⏩ Paper: arxiv.org/abs/2009.05617
⏩ Author: Michele Tufano, Dawn Drain, Alexey Svyatkovskiy, Shao Kun Deng, Neel Sundaresan
⏩ Organisation: Microsoft

⏩ IMPORTANT LINKS
Full Playlist on BERT usecases in NLP:    • Text Summarization of COVID-19 Medica...  
Full Playlist on Text Data Augmentation Techniques:    • Data Augmentation using Pre-trained T...  
Full Playlist on Text Summarization:    • Text Summarization of COVID-19 Medica...  
Full Playlist on Machine Learning with Graphs:    • DEEPWALK: Online Learning of Social R...  
Full Playlist on Evaluating NLG Systems:    • Evaluation of Text Generation: A Surv...  

*********************************************
If you want to support me financially which totally optional and voluntary :) ❤️
You can consider buying me chai ( because i don't drink coffee :) ) at www.buymeacoffee.com/TechvizCoffee

*********************************************
⏩ Youtube - youtube.com/c/TechVizTheDataScienceGuy
⏩ Blog - prakhartechviz.blogspot.com/
⏩ LinkedIn - linkedin.com/in/prakhar21
⏩ Medium - medium.com/@prakhar.mishra
⏩ GitHub - github.com/prakhar21
⏩ Twitter - twitter.com/rattller
*********************************************

Please feel free to share out the content and subscribe to my channel :)
⏩ Subscribe -    / @techvizthedatascienceguy  

Tools I use for making videos :)
⏩ iPad - tinyurl.com/y39p6pwc
⏩ Apple Pencil - tinyurl.com/y5rk8txn
⏩ GoodNotes - tinyurl.com/y627cfsa

#techviz #datascienceguy #ai #researchpaper #naturallanguageprocessing #bart