Adding bugs to create a more effective computer bug-finder

Washington, 10 July (ANI): A group of academics has developed a novel approach for training bug-finding systems to detect more vulnerabilities. Researchers at New York University Tandon School of Engineering, in partnership with MIT Lincoln Laboratory and Northeastern University, are adopting an unconventional approach: instead of detecting and fixing faults, they are introducing hundreds of thousands of them. Brendan Dolan-Gavitt is one of the co-inventors of LAVA, or Large-Scale Automated Vulnerability Addition, a technique for purposefully introducing vulnerabilities to a program’s source code to test the boundaries of bug-finding tools and eventually assist engineers improve them. Experiments with LAVA revealed that several common bug finders discover just 2% of vulnerabilities. According to Dolan-Gavitt, the effectiveness of bug-finding systems is measured by two metrics: the false positive rate and the false negative rate, both of which are notoriously difficult to assess.

It’s fairly uncommon for a software to discover a flaw that subsequently turns out to be fake – a false positive – and to overlook vulnerabilities that are genuinely there – a false negative. There is no way to judge how well these tools operate without knowing the entire number of true bugs. “The only way to assess a bug finder is to regulate the amount of defects in a software, which is precisely what LAVA does,” Dolan-Gavitt explained. The automated method introduces known quantities of unique vulnerabilities that are synthetic yet share many of the same characteristics as real-world computer problems. Dolan-Gavitt and his colleagues avoided the customary five-figure price tag for human, custom-designed vulnerabilities by developing an automated approach that makes smart alterations in the source code of genuine systems.

As a result, there exist hundreds of thousands of unstudied, extremely realistic vulnerabilities that are low-cost, span a program’s execution lifetime, are incorporated in regular control and data flow, and appear only for a small percentage of inputs lest they bring down the entire programme. To have a big enough sample size to investigate the strengths and weaknesses of bug-finding software, the researchers had to manufacture unique defects in huge numbers. Previously discovered flaws would readily trip up current bug finders, skewing the results. The researchers analysed existing bug-finding technologies and discovered that just 2% of the defects generated by LAVA were spotted. Dolan-Gavitt emphasised that automated problem discovery is a difficult endeavour that engineers are always working to improve. The researchers will share their findings in order to aid in these efforts. The study was presented at the IEEE Symposium on Security and Privacy and was later published in the proceedings. (ANI)


Leave a Reply

Your email address will not be published. Required fields are marked *