The Communications of ACM published an article on automata learning in software engineering last February. The techniques described in the article are used to obtain models for the (input/output) behaviour of software. Even without access to source code, one can now use model checking or other bug finding tools on these models. The article shows many successful applications. Why is this possible at all?
This year a team from the Radboud University and TU Delft used automata learning to compete in the RERS challenge 2016. The challenge provides (generated) source code where the challenge is to (dis)prove certain LTL formulas and to analyze which error states are reachable. Information on this challenge can be found here: http://www.rers-challenge.org/2016/. Commonly, learning is not used in this competition and only white-box methods are used.
This year, however, automata learning was applied to great success. For the problems where LTL formulas had to be (dis)proven, the team managed to get a perfect score. Other teams did not manage to get so many results here. For the reachability problems, they performed well but did not win in the rankings. The team applied state of the art learning algorithms, but did not tweak or alter them for the challenge.
It is interesting that a black box technique can get such good scores, compared to white box methods. Indeed, less information is used and by using black box techniques one cannot have 100% guarantees. But more results are obtained. So it seems one can trade confidence for scaling to bigger problems.
More information will follow.
We made a page with some useful resource on automata learning. Of course, any help is welcome. So if you know some tools, benchmarks, use cases, or anything related, please contact us.