The Cross-Entropy Method

This book is a comprehensive and accessible introduction to the cross-entropy (CE) method. The CE method started life around 1997 when the first author proposed an adaptive algorithm for rare-event simulation using a cross-entropy minimization technique. It was soon realized that the underlying ideas had a much wider range of application than just in rare-event simulation; they could be readily adapted to tackle quite general combinatorial and multi-extremal optimization problems, including many problems associated with the field of learning algorithms and neural computation. The book is based on an advanced undergraduate course on the CE method, given at the Israel Institute of Technology (Technion) for the last three years. It is aimed at a broad audience of engineers, computer scientists, mathematicians, statisticians and in general anyone, theorist or practitioner, who is interested in smart simulation, fast optimization, learning algorithms, image processing, etc. Our aim was to write a book on the CE method which was accessible to advanced undergraduate students and engineers who simply want to apply the CE method in their work, while at the same time accentu- ating the unifying and novel mathematical ideas behind the CE method, so as to stimulate further research at a postgraduate level.