In this abstract, we discuss an alternative method to sampling for computing the answer to probabilistic expectation queries: enumerating values in the support of the model and adding their contribution to the expectation. We propose several criteria for a good enumeration scheme and discuss several issues that arise when implementing this idea. We present a method for enumerating the support of continuous variables that meets these criteria. We also present a general method for enumerating the support of models consisting of many variables. Preliminary experiments show that this method can be better than sampling methods on some queries.
Extended abstract: Support Method
Probabilistic program inference often involves choices between various strategies. Rather than try to make the choices in advance or delegate them to the user, we can use reinforcement learning to try different strategies and see which performs well. When a compositional inference process is being used, we get a network of reinforcement learners. In our approach, the solution to an inference task is represented as a stream of successive approximations. We present strategies for choosing between a fixed set of such streams, for combining multiple streams to produce a single output stream, and for merging a stream of streams into a single stream.
Extended abstract: Reinforcement Learning for Inference