Title: Estimating Density Models with Truncation Boundaries using Score Matching.
Abstract:
Truncated densities are probability density functions defined on truncated domains. They share the same parametric form with their non-truncated counterparts up to a normalizing constant. Since the computation of their normalizing constants is usually infeasible, Maximum Likelihood Estimation cannot be easily applied to estimate truncated density models. Score Matching (SM) is a powerful tool for fitting parameters using only unnormalized models. However, it cannot be directly applied here as boundary conditions used to derive a tractable SM objective are not satisfied by truncated densities. In this paper, we study parameter estimation for truncated probability densities using SM. The estimator minimizes a weighted Fisher divergence. The weight function is simply the shortest distance from the data point to the boundary of the domain. We show this choice of weight function naturally arises from minimizing the Stein discrepancy as well as upper-bounding the finite-sample statistical estimation error. The usefulness of our method is demonstrated by numerical experiments and a Chicago crime dataset. We also show that proposed density estimation can correct the outlier-trimming bias caused by aggressive outlier detection methods.