Abstract: Novelty detection detects outliers located at any location, such as abnormalities (i.e., far distance outliers) and novel/unobserved patterns (i.e., close distance outliers). While many novelty detection approaches have been proposed in the literature, they generally focus on detecting one specific type of outlier, e.g., Multi-Class Open Set Recognition (MCOSR) and One-Class Novelty Detection (OCND) approaches are applied for far and close distance outlier detection, respectively. However, in practice, it is difficult to measure in advance whether the distance between outliers and inliers is far or close. Recent work on outlier detection at any location with a unified model has yielded mixed performance. In this paper, we propose a new unified model, named Calibrated Reconstruction Based Adversarial AutoEncoder (CRAAE), for location agnostic outlier detection. The key idea is to integrate implicit and explicit confidence calibration strategies into a reconstruction based model for building a more accurate decision boundary. We leverage the category information disentangled from feature space to calibrate the decision metric (i.e., reconstruction error) constructed in the original data space. CRAAE also adds Uniform or Dirichlet noise into the artificial outlier generation process to represent various outliers. Experimental results show that CRAAE can outperform state-of-the-art unified models (e.g., GPND) and achieve similar performance with OCND and MCOSR methods in close and far distance outlier detection, respectively.