Towards Optimal Model Evaluation: Enhancing Active Testing With Actively Improved Estimators

This article discusses a proposed model re-calibration process for updating the sampling probability in order to improve the performance of the AIIPW estimator. The proposed algorithm is shown to outperform existing methods and is robust to variations in subsample size and number of sampling steps. Two datasets, Fashion-MNIST and CIFAR-10, are used to demonstrate the effectiveness of the proposed algorithm.