This paper presents FactorMatte, a novel framework for video matting that combines classical matting priors with conditional ones based on expected deformations in a scene. The proposed framework is easy-to-use and can be used to factor video into more independent components for downstream editing tasks. It is based on a Bayes formulation and is capable of producing high-quality results.