Eastern Cape police have recovered four of the five paintings stolen from a Pretoria Art Museum.