What can local descriptors tell us about an image?
Local descriptors are designed to optimize their performance in image patch retrieval tasks. As such, they’re very likely to embed useful and descriptive information about the local image content. Surprisingly however, this property has not been subject to many studies.
In this project, we leverage variational approaches to directly invert local image descriptors. We demonstrate that it’s possible to reconstruct meaningful images from modern real and binarized descriptors (BRIEF and FREAK) using an inverse problem framework and a generic sparsity prior as regularizer.
Related work
This work was inspired by the pioneering reconstruction of SIFT descriptors. we developed it at the same time (though independently) to C. Vondrick’s HOGgles. Both of these rely on knowing (or learning) descriptors to image relationship, thus avoiding direct inversion.