State-of-the-art Convolutional Neural Networks (CNN) are used to process images. In most cases, videos are streamed and processed frame by frame using a CNN. In this paper we present a two-step approach to process images in a real-life streaming environment. We exploit size-reduction and data encoding to reduce the computational and communication load. A near-sensor architecture is proposed. The final design reaches 14 EPS for the full Faster R-CNN pipeline.