Modern System-on-Chip (SoC) designs face many challenges. Choosing the best communication protocol among the different processing nodes is one of the most important design decisions. On-chip communication architectures can have a significant impact on the performance of SoC designs. However, in most of the existing design tools, only the computation cost is accurately estimated. To address this challenge, we present a high-level analytical tool to estimate the data communication cost for hybrid CPU-FPGA architectures. The proposed model allows to estimate, rapidly and accurately, both computation and communication cost of applications containing multiple nested loops. This paper also explores the benefits of applying various optimization pragmas including dataflow and loop pipelining, at the compilation phase. Experimental results show that the proposed model provides accurate data communication estimation for hybrid CPU-FPGA architectures.