It is challenging to parallelize the problems with irregular computation and communication. In this paper, we proposed a scalable and efficient algorithm for solving irregular inter-processor data dependency in image understanding tasks on distributed memory machines. Depending on the input data, in the scatter phase, each processor distributes search requests to collect the remote data satisfying certain geometric constraints. In the gather phase, the requested remote data are collected by using a DataZone data structure and are returned to each processor. For demonstrating the usefulness of our algorithm, we conducted experiments on an IBM SP2. The experimental results were consistent with the theoretical analyses, and showed the scalability and efficiency of the proposed algorithm. Our code using C and MPI is portable onto other High Performance Computing(HPC) platforms.