Hardware-software co-design is the new trend for deep neural network and FPGA accelerator development, which iteratively revises and tunes the full system. The bottleneck of the approach lies in the time-consuming hardware synthesis. In this paper, we propose an incremental synthesis framework Acoda to rapidly design DNN accelerators on FPGAs. Based on the observation that most revisions to DNNs are minor and local, Acoda reuses existing hardware modules and incrementally modifies the accelerator. It first detects the software revisions using a graph edit distance algorithm. Then, it maps the software revisions to hardware revisions through a multi-level reuse hierarchy. As a result, Acoda speeds up the design process by 9.31X to 34.17X and achieves performance results comparable to off-the-shelf accelerators.