Unmanned aerial vehicles (UAVs) are attracting more attention in both academic and commercial research. An open question is how to evaluate UAVs for a particular task. We address this issue by employing ideas from Algorithmic Information Theory (AIT) and propose a quantitative method of effectiveness evaluation of UAVs search in urban environment. AIT employs computer science to investigate the relationship between computation, information, and randomness. This enables the settings to be applied to numerous artificial intelligence fields. The evaluation method we proposed is based on interpreting UAVs search as their capacity to succeed in a variety of environments. We investigate the complexity of the UAVs search and introduce the complexity factor into the environment, and use coarse-grained stratified sampling to get the final environments. Two metrics are designed and are weighted by the CRiteria Importance Through Intercriteria Correlation (CRITIC) method to quantify the effectiveness of UAVs search in urban environment. The quantitative method we proposed incorporates the result function, metrics, and complexity factor. We illustrate the method with a pygame-based urban search platform. It has been demonstrated that the procedure is adaptable and reproducible, making it easier to discover flaws and thereby improve the system's effectiveness.