Research

A sky full of drones -- how to keep them separated

I am applying artificial intelligence to air traffic control for UAVs, as well as to other exciting new applications.

Unmanned aerial vehicles (UAVs, or drones) have enormous potential in many applications, including delivery, surveillance, communications, disaster relief, wildlife management and more.

Look out below!

But as the skies become crowded with drones, the possibility of collisions becomes a serious issue, especially in urban areas, where they could pose a danger to people below.

I have been applying my research in artificial intelligence (AI) to the problem.

The key is "conflict detection and resolution" (CDR). Before the drone takes off, the system projects the possibility of collision, and "de-conflicts" by delaying the drone or changing its route. When the drones are already in flight, to avoid a collision, the system decides what to do -- reroute, change altitude or what have you.

There are so many operations going on at the same time, with multiple individual vehicles of different sizes, speeds, routes and altitudes, it becomes a multi-agent pathfinding problem. When the area is crowded with hundreds or thousands of vehicles already in the air, if you use a first-come-first-served system, it could be a long wait for a drone to be processed and integrated into the airspace.

The solution is batch processing -- the system computes a solution for a group of drones, which are then injected into the airspace together. This affords much greater throughput -- the number of vehicles that can go smoothly through the space -- thus reducing congestion.

The method is also quickly scalable, which could be crucial after a disaster, for example, when a small area might be flooded with drones, which would have to be processed quickly.

Deep learning in Okutama

To test the system in a real world setting, I needed a real world area for field testing the system, and I found enthusiastic interest from the government of Okutama, a mountainous and rural part of Tokyo.

For many places in rural Japan, UAVs make sense. For instance, after a disaster, roads might be blocked, so drones could be used for carrying food and medicine, as well as for communications and damage assessment.

Okutama's enthusiasm enabled me to investigate deep learning applications, which allow for situational awareness by drones.

We developed a dynamic map to see what's going on on the ground. We wanted to automatically detect and categorize the actions of people, such as reading, drinking, pushing, carrying, running, walking and so on.

The input is an image and the output is a label or semantic segmentation. The drone flies over and determines what is on the ground -- house, paved area, vegetation, water, etc.

The map can be used for damage assessment, by comparing images. For example, if there was a house there and now there isn't, you can conclude that the house was destroyed. Assessments can be done extremely quickly and accurately.

There is also potential in wildlife control. They have a lot of monkeys in Okutama, and they have to keep them away from people. The drones can recognise them and shoo them away by flashing lights.

Given the current progress in drones and AI, you can expect a revolution in the sky, as drones equipped with AI capability come to an airspace near you -- very soon.