Qin’s breakthrough technology will enhance Air Force surveillance missions
New technology developed by a College of Engineering researcher at The Ohio State University will provide Air Force personnel with unprecedented realistic situational awareness, an advancement that will enhance mission success and help save lives.
Led by Rongjun Qin, an associate professor in the Departments of Civil, Environmental and Geodetic Engineering and Electrical and Computer Engineering, his proposal “Live 3D Gaussian Splatting for ISR” will receive $200,000 in funding over two years from the Air Force Research Laboratory (AFRL) Regional Network - Midwest.
Qin’s project proposes a unique, photorealistic live 3D intelligence, surveillance and reconnaissance (ISR) capability. Not only will the breakthrough technology provide airmen, soldiers and their commanders with advanced realistic situational awareness over areas both large and small, it will also provide very-high data compression to facilitate tactical utility.
“The technology, if developed, expands the pilots’ field of view at surveillance through 3D augmentation, and allows them to replay the 2D video recordings in a 3D navigable environment,” said Qin.
Robert Ewing, director of the Center for Innovative Radar Engineering for the AFRL Sensors Directorate is co-PI for the project, and Steven Suddarth, founder of Transparent Sky, LLC is a collaborator.
ISR systems require Air Force users to make instantaneous decisions critical to mission success and preservation of life. Accuracy and photorealism are pivotal for informed decisions, according to Qin and the team. Their project targets the immediate need for generating dense, dynamic and visualizable 3D models in real- or near-real time.
The project utilizes a recently introduced concept called 3D Gaussian Splatting (3DGS). Unlike traditional 3D polygon representations from 2D frames, such as those used in terrain modeling and gaming, 3DGS represents a sibling of the field-based methods that model the appearance of the scene from novel perspectives, where structural information can be subsequently extracted as needed. The key is the use of deep models to generate a field of gaussian “fuzzy ellipsoids” that interact to create photorealistic views.
According to Qin’s team, 3DGS is appealing for ISR because it generates geometrically consistent and modifiable models for injecting information and it generates views at roughly the same resolution as the input imagery. It’s also fast, providing dozens and hundreds of frames per second, and it works for difficult surfaces such as those of reflective, transparent, translucent, and geometrically complex objects, including fence-like objects.
3DGS can be applied to most imagery types used by the Air Force, including visible, infrared, and even radio frequency imagery of certain types.
Despite advancements in field-based methods, their practical application—particularly in real-time ISR— poses challenges, especially concerning dynamic model updates requiring affordable computations for onboard devices.
By offering a photorealistic live 3D model and animating moving components in real-time, the team’s 3DGS technology allows combatants to observe non-occluded areas from any viewpoint within sensor coverage. Also, since it does not rely on tracking for frame-to-frame novelty detection, it enables detection of slow or small changes from the initial frames.”
In 2023, Qin received funding from the Office of Naval Research to develop next-generation 3D geoinformation techniques to make synthetic battlefield environments as accurate and as realistic as possible.
- by Meggie Biss, College of Engineering Communications | biss.11@osu.edu
