Seeing when vision is down to zero – Firefighters are literally groping in the dark when working their way forward through smoke-filled rooms to save people. Researchers and engineers all over the world are working on improving the orientation of rescue squads in closed rooms under extremely challenging visual conditions. That’s what they’re doing also at Qwake Technologies, a startup company that in collaboration with U.S. Homeland Security is developing the real-time visualization system C-THRU.

C-THRU has two components: the navigator unit consisting of a thermal imager, display, and computing unit that can be attached to any standard fire helmet, and the “Visual Command” tablet that the on-scene commander uses to retrieve in real time the information logged by the members of their rescue squad. That enables the leader to coordinate activities faster and more reliably under frequently chaotic conditions.

The augmented reality overlay projected into the firefighter’s field of vision represents surfaces, objects, and people as contours in the environment. Captain Justin Quarisa, a firefighter from California, was one of the first to test the system in 2023. His conclusion: “The wow factor is just the incredible technology. Obviously, we have thermal imagers, but putting that on your helmet with a heads-up display and then the firefighter tracking is something that everybody’s been trying to solve. So just having those capabilities and then the AI component where it enhances what you’re seeing with the green image or you can, you know, take it off if you don't like it, but it really just gives you a lot better picture of a room, almost like you’re watching a video game.”

AI coordinates without stress and emotions

Talking about AI: Generally, AI can help on-scene commanders capture and analyze the situation at the scene. Unencumbered by stress and emotions, the computers analyze the information transmitted by sensors and cameras, compare it with stored experience-based values, and in fractions of a second provide possible rescue scenarios helping the on-site commander make fast effective decisions and guide the rescue squad. Moreover, the AI unit can support the commander in performing multi-tasking jobs and, for instance, warn individual team members if toxic gases are forming in the immediate surroundings. Ideally, the AI autonomously navigates the firefighters out of the building quickly and safely.

AI can also provide coordination support beyond the scene of the incident, for instance by allocating accident victims to hospitals specializing in treating the relevant injuries or having adequate capacities for taking care of patients. AI-assisted hazard management systems like these for inspecting and leading rescue missions have already been deployed in some cases.

Green wave

In the field of rescue mobility, techniques enabling rescue squads to get to the scene of an incident faster and more reliably by means of real-time data transmission using the 5G mobile network are being tested, for instance at the DLR Institute of Transportation Systems in Germany. “We’ve equipped various intersections in Braunschweig and in Wolfsburg with a receiver and a transmitter module. It makes it possible for rescue vehicles of the fire department to automatically log into these traffic lights and switch them to green,” explains project leader Anna Schieben. Due to the smartly switched “green wave,” they can also aptly pass other traffic.

In the future, other vehicles, pedestrians, or bicycle riders could receive messages about enroute fire runs on their smartphones and be instructed to form a rescue lane. For that purpose, the fire truck transmits the information about the planned route to the 5G system.

Rescuing in the metaverse

Navigation systems with traffic information in real time indicate the quickest way to the scene. However, inside the building at the scene rescue squads typically have to find their own way. That’s dangerous and time-consuming. Digital twins of buildings are supposed to help in the future. Using the digital plans and resulting route guidance that’s projected onto a display in the firefighter’s field of vision, rescue squads are supposed to be able to navigate their way through a building quickly and safely while avoiding potential bottlenecks and hazardous areas. Ideally, the system uses data provided by sensors installed in the building that complete the extensive overview of the situation. Or the digital twin points out potential sources of hazards stored in it such as ignitable substances etc. If the digital twin encompasses entire building complexes, neighborhoods, or even cities preventive actions could be initiated, such as keeping flames from jumping over, supply of firefighting water, or warnings of toxic smoke, for example. Knowledge pertaining to the scene also helps firefighters to get the requisite equipment ready and to optimally position it there.

With digital twins of large buildings such as shopping malls or arenas, rescue squads can also go through and practice a variety of emergency scenarios. Plus, as early as in the planning stage, subsequent fire behaviors can be simulated on the digital twin and the plans be optimized accordingly. Later, the simulations can provide information for real-world operations.

By providing aerial photos the compact and uncrewed aircraft help assess the situation. But there’s even greater potential: in the future, cable-bound drones are supposed to illuminate vast areas, large cargo drones could dump firefighting water, and even rescue operations from roofs or balconies are conceivable.

The Swiss research institution EMPA together with the Imperial College London has developed a drone that has withstood temperatures of up to 200 degrees centigrade (392 degrees Fahrenheit) in initial tests in 2023. With that kind of heat resistance, the multi-wing aircraft could, for instance, be deployed into burning buildings as an advance party for human firefighters. The prototype is equipped with a heat-reflecting aluminum skin under which an insulating layer of so-called aerogel tiles is located. The third protective component is a cooling “perspiration effect” in which liquid CO2 is evaporated.

Space-based support

The number of devastating forest fires has been increasing year after year. This is another area in which high-tech help from above is on the horizon. Together with relevant authorities around the world Google is planning to launch “FireSat,” a satellite-based early warning system. The objective is to localize a fire source with a mere size of 25 square meters (269 square feet) anywhere on the globe by means of infrared scanning.

Currently, the warning threshold that can be captured by satellites has a size of two to three football fields, which makes containment clearly more difficult. In the case of “FireSat,” artificial intelligence supports the sensitive search grid by matching real-time and stored footage with local information such as reported artificial fires or other events.

Robo rescuers

The technologies enabling drones to fly through fires might also be the key to the next level in rescue operations: the heat-resistant humanoid robo firefighter. Istituto Italiano di Technologia (IIT) even takes that a step further: scientists there are planning to deploy a flying humanoid into the blaze. Their iRonCub MK3 equipped with jet engines on its arms and legs is reminiscent of Marvel hero Iron Man. The first full-fledged test flight is still a ways away. Plus, the question arises whether jet engines developing heat of up to 1,000 degrees centigrade (1,832 degrees Fahrenheit) are a suitable solution for indoor operations. A fireproof wheel- or leg-driven humanoid is probably the better alternative.

Automated multi-terrain rescuer

Mud, rubble, water – in a nutshell, obstacles of any kind: Thanks to its four ample low-pressure tires, the Sherp amphibious vehicle handles them with consummate ease. On land, the all-terrain vehicle with a mere length of 3.4 meters (11.1 feet) and substantial width of 2.5 meters (8.2 feet) reaches a speed of 40 km/h (24.8 mph) and as much as 6 km/h (3.7 mph) when floating on water. Like a tank, the vehicle is steered by lateral shifting of propulsion. And in a new prototype version named “Ahead” (which stands for Autonomous Humanitarian Emergency Aid Devices) from the DLR Institute for Robotics and Mechatronics it operates without any drivers on board. In partly autonomous maneuvering events, a tele-operator supports the various assistance systems such as radio communication, perception sensors, depth cameras, and a lidar-laser system. Equipped like that, people in need of help can be freed even from extreme emergency situations, such as in areas struck by flooding or epidemics, without endangering rescue squads. “AHEAD helps us cope more effectively with natural catastrophes that occur increasingly often. Automated driving poses no hazards to driver crews and we can reach crisis-struck areas that are unpassable due to roads having been washed away or destroyed bridges. In addition, the on-board technology of this special-purpose vehicle provides data enabling us to quickly create a digital overview of the situation and running search and rescue operations even more systematically than before,” says Uwe Kippnich from the Bavarian Red Cross that is involved in the project.

Read more details here.