This new AI technology improves video analytics by detecting human actions in real time
- A new AI has been developed that can recognize complex visual data
- SMAST can learn and predict complex human actions
- The tool could be used in surveillance, healthcare and autonomous driving, researchers say
Researchers at the University of Virginia’s School of Engineering and Applied Science have taken AI’s visual data capabilities a step further with their latest innovation: an AI-powered video analyzer called the Semantic and Motion-Aware Spatiotemporal Transformer Network (SMAST).
This system offers precision in detecting human actions and promising applications in areas such as public safety, motion tracking and even autonomous vehicle navigation.
At the heart of SMAST’s capabilities is its ability to process complex video footage by focusing on the most relevant parts of a scene.
The system integrates a multi-feature selective attention model and a motion-aware positional 2D coding algorithm. These features work together to ensure that the AI can accurately detect and interpret human actions.
The selective attention model allows SMAST to focus on crucial elements, such as a person or a moving object, while ignoring irrelevant details. For example, it can distinguish between someone throwing a ball and someone simply raising their arm.
Meanwhile, the motion-aware algorithm allows the AI to track movements over time and remember how objects and people have shifted within a scene. This gives SMAST the ability to understand the relationships between different actions, making it more effective at recognizing complex behavior.
In the security and surveillance sectors, the SMAST system can increase public safety by detecting potential threats in real time. For example, it can identify suspicious behavior in a busy space or recognize whether someone is in distress. In healthcare, the technology could be used to track patients’ movements, allowing better movement analysis for rehabilitation or monitoring during surgery.
The researchers claim that SMAST stands out in its ability to process chaotic, raw images. SMAST’s AI-driven approach apparently makes it possible to learn from data, adapt to different environments and improve action detection capabilities. The tool was subjected to several academic benchmarks, including AVA, UCF101-24, and EPIC-Kitchens, and performed quite well.
“This AI technology opens doors to real-time action detection in some of the most demanding environments,” said Scott T. Acton, professor and chair of the Department of Electrical and Computer Engineering. “It’s the kind of progress that could help prevent accidents, improve diagnostics and even save lives.”
Via TechXplore