BSc in Computer Science Thesis - Eötvös Loránd University
Medical and pharmaceutical research at Semmelweis University requires the analysis of leg movements in rats following spinal cord injury to evaluate rehabilitation effectiveness. Traditionally, this involves manual annotation of thousands of video frames. This would be a monotonous, expensive, and time-consuming process. This thesis presents Soterats, an automated system designed to replace human annotation using deep learning and machine vision.
The system utilizes the YOLOv8-pose estimation model to track 13 specific keypoints and the Segment Anything Model (SAM) to generate binary masks. A novel synthetic skeleton-based correction algorithm improves toe-point accuracy, while optical flow filters out frames with insufficient movement. The application generates normalized time-series data of leg movements to classify subjects into five categories ranging from "crippled" to "healthy."
The resulting application features a Streamlit-based graphical interface, allowing researchers without extensive IT backgrounds to train custom models on as little as 10% of their dataset and perform high-throughput automated analysis. This significantly increases research efficiency and provides objective, reproducible metrics for biological studies.