Markerless human pose estimation for biomedical applications: a survey
Abstract: Markerless Human Pose Estimation (HPE) proved its potential to support decision making and assessment in many fields of application. HPE is often preferred to traditional marker-based Motion Capture systems due to the ease of setup, portability, and affordable cost of the technology. However, the exploitation of HPE in biomedical applications is still under investigation. This review aims to provide an overview of current biomedical applications of HPE. In this paper, we examine the main features of HPE approaches and discuss whether or not those features are of interest to biomedical applications. We also identify those areas where HPE is already in use and present peculiarities and trends followed by researchers and practitioners. We include here 25 approaches to HPE and more than 40 studies of HPE applied to motor development assessment, neuromuscolar rehabilitation, and gait & posture analysis. We conclude that markerless HPE offers great potential for extending diagnosis and rehabilitation outside hospitals and clinics, toward the paradigm of remote medical care.
Summary
- The paper highlights markerless HPE's advancements in biomedical applications using both 2D and 3D techniques.
- It compares top-down and bottom-up approaches, emphasizing their trade-offs in handling occlusions and false positives.
- The survey underscores practical deployments in motor assessment, rehabilitation, and gait analysis while calling for wider evaluation studies.
Markerless Human Pose Estimation for Biomedical Applications
Human pose estimation (HPE) has progressed significantly over the years, becoming an indispensable tool in various fields, notably in biomedical applications for its ability to facilitate motion analysis without the need for intrusive markers. The advancement of markerless HPE technologies offers several advantages, such as reduced setup complexity and lower costs compared to traditional marker-based systems.
Overview of Human Pose Estimation Approaches
Methodologies and Taxonomy
Markerless HPE encompasses a wide range of methodologies, which can largely be categorized into 2D and 3D pose estimation frameworks. These methods typically harness imaging data from RGB and RGB-D sensors and apply computer vision techniques, including deep learning models such as CNNs and transformers. A taxonomy of these approaches highlights the differences based on the input type (RGB or RGB-D), estimation space (2D or 3D), and processing views (single-view or multi-view).
Top-down vs. Bottom-up Approaches
A critical distinction in HPE methods is between top-down and bottom-up approaches. Top-down methods first detect individuals and then estimate keypoints, providing robustness against false positives Figure 1. However, these approaches can struggle with occlusions. Conversely, bottom-up methods begin by locating keypoints and grouping them to individuals, which can efficiently handle occlusions but are prone to false positives in noisy environments.
Figure 1: Top-down and bottom-up approaches in HPE; the top-down approach manages false positives effectively, while bottom-up excels in occlusion scenarios.
Single-view vs. Multi-view Approaches
Single-view approaches, despite providing valuable insights, often suffer from depth ambiguity since they rely solely on monocular image cues. Multi-view approaches, conversely, use geometric triangulation to achieve more accurate spatial representation Figure 2, though they necessitate complex setups and camera calibration.
Figure 2: Single-view and multi-view HPE methodologies compared; multi-view setups offer improved accuracy in spatial representation.
Applications in Biomedical Fields
Motor Development Assessment
HPE technologies are transforming how clinicians assess motor development in infants by automating video analysis and reducing manual coding time. The studies reveal the importance of tailoring models to accommodate the anatomical proportions of infants, with newer techniques adept at identifying markers of neurological disorders.
Neuromuscular Rehabilitation
Markerless HPE provides a promising alternative to conventional motion capture in rehabilitation settings. These methods can accurately track patient progress in clinical and telerehabilitation scenarios, offering the potential for enhanced home healthcare solutions. Innovative models are being developed specifically to cater to the computational and environmental constraints encountered in remote assessment setups.
Gait and Posture Analysis
In the domain of gait analysis, HPE offers the potential to quantify movement dynamics for diagnosing conditions such as Parkinson's disease. The integration of temporal reasoning with pose estimation enriches the precision of gait metrics, aiding in clinical evaluations and movement disorder diagnosis.
Limitations and Future Directions
Despite the promise these systems hold, practical application is hindered by challenges such as the need for large-scale evaluation studies to affirm their reliability across diverse patient populations, improvements in the algorithmic handling of anthropometric variability, and the development of standard frameworks for widespread clinical adoption. Future research may focus on robust generalization techniques and computational efficacy to enable practical deployment in real-world healthcare environments.
Conclusion
The survey of markerless human pose estimation reveals significant potential for its application in the biomedical field. While challenges remain, the unique advantages in terms of setup simplicity and reduced invasiveness position HPE as a crucial tool in extending medical diagnostics and rehabilitation into non-hospital environments. This advance towards remote care paradigms represents a pivotal change in how patient movement is monitored and analyzed, paving the way for broader accessibility and improved patient outcomes.
Whiteboard
Paper Prompts
Sign up for free to create and run prompts on this paper.
Explain it Like I'm 14
Overview
This paper is a survey, which means it reviews and organizes lots of earlier studies instead of running one new experiment. Its topic is “markerless human pose estimation” (HPE) for health and medicine. That’s a way for computers to find where your body joints (like knees, elbows, and shoulders) are by looking at normal videos, without needing sticky markers on your skin. The authors look at how well this works today, where it’s already being used in healthcare, what the main tools and tricks are, and what still needs improvement.
What questions did the paper ask?
In simple terms, the authors wanted to know:
- What are the main kinds of markerless pose estimation methods, and how do they differ?
- Where in medicine are these methods already being used (for example, with babies, in rehabilitation, or in walking/posture analysis)?
- How good are they for real clinical use, and what trade-offs exist (like accuracy versus cost or ease of use)?
- What trends and future directions could make these tools more helpful in everyday healthcare, including at home?
How did they study it?
This is a review of the field:
- They examined 25 different pose-estimation approaches and over 40 biomedical studies that used them.
- They grouped and compared methods by important features and explained how those choices affect healthcare use.
To make the technical terms approachable, here are the main ideas they explain and compare:
Human pose estimation, in everyday words
- Think of the computer trying to draw a stick figure on a person in a video, putting dots on joints and connecting them.
- “Markerless” means it does this from regular cameras—no special reflective markers, no lab-only equipment.
Key design choices, explained with analogies
- 2D vs 3D:
- 2D is like drawing the stick figure on a flat photo.
- 3D is like placing the stick figure in real space, with depth. 3D is more useful clinically but harder to get right.
- Top-down vs bottom-up:
- Top-down: first find each person in the image, then place the joints on that person. Like spotting people in a crowd, then labeling their body parts.
- Bottom-up: first find all joints in the image, then group them into people. Like collecting puzzle pieces first, then assembling the people.
- Single-view vs multi-view:
- Single-view uses one camera—cheaper and simpler, but depth is guessed.
- Multi-view uses two or more cameras—by “crossing” lines from each camera (triangulation), it can locate joints in 3D more accurately.
- RGB vs RGB-D:
- RGB is a normal color camera.
- RGB-D adds a depth sensor that measures how far things are (like a built-in ruler). This can help with 3D but still struggles when body parts block each other.
- Temporal reasoning:
- Videos can look “jittery” frame-by-frame. Methods use smoothing or look at several frames in a row to keep the stick figure steady over time.
How the review connects to healthcare
- The paper looks at how these choices play out in three areas:
- Motor development in babies and toddlers
- Neuromuscular rehabilitation (e.g., after injury or stroke)
- Gait and posture analysis (e.g., walking patterns)
It also discusses practical realities: what works on smartphones, what needs more cameras, where accuracy is “good enough,” and where it isn’t yet.
What did they find, and why is it important?
Below is a brief, readable summary. Each topic includes what researchers tried and what it means for patients and clinicians.
1) Motor development assessment (especially newborns and infants)
- Goal: Spot early signs of movement problems (like risk of cerebral palsy) by analyzing how babies move.
- What’s hard: Babies are tiny, move unpredictably, and have different body proportions than adults—most AI models were trained on adults.
- What researchers did:
- Used simple setups (often a camera above a bed) and sometimes depth cameras.
- Used pose tools like OpenPose, HRNet, and newer infant-specific methods.
- Focused on “fidgety” and “writhing” movements in early months, which are important clinical signals.
- Explored at-home videos from phones to make screening more accessible.
- Why it matters: These systems can help flag babies who may need further evaluation, possibly earlier and outside the hospital. That can lead to faster interventions when they matter most.
- Key caution: Doctors need to understand why the AI makes certain calls. Newer work adds visual explanations to improve trust.
2) Neuromuscular rehabilitation
- Goal: Help patients do exercises correctly, track recovery, and even guide robot-assisted therapy.
- What’s hard: People may have unusual body shapes or movement limits (e.g., post-surgery), which can confuse generic pose models. Safety is crucial.
- What researchers did:
- Built systems that run on phones, browsers, edge devices, or in the cloud to make home use realistic.
- Mixed RGB and depth cameras when needed for better 3D accuracy.
- Measured joint angles, detected mistakes, and provided feedback in real time or near-real time.
- Compared different pose models and found some are fast and accurate enough for clinical angles (like elbows or shoulders).
- Why it matters: Patients could get quality guidance at home, reducing travel and costs. Clinicians can monitor progress more objectively.
3) Gait and posture analysis
- Goal: Measure how people walk and stand to spot problems and plan treatment.
- What’s hard: Self-occlusion (body parts hiding others) with a single camera; getting true 3D angles; precise force/torque estimates often need additional sensors.
- What researchers did:
- Used one or two cameras, often placed to capture side and front views, and focused on key angles rather than the whole body.
- Found that while markerless systems are less precise than lab-grade motion capture, many still perform well enough for practical clinical measures.
- Explored ways to combine 2D video with models or depth to estimate gait parameters over time.
- Why it matters: Clinics and even small practices can assess gait more easily, and patients can be monitored outside the lab.
General lessons and trade-offs
- Accuracy vs accessibility: Multi-camera or lab systems are more accurate but cost more and require expertise. Single-camera, phone-based systems are cheaper and easier but less precise.
- 2D-to-3D “lifting” can work, but it’s an estimate. Multi-view 3D is more reliable.
- Depth cameras help but aren’t a magic fix—occlusions and noise still matter.
- For some tasks, “good enough” accuracy is already helpful, especially if the system assists a clinician rather than replacing one.
- For detailed physics (like forces and torques), current markerless methods often aren’t precise enough without extra sensors.
What’s the big picture?
- Markerless HPE can move parts of diagnosis and rehabilitation out of specialized labs and into homes or regular clinics. This supports remote care, which is more convenient and can reach more people.
- It’s cheaper, easier to set up, and less intrusive than traditional marker-based systems.
- It’s not perfect yet. Future work should:
- Improve reliability for babies and for patients with atypical body shapes.
- Make models faster and lighter for phones and low-cost devices.
- Provide better explanations so clinicians trust and understand results.
- Bridge the gap for biomechanics that need very precise 3D and force estimates.
In short, the survey shows that markerless pose estimation is already useful in several medical areas and has strong potential to make care more accessible—especially through remote monitoring and at-home assessments—while highlighting where more progress is needed to meet clinical standards fully.
Practical Applications
Immediate Applications
The following items can be deployed today using existing markerless HPE methods and workflows described in the paper. Each item notes sector(s), indicative tools/workflows, and key assumptions/dependencies that may affect feasibility.
- Home-based infant general movement assessment (GMA) via smartphone video: healthcare, mobile software; OpenPose/EfficientPose/HRNet for 2D single-view infant pose (7–25 joints), pose-derived features (e.g., HOJO2D, HOJD2D), clinician dashboard for FM scoring; assumes standardized recording protocols (camera position, lighting), domain adaptation to infants (non-adult proportions), privacy/consent, sufficient video quality and parent guidance.
- NICU limb pose tracking for preterm infants (bed-mounted RGB-D): healthcare (neonatology), hospital IT; Kinect/RealSense, Mask R-CNN or JFC-based limb pose, RGB–depth alignment, periodic recordings; assumes device placement sterility, sensor calibration, limited occlusion, workflow integration with NICU routines.
- Clinic gait and posture evaluation with two low-cost RGB cameras (sagittal/coronal): healthcare (physiotherapy/orthopedics); OpenPose or DeepLabCut for 2D single-view poses, planar constraints, knee/hip angle extraction; assumes controlled walkway, occlusion management (two-camera placement), tolerance for higher error vs. marker-based systems.
- Telerehabilitation exercise monitoring with lightweight models: healthcare (rehab), patient apps; HRNet variants (EESD), MobileNet/ESPNetv2 modules, real-time browser deployment (PoseNet via TensorFlow.js/WebGL) for feedback and error detection; assumes modern smartphones/laptops, stable connectivity, clear backgrounds, sufficient frame rate for temporal consistency.
- Edge-based rehab monitoring on dedicated devices: healthcare devices, IoT; DeepRehab-style fully convolutional 2D HPE (ResNet101 backbone) on Edge TPU with companion smartphone app; assumes procurement and maintenance of edge hardware, local power, on-device updates, environment free of heavy occlusions.
- Robot-assisted therapy safety monitoring (upper-limb): robotics in healthcare; OpenPose for upper-body joint angles at ~10 fps, depth channel mapping for 3D angles and safety thresholds; assumes accurate angle estimation in clinical range, depth sensing availability, clinical validation of thresholds, safety certification of the integrated system.
- Cloud-supported motion assessment pipeline: healthcare IT/cloud; smartphone capture → OpenPose → heuristic 3D lifting (VideoPose3D optional) → joint angles → clinician-readable report; assumes secure data transfer, GDPR/HIPAA compliance, latency tolerances for non-real-time review, robust anonymization.
- Fall detection in home security using markerless HPE: consumer/home security; top-down person detection followed by pose-based fall classification; assumes permission and privacy controls, indoor lighting adequacy, tolerance to false positives/negatives, model robustness to household occlusions.
- Sports coaching for planar tasks (jumping, running) with sagittal 2D HPE: sports/fitness industry; DeepLabCut/OpenPose for 2D poses, optical-flow patch CNNs to derive gait/jump descriptors; assumes consistent camera placement, lighting, athlete compliance, acceptance of moderate error.
- Rapid video coding in developmental research: academia (developmental psychology/neuroscience); batch pose extraction (OpenPose/HRNet), semi-supervised infant body parsing (SiamParseNet + FVGAN) to reduce manual labeling burden; assumes IRB approvals, data governance, annotator validation workflows, computational resources.
- CP risk scoring from infant pose sequences: healthcare (pediatrics/neonatology); EfficientPose + GCN/CTR-GCN on infant joint trajectories for binary/graded risk prediction with visualization of salient movements; assumes representative training data, clinician-in-the-loop review, model interpretability aids, bias assessment.
- Workplace posture screening via RGB cameras: occupational health; 2D HPE for posture metrics, ergonomic risk flags and dashboards; assumes employee consent, privacy safeguards, standardized camera setups, policy-compliant data retention.
Long-Term Applications
These opportunities require further research, scaling, validation, and/or policy frameworks before widespread deployment.
- Hospital-at-home multiview RGB-D motion labs for comprehensive remote diagnostics: healthcare; synchronized consumer-grade camera arrays (multiview RGB/RGB-D), PlaneSweepPose/FasterVoxelPose for robust 3D, clinician dashboards; depends on calibration wizards, affordable hardware bundles, standardized home protocols, service reimbursement, remote technical support.
- Plug-and-play generalizable 3D HPE in unconstrained environments: software/AI platforms; MetaPose/FLEX-style models with attention and meta-learning to reduce calibration and labeled-data needs; depends on larger, diverse training corpora (infants, elderly, post-surgery), robust domain adaptation, rigorous external validation.
- Clinically validated inverse dynamics (forces/torques) from markerless HPE: healthcare biomechanics; fusing 3D HPE with learned ground-reaction estimation or minimal sensor augmentation to replace/augment force plates; depends on multimodal fusion accuracy, longitudinal validation against gold standards, regulatory clearance.
- Closed-loop control of exoskeletons and assistive robots via markerless HPE: robotics/medical devices; low-latency multiview 3D estimation with temporal filtering (Kalman, VideoPose3D) feeding controller constraints; depends on sub-50 ms end-to-end latency, safety certs, fail-safes for occlusion, robust edge inference.
- Automated screening for neurodegenerative diseases from gait using smartphones: healthcare/public health; longitudinal gait feature extraction (stride variability, asymmetry) via HPE, population-scale risk models; depends on large, diverse cohorts, confounder control (footwear, flooring), clinical studies demonstrating predictive value.
- Standardized infant pose datasets and benchmarks for global neonatal screening: academia/healthcare consortia; shared labeled datasets (0–12 months) with pose, movement categories, outcomes; depends on multicenter collaborations, ethical approvals, de-identification infrastructure, annotation standards.
- Telehealth standards and reimbursement policy for markerless motion analysis: policy/healthcare payers; guidelines for camera setups, accuracy thresholds, clinician workflows, CPT codes; depends on cost-effectiveness evidence, stakeholder consensus, privacy frameworks, liability clarity.
- Privacy-preserving, on-device HPE with federated learning: software/edge AI; model training across clinics/devices without centralizing raw video, secure aggregation; depends on edge compute capability, federated optimization robustness, differential privacy guarantees, IT acceptance.
- Multi-modal fusion (RGB-D + IMU) to overcome occlusions and improve accuracy: medical devices/software; wearable IMUs paired with cameras, alignment algorithms for robust 3D reconstruction; depends on user compliance with wearables, synchronization protocols, fusion models.
- Patient-specific digital twins for personalized rehab planning: healthcare/digital health; biomechanical models parameterized by patient HPE data to simulate therapies and predict outcomes; depends on modeling fidelity, integration with EMR, clinical validation of benefits.
- Community-based fall detection networks with governance frameworks: public health/smart cities; ambient cameras in shared spaces with strict privacy/consent, pose-based risk alerts; depends on public acceptance, privacy-by-design architecture, clear governance and opt-out mechanisms.
- Automated clinical documentation and billing from motion metrics: healthcare operations; standardized motion summaries auto-attached to EMR, coding support for tele-rehab sessions; depends on policy alignment, auditability, interoperability (HL7/FHIR), clinician trust.
- Education and ergonomics programs leveraging low-cost HPE kits: education/occupational health; school PE assessment and workplace training using 2D HPE; depends on educator training, simple deployment guides, consent policies, curriculum integration.
Each long-term application will benefit from ongoing advances noted in the paper—better multiview triangulation (PlaneSweepPose/FasterVoxelPose), temporal consistency (VideoPose3D), transformer-based 2D HPE, and models designed for limited labels (weakly/semi-supervised)—as well as sector-specific validation, standards, and infrastructure.