WorkMotionCase file 01, 1 of 3 in this programme
Markerless 3D pose with public models
Experiments with public pose and 3D-lifting models: extending a 2D pose model without breaking it, lifting 2D points to 3D, and checking what the scores can and cannot see.
- Where
- IdeasLab Formosa, 2025 — present
- My part
- Research engineer: experiments, training, evaluation
- Built with
- PyTorch, 2D pose estimation, 2D-to-3D lifting, Multi-view geometry, Evaluation design
Public pose models are trained on everyday images and studio motion. Applied to new kinds of footage, they need new keypoints and better behaviour on fast motion, and the obvious way to get either, fine-tuning, can quietly damage what already worked while the validation score improves.
- 0.000
- change on the original keypoints after adding new ones
- 0.834 vs 0.733
- AP on the new keypoints: side branch against a shared head
6 more sections
The rest of this study is private
Reach out on LinkedIn or by email for research collaboration or a deeper technical discussion.
The failure the validation score missed
Adding keypoints without touching the old ones
Reading the shape of the error
Labelling what the data does and doesn't know
Lifting to 3D, and what the public numbers say
Testing 3D models against independent measurements
What it taught
A pose model scored better while getting worse. Its validation set held only the narrow footage it was tuned on, where the new bias happened to be right.
Freeze the original model's weights and its output cannot change, which a weight diff proves. A penalty term in the loss only makes a change less likely.
Error that grows toward the edge of what the network can see means it cannot see far enough; a bigger model will not help. Error that stays flat across sessions but groups by camera setup comes from how points were labelled, not from scale.