Evaluation toolkit
Single Person
Evaluation is performed on sufficiently separated people only ("Single Person" subset).
Overall performance
PCKh evaluation measure
PCKh: PCK measure that uses the matching threshold as 50% of the head segment length.
PCKh @ 0.5
| Method | Head | Shoulder | Elbow | Wrist | Hip | Knee | Ankle | PCKh |
|---|---|---|---|---|---|---|---|---|
| Pishchulin et al., ICCV'13 | 74.3 | 49.0 | 40.8 | 34.1 | 36.5 | 34.4 | 35.2 | 44.1 |
| Tompson et al., NIPS'14 | 95.8 | 90.3 | 80.5 | 74.3 | 77.6 | 69.7 | 62.8 | 79.6 |
| Carreira et al., CVPR'16 | 95.7 | 91.7 | 81.7 | 72.4 | 82.8 | 73.2 | 66.4 | 81.3 |
| Tompson et al., CVPR'15 | 96.1 | 91.9 | 83.9 | 77.8 | 80.9 | 72.3 | 64.8 | 82.0 |
| Hu&Ramanan., CVPR'16 | 95.0 | 91.6 | 83.0 | 76.6 | 81.9 | 74.5 | 69.5 | 82.4 |
| Pishchulin et al., CVPR'16* | 94.1 | 90.2 | 83.4 | 77.3 | 82.6 | 75.7 | 68.6 | 82.4 |
| Lifshitz et al., ECCV'16 | 97.8 | 93.3 | 85.7 | 80.4 | 85.3 | 76.6 | 70.2 | 85.0 |
| Gkioxary et al., ECCV'16 | 96.2 | 93.1 | 86.7 | 82.1 | 85.2 | 81.4 | 74.1 | 86.1 |
| Rafi et al., BMVC'16 | 97.2 | 93.9 | 86.4 | 81.3 | 86.8 | 80.6 | 73.4 | 86.3 |
| Belagiannis&Zisserman, FG'17** | 97.7 | 95.0 | 88.2 | 83.0 | 87.9 | 82.6 | 78.4 | 88.1 |
| Insafutdinov et al., ECCV'16 | 96.8 | 95.2 | 89.3 | 84.4 | 88.4 | 83.4 | 78.0 | 88.5 |
| Wei et al., CVPR'16* | 97.8 | 95.0 | 88.7 | 84.0 | 88.4 | 82.8 | 79.4 | 88.5 |
| Bulat&Tzimiropoulos, ECCV'16 | 97.9 | 95.1 | 89.9 | 85.3 | 89.4 | 85.7 | 81.7 | 89.7 |
| Newell et al., ECCV'16 | 98.2 | 96.3 | 91.2 | 87.1 | 90.1 | 87.4 | 83.6 | 90.9 |
| Tang et al., ECCV'18 | 97.4 | 96.4 | 92.1 | 87.7 | 90.2 | 87.7 | 84.3 | 91.2 |
| Ning et al., TMM'17 | 98.1 | 96.3 | 92.2 | 87.8 | 90.6 | 87.6 | 82.7 | 91.2 |
| Luvizon et al., arXiv'17 | 98.1 | 96.6 | 92.0 | 87.5 | 90.6 | 88.0 | 82.7 | 91.2 |
| Chu et al., CVPR'17 | 98.5 | 96.3 | 91.9 | 88.1 | 90.6 | 88.0 | 85.0 | 91.5 |
| Chou et al., arXiv'17 | 98.2 | 96.8 | 92.2 | 88.0 | 91.3 | 89.1 | 84.9 | 91.8 |
| Chen et al., ICCV'17 | 98.1 | 96.5 | 92.5 | 88.5 | 90.2 | 89.6 | 86.0 | 91.9 |
| Yang et al., ICCV'17 | 98.5 | 96.7 | 92.5 | 88.7 | 91.1 | 88.6 | 86.0 | 92.0 |
| Ke et al., ECCV'18 | 98.5 | 96.8 | 92.7 | 88.4 | 90.6 | 89.3 | 86.3 | 92.1 |
| Tang et al., ECCV'18 | 98.4 | 96.9 | 92.6 | 88.7 | 91.8 | 89.4 | 86.2 | 92.3 |
| Zhang et al., arXiv'19 | 98.6 | 97.0 | 92.8 | 88.8 | 91.7 | 89.8 | 86.6 | 92.5 |
| Su et al., arXiv'19*** | 98.7 | 97.5 | 94.3 | 90.7 | 93.4 | 92.2 | 88.4 | 93.9 |
| Bulat et al., FG'2020*** | 98.8 | 97.5 | 94.4 | 91.2 | 93.2 | 92.2 | 89.3 | 94.1 |
* methods trained when adding LSP training and LSP extended sets to the MPII training set
** methods trained on MS COCO training and finetuned on MPII training set
*** methods trained on HSSK training and MPII training sets
Performance vs. complexity measures
Performance by pose
Shown are medoids of body pose clusters aranged according to pose complexity.
Performance by viewpoint and activity
Shown are medoids of 3D torso orientation clusters arranged according to the cluster size:
cluster 1 represents upright frontal torso,
cluster 2 represents slightly rotated upright backward facing torso,
cluster 6 represents torso bending towards the camera, etc.
Multi-Person
Evaluation is performed on groups of multiple people ("Multi-Person" subset).
mAP evaluation measure
Mean Average Precision (mAP) based evaluation of body joint predictions forming cosistent body pose configurations.
Performance on full set
mAP @ 0.5
| Method | Head | Shoulder | Elbow | Wrist | Hip | Knee | Ankle | mAP |
|---|---|---|---|---|---|---|---|---|
| Iqbal&Gall, ECCVw'16 | 58.4 | 53.9 | 44.5 | 35.0 | 42.2 | 36.7 | 31.1 | 43.1 |
| Insafutdinov et al., ECCV'16* | 78.4 | 72.5 | 60.2 | 51.0 | 57.2 | 52.0 | 45.4 | 59.5 |
| Insafutdinov et al., arXiv'16a* | 89.4 | 84.5 | 70.4 | 59.3 | 68.9 | 62.7 | 54.6 | 70.0 |
| Levinkov et al., CVPR'17 | 89.8 | 85.2 | 71.8 | 59.6 | 71.1 | 63.0 | 53.5 | 70.6 |
| Varadarajan et al., arXiv'17 | 92.1 | 85.9 | 72.9 | 61.7 | 72.0 | 64.6 | 56.6 | 72.2 |
| Insafutdinov et al., CVPR'17 | 88.8 | 87.0 | 75.9 | 64.9 | 74.2 | 68.8 | 60.5 | 74.3 |
| Cao et al., CVPR'17 | 91.2 | 87.6 | 77.7 | 66.8 | 75.4 | 68.9 | 61.7 | 75.6 |
| Fang et al., arXiv'16 | 88.4 | 86.5 | 78.6 | 70.4 | 74.4 | 73.0 | 65.8 | 76.7 |
| Newell et al., NIPS'17 | 92.1 | 89.3 | 78.9 | 69.8 | 76.2 | 71.6 | 64.7 | 77.5 |
| Fieraru et al., CVPRw'18 | 91.8 | 89.5 | 80.4 | 69.6 | 77.3 | 71.7 | 65.5 | 78.0 |
* methods trained when adding LSP training and LSP extended sets to the MPII training set
Performance on subset of 288 testing images
mAP @ 0.5
| Method | Head | Shoulder | Elbow | Wrist | Hip | Knee | Ankle | mAP |
|---|---|---|---|---|---|---|---|---|
| Pishchulin et al., CVPR'16* | 73.1 | 71.7 | 58.0 | 39.9 | 56.1 | 43.5 | 31.9 | 53.5 |
| Iqbal&Gall, ECCVw'16 | 70.0 | 65.2 | 56.4 | 46.1 | 52.7 | 47.9 | 44.5 | 54.7 |
| Insafutdinov et al., ECCV'16* | 87.9 | 84.0 | 71.9 | 63.9 | 68.8 | 63.8 | 58.1 | 71.2 |
| Newell&Deng, arXiv'16 | 91.5 | 87.2 | 75.9 | 65.4 | 72.2 | 67.0 | 62.1 | 74.5 |
| Insafutdinov et al., arXiv'16a* | 92.1 | 88.5 | 76.4 | 67.8 | 73.6 | 68.7 | 62.3 | 75.6 |
| Varadarajan et al., arXiv'17 | 92.9 | 88.8 | 77.7 | 67.8 | 74.6 | 67.0 | 63.8 | 76.1 |
| Cao et al., CVPR'17 | 92.9 | 91.3 | 82.3 | 72.6 | 76.0 | 70.9 | 66.8 | 79.0 |
| Fang et al., arXiv'16 | 89.3 | 88.1 | 80.7 | 75.5 | 73.7 | 76.7 | 70.0 | 79.1 |
| Insafutdinov et al., CVPR'17 | 92.2 | 91.3 | 80.8 | 71.4 | 79.1 | 72.6 | 67.8 | 79.3 |
* methods trained when adding LSP training and LSP extended sets to the MPII training set