# Model evaluation report

Split: `test` | bootstrap: 1000 resamples, seed 42 | smoothing window: 12

## Summary

| model | accuracy [95% CI] | within-one accuracy [95% CI] | training time (min) |
|---|---|---|---|
| siglip2_base_224 | 0.6680 [0.6332, 0.7038] | 0.9497 [0.9381, 0.9619] | 25.3 |
| siglip2_base_384 | 0.6893 [0.6599, 0.7223] | 0.9561 [0.9461, 0.9661] | 102.4 |
| siglip2_so400m_384 | 0.6807 [0.6476, 0.7108] | 0.9382 [0.9237, 0.9514] | 447.9 |
| convnext_large | 0.6400 [0.6050, 0.6750] | 0.9358 [0.9193, 0.9502] | 62.5 |
| swin_b | 0.6516 [0.6161, 0.6857] | 0.9420 [0.9269, 0.9548] | 68.7 |
| resnet50 | 0.5984 [0.5629, 0.6339] | 0.9149 [0.8962, 0.9325] | 17.3 |
| vit_b_16 | 0.5743 [0.5359, 0.6116] | 0.9161 [0.8986, 0.9329] | 51.1 |
| vit_l_16 | 0.6319 [0.5976, 0.6677] | 0.9435 [0.9274, 0.9580] | 92.2 |

## Per-model breakdowns

### siglip2_base_224

**Total** — accuracy 0.6680 95% CI [0.6332, 0.7038] (27952 frames, 138 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.5240 | [0.3521, 0.6619] | 0.3306 | [0.1940, 0.4596] | 0.4055 | [0.2568, 0.5201] |
| l | 0.6340 | [0.5695, 0.6921] | 0.7978 | [0.7422, 0.8456] | 0.7065 | [0.6556, 0.7500] |
| m | 0.7215 | [0.6625, 0.7789] | 0.8272 | [0.7809, 0.8727] | 0.7707 | [0.7259, 0.8133] |
| mcu | 0.6504 | [0.5647, 0.7387] | 0.6480 | [0.5677, 0.7222] | 0.6492 | [0.5817, 0.7128] |
| ml | 0.6264 | [0.5275, 0.7107] | 0.4335 | [0.3530, 0.5197] | 0.5124 | [0.4283, 0.5899] |
| na | 0.9399 | [0.6554, 1.0000] | 1.0000 | [1.0000, 1.0000] | 0.9690 | [0.7918, 1.0000] |
| xcu | 0.2857 | [0.0000, 0.8425] | 0.1081 | [0.0000, 0.3478] | 0.1569 | [0.0000, 0.4325] |
| xls | 0.6170 | [0.4770, 0.7455] | 0.2565 | [0.1678, 0.3561] | 0.3624 | [0.2572, 0.4674] |

**Animated** — accuracy 0.6061 95% CI [0.5526, 0.6603] (9526 frames, 41 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.4218 | [0.2048, 0.6132] | 0.3080 | [0.1314, 0.4788] | 0.3560 | [0.1645, 0.5030] |
| l | 0.6461 | [0.5673, 0.7223] | 0.8262 | [0.7589, 0.8813] | 0.7251 | [0.6614, 0.7804] |
| m | 0.5763 | [0.4760, 0.6648] | 0.7735 | [0.6870, 0.8423] | 0.6605 | [0.5709, 0.7321] |
| mcu | 0.5893 | [0.4273, 0.7560] | 0.4681 | [0.3432, 0.5985] | 0.5217 | [0.3973, 0.6309] |
| ml | 0.5558 | [0.4177, 0.6912] | 0.2657 | [0.1785, 0.3754] | 0.3595 | [0.2579, 0.4690] |
| na | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.6303 | [0.4350, 0.7957] | 0.3122 | [0.1748, 0.4916] | 0.4175 | [0.2604, 0.5723] |

**Live-action** — accuracy 0.6999 95% CI [0.6571, 0.7423] (18426 frames, 97 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.5763 | [0.3656, 0.7255] | 0.3400 | [0.1722, 0.5394] | 0.4277 | [0.2377, 0.5923] |
| l | 0.6205 | [0.5205, 0.7094] | 0.7669 | [0.6813, 0.8440] | 0.6860 | [0.6072, 0.7502] |
| m | 0.7726 | [0.7030, 0.8361] | 0.8426 | [0.7850, 0.8904] | 0.8061 | [0.7522, 0.8545] |
| mcu | 0.6670 | [0.5569, 0.7680] | 0.7137 | [0.6178, 0.8022] | 0.6895 | [0.6047, 0.7543] |
| ml | 0.6484 | [0.5361, 0.7496] | 0.5220 | [0.4152, 0.6381] | 0.5784 | [0.4770, 0.6704] |
| na | 0.9399 | [0.6389, 1.0000] | 1.0000 | [1.0000, 1.0000] | 0.9690 | [0.7796, 1.0000] |
| xcu | 0.5714 | [0.0000, 1.0000] | 0.1667 | [0.0000, 0.5581] | 0.2581 | [0.0000, 0.6400] |
| xls | 0.5982 | [0.4110, 0.7724] | 0.2026 | [0.1133, 0.3041] | 0.3027 | [0.1792, 0.4176] |

### siglip2_base_384

**Total** — accuracy 0.6893 95% CI [0.6599, 0.7223] (27952 frames, 138 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.5825 | [0.4190, 0.7229] | 0.3428 | [0.2074, 0.5036] | 0.4316 | [0.2840, 0.5723] |
| l | 0.6342 | [0.5798, 0.6930] | 0.8331 | [0.7807, 0.8769] | 0.7202 | [0.6762, 0.7621] |
| m | 0.7575 | [0.7033, 0.8083] | 0.8403 | [0.7941, 0.8827] | 0.7968 | [0.7563, 0.8397] |
| mcu | 0.6683 | [0.5840, 0.7559] | 0.6732 | [0.5940, 0.7465] | 0.6707 | [0.6068, 0.7317] |
| ml | 0.6698 | [0.5742, 0.7557] | 0.4904 | [0.4152, 0.5786] | 0.5662 | [0.4898, 0.6386] |
| na | 0.9399 | [0.4762, 1.0000] | 1.0000 | [0.9750, 1.0000] | 0.9690 | [0.6398, 1.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.5788 | [0.4000, 0.7167] | 0.1843 | [0.1056, 0.2796] | 0.2796 | [0.1712, 0.3969] |

**Animated** — accuracy 0.6336 95% CI [0.5844, 0.6849] (9526 frames, 41 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.5057 | [0.2712, 0.7208] | 0.3045 | [0.1477, 0.4691] | 0.3801 | [0.1910, 0.5265] |
| l | 0.6504 | [0.5751, 0.7258] | 0.8688 | [0.8160, 0.9168] | 0.7439 | [0.6854, 0.7973] |
| m | 0.6237 | [0.5237, 0.7194] | 0.7720 | [0.6965, 0.8373] | 0.6900 | [0.6064, 0.7661] |
| mcu | 0.6027 | [0.4475, 0.7503] | 0.5350 | [0.4326, 0.6386] | 0.5668 | [0.4525, 0.6617] |
| ml | 0.6138 | [0.4622, 0.7591] | 0.2987 | [0.2094, 0.3976] | 0.4019 | [0.3004, 0.5068] |
| na | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.6765 | [0.4721, 0.8422] | 0.3091 | [0.1594, 0.4920] | 0.4243 | [0.2444, 0.5922] |

**Live-action** — accuracy 0.7181 95% CI [0.6765, 0.7619] (18426 frames, 97 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.6152 | [0.3867, 0.7781] | 0.3586 | [0.1876, 0.5800] | 0.4531 | [0.2684, 0.6395] |
| l | 0.6160 | [0.5213, 0.7013] | 0.7944 | [0.7108, 0.8677] | 0.6939 | [0.6202, 0.7612] |
| m | 0.8017 | [0.7413, 0.8558] | 0.8598 | [0.7990, 0.9094] | 0.8297 | [0.7808, 0.8727] |
| mcu | 0.6885 | [0.5830, 0.7947] | 0.7236 | [0.6273, 0.8107] | 0.7056 | [0.6234, 0.7750] |
| ml | 0.6864 | [0.5617, 0.7796] | 0.5914 | [0.4887, 0.6881] | 0.6354 | [0.5389, 0.7186] |
| na | 0.9399 | [0.6558, 1.0000] | 1.0000 | [1.0000, 1.0000] | 0.9690 | [0.7921, 1.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.3443 | [0.1228, 0.5420] | 0.0635 | [0.0155, 0.1194] | 0.1072 | [0.0283, 0.1903] |

### siglip2_so400m_384

**Total** — accuracy 0.6807 95% CI [0.6476, 0.7108] (27952 frames, 138 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.4566 | [0.3167, 0.5936] | 0.4793 | [0.3348, 0.5971] | 0.4677 | [0.3386, 0.5783] |
| l | 0.6262 | [0.5629, 0.6832] | 0.7797 | [0.7246, 0.8253] | 0.6946 | [0.6444, 0.7392] |
| m | 0.7818 | [0.7265, 0.8332] | 0.7708 | [0.7171, 0.8208] | 0.7763 | [0.7281, 0.8170] |
| mcu | 0.6684 | [0.5911, 0.7447] | 0.6179 | [0.5377, 0.6976] | 0.6422 | [0.5799, 0.6987] |
| ml | 0.6372 | [0.5426, 0.7193] | 0.6355 | [0.5575, 0.7101] | 0.6363 | [0.5590, 0.6989] |
| na | 0.9942 | [0.9474, 1.0000] | 1.0000 | [1.0000, 1.0000] | 0.9971 | [0.9730, 1.0000] |
| xcu | 0.3261 | [0.0000, 0.8571] | 0.2027 | [0.0000, 0.5172] | 0.2500 | [0.0000, 0.5661] |
| xls | 0.6189 | [0.4727, 0.7546] | 0.2345 | [0.1494, 0.3251] | 0.3401 | [0.2317, 0.4446] |

**Animated** — accuracy 0.6363 95% CI [0.5858, 0.6841] (9526 frames, 41 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.3393 | [0.1875, 0.4588] | 0.3910 | [0.2459, 0.5218] | 0.3633 | [0.2227, 0.4553] |
| l | 0.6569 | [0.5710, 0.7266] | 0.8247 | [0.7679, 0.8737] | 0.7313 | [0.6633, 0.7830] |
| m | 0.6345 | [0.5414, 0.7171] | 0.7412 | [0.6632, 0.8102] | 0.6837 | [0.6073, 0.7482] |
| mcu | 0.5946 | [0.4437, 0.7209] | 0.4934 | [0.3556, 0.6255] | 0.5393 | [0.4114, 0.6435] |
| ml | 0.6635 | [0.5366, 0.7702] | 0.4403 | [0.3382, 0.5500] | 0.5293 | [0.4289, 0.6162] |
| na | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.6982 | [0.4918, 0.8620] | 0.3226 | [0.1795, 0.4812] | 0.4413 | [0.2694, 0.5856] |

**Live-action** — accuracy 0.7036 95% CI [0.6610, 0.7461] (18426 frames, 97 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.5121 | [0.3229, 0.6814] | 0.5157 | [0.3302, 0.7055] | 0.5139 | [0.3372, 0.6676] |
| l | 0.5923 | [0.4943, 0.6818] | 0.7309 | [0.6413, 0.8112] | 0.6543 | [0.5739, 0.7232] |
| m | 0.8345 | [0.7668, 0.8928] | 0.7793 | [0.7058, 0.8399] | 0.8060 | [0.7474, 0.8521] |
| mcu | 0.6917 | [0.5969, 0.7924] | 0.6633 | [0.5760, 0.7618] | 0.6772 | [0.6082, 0.7414] |
| ml | 0.6293 | [0.5067, 0.7277] | 0.7384 | [0.6479, 0.8182] | 0.6795 | [0.5885, 0.7514] |
| na | 1.0000 | [0.9750, 1.0000] | 1.0000 | [0.9750, 1.0000] | 1.0000 | [0.9750, 1.0000] |
| xcu | 0.4839 | [0.0000, 1.0000] | 0.3125 | [0.0000, 0.6522] | 0.3797 | [0.0000, 0.7143] |
| xls | 0.5000 | [0.2786, 0.6953] | 0.1492 | [0.0585, 0.2416] | 0.2298 | [0.0982, 0.3477] |

### convnext_large

**Total** — accuracy 0.6400 95% CI [0.6050, 0.6750] (27952 frames, 138 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.4414 | [0.2837, 0.5859] | 0.3428 | [0.1920, 0.5183] | 0.3859 | [0.2324, 0.5289] |
| l | 0.6104 | [0.5487, 0.6685] | 0.7445 | [0.6866, 0.7945] | 0.6708 | [0.6216, 0.7149] |
| m | 0.6933 | [0.6312, 0.7507] | 0.8227 | [0.7748, 0.8632] | 0.7525 | [0.7037, 0.7957] |
| mcu | 0.6407 | [0.5463, 0.7326] | 0.5566 | [0.4732, 0.6331] | 0.5957 | [0.5238, 0.6560] |
| ml | 0.5839 | [0.4875, 0.6736] | 0.4408 | [0.3447, 0.5467] | 0.5023 | [0.4102, 0.5905] |
| na | 0.9829 | [0.8571, 1.0000] | 1.0000 | [1.0000, 1.0000] | 0.9914 | [0.9231, 1.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.5785 | [0.4094, 0.7311] | 0.2094 | [0.1186, 0.3202] | 0.3075 | [0.1842, 0.4327] |

**Animated** — accuracy 0.5663 95% CI [0.5151, 0.6216] (9526 frames, 41 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.3954 | [0.1511, 0.6001] | 0.3599 | [0.1696, 0.5124] | 0.3768 | [0.1757, 0.5063] |
| l | 0.6345 | [0.5592, 0.7126] | 0.7848 | [0.7118, 0.8537] | 0.7017 | [0.6359, 0.7597] |
| m | 0.5202 | [0.4132, 0.6194] | 0.7261 | [0.6249, 0.8089] | 0.6061 | [0.5037, 0.6929] |
| mcu | 0.5734 | [0.4065, 0.7525] | 0.4195 | [0.2904, 0.5419] | 0.4845 | [0.3582, 0.5941] |
| ml | 0.4109 | [0.2727, 0.5364] | 0.2100 | [0.1231, 0.3056] | 0.2779 | [0.1773, 0.3772] |
| na | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.6532 | [0.4055, 0.8313] | 0.3039 | [0.1481, 0.5023] | 0.4148 | [0.2245, 0.6001] |

**Live-action** — accuracy 0.6780 95% CI [0.6351, 0.7209] (18426 frames, 97 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.4653 | [0.2482, 0.6373] | 0.3357 | [0.1513, 0.5688] | 0.3900 | [0.1900, 0.5755] |
| l | 0.5835 | [0.4952, 0.6723] | 0.7008 | [0.6130, 0.7768] | 0.6368 | [0.5638, 0.6990] |
| m | 0.7546 | [0.6874, 0.8154] | 0.8503 | [0.7926, 0.8973] | 0.7996 | [0.7461, 0.8445] |
| mcu | 0.6602 | [0.5456, 0.7790] | 0.6067 | [0.5054, 0.7077] | 0.6324 | [0.5456, 0.7141] |
| ml | 0.6367 | [0.5198, 0.7397] | 0.5623 | [0.4440, 0.6811] | 0.5972 | [0.4902, 0.6927] |
| na | 0.9829 | [0.8565, 1.0000] | 1.0000 | [1.0000, 1.0000] | 0.9914 | [0.9227, 1.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.4500 | [0.2362, 0.6281] | 0.1179 | [0.0394, 0.2045] | 0.1869 | [0.0682, 0.2940] |

### swin_b

**Total** — accuracy 0.6516 95% CI [0.6161, 0.6857] (27952 frames, 138 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.5107 | [0.3460, 0.6433] | 0.3610 | [0.2150, 0.5355] | 0.4230 | [0.2809, 0.5624] |
| l | 0.6132 | [0.5538, 0.6691] | 0.7334 | [0.6684, 0.7882] | 0.6680 | [0.6196, 0.7120] |
| m | 0.7277 | [0.6648, 0.7842] | 0.7945 | [0.7333, 0.8425] | 0.7596 | [0.7093, 0.8045] |
| mcu | 0.6433 | [0.5418, 0.7438] | 0.6173 | [0.5453, 0.6855] | 0.6301 | [0.5608, 0.6917] |
| ml | 0.5852 | [0.4982, 0.6665] | 0.4996 | [0.4007, 0.5946] | 0.5390 | [0.4540, 0.6156] |
| na | 0.9773 | [0.8059, 1.0000] | 1.0000 | [1.0000, 1.0000] | 0.9885 | [0.8925, 1.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.5551 | [0.4224, 0.6729] | 0.2760 | [0.1785, 0.3772] | 0.3687 | [0.2595, 0.4694] |

**Animated** — accuracy 0.5567 95% CI [0.5062, 0.6057] (9526 frames, 41 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.3590 | [0.1867, 0.5297] | 0.3391 | [0.1796, 0.4734] | 0.3488 | [0.1904, 0.4604] |
| l | 0.6175 | [0.5333, 0.6978] | 0.7457 | [0.6632, 0.8133] | 0.6756 | [0.6075, 0.7341] |
| m | 0.5435 | [0.4430, 0.6499] | 0.6782 | [0.5883, 0.7626] | 0.6034 | [0.5129, 0.6880] |
| mcu | 0.5778 | [0.4515, 0.7166] | 0.4326 | [0.3173, 0.5487] | 0.4948 | [0.3866, 0.5968] |
| ml | 0.4091 | [0.2716, 0.5289] | 0.2651 | [0.1544, 0.4182] | 0.3217 | [0.2034, 0.4492] |
| na | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.5801 | [0.3938, 0.7449] | 0.3465 | [0.1812, 0.5339] | 0.4339 | [0.2588, 0.5988] |

**Live-action** — accuracy 0.7006 95% CI [0.6595, 0.7411] (18426 frames, 97 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.6080 | [0.3846, 0.7793] | 0.3700 | [0.1843, 0.5927] | 0.4600 | [0.2544, 0.6339] |
| l | 0.6085 | [0.5120, 0.6939] | 0.7201 | [0.6269, 0.8009] | 0.6596 | [0.5803, 0.7264] |
| m | 0.7905 | [0.7274, 0.8513] | 0.8277 | [0.7685, 0.8830] | 0.8087 | [0.7564, 0.8538] |
| mcu | 0.6606 | [0.5491, 0.7741] | 0.6848 | [0.6056, 0.7676] | 0.6725 | [0.5942, 0.7423] |
| ml | 0.6477 | [0.5470, 0.7435] | 0.6232 | [0.5090, 0.7192] | 0.6352 | [0.5400, 0.7184] |
| na | 0.9773 | [0.8400, 1.0000] | 1.0000 | [1.0000, 1.0000] | 0.9885 | [0.9130, 1.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.5189 | [0.3150, 0.6921] | 0.2077 | [0.0967, 0.3213] | 0.2966 | [0.1478, 0.4267] |

### resnet50

**Total** — accuracy 0.5984 95% CI [0.5629, 0.6339] (27952 frames, 138 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.4379 | [0.2660, 0.6031] | 0.2922 | [0.1681, 0.4617] | 0.3505 | [0.2124, 0.4934] |
| l | 0.5968 | [0.5358, 0.6594] | 0.7181 | [0.6613, 0.7741] | 0.6518 | [0.6020, 0.6992] |
| m | 0.6566 | [0.5899, 0.7165] | 0.7440 | [0.6918, 0.7883] | 0.6975 | [0.6443, 0.7444] |
| mcu | 0.6018 | [0.5037, 0.6958] | 0.4959 | [0.4075, 0.5910] | 0.5438 | [0.4707, 0.6194] |
| ml | 0.4978 | [0.3935, 0.5876] | 0.4119 | [0.3311, 0.5006] | 0.4508 | [0.3635, 0.5280] |
| na | 0.8542 | [0.4184, 1.0000] | 0.9535 | [0.8095, 1.0000] | 0.9011 | [0.5873, 1.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.5042 | [0.3547, 0.6457] | 0.3041 | [0.1970, 0.4130] | 0.3794 | [0.2578, 0.4818] |

**Animated** — accuracy 0.5201 95% CI [0.4732, 0.5676] (9526 frames, 41 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.3828 | [0.1902, 0.5440] | 0.2768 | [0.1591, 0.3671] | 0.3213 | [0.1799, 0.4153] |
| l | 0.5939 | [0.5145, 0.6703] | 0.7241 | [0.6477, 0.7945] | 0.6526 | [0.5831, 0.7096] |
| m | 0.4881 | [0.3859, 0.5837] | 0.6820 | [0.6000, 0.7556] | 0.5690 | [0.4747, 0.6493] |
| mcu | 0.5117 | [0.3339, 0.6789] | 0.2877 | [0.1727, 0.4016] | 0.3684 | [0.2397, 0.4819] |
| ml | 0.3978 | [0.2370, 0.5508] | 0.2280 | [0.1423, 0.3066] | 0.2898 | [0.1826, 0.3847] |
| na | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.4725 | [0.2486, 0.6774] | 0.2862 | [0.1397, 0.4691] | 0.3564 | [0.1849, 0.5297] |

**Live-action** — accuracy 0.6389 95% CI [0.5954, 0.6836] (18426 frames, 97 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.4634 | [0.2299, 0.6904] | 0.2986 | [0.1244, 0.5296] | 0.3632 | [0.1676, 0.5582] |
| l | 0.6000 | [0.5070, 0.6970] | 0.7116 | [0.6243, 0.7908] | 0.6510 | [0.5760, 0.7194] |
| m | 0.7202 | [0.6454, 0.7886] | 0.7617 | [0.6996, 0.8170] | 0.7404 | [0.6815, 0.7915] |
| mcu | 0.6219 | [0.5252, 0.7249] | 0.5720 | [0.4700, 0.6783] | 0.5959 | [0.5105, 0.6745] |
| ml | 0.5292 | [0.4122, 0.6383] | 0.5089 | [0.3989, 0.6238] | 0.5189 | [0.4155, 0.6085] |
| na | 0.8542 | [0.4184, 1.0000] | 0.9535 | [0.8095, 1.0000] | 0.9011 | [0.5899, 1.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.5352 | [0.3212, 0.6935] | 0.3216 | [0.1720, 0.4645] | 0.4018 | [0.2315, 0.5307] |

### vit_b_16

**Total** — accuracy 0.5743 95% CI [0.5359, 0.6116] (27952 frames, 138 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.3902 | [0.2359, 0.5535] | 0.3054 | [0.1897, 0.4343] | 0.3426 | [0.2216, 0.4558] |
| l | 0.5769 | [0.5095, 0.6405] | 0.6208 | [0.5588, 0.6734] | 0.5981 | [0.5387, 0.6465] |
| m | 0.6277 | [0.5619, 0.6878] | 0.7944 | [0.7453, 0.8368] | 0.7013 | [0.6469, 0.7471] |
| mcu | 0.5948 | [0.5185, 0.6838] | 0.4786 | [0.3903, 0.5750] | 0.5304 | [0.4521, 0.6040] |
| ml | 0.4337 | [0.3346, 0.5282] | 0.3237 | [0.2442, 0.4017] | 0.3707 | [0.2858, 0.4424] |
| na | 1.0000 | [1.0000, 1.0000] | 1.0000 | [1.0000, 1.0000] | 1.0000 | [1.0000, 1.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.4789 | [0.3328, 0.6073] | 0.2903 | [0.1815, 0.4027] | 0.3615 | [0.2416, 0.4712] |

**Animated** — accuracy 0.4665 95% CI [0.4080, 0.5254] (9526 frames, 41 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.2143 | [0.0698, 0.4012] | 0.2803 | [0.1244, 0.4233] | 0.2429 | [0.0922, 0.3728] |
| l | 0.5694 | [0.4784, 0.6582] | 0.6054 | [0.5229, 0.6819] | 0.5869 | [0.5074, 0.6539] |
| m | 0.4372 | [0.3307, 0.5438] | 0.6332 | [0.5310, 0.7221] | 0.5172 | [0.4147, 0.6039] |
| mcu | 0.4339 | [0.2850, 0.5802] | 0.3323 | [0.2031, 0.4695] | 0.3764 | [0.2483, 0.4944] |
| ml | 0.3110 | [0.1966, 0.4305] | 0.1827 | [0.1108, 0.2658] | 0.2302 | [0.1450, 0.3195] |
| na | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.4717 | [0.2276, 0.6749] | 0.3205 | [0.1438, 0.5182] | 0.3817 | [0.1836, 0.5522] |

**Live-action** — accuracy 0.6300 95% CI [0.5853, 0.6693] (18426 frames, 97 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.5581 | [0.3309, 0.7540] | 0.3157 | [0.1730, 0.4880] | 0.4033 | [0.2300, 0.5570] |
| l | 0.5849 | [0.4951, 0.6790] | 0.6376 | [0.5448, 0.7202] | 0.6101 | [0.5268, 0.6762] |
| m | 0.6928 | [0.6157, 0.7589] | 0.8405 | [0.7959, 0.8812] | 0.7595 | [0.7019, 0.8071] |
| mcu | 0.6498 | [0.5577, 0.7497] | 0.5320 | [0.4253, 0.6466] | 0.5850 | [0.4965, 0.6696] |
| ml | 0.4796 | [0.3500, 0.5850] | 0.3979 | [0.2894, 0.5069] | 0.4349 | [0.3231, 0.5240] |
| na | 1.0000 | [1.0000, 1.0000] | 1.0000 | [1.0000, 1.0000] | 1.0000 | [1.0000, 1.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.4878 | [0.3068, 0.6531] | 0.2611 | [0.1476, 0.3902] | 0.3401 | [0.2058, 0.4703] |

### vit_l_16

**Total** — accuracy 0.6319 95% CI [0.5976, 0.6677] (27952 frames, 138 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.4643 | [0.2850, 0.6463] | 0.3751 | [0.2266, 0.5225] | 0.4150 | [0.2646, 0.5511] |
| l | 0.5988 | [0.5357, 0.6589] | 0.6964 | [0.6257, 0.7622] | 0.6439 | [0.5889, 0.6942] |
| m | 0.6910 | [0.6329, 0.7427] | 0.8334 | [0.7916, 0.8678] | 0.7555 | [0.7092, 0.7930] |
| mcu | 0.6902 | [0.6045, 0.7838] | 0.5350 | [0.4555, 0.6130] | 0.6027 | [0.5346, 0.6705] |
| ml | 0.5352 | [0.4441, 0.6232] | 0.4478 | [0.3658, 0.5309] | 0.4876 | [0.4080, 0.5592] |
| na | 1.0000 | [1.0000, 1.0000] | 1.0000 | [1.0000, 1.0000] | 1.0000 | [1.0000, 1.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.5275 | [0.3482, 0.6932] | 0.2115 | [0.1199, 0.3166] | 0.3019 | [0.1805, 0.4210] |

**Animated** — accuracy 0.5607 95% CI [0.5108, 0.6138] (9526 frames, 41 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.2980 | [0.0925, 0.5633] | 0.3599 | [0.1433, 0.5653] | 0.3260 | [0.1139, 0.5056] |
| l | 0.6265 | [0.5478, 0.7063] | 0.7588 | [0.6732, 0.8323] | 0.6864 | [0.6161, 0.7454] |
| m | 0.5306 | [0.4156, 0.6371] | 0.6943 | [0.6018, 0.7741] | 0.6015 | [0.4966, 0.6921] |
| mcu | 0.6519 | [0.5189, 0.7785] | 0.3739 | [0.2432, 0.5059] | 0.4752 | [0.3401, 0.5947] |
| ml | 0.4330 | [0.2996, 0.5711] | 0.3057 | [0.2116, 0.4030] | 0.3584 | [0.2573, 0.4449] |
| na | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.5744 | [0.3262, 0.7763] | 0.2851 | [0.1273, 0.4828] | 0.3811 | [0.1893, 0.5678] |

**Live-action** — accuracy 0.6687 95% CI [0.6256, 0.7129] (18426 frames, 97 movies)

| class | P | 95% CI | R | 95% CI | F1 | 95% CI |
|---|---|---|---|---|---|---|
| cu | 0.5933 | [0.3314, 0.8321] | 0.3814 | [0.1919, 0.5829] | 0.4643 | [0.2624, 0.6437] |
| l | 0.5661 | [0.4642, 0.6617] | 0.6287 | [0.5156, 0.7295] | 0.5957 | [0.5031, 0.6735] |
| m | 0.7420 | [0.6759, 0.8053] | 0.8731 | [0.8277, 0.9106] | 0.8022 | [0.7547, 0.8445] |
| mcu | 0.6997 | [0.5962, 0.8015] | 0.5938 | [0.5007, 0.6887] | 0.6424 | [0.5648, 0.7099] |
| ml | 0.5771 | [0.4611, 0.6895] | 0.5226 | [0.4144, 0.6314] | 0.5485 | [0.4473, 0.6420] |
| na | 1.0000 | [1.0000, 1.0000] | 1.0000 | [1.0000, 1.0000] | 1.0000 | [1.0000, 1.0000] |
| xcu | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] | 0.0000 | [0.0000, 0.0000] |
| xls | 0.4542 | [0.2128, 0.7023] | 0.1401 | [0.0471, 0.2543] | 0.2142 | [0.0807, 0.3488] |