Advent of Alpha Day 18: Gait, Weight & Computer Vision

ITV Racing love a paddock judge. I think I get why: one of the best reasons to actually go racing is to see lines of muscle conformation, how a head is carried, where the feet are being placed, etc., so by offering that via an expert is helpful.

It also feels a bit mysterious: how many people know how to look at a group of horses and tell you based on how they’re walking around a paddock, how well they’ll race each other? Not many. I’m not sure even the people who ITV employ, if we’re honest.

But we know from what we hear from successful syndicates like Bill Benter’s Hong Kong operation that video analysis is key.

20 years ago the idea of using computer vision (CV) to assess events was considered ridiculous. Even seven years ago, when I discussed the idea of using CV to look at how football players moved to assess fitness it seemed too hard a problem.

Today, this is doable using consumer-grade hardware across a range of sports. There are some good CV examples coming out for table tennis which means tennis can’t be far behind (take the video feed and using nothing but TV pictures, figure out speeds, angles, a full kinematic model of a player and how they’re evolving through the game as energy and mindsets change, no Hawkeye needed), and cricket is an obvious target too.

I also suspect we can now do more/better analysis in horse racing from paddock feeds (accuracy of foot placement is often all you need to know how well conformed a horse is), but I think we can also start to get a better understanding of weight gain/loss between races, whether a winter or summer coat is in, head carriage and other factors that paddock experts look for.

I don’t even think you need a labelled data set for those features, I think you can just take some CV models, create tabulated data using race outcome as labels in a supervised learning model. Never tried it, but I’d be amazed if this didn’t yield.

It all sounds ridiculous, or too much hard work, but I’ve seen much more complex systems than what I just described working quite nicely in the CV space.

I’d start small in terms of spending: use hardware you already have, don’t start experimenting with expensive cloud instances; and consider events where participants are constant features (ATP250 for tennis, T20 games for cricket, HK or US racing for horses), because I think you’ll get further, quicker, and more accurately.