Which cat is at the bowl

One of our cats has a kidney condition and is meant to eat a specific food, and cats do not respect this. The vet asked how much he was actually eating and the honest answer was that nobody knew, because there are three of them and they eat when we are asleep. I wanted a real answer rather than a guess, and I also wanted an excuse to do a computer-vision project small enough to finish. It took about five weekends to get to something useful and I have been fiddling with it ever since, which is the normal shape of these things.

It is entirely for my household and, at a stretch, for our vet. An old phone on a shelf watches the feeding area, and when there is movement it works out which cat is present and for how long, then logs a visit. Over a week that produces a picture of who is eating from which bowl and roughly how often, which is what the vet actually asked for. It does not weigh anything and it cannot tell how much was eaten, which I was careful to say when I showed the vet the chart, because a confident-looking graph invites people to believe more than it says.

All of it. I labelled about twelve hundred frames myself over two evenings, which is the least glamorous and most decisive part. The classifier is small and fine-tuned rather than trained from scratch — three classes plus a not-a-cat class that exists because my own feet were being confidently classified as the tabby. The detection and cropping runs before it so the classifier only sees a cat-sized region. I wrote the visit aggregation, which turns a stream of per-frame guesses into visits with a start and an end, and that logic is more of the accuracy than the model is.

I decided to run everything on the old phone rather than send frames anywhere, partly on principle and mostly because uploading video of my kitchen all day was never going to be worth it. That forced the model small, which forced the detection-then-classify split, which is why it works at all. I chose to log low-confidence frames as unknown rather than force a guess, and the unknown rate is a number I watch — when it climbs it usually means the light has changed or something has moved, not that the cats have. I also deliberately did not build any feeder control, which is where every project like this wants to go and where the consequences of being wrong are a hungry cat.

Two of the three cats are both grey tabbies and are genuinely difficult to tell apart from above at night, which is when most eating happens. My first version was around sixty per cent on those two, which is barely better than a coin toss and worse than useless because it looked authoritative. Three things fixed it: infrared lighting so night frames are not noise, adding size as a feature since one is noticeably larger, and using the visit aggregation to smooth — a visit is classified by the whole sequence rather than by a frame, so a few bad frames do not flip it. That got it to around ninety-one per cent on a held-out week I labelled by hand, and I am reasonably confident in that number because I built the test set before I built the fixes.

Ten months of logs. The immediate outcome was finding that the sick cat was eating the other food most nights, which is why we now feed him separately, and his last two blood panels have been better — I am not claiming credit for that, but the vet did use the visit chart to make the change. I have also learned more about evaluation from this than from anything at work, mostly by being wrong in ways I could check within a day. It runs on a phone from 2019 propped against a jar.

None, though my flatmate labelled about two hundred of the night frames and remains the only person who can reliably tell the two tabbies apart, which is how I got a trustworthy test set.