Key takeaways
- Free tables: 38 of 38 tables called correctly on three overhead frames, in about 10 seconds per room.
- Supplier invoices: every item price read correctly from phone photos, and the setup we chose read all six documents with zero errors.
- Kitchen hygiene: every smoke break caught. Gloves and hair covers on wide shots are next, checked cook by cook.
- Guest complaints about the wrong pizza: the right menu item found 3 times out of 3, in under a fifth of a second.
- All of it ran on one $749 GPU, with no cloud fees and no face data.
- It works without Chinese models too: Google's Gemma 4 read all six documents with zero errors, and Mistral's Ministral 3 called all 38 tables.
- Each task gets its own model. We test candidates on real photos, keep the best one for each job and change the setup when a test shows a gap.
What restaurant video analytics needs besides a model
Most chains already have the hardware for AI video analytics: cameras over the dining room, the line and the back door, recording footage nobody has time to watch. The detection itself is now the cheap part. Open models that run on a single gaming GPU can tell an occupied table from a free one, a gloved hand from a bare one, and a supplier invoice line from the one below it.
What decides whether a chain gets value is what happens after the detection. A "table 7 is free" flag is useful when it reaches the host stand and the reservation book. An invoice read from a photo is useful when its lines land in inventory and get checked against the purchase order. Restaurant Business reported on February 27, 2025 that operators get more use cases out of computer vision once it is connected to the POS. Our tests point the same way. The model work took days. The integration work is where a project spends its time, and our guide to POS integration for restaurants and retailers shows what that work involves.
Running AI on the phone line too?
Our overview covers what voice AI does for restaurant chains.
How we ran the test
Ilya, the dev.family engineer who ran the tests, picked five tasks that come up with restaurant groups and food retailers: free tables, kitchen hygiene, matching a guest's photo to the menu, naming several pizzas on one table, and reading invoices and receipts. For each task he built a set of photos where the right answer was known in advance and ran four to nine open models on it. He kept the model that came closest to the known answers. When the first try fell short, he changed the setup, such as how the question is asked or which model handles which step, and ran the test again.
I don't pick models from leaderboards. I take photos where I already know the answer and see which model gets closest. A model that tops a ranking can still return junk on your own pictures.Ilya, dev.family engineer
Two kinds of models did the work:
- Embedding models turn a photo, or a phrase like "person smoking a cigarette," into a list of numbers and measure how similar two of them are. They only compare, and they do it in a fraction of a second per frame.
- Vision-language models look at a photo and answer a question in words: "Is anyone sitting at this table?" They understand the whole scene and take a few seconds per frame.
Everything ran on one NVIDIA RTX 5070 Ti with 16 GB of memory, a consumer card that launched at $749 in February 2025. The compressed models needed less than 5 GB of that memory, and the full-size model for documents about 10 GB, so everything fit on the one card. Nothing was sent to a cloud API.
The photo sets were small: three dining-room frames, four kitchen frames, six invoices and receipts. Where we didn't have real footage, we used images generated by Google Gemini and say so in every caption. A production rollout repeats the same test on 20 to 30 frames from the chain's own cameras before anything goes live.
1. Free tables from a ceiling camera
Why a chain wants it: hosts and guests see open tables without walking the floor, and managers see occupancy and turn times by hour for every location.

How it works: the camera sends one frame a minute, which is plenty for tables. When the camera is set up, the tables in the frame are outlined once and numbered. In our test an AI assistant did this from the frame, so no one drew boxes by hand. From then on, the vision model answers one question per table: occupied or free.
Our first version showed the model the whole room and asked about every table at once. Qwen3.5-4B handled it, but took about 19 seconds per room, and other models mixed tables up. So we changed the question: each table is cropped out of the frame and asked about on its own, ten short questions instead of one long one. With that setup Qwen3.5-4B from Alibaba, the model we chose, called 38 of 38 tables correctly across three frames, in about 9.4 seconds for a room of 10 to 14 tables. Most of the other models we tried also got noticeably better with crops.


Cropping also cut the time in half. A short question about a small picture takes less time than a long one about the whole room. At one frame a minute, a single camera keeps the GPU busy about 9.4 seconds out of every 60, so one card can cover 5 to 6 cameras.
From here the integration does the useful part: table status goes to the host stand and into restaurant reservation software, and turn times by table feed the same reports as covers and checks. If you've built reservations yourself, as we did for Foodclick's reservation and pre-order app, camera status is one more input to a system you already own. Our piece on why restaurant franchises outgrow OpenTable covers the build-or-buy side of reservations.
Building reservations in-house?
See the reservation and pre-order app we built for Foodclick.
Want table status in your host stand and reservation system?
We connect cameras, reservations and the POS across every location.
2. Gloves, hair covers and smoking in the kitchen
Why a chain wants it: brand and food-safety standards hold at every franchise location between audits, one of the operational challenges restaurant franchises face as they grow, and the camera sends the GM a frame when something breaks the rules.
How it works: every few seconds an embedding model compares the kitchen frame with phrases like "person smoking a cigarette," "hands in disposable gloves" and "cook wearing a hat or hairnet." That takes about 0.3 seconds. Frames that look like a problem go to the vision model, which takes about 4 seconds to answer three questions: is anyone smoking, are gloves on, is hair covered.
Qwen3.5-4B caught every smoke break in our test frames. On wide shots with several cooks, it sometimes answered for the kitchen as a whole, so one bare head or one pair of bare hands next to a fully gloved team could slip past. The next step is the one that worked for tables, and it wasn't part of this round: the system finds each cook, crops them out and asks about each person separately. The fast filter picked out smoking by a wide margin in our frames, while gloves and hair covers go to the vision model.

A chain rolling this out across locations also needs to tell staff where the cameras are and what they flag, and to run the setup past counsel before it goes live.
Want to know what your own cameras can catch?
Tell us what you need to track, and we'll tell you whether a model can do it on your footage.
3. Is that the right pizza? Matching a guest's photo to your menu
Why a chain wants it: a guest sends a photo with "you sent the wrong pizza." The system compares the photo with the menu and tells the agent which item it is. The same check works on the pass before an order goes out.
How it works: every pizza on the menu already has a photo. An embedding model compares the guest's photo with all of them, and the closest one is the answer. That takes 0.1 to 0.2 seconds. If two items come out almost equally close, the vision model looks at the toppings and checks them against the ingredient list.
We took three photos of pizzas shot at the table and asked the system to find each one among a 42-item pizzeria menu. Qwen3-VL-Embedding-2B found all three, with a clear gap between the right pizza and the runner-up. A Google model we tried also found all three, but with a thinner gap, so we kept the Qwen model for this step. Menus with photos work best for food image recognition. Text-only menus work less well, so a chain that wants this check should keep a photo for every menu item.

4. Naming every pizza on a crowded table
Why a chain wants it: order checks at the pass, complaint reviews and catering counts all start with a photo that has more than one item in it.
How it works: the vision model finds every pizza in the photo and draws a box around it. Each pizza is then cropped and matched on its own, and an embedding model suggests the three closest items from the menu photos.
On three photos of office lunch tables covered with pizza boxes from two chains, Qwen3.5-4B found and boxed all 19 pizzas, including ones half hidden behind bottles. Naming the exact menu item from a side shot is harder: pizzas from one kitchen share the same dough, cheese and sauce, and the toppings hide under cheese. So the system works the way a good assistant would. It shows a staff member the three closest menu items, and for the chain whose menu had photos, the right one was among them 7 times out of 8. For fully automatic checks, each pizza needs to fill the frame, as in test 3, where the match was 3 for 3.

5. Supplier invoices and receipts straight into inventory
Why a chain wants it: a supplier drops off product, someone photographs the invoice, and the lines with prices land in inventory and accounts payable without anyone typing them in. Expense receipts work the same way.
How it works: the vision model reads the document from the photo: supplier, date, number, every line with its price, and the total. Then the system adds up the lines and compares the sum with the total on the document. If they match, the document goes into the books. If not, it goes to a person.

We ran this on six real documents from New York, photographed on a phone: two supplier invoices and four receipts from stores and cafés, the longest with 23 lines. The compact version of Qwen3.5-4B read all 37 item prices correctly. On one document, a 23-line grocery receipt, it got every price and the total right but attached five prices to the item one line up. Every price was still there, so the lines added up to the total, and a simple sum check would have let it through.

Two things close that gap. The full-size version of the same model read all six documents, this receipt included, with zero errors. The second belongs in the production setup and wasn't part of this test: each line gets checked against the purchase order and the supplier's price list in the ERP, which catches a price sitting next to the wrong item.
The sum check catches a misread amount. It doesn't catch an amount sitting next to the wrong item. For that you need a second source of truth.Ilya, dev.family engineer
In a restaurant chain that second source already exists: purchase orders and price lists sit in the ERP, and our guide to ERP integration for restaurant chains covers how they connect to the POS and inventory. This is where an invoice-reading model becomes part of the stack you already run, and the reason we treat it as integration work.
Two setup rules came out of the same test:
- Let the model read the photo itself. Feeding it text from a separate OCR program lost track of which price belonged to which line.
- Send the photo as a file. Messaging apps compress images, and compressed receipts lose small print.
The full-size model takes about 19 seconds for a typical document and 25 seconds on average once long receipts are counted, since it writes its answer line by line. That works out to roughly 140 documents an hour on one GPU.
Supplier invoices still typed in by hand at every location?
We connect invoice capture to purchase orders, price lists and your ERP.
When the first try falls short, we change the setup
The best model differed from task to task, and in several tasks the first version needed work. Each time, the answer was a change in how the system is built:
- Ask about one thing at a time. Cropping tables one by one cut the time in half for our model and lifted accuracy for most others. The same approach fits cooks in the kitchen and pizzas on a table.
- Split the work between two models. A fast embedding model screens every frame in 0.1 to 0.3 seconds and handles about 13,000 frames an hour on one card. A vision model looks only at the frames it flags.
- Combine models in one step. One model finds the pizzas, another matches each to the menu.
- Pick the model size for the job. The full-size model read every document without an error, so documents get the larger model.
- Check against a second source. Invoice lines are matched to purchase orders and price lists in the ERP.

What it costs to run on your own GPU
One RTX 5070 Ti handled every task in this test. Here is what that card keeps up with in an hour, based on measured time per item:
Task | What runs | Time per item | One GPU per hour |
|---|---|---|---|
Free tables | Vision model checks every table | 9.4 s per room of 10–14 tables | ~380 rooms, or 5–6 cameras at one frame a minute |
Kitchen hygiene, filter | Embedding model screens every frame | 0.28 s per frame | ~13,000 frames |
Kitchen hygiene, check | Vision model re-checks flagged frames | 4.0 s per frame | ~900 frames |
Right pizza | Embedding model matches a photo to the menu | 0.1–0.2 s per photo | 18,000–36,000 photos |
Pizzas on a table | Vision model finds every pizza, then names each | ~36 s to find all, ~3 s to name each | ~60 table photos |
Invoices and receipts | Full-size vision model reads the document | ~19 s typical, 25 s average | ~140 documents |

The math that favors your own hardware is simple. One camera at one frame a minute produces 43,200 frames a month. A cloud API bills every one of those calls, so the bill grows with every camera and location you add. A GPU costs the same whether it processes ten frames or ten million, and the frames never leave your network.
The trade-off is ownership. Someone has to keep the server running, retune thresholds when a location changes its lighting, swap in a better model when one comes out, and watch for drift. That is ongoing operations work. A chain either staffs it or has a partner run it.
Which model does which job
Task | Setup we chose | Result in our test | Option without Chinese models |
|---|---|---|---|
Free tables | Qwen3.5-4B, one question per table | 38 of 38 tables | Ministral 3 8B (Mistral AI) or Gemma 4 E4B (Google) |
Kitchen hygiene | Qwen3-VL-Embedding-2B filter, Qwen3.5-4B check per frame | Every smoke break caught | SigLIP2 filter and Gemma 4 E4B check (Google) |
Right pizza | Qwen3-VL-Embedding-2B against menu photos | 3 of 3 pizzas | SigLIP2 (Google) |
Pizzas on a table | Qwen3.5-4B finds, Qwen3-VL-Embedding-2B suggests the top three | 19 of 19 pizzas found | Gemma 4 E4B finds, SigLIP2 suggests (Google) |
Invoices and receipts | Qwen3.5-4B, full size, reading the photo | 6 of 6 documents, 37 of 37 prices | Gemma 4 E4B (Google) |
For each task we tested four to nine models on the same photos, from Alibaba, Google, Microsoft, Mistral AI, Hugging Face and the Shanghai AI Laboratory. The winner was a different model or combination for different tasks, and how we asked mattered as much as which model answered.
Our first picks come from Alibaba's Qwen family. They run entirely on your own hardware and send nothing out, but some US companies don't allow models from Chinese developers at all. So we ran the same photos through models from Google, Microsoft and France's Mistral AI. Google's Gemma 4 E4B came closest: it read all six documents with zero errors, called 37 of 38 tables and boxed 18 of 19 pizzas. Mistral's Ministral 3 8B called all 38 tables and read five of six documents with zero errors. Both need 10 to 13 GB of video memory instead of 4 to 5, and both still fit on one gaming card. Microsoft's Phi-4-multimodal reads documents as accurately and twice as fast, which makes it a good pick for invoices, but it isn't built for camera work. Asking in English didn't change the results. We left out Meta's Llama 3.2 Vision, whose license doesn't cover companies based in the EU.
The pipeline stays the same whichever model runs inside it. Swapping a model means rerunning the same known-answer test and adjusting thresholds.
Does your security team rule out models from Chinese developers?
Tell us your constraints, and we'll put together a setup that passes review.
What the system tracks: tables, stations and documents
All five tests work on objects and events. The table test answers "is someone sitting here." The kitchen test answers "are those hands gloved." That keeps the system away from face geometry, the data that biometric privacy laws regulate. Illinois' Biometric Information Privacy Act, for example, requires a business to get a person's consent before it collects a face scan. We design for status and events: a table, a station, a document. If a project calls for anything closer to identity, it starts with counsel.
How to start: one camera, one question, one system
The fastest way to find out whether this works for your chain is a short pilot on your own footage and documents:
- Pick one scenario with a clear payoff: free tables in your busiest locations, hygiene at the line, or supplier invoices.
- Collect 20 to 30 frames or documents from real locations, at different times of day, with the right answers marked.
- Test the candidate models on your data, pick the best setup and tune it until the results hold.
- Connect the result to one system: the reservation book, a manager alert or the ERP.
Our AI Operations Pilot does this in 2 to 4 weeks and ends with a working prototype and a fixed estimate for the build. The same page covers restaurant voice AI and the other AI scenarios we run for chains.
Not sure which scenario pays back first in your locations?
A 2 to 4 week assessment tests it on your data and ends with a fixed build estimate.
FAQ
Last updated: November 11, 2026








