Reading a dining room without recognising a face
Restaurants already run cameras, and almost all of that footage is only ever watched after something has gone wrong. We built a vision system that reads those same feeds while the shift is running and reports what the floor can act on, with no face model anywhere in it.
- IndustryHospitality and food service
- EngagementVision system and operations backend, designed and built in house
- StatusWorking build; packaging for site installation still ahead
The engineering is in what it refuses to do
The system takes the ordinary camera feeds from the room a venue already watches, blacks out everything outside the dining area the operator has drawn, and runs detection only on what is left. Out of that come the three things a floor manager can actually use: which tables are seated, which are empty but still carrying plates and glasses, and which have been seated too long without a staff visit.
The decisions worth describing are the restraints. Staff are identified by their uniform, with no face model anywhere in the system. What crosses to the operations side is structured events. A table will not flip between states on a single frame, and no recommendation is offered until the system holds enough of the venue's own trading history to justify one. Each of those was the harder build, and together they are why the system can be explained to the people it watches.
What a dining room asks of a vision system
The detections are the straightforward part. What shaped this build is the limits a room full of guests puts on how you are allowed to solve it.
Cameras that only look backwards
A venue's cameras record continuously and the footage gets reviewed after a complaint. While the thing that will be complained about is still fixable, nobody on the floor is told anything.
The shortcut nobody should take
The quickest way to tell a server from a guest is to recognise faces. In a dining room that is the wrong trade: it puts everyone present into a biometric system in return for an operational convenience, and it is not a trade a venue can explain to its guests or to its staff.
A detection is not a state
Run a detector frame by frame and the same table reads as occupied, free and occupied again within a minute, purely because people walk past it. A floor plan that changes every second carries no information, and staff stop looking at it after the first shift.
Four decisions the rest of the system is built on
Each one trades a flashier claim for one a venue can check for itself.
Identity from a uniform, not a face
Staff are told apart from guests by a detection model we trained ourselves on exactly one thing: the apron. The uniform is matched to the person by overlap, and the classification survives a few seconds of the apron sitting behind a chair or another body. There is no face detection, no face recognition and no person re-identification model anywhere in the system.
- No face model at any point in the pipeline
- The apron is what the model detects, and staff status follows from it
- Everything outside the drawn dining area is blacked out before analysis
A table state machine with hysteresis
Occupancy is not a per frame reading. Each table runs a state machine with separate thresholds for filling and for emptying, so seating a table takes sustained evidence and releasing it takes rather more. On top of that sits the distinction that turns the output into a worklist: a vacated table still carrying plates and glasses is not clean, and it says so until the objects are gone.
- Separate confirmation thresholds for filling and for emptying
- Empty and empty but not cleared are different states
- A seated table with no staff visit inside its window raises an alert that escalates
People read as geometry
Sitting and standing are computed from body geometry, and no learned posture classifier is involved: spine tilt against vertical, spine to leg alignment, hip angle and knee angle, voted independently for each side of the body and smoothed over recent frames, with a fallback for when the keypoints are missing. Association to a table is tuned for the seated case, where a standing height box overlaps a table badly.
- Posture from joint angles, voted per side and smoothed over time
- Seated guests matched to a table by the lower part of the body
- Every threshold is a per venue setting
Metadata out, names only by approval
What crosses from the vision process to the operations side is a small structured event: a table's state, a staff visit and its duration, a request to recalibrate after the room has been rearranged. Frames are decoded, analysed and dropped. The vision core emits only an anonymous per camera number, and attaching a name to it happens against the venue's own staff roster.
- Table states and visit durations are what cross the wire
- The link from a tracked person to a named employee is proposed, then approved by a person
- When the room has been rearranged the system flags it instead of reporting against a stale plan
What the restraints bought
No figure appears on this page. Nothing here has been measured against a published baseline, so what follows is what the system does, described in words.
Structured events
What the vision process sends
The architecture keeps video on the venue's own hardware: the vision process transmits table states, visit durations and recalibration requests. The exceptions are the operator steps that need a picture, such as drawing table boundaries over a still frame. Those are deliberate, switchable and never a continuous stream.
No faces
How people are told apart
Staff are identified by a uniform and guests are not identified at all. There is no face detection, no face recognition and no re-identification model in the system, so this is a property of the architecture rather than a policy written on top of one.
Says when it cannot
Recommendations
The recommendation side is association rule statistics that can be read and argued with, and it refuses to train or predict until it holds a long enough run of the venue's own trading history. Until then it reports how far along it is instead of returning a confident guess.
Have a camera estate that only looks backwards?
If you are weighing a vision system for an operation with people in it, the constraints are the interesting part of the conversation. Tell us about the room and what you need it to report.
Discuss your project