Watch for the Thing You Actually Worry About - Shobdo Blog
← Back to all posts

Watch for the Thing You Actually Worry About

· Iftekhar Tanveer
Watch for the Thing You Actually Worry About

Open the manual for a modern "smart" security camera and look at the list of things it can detect. Here is the complete event list from a current 4 MP camera datasheet: motion detection with human and vehicle classification, video tampering, exception, scene change, line crossing, intrusion, region entrance, region exiting. The datasheet is refreshingly blunt about the scope: "the system focuses on human and vehicle targets."

That is the menu. Eight items, most of them geometric. Now think about what actually worries you in your own store.

The menu and the worry list do not overlap

Two lists compared, under the label 'the menu and the worry list'. The top panel, titled "What the camera can detect", shows every event on one current 4 MP camera's datasheet as eight menu chips, each with a small symbol: human or vehicle motion, video tampering, exception, scene change, line crossing, intrusion, region entrance and region exiting. The last four are drawn as a line or a dashed region. The bottom panel, titled "What a store owner worries about", notes that these are situations made of what, where, when and who is not there, and lists six worries: the lottery terminal used with no customer at the counter, a beer delivery left at the back door unsigned, someone eating in the aisle, the back door propped open after ten at night, a line at the register with nobody opening the second till, and a cooler door left open overnight. Next to each worry, a column headed "on the menu?" shows a red cross. An orange tally at the bottom reads "On both lists: 0".
The camera's full event list against a store owner's worry list. None of the six worries is on the menu.

Ask a convenience store owner what keeps them up and you get answers like these. The lottery terminal being used when no customer is at the counter. A beer delivery left at the back door without a signature. Someone eating in the aisle. The back door propped open after ten at night. A line at the register with nobody stepping up to the second till. A cooler door left open overnight.

Not one of those is "line crossing." They are not geometry. They are situations, and a situation is a combination of what, where, when and who-is-not-there.

The gap matters because these are not hypothetical losses. Lottery is a good example. A Colorado grand jury indicted four people over at least 45 scratch-ticket thefts totalling more than $150,000 across sixteen months; the method was to get the clerk away from the counter - a propane exchange, a "stuck" card, spilled gas - and take tickets from the dispenser. In Florida, a compliance sting caught a clerk who scanned a customer's test ticket showing a $1,000 win and kept both the ticket and the claim documentation, selling the claim for $800.

Deliveries are the same story from the supply side. A News4 I-Team investigation into a county beverage delivery operation found crews keeping cases on the truck and filing shortage claims, with two-thirds of businesses shorted, some more than once a week. The fix the county reached for included reviewing loading-dock video - after the fact, which is the only way that has ever been available.

And the internal category is large in aggregate: the National Retail Federation's security survey attributes 29 percent of shrink to internal theft, with an average loss per investigation of $2,180, against total US shrink of $112 billion.

A camera pointed at the lottery dispenser sees all of this happen. The analytics menu just has no word for it.

Why the menu was fixed in the first place

Diagram, illustrative. One new worry, "beer delivery left unsigned", splits into two paths. The left path is labelled "Before, a fixed list" with the heading "A programming task". It has four steps. Three are coral boxes: collect a dataset, label it, and retrain the AI. The fourth is an outlined coral box: one more item on the menu. A dashed coral arrow loops from the last step back to the first. A note says: repeat for every new item, that is why the list stayed at person, vehicle and line. The right path is labelled "Now, open vocabulary" with the heading "A writing task". It has two steps. The first is write it as one sentence: "Tell me if a delivery is left at the back door and nobody signs for it". The second is an amber box: "Watching for it". A note says: next worry, write another sentence. The right path is two boxes long and the left path is four.
One new worry, two ways. Before, it meant new training data and a retrained AI. Now it is one sentence.

For most of the history of video analytics, a detector had to be trained on a closed list of categories decided in advance. As the paper behind CLIP put it in 2021, "state-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories". Adding "beer delivery left unsigned" to that list meant collecting a dataset, labelling it and retraining a model - which is why the list stayed at person, vehicle and line.

That constraint is what changed. CLIP was trained on 400 million image-and-text pairs and could classify things it had never been explicitly trained on. Open-set detectors followed: Grounding DINO, published in 2023, detects "arbitrary objects with human inputs such as category names or referring expressions". Vision-language models went further, describing a scene in sentences rather than picking from a list. The practical consequence is that "what to watch for" stopped being a programming task and became a writing task.

Writing the instruction

One instruction, one night (illustrative). A flow from top to bottom. At the top, a speech bubble says: You type, "Tell me if the back door is open after ten." An arrow labelled "checked against the running log" points to a panel titled "What the first AI writes, camera by camera", with four lines: 9:52 pm, Front counter, A clerk restocks the shelf. 10:04 pm, Aisle 2, A customer picks up chips. 10:12 pm, Back door, The back door is propped open with a crate. No one is nearby. 10:15 pm, Parking lot, A car parks by the entrance. The 10:12 pm line is highlighted and tagged "match". An arrow leads to a box: Second AI pulls the 10:12 pm clip and checks it. A side branch reads: Not confirmed, no text is sent. A second arrow with a tick, marked Confirmed, leads to an amber card labelled Text to you, with a video clip thumbnail and the message: Back door open at 10:12 pm. A crate is holding it open and no one is nearby. You asked to hear if the back door is open after ten.
One illustrative night. A line in the log matches your sentence, and a second AI checks the clip before you get a text.

In our system, you type the thing you are worried about, in your own words, for one camera or for the whole store. "Tell me if the lottery terminal is used and there is no customer at the counter." "Tell me if a delivery is left at the back door and nobody signs for it." "Tell me if the back door is open after ten."

The first AI is already writing down what happens on every camera in plain English. Your instruction is checked against that running description, and when something matches, a second AI pulls the actual clip and confirms it before you are texted. You get the clip and a sentence saying what it saw and why it matched what you asked for.

Two honest caveats. Specific beats vague: "someone is behind the counter who is not wearing a store shirt after closing" works better than "watch for anything suspicious", because the second one is not a description of anything. And the system is not omniscient - a camera that cannot see the terminal cannot tell you about it, and an instruction about something off-camera will never fire. The first thing worth doing is walking the store and asking, camera by camera, what this one can actually see.

A grid with four columns: What, Where, When and Who is not there. Five instructions from the post are split into those four parts. "Tell me if the lottery terminal is used and there is no customer at the counter" splits into terminal used, lottery counter, any time, a customer. "Tell me if a delivery is left at the back door and nobody signs for it" splits into delivery left, back door, any time, someone to sign. "Tell me if the back door is open after ten" splits into door open, back door, after 10 pm, and the last part is not needed. "Someone is behind the counter who is not wearing a store shirt after closing" splits into someone, behind the counter, after closing, store staff. Parts the instruction does not need, such as any time or not needed, are greyed out. The last row is shaded coral: "Watch for anything suspicious." All four of its parts are empty dashed outlines, with a coral X and the note: nothing to match.
Instructions split into what, where, when and who is not there. "Anything suspicious" fills none of them.
An illustrative top-down floor plan of a small convenience store. The front door and windows run along the bottom. Near the front is a counter with the lottery terminal, the register and a second till, with a short register line in front of it. Three aisles fill the middle. A wall of cooler doors runs down the right side, and a stockroom with the back door sits at the back. Four numbered cameras each have a shaded field of view. Camera 1 covers the counter and the lottery terminal. Camera 2 covers the register line and the front door. Camera 3 covers the aisles. Camera 4 covers the back door. No shading reaches the cooler doors, which are marked in coral as a blind spot. A checklist below reads: lottery terminal, camera 1. Register line, camera 2. Back door, camera 4. Cooler doors, no camera, so "tell me if a cooler door is left open" will never fire.
One illustrative store. No camera sees the cooler doors, so an instruction about them can never fire.

Ask for a demo

When getting a new camera-AI service, test this service. Ask the salesperson to type your concern - the real one, in your words - into their system while you watch, and then show you what happens. Not a prepared demo. Yours.

A vendor whose system takes a fixed menu will change the subject to their detection accuracy. That answer tells you what you need to know: the product can watch for their list, and your list is a different list.

Shobdo VideoRAG is an AI agent for the security cameras your store already owns. It writes down what it sees and texts you only when something matters. Learn more or book a conversation.

Surveillance AI Retail Loss Prevention VideoRAG
← Back to all posts