[David Strom's Web Informant] 11 January 2010: Automated video image analysis

[David Strom]

Web Informant 11 January 2010:
Automated video image analysis will be the next big breakthrough

Remember Chance the Gardener character in Being There? "I like to
watch tv" was his famous line. The problem with most business videos
is that we produce a lot of unwatchable footage, especially those
taken by security camera systems. And there aren't any Chances around
that want to watch them, either.

The trouble is that trying to find the one or two actionable events in
all of that footage isn't easy. As an example, take the story in
today's NY Times that mentions how the military is being buried under
a massive pile of video footage from its Predator drones that are
flying over Pakistan and Afghanistan. The situation is only going to
get worse, as newer drone models will be sending streams from dozens
of cameras within a few years:
http://www.nytimes.com/2010/01/11/business/11drone.html?pagewanted=2&ref=technology

The story has already been told about how the drones' video feeds are
available to anyone who has some minor software skills – the actual
control channels are encrypted but the video transmissions aren't.
This is because many legit people need to see what they are
broadcasting and the military hasn't been able to implement any
encrypted viewing packages on these streams.

In the Times article, a bunch of soldiers based in Hampton, Virgina
sit in front of the screens and see it in real time, and then make
screengrabs available to the right people via computer chat rooms. I
hope for the sake of everyone involved that these chat rooms are
encrypted, but the article didn't say.

So how do we implement our automated Chance Gardner? There are a
couple of technologies that can help here, but they aren't easy or
cheap to implement. One is the telestrator, the device made popular by
John Madden and Monday Night Football where a commentator draws on the
screen and you seen colored squiggles to highlight what is going on.
The ones that Madden and the pros use are very expensive, but there
are dozens of products available for the PC market, with some freeware
products such as VideoMage Producer.
http://www.thedvshow.com/just-in-time-for-football-diy-telestrator/

The telestrators are nice, but again, someone has to be watching the
video and doing the electronic doodling. You need more than the
fast-forward button to do this – ideally, you want some kind of
automated system that can identify actionable moments on the video.
This is what the next class of products does, called intelligent image
analysis. They have computers to look at the stream and highlight
particular activities that a human operator can come back to and
review later.

This is what the company stoplift.com is doing with its retail
checkout analysis systems. Typically, a retail store installs video
cameras above each checkout aisle and records what the checker is
doing as items pass through the point of sale scanning devices. There
are all sorts of scams that can be used, such as "sweethearting" (a
confederate is giving free items that aren't scanned) and looking like
you are scanning a bar code when you are just passing the item around
and over to the bagger. So what is needed is a system that ties into
your point of sale and can flag when these items aren't rung up at the
register. I got to see a demo last week and thought this was way cool.
The company claims their software can have a six-month ROI and
significantly reduce the cost of stolen goods.  And the good news is
that no one has to watch all the security tapes to see those few
sweetheart moments.