Friday, October 14, 2016

Adventures in Raspberry Pi Video

So I am working on a project for work which will involve using a built-in camera or webcam with a raspberry pi.  Pretty simple stuff, but it got me thinking what I could use that hardware for at home.  I had the idea of a video doorbell system.

Sure, it's been done, but not quite how I'm envisioning it:
1) Camera is watching 24/7, not just when you ring the doorbell
2) Motion detection lets you know before someone even gets to the door, sends Pushover request to ping your devices.
3) Facial Recognition API runs on a motion detected frame to tell you who it is (you obviously have to train this data first, so see step 4)
4) If unknown person, save the image(s) for training later
5) Web Page stores images from 3 or 4 for an access log and to tag photos to send to web service to train images
6) When motion is detected, stream webcam to AppleTV/FireTV through AirPlay or Android Cast.
7) When person hits doorbell, show person's name/time on a display on doorbell to subtly let person know we know who it is and we've recorded their visit, then also trigger an action like play ring sound through AirPlay, send another pushover notification, whatever...
8) Streams audio/video 24/7 with RTSP
9) Receives audio anytime to play through a speaker included in doorbell

Sounds like a tall order, right?  I haven't even received my Raspberry Pi hardware yet and I've already completed a few of these things.

1) Webcam connected to home CentOS server - it's a logitech webcam, so it sets it up as a v4l device.
2) Uses 'motion' package from CentOS repo, configured to run a custom node.js script written to send pushover notifications (using pushover-notifications npm library).
3) Haven't connected this back up yet, but I'm thinking of using free Facial Recognition API.

4) Motion is set up to record to local mysql database on server.  With all the scripts it can run it would be trivial to also save files on Amazon Cloud or s3 also.. this can take care of #5

6) For this, we need motion to loop back video to another v4l device so that ffmpeg can stream it as an http server... here's how to split the video:
Then use the ffmpeg location for casting: or AirPlay (node)

7) Simple button/display to trigger these actions... eInk would be cool:

8) Simply run rtsp streamer on secondary v4l device:

9) Haven't decided this yet.  Ideally this connects to something and just sits waiting for audio, then plays whatever comes through.  Almost every solution is going to be gui-based though and I don't want to have to vnc into my doorbell to click 'ok' when a server can't be found.