[Monitoring Camera System with Passive Infrared Sensor]
[Block Diagram]
Raspberry Pi2
USB power supply
KSY UB310-0520
Case
USB Camera
C270
USB Wireless LAN
WLI-UC-G301N
USB hub
U2H-TZ420SBK
Passive infrared sensor
HC-SR501
Jumper wire
female-female 10cmx3
[Functions]
Pi detects a human by a passive infrared sensor.
Pi records a movie for a while.
Pi converts the recorded AVI file to Docomo 3gp file.
Pi sends it to the mobile phone by email.
[Install Raspbian]
Install OS to SD card. It needs 4.7GB totally in the end. SD card size should be 8GB or more.
Downloads the Raspbian image file from Raspberry Pi Foundation. Write it to the SD card by Win32DiskImager.
Power on with LAN cable, HDMI connection to TV, USB mouse and USB keyboard.
You will see the Config dialog first. Select Expand Filesystem to expand SD card actual size.
Change User Password.
Select Internationalization Options for your location.
Enable SSH in Advanced Options for the remote accress from your PC.
Reboot by Finish.
[Remote access]
Check IP address at "inet addr" with ifconfig command.
Install a terminal software like TeraTerm on your PC.
Connect Pi with the IP address. Input default ID "pi" and password.
[Install FTP server]
Install vsftpd to check files on PC. Before the install, apt-get update and upgrade should be done.
USB power supply from Raspberry Pi is not enough even though the USB power supply to Raspberry Pi is enough. USB camera should be connected through the USB hub with the another power supply. USB camera would be unstable when it is connected directly.
Check connection by lsusb command
Bus 001 Device 002: ID 0424:9514 Standard Microsystems Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp.
Bus 001 Device 004: ID 05e3:0608 Genesys Logic, Inc. USB-2.0 4-Port HUB
Bus 001 Device 005: ID 1c4f:0027 SiGma Micro
Bus 001 Device 006: ID 046d:0825 Logitech, Inc. Webcam C270
Bus 001 Device 007: ID 093a:2510 Pixart Imaging, Inc. Optical Mouse
pi@raspberrypi ~ $ fswebcam test.jpg
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
Adjusting resolution from 384x288 to 352x288.
--- Capturing frame...
Corrupt JPEG data: 1 extraneous bytes before marker 0xd6
Captured frame in 0.00 seconds.
--- Processing captured image...
Writing JPEG image to 'test.jpg'.
No real problem about the message "Corrupt JPEG data".
C270 outputs the following message at the second capture with an error but it disappeared by a reboot.
Unable to find a compatible palette format.
Check the image on PC with ftp software like FFFTP. (You can also check on Desktop on Raspberry Pi directly.)
[USB Wireless LAN]
Connect through the USB hub. Check lsusb.
pi@raspberrypi ~ $ lsusb
Bus 001 Device 002: ID 0424:9514 Standard Microsystems Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp.
Bus 001 Device 004: ID 05e3:0608 Genesys Logic, Inc. USB-2.0 4-Port HUB
Bus 001 Device 005: ID 1c4f:0027 SiGma Micro
Bus 001 Device 008: ID 0411:016f BUFFALO INC. (formerly MelCo., Inc.) WLI-UC-G301N Wireless LAN Adapter [Ralink RT3072]
Bus 001 Device 006: ID 046d:0825 Logitech, Inc. Webcam C270
Bus 001 Device 007: ID 093a:2510 Pixart Imaging, Inc. Optical Mouse
Configure the WiFi software "wpa_supplicant". /etc/wpa_supplicant/wpa_supplicant.conf configuration. See official website.
But, WiFi connection is not so stable for a long time. This method can not recover such disconnection. I uses wicd now. wicd is also written in Debian Wiki.
[Install OpenCV]
OpenCV is Computer Vision library. It includes the interest libaries like a face detection but it requires very high speed processing. It was too slow on Raspberry Pi2. Now, I use OpenCV just for recording a movie. Install with the "Installation in Linux" on opencv.org. I removed some options like libtbb2 because they are not found.
The last words ".." means "parent directory". It does not mean "please take care by yourself though there are more arguments." I had misread...
My configuration result. There are many 'not found's but it is ok for this system.
-- General configuration for OpenCV 3.0.0-dev =====================================
-- Version control: 3.0.0-1-g424c2bd
--
-- Platform:
-- Host: Linux 3.18.11-v7+ armv7l
-- CMake: 2.8.9
-- CMake generator: Unix Makefiles
-- CMake build tool: /usr/bin/make
-- Configuration: RELEASE
--
-- C/C++:
-- Built as dynamic libs?: YES
-- C++ Compiler: /usr/bin/c++ (ver 4.6.3)
-- C++ flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG -DNDEBUG
-- C++ flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -g -O0 -DDEBUG -D_DEBUG
-- C Compiler: /usr/bin/gcc
-- C flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fvisibility=hidden -O3 -DNDEBUG -DNDEBUG
-- C flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fvisibility=hidden -g -O0 -DDEBUG -D_DEBUG
-- Linker flags (Release):
-- Linker flags (Debug):
-- Precompiled headers: YES
-- Extra dependencies: dl m pthread rt
-- 3rdparty dependencies:
--
-- OpenCV modules:
-- To be built: hal core flann imgproc ml photo video imgcodecs shape videoio highgui objdetect superres ts features2d calib3d stitching videostab python2
-- Disabled: world
-- Disabled by dependency: -
-- Unavailable: cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev java python3 viz
--
-- GUI:
-- QT: NO
-- GTK+ 2.x: YES (ver 2.24.10)
-- GThread : YES (ver 2.40.0)
-- GtkGlExt: NO
-- OpenGL support: NO
-- VTK support: NO
--
-- Media I/O:
-- ZLib: /usr/lib/arm-linux-gnueabihf/libz.so (ver 1.2.7)
-- JPEG: libjpeg (ver 90)
-- WEBP: build (ver 0.3.1)
-- PNG: /usr/lib/arm-linux-gnueabihf/libpng.so (ver 1.2.49)
-- TIFF: build (ver 42 - 4.0.2)
-- JPEG 2000: build (ver 1.900.1)
-- OpenEXR: build (ver 1.7.1)
-- GDAL: NO
--
-- Video I/O:
-- DC1394 1.x: NO
-- DC1394 2.x: NO
-- FFMPEG: YES
-- codec: YES (ver 54.35.0)
-- format: YES (ver 54.20.4)
-- util: YES (ver 52.3.0)
-- swscale: YES (ver 2.1.1)
-- resample: NO
-- gentoo-style: YES
-- GStreamer: NO
-- OpenNI: NO
-- OpenNI PrimeSensor Modules: NO
-- OpenNI2: NO
-- PvAPI: NO
-- GigEVisionSDK: NO
-- UniCap: NO
-- UniCap ucil: NO
-- V4L/V4L2: Using libv4l1 (ver 1.0.0) / libv4l2 (ver 1.0.0)
-- XIMEA: NO
-- Xine: NO
-- gPhoto2: NO
--
-- Other third-party libraries:
-- Use IPP: NO
-- Use Eigen: NO
-- Use TBB: NO
-- Use OpenMP: NO
-- Use GCD NO
-- Use Concurrency NO
-- Use C=: NO
-- Use pthreads for parallel for:
-- YES
-- Use Cuda: NO
-- Use OpenCL: YES
--
-- OpenCL:
-- Version: dynamic
-- Include path: /home/pi/opencv/3rdparty/include/opencl/1.2
-- Use AMDFFT: NO
-- Use AMDBLAS: NO
--
-- Python 2:
-- Interpreter: /usr/bin/python2.7 (ver 2.7.3)
-- Libraries: /usr/lib/libpython2.7.so (ver 2.7.3)
-- numpy: /usr/lib/pymodules/python2.7/numpy/core/include (ver 1.6.2)
-- packages path: lib/python2.7/dist-packages
--
-- Python 3:
-- Interpreter: /usr/bin/python3 (ver 3.2.3)
--
-- Python (for build): /usr/bin/python2.7
--
-- Java:
-- ant: NO
-- JNI: NO
-- Java wrappers: NO
-- Java tests: NO
--
-- Matlab:
-- mex: NO
--
-- Documentation:
-- Doxygen: NO
-- PlantUML: NO
--
-- Tests and samples:
-- Tests: YES
-- Performance tests: YES
-- C/C++ Examples: NO
--
-- Install path: /usr/local
--
-- cvconfig.h is in: /home/pi/opencv/release
-- -----------------------------------------------------------------
run make. It takes about 3 hours.
pi@raspberrypi ~/opencv/release $ make
Scanning dependencies of target libtiff
[ 0%] Building C object 3rdparty/libtiff/CMakeFiles/libtiff.dir/tif_aux.c.o
.
(skip)
.
[100%] Built target opencv_annotation
Install
pi@raspberrypi ~/opencv/release $ sudo make install
(skip)
Passive infrared sensor is connected to VCC/GND and GPIO4 (pin number 7). I used eLinux-RPi Low-level peripherals for GPIO controls.
Use daemon() to run this program for the monitoring system. The output directory is defined directly by chdir(). Use syslog() for logging. But, the daemon should be added at the end after all debugs.
I added some dummy read of VideoCapture because some frames remain in somewhere and they will appear at the first frames in the next movie.
No video capturing when the passive infrared sensor does not detect human for low power consumption. Loop with 100mS sleep. After human detection, record without sleep.
The program code execution itself takes time. The loop rate is around 8fps. You can measure the loop time by cvGetTickCount function. Record AVI file with MP42 codec with 8fps.
Convert AVI file to NTT Docomo 3gp file (this is not applicable for the other countries.) by ffmpeg (avconv). Send it by email script.
Recording time is about 15 seconds. The next recording will be enable after the continuous one minute no detection considering the working near the door. Set the maximum number of movies for the unexpected infrared detections.
The detection at morning and the detection at midnight will be ignored. It must be me.
source code
#include <iostream>
#include <string>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <time.h>
#include <sys/time.h>
#include <sys/mman.h>
#include <unistd.h>
#include <fcntl.h>
#include <syslog.h>
#define BCM2708_PERI_BASE 0x3F000000
#define GPIO_BASE (BCM2708_PERI_BASE + 0x200000) /* GPIO controller */
#define PAGE_SIZE (4*1024)
#define BLOCK_SIZE (4*1024)
#define MOVIEFPS 8
#define MAXMOVIENUM 5
#define MOVIETIME 100
#define DETECTINT 600
using namespace std;
using namespace cv;
int mem_fd;
void *gpio_map;
// I/O access
volatile unsigned *gpio;
// GPIO setup macros. Always use INP_GPIO(x) before using OUT_GPIO(x) or SET_GPIO_ALT(x,y)
#define INP_GPIO(g) *(gpio+((g)/10)) &= ~(7<<(((g)%10)*3))
#define OUT_GPIO(g) *(gpio+((g)/10)) |= (1<<(((g)%10)*3))
#define SET_GPIO_ALT(g,a) *(gpio+(((g)/10))) |= (((a)<=3?(a)+4:(a)==4?3:2)<<(((g)%10)*3))
#define GPIO_SET *(gpio+7) // sets bits which are 1 ignores bits which are 0
#define GPIO_CLR *(gpio+10) // clears bits which are 1 ignores bits which are 0
#define GET_GPIO(g) (*(gpio+13)&(1<<g)) // 0 if LOW, (1<<g) if HIGH
#define GPIO_PULL *(gpio+37) // Pull up/pull down
#define GPIO_PULLCLK0 *(gpio+38) // Pull up/pull down clock
int movienumber;
int timecount;
bool rec;
void setup_io();
int main(int argc, char *argv[])
{
if (daemon(1,0) == -1) {
syslog(LOG_USER|LOG_INFO,"failed to launch daemon.\n");
return -1;
}
VideoWriter outputVideo;
char outfile[20];
char workcommand[150];
#ifdef DEBUG
double ticFrequency;
double processTime;
int startTic;
int stopTic;
ticFrequency = cvGetTickFrequency();
#endif
struct timespec wt={0,100000000}; // sleep time of main loop 100mS.
struct tm *s_time;
time_t the_time;
chdir("/home/pi/cpp/humandetector"); // Movie output directory
syslog(LOG_USER|LOG_INFO,"Start");
setup_io();
INP_GPIO(4);
movienumber = 0;
timecount = 0;
rec = false;
VideoCapture inputVideo(0);
if (!inputVideo.isOpened())
{
syslog(LOG_USER|LOG_INFO,"Could not open the input video.\n");
return -1;
}
Size S = Size((int) inputVideo.get(CV_CAP_PROP_FRAME_WIDTH), (int) inputVideo.get(CV_CAP_PROP_FRAME_HEIGHT));
Mat frame,clrframe;
inputVideo >> clrframe; // Dummy read to start camera
for (;;)
{
#ifdef DEBUG
startTic = cvGetTickCount();
#endif
if (rec){
inputVideo >> frame;
if(!frame.empty()){
outputVideo << frame;
}
timecount++;
if(timecount>MOVIETIME){
rec = false;
outputVideo.release();
syslog(LOG_USER|LOG_INFO,"released. convert avi to 3gp.");
sprintf(workcommand, "avconv -y -i %s -r:v 30 -codec:v mpeg4 -b:v 64k -s:v 320x240 -codec:a libaac -ar:a 48000 000.3gp", outfile);
system(workcommand);
syslog(LOG_USER|LOG_INFO,"send 3gp to docomo.");
system("sh mymail.sh 000.3gp");
if(movienumber>=MAXMOVIENUM){break;}
timecount=DETECTINT;
syslog(LOG_USER|LOG_INFO,"done.");
}
}else if(GET_GPIO(4) > 0){
(void) time(&the_time);
s_time=localtime(&the_time);
if(timecount>0){
timecount=DETECTINT;
}else if(s_time->tm_hour>=8 && s_time->tm_hour<=21){
rec = true;
timecount=0;
sprintf(outfile, "%03d.avi", movienumber++);
outputVideo.open(outfile, CV_FOURCC('M', 'P', '4', '2'), MOVIEFPS, S, true);
if (!outputVideo.isOpened())
{
syslog(LOG_USER|LOG_INFO,"Could not open the output video.");
return -1;
}
inputVideo >> clrframe; // Dummy reads to skip old cache.
inputVideo >> clrframe;
inputVideo >> clrframe;
inputVideo >> clrframe;
syslog(LOG_USER|LOG_INFO,"detected.");
}
}else{
nanosleep(&wt,NULL);
if(timecount>0){timecount--;}
}
#ifdef DEBUG
stopTic = cvGetTickCount();
processTime = (stopTic-startTic)/ticFrequency;
printf("time=%fus\n",processTime);
#endif
}
outputVideo.release();
inputVideo.release();
syslog(LOG_USER|LOG_INFO,"Finished.");
return 0;
}
//
// Set up a memory regions to access GPIO
//
void setup_io()
{
/* open /dev/mem */
if ((mem_fd = open("/dev/mem", O_RDWR|O_SYNC) ) < 0) {
syslog(LOG_USER|LOG_INFO,"can't open /dev/mem ");
exit(-1);
}
/* mmap GPIO */
gpio_map = mmap(
NULL, //Any adddress in our space will do
BLOCK_SIZE, //Map length
PROT_READ|PROT_WRITE,// Enable reading & writting to mapped memory
MAP_SHARED, //Shared with other processes
mem_fd, //File to map
GPIO_BASE //Offset to GPIO peripheral
);
close(mem_fd); //No need to keep mem_fd open after mmap
if (gpio_map == MAP_FAILED) {
syslog(LOG_USER|LOG_INFO,"mmap error %d", (int)gpio_map);//errno also set!
exit(-1);
}
// Always use volatile pointer!
gpio = (volatile unsigned *)gpio_map;
} // setup_io
pi@raspberrypi ~/cpp/humandetector $ cmake .
-- The C compiler identification is GNU 4.6.3
-- The CXX compiler identification is GNU 4.6.3
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/pi/cpp/humandetector
make
pi@raspberrypi ~/cpp/humandetector $ make
Scanning dependencies of target humandetector
[100%] Building CXX object CMakeFiles/humandetector.dir/humandetector.cpp.o
Linking CXX executable humandetector
[100%] Built target humandetector
/etc/ssmtp/ssmtp.conf configration. Use Gmail. Do not deactivate Google 2 step verification. Use App Password to set AuthPass.
#
# Config file for sSMTP sendmail
#
# The person who gets all mail for userids < 1000
# Make this empty to disable rewriting.
root=xxxxx@gmail.com
# The place where the mail goes. The actual machine name is required no
# MX records are consulted. Commonly mailhosts are named mail.domain.com
mailhub=smtp.gmail.com:587
AuthUser=xxxxx@gmail.com
AuthPass=xxxxxxx
AuthMethod=LOGIN
UseSTARTTLS=YES
UseTLS=Yes
# Where will the mail seem to come from?
#rewriteDomain=
# The full hostname
hostname=localhost
# Are users allowed to set their own From: address?
# YES - Allow the user to specify their own From: address
# NO - Use the system generated From: address
#FromLineOverride=YES
Create humandetector under /etc/init.d/. Comment sentences in INIT INFO are necessary for update-rc.d command later. /lib/lsb/init-functions is also necessary for status_of_proc command.
#! /bin/sh
### BEGIN INIT INFO
# Provides: humandetector
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
### END INIT INFO
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="humandetector"
NAME=humandetector
DAEMON=/home/pi/cpp/humandetector/$NAME
SCRIPTNAME=/etc/init.d/$NAME
# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0
# Define LSB log_* functions.
# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
# and status_of_proc is working.
. /lib/lsb/init-functions
#
# Function that starts the daemon/service
#
do_start()
{
# Return
# 0 if daemon has been started
# 1 if daemon was already running
# 2 if daemon could not be started
start-stop-daemon --start --quiet --exec $DAEMON --test > /dev/null \
|| return 1
start-stop-daemon --start --quiet --exec $DAEMON \
|| return 2
return 0
}
#
# Function that stops the daemon/service
#
do_stop()
{
# Return
# 0 if daemon has been stopped
# 1 if daemon was already stopped
# 2 if daemon could not be stopped
# other if a failure occurred
start-stop-daemon --stop --quiet --name $NAME
RETVAL="$?"
return "$RETVAL"
}
case "$1" in
start)
do_start
;;
stop)
do_stop
;;
status)
status_of_proc "$DAEMON" "$NAME" && exit 0 || exit $?
;;
*)
echo "Usage: $SCRIPTNAME {start|stop|status}" >&2
exit 3
;;
esac
:
Make link files by update-rc.d command. The link files will be created in /etc/rc0.d/~/etc/rc6.d/. After that, humandetector will run automatically after the boot.
pi@raspberrypi ~ $ sudo update-rc.d humandetector defaults
update-rc.d: using dependency based boot sequencing
pi@raspberrypi ~ $ ls /etc/rc[0-6].d/*humandetector*
/etc/rc0.d/K01humandetector /etc/rc2.d/S02humandetector /etc/rc4.d/S02humandetector /etc/rc6.d/K01humandetector
/etc/rc1.d/K01humandetector /etc/rc3.d/S02humandetector /etc/rc5.d/S02humandetector
service command can controll humandetector daemon.
pi@raspberrypi ~ $ sudo service humandetector start
pi@raspberrypi ~ $ sudo service humandetector status
[ ok ] humandetector is running.
pi@raspberrypi ~ $ sudo service humandetector stop
pi@raspberrypi ~ $ sudo service humandetector status
[FAIL] humandetector is not running ... failed!