After setting up the cams at home and software on the server we can install and start the main recording script.
My first idea was just let the cam to upload the images via FTP as often as possible and then create video files from those single images by Ffmpeg in a daily job. This idea works, but we only get a video stream at max. 1 fps as the cam uploads image by image. For each image it takes a lot of time for FTP login and logout. Even on the very fast FTP servers it wouldn’t be possible to go much higher than 1 fps. Another problem is that the server needs lots of storage space for the images (30-40 KB per 640×480 px image) and then it takes hours of time with ~100% CPU load every night to compress the images to video files.
So I tried the live recording and encoding by Ffmpeg from the MJPEG stream in real time. I still need FTP upload in order to start recording the MJPEG stream only at times when motion is detected by the cam. The idea is very easy: start recording the stream when the cam connects to the FTP server and end recording automatically after one minute (if no other FTP connection was done by the cam in the meantime).
Fortunately Ffmpeg can use MJPEG streams directly as input format and convert them to x264 encoded mp4 files in real time. So recording the stream to a file is almoust easy. However there were some challenges:
- Start recording when the cam connects to the FTP server. This is done by monitoring the FTP logfile by inotify and starting Ffmpeg task when ipcam user logs in if that cam is not being recorded now. The PID of the ffmpeg process is stored in the pid file /var/run/ipcam?.pid. If the cam process is already runing it make just a touch on the pid file. So the last FTP connection time is the mtime of the pid file.
- Stop recording after 1 minute after the last FTP connection from the cam. This will done by a cronjob which runs every minute and checks when the last “FTP ping” was received from the cam. If the mtime of the pid file is older than 1 minute the recording process of that pid will be stopped. Cronjob is not the ideal solution as it runs only each minute. So the recording will run 1-2 minutes (and not exactly 1 minute) after the last FTP connection.
- Starting Ffmpeg job for indefinite time is easy, but how to stop it when it runs in the background? Just killing Ffmpeg process works mostly good, but in some cases it produces a corrupted mp4 file (no moov atom found). There seems to be no possibility to pause Ffmpeg and continue on the next FTP connection. The clean solution to end Ffmpeg is to send a keystroke “q” to the process. This is done through a fifo buffer.
- Each Ffmpeg job will produce a single mp4 file as Ffmpeg seems to have no ability to append to the existing mp4 file. At the end of the day we’ll have a set of mp4 files for each cam, one file for each recording process. If I want to have one mp4 file per day and cam, I can merge the mp4 files collected during the day into one very fast using MP4Box. This is done by finalizing job.
These are the basic requirements. I also implemented some extra features with Ffmpeg:
- Adding current timestamp (in a visible textbox) to the video is done by drawtext filter.
- Better video quality and much better compression become possible by using noise reduction for the ipcam images. This can be done by ffmpeg itself (use ‘-nr 1000’ configuration key) or even better with High precision/quality 3d denoise filter hqdn3d.
The main script itself is called ipcam2rec.sh and can be placed anywhere.
#!/bin/bash
# IPcam2rec: IPCam2rec: Surveillance IPcam Recorder
# Version 1.3
# Look at http://www.ozerov.de/ipcam2rec for more info
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version.
FTPDIR="/var/ip-cam"
FINDIR="/root/ip-cam"
BADDIR="/root/"
PIDDIR="/var/run"
LOGFILE="/var/log/ipcam.log"
FTPLOGFILE="/var/log/proftpd/proftpd.log"
CAMHOST="http://user:pass@mynick.dyndns.org"
DATETIME=`/bin/date +%Y%m%d-%H%M%S`
###################################
# FUNCTION start camera (give camid as param)
###################################
startcam () {
if [[ $1 =~ 'ipcam1' ]]; then cam='ipcam1'; port=1111; fi
if [[ $1 =~ 'ipcam2' ]]; then cam='ipcam2'; port=1112; fi
if [[ $1 =~ 'ipcam3' ]]; then cam='ipcam3'; port=1113; fi
if [ "$cam" != "" ]; then
PIDFILE="$PIDDIR/$cam.pid"
DATETIME=`/bin/date +%Y%m%d-%H%M%S`
if [ -f $PIDFILE ]; then
echo $DATETIME "Continue recording cam" $cam >> $LOGFILE
touch $PIDFILE
else
/usr/bin/mkfifo $PIDFILE.fifo 1>/dev/null 2>/dev/null
FONTFILE="/usr/share/fonts/truetype/freefont/FreeSans.ttf"
DRAWTEXT="fontfile=$FONTFILE:text='%Y-%m-%d %H-%M-%S':x=10:y=10:fontsize=13:fgcolor=yellow@0.9:box=1:bgcolor=blue@0.4"
nohup /usr/local/bin/ffmpeg -r 4 -er 4 -y -analyzeduration 0 -f mjpeg -an -i \
$CAMHOST:$port/videostream.cgi -vf hqdn3d,drawtext="$DRAWTEXT" \
-vcodec libx264 -vpre slow -crf 22 $FTPDIR/$cam/$cam-$DATETIME.mp4 1>/dev/null 2>&1 <$PIDFILE.fifo &
echo $! > $PIDFILE # Write process ID in the pid file
echo > $PIDFILE.fifo # Need this to start ffmpeg process
echo $DATETIME "Start recording cam" $cam "PID" $(cat $PIDFILE) >> $LOGFILE
fi
fi
}
###################################
# FUNCTION stop camera (give camid as param)
###################################
stopcam () {
PIDFILE="$PIDDIR/$1.pid"
PID=`cat $PIDFILE`
if [ "`ps -p $PID --no-headers`" != "" ];
then
echo $DATETIME "Stop recording cam" $1 "PID" $PID >> $LOGFILE
echo q > $PIDFILE.fifo # Keystroke q stops ffmpeg
else
echo $DATETIME "Cam already stopped" $1 "PID" $PID >> $LOGFILE
fi
/bin/rm -f $PIDFILE # Remove pid file, but let fifo file
}
###################################
# FUNCTION restart all running cams
###################################
restartallcams () {
if [ -f $PIDDIR/ipcam?.pid ]; then
for i in $PIDDIR/ipcam?.pid; do
cam=`echo $i | sed -ne 's/.*\(ipcam[0-9]\).*/\1/p'`
stopcam $cam
startcam $cam
done
fi
}
case "$1" in
###################################
# restartallcams
###################################
restartallcams)
restartallcams
;;
###################################
# install
###################################
install)
echo "Not implemented yet... Please install cronjobs by hand!"
;;
###################################
# stop camera (give PID filename as param)
###################################
stopcam)
cam=`echo $2 | sed -ne 's/.*\(ipcam[0-9]\).*/\1/p'`
stopcam $cam
;;
###################################
# stop daemon
###################################
stop)
# Kill daemon by killing the inotifywait process
# Note: daemon will be restarted by cronjob within a minute
if [ -f $PIDDIR/ipcam?.pid ]; then
for i in $PIDDIR/ipcam?.pid; do
cam=`echo $i | sed -ne 's/.*\(ipcam[0-9]\).*/\1/p'`
stopcam $cam
done
fi
sleep 5 # There will be a problem if a cam starts within this time
kill `pidof -s inotifywait` # Not 100% clean if more than one process
/bin/rm -f $PIDDIR/ipcam?.pid # Just to be sure...
echo $DATETIME "Stop daemon" >> $LOGFILE
;;
###################################
# start daemon
###################################
start)
echo $DATETIME "Start daemon" >> $LOGFILE
# NB: inotifywait looses log watching when log is rotating
# need to restart daemon in that case
inotifywait -mrq -e modify $FTPLOGFILE | while read file
do
cam=`tail -n 1 $FTPLOGFILE | sed -ne 's/.*USER \(ipcam[0-9]\).*/\1/p'`
if [ "$cam" != "" ]; then
startcam $cam
fi
done
;;
###################################
# finalize videos for a day
###################################
finalize)
# Job to be done each night at 0:00
restartallcams # New day new recording file
YESTERDATE=$2
if [ "$YESTERDATE" == "" ]; then
echo "Date YYYYMMDD not specified for finalizing"
exit 1
fi
if [ "$YESTERDATE" == "YESTERDAY" ]; then
YESTERDATE=`date --date "yesterday" +%Y%m%d`
fi
echo Processing cam records...
# Picture records from FTP upload: add datetime, convert to video
for cam in "cam1" "cam2" "cam3"; do
if [ "$(ls -A $FTPDIR/$cam 2>/dev/null)" ]; then
mkdir /tmp/ip-cam
x=1;
for i in $FTPDIR/$cam/*jpg; do # Not very clean, need to take only files of specified date
counter=$(printf %06d $x);
if [ "$3" == "clean" ]; then
mv "$i" /tmp/ip-cam/img"$counter".jpg;
else
cp "$i" /tmp/ip-cam/img"$counter".jpg;
fi
MTIME=$(stat -c "%x" /tmp/ip-cam/img"$counter".jpg);
PRIMITIVE="text 2,12 \"$MTIME\"";
/usr/bin/mogrify -draw "$PRIMITIVE" -fill blue -bordercolor blue /tmp/ip-cam/img"$counter".jpg 1>/dev/null 2>&1
x=$(($x+1));
done
/usr/local/bin/ffmpeg -r 3 -i /tmp/ip-cam/img%06d.jpg -vcodec libx264 -vpre slow -crf 22 /tmp/ip-cam/output_tmp.mp4 1>/dev/null 2>&1
/usr/bin/qt-faststart /tmp/ip-cam/output_tmp.mp4 $FINDIR/$cam/$cam-$YESTERDATE.mp4 1>/dev/null 2>&1
ls -l $FINDIR/$cam/$cam-$YESTERDATE.mp4
/bin/rm -fR /tmp/ip-cam;
fi
done
# MP4 records from deamon: check and merge mp4 files
for cam in "ipcam1" "ipcam2" "ipcam3"; do
if [ "$(ls -A $FTPDIR/$cam/$cam-$YESTERDATE*.mp4 2>/dev/null)" ]; then
for i in $FTPDIR/$cam/$cam-$YESTERDATE*.mp4; do
if [ "`/usr/bin/AtomicParsley $i -T | grep moov`" == "" ]; then
echo Atom moov not found in $i.
echo Corrupted MP4 file moved to $BADDIR.
mv $i $BADDIR
fi
done
/root/mergeMP4.sh -i "$FTPDIR/$cam/$cam-$YESTERDATE*.mp4" -o $FINDIR/$cam/$cam-$YESTERDATE.mp4 1>/dev/null
ls -l $FINDIR/$cam/$cam-$YESTERDATE.mp4
if [ "$3" == "clean" ]; then
/bin/rm -fR $FTPDIR/$cam/$cam-$YESTERDATE*.mp4;
fi
fi
done
;;
*)
echo "Usage: $0 {install|start|stop|stopcam id|finalize YYYYMMDD [clean]}"
exit 1
;;
esac
exit 0
Proceed to the configuring section.