10/24/2013

Turning Raspberry Pi into a Spycam using just a webcam (my first adventure with pi)

Start by going to http://www.raspberrypi.org/downloads and downloading the Noobs. Download the formatting tool and use it to format the SD card. To install one of the images Win32DiskImager will be needed. However, NOOBS does not require the disk image tool. Place the unzipped files on the SD card.

 

After booting it, the NOOBS will give you the option to install various different operating systems choose a flavor [raspbian for this tutorial]. The installation process should take some amount of time. It will reboot after installation and ask about a few more options in raspi-config. If for some reason you would like to change these options later:

 

sudo raspi-config

 

Ideally do not turn on the HDMI-GUI output “boot to desktop”, this saves just a little bit of system resources (but not much). There may also be an option to enable SSH, if so enable it. If you choose not to set a new password in raspi-config you can do it later with:

 

passwd pi

 

However, it’s wise to set a new password as soon as possible.

 

First off we need to make sure that openssh is working, so that we don’t have to type everything in. Putty allows us to access the command line from a remote computer as long as it is plugged into a local router or connected through a local wireless router. Later we will go over connecting wireless, but for now it might be best to have Raspberry Pi connected directly into a router. This way you can configure it from a desktop/laptop.

 

http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

 

The following command will give you a printout where you can find your IP address on the local

network provided by your local router. Use this IP address to configure Putty for connecting to Raspberry Pi remotely. Usually, the port is 22.

 

ifconfig

 

Upon first connect with Putty it will ask you if the public key is acceptable to you. If want to check if it is go back over to Raspberry Pi and type in this to verify it:

 

ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub

 

Since you are on your home router it is unlikely that anything is out of order here. Usually people just accept it without checking. If someone was man-in-the-middle attacking you the numbers would be different. The long string of letters and numbers should be exactly the same.

 

Once verified and connected these two commands are for good measure, they update the software packages and upgrade them. Also, it shows you that your openssh-Putty connection is working well. It would be appropriate to disconnect the monitor now, because we won’t be using it anymore. Everything from here on out will be done over the network.

 

sudo apt-get update && sudo apt-get upgrade

 

Once Putty is working and updates/upgrades are done we need to install motion.

 

sudo apt-get install motion

 

There are quite a few options for motion, it is highly customizable. It may seem daunting at first, but it’s not terribly complicated. We want to take advantage of all the customizable options. Read this document a few times to get familiar with it: http://linux.die.net/man/1/motion

 

To edit the .conf file:

 

sudo nano /etc/motion/motion.conf

 

Alternately if you would like to use a file other than the default provided:

 

sudo cp /etc/motion/motion.conf /etc/motion/motion.conf.originalbackup

sudo rm /etc/motion/motion.conf

sudo nano /etc/motion/motion.conf

 

Now copy/paste in your desired .conf file. To make it easy for a single webcam setup this is a good /etc/motion/motion.conf file. It is what we will be using for this tutorial.

 

# Advanced single webcam /etc/motion/motion.conf file

# credit to Phillip Moxley

# modified from original generated file most helpful comments are removed

############################################################

daemon on

process_id_file /var/run/motion/motion.pid

setup_mode off

#######################################################

#Capture device

videodevice /dev/video0

v4l2_palette 8

input 8

frequency 0

rotate 0

width 640

height 480

framerate 2

minimum_frame_time 0

netcam_tolerant_check off

auto_brightness off

brightness 0

contrast 0

saturation 0

hue 0

#############################################

#Round Robin

# This is for multiple webcams on the same device

roundrobin_frames 1

roundrobin_skip 1

switchfilter off

###############################################

#Motion Detection Settings:

threshold 1500

threshold_tune off

noise_level 32

noise_tune on

despeckle EedDl

smart_mask_speed 0

lightswitch 0

 

## These motion detect settings are important for our purposes

############################################

# Picture frames must contain motion at least the specified number of frames

# in a row before they are detected as true motion. At the default of 1, all

# motion is detected. Valid range: 1 to thousands, recommended 1-5

minimum_motion_frames 1

 

# Specifies the number of pre-captured (buffered) pictures from before motion

# was detected that will be output at motion detection.

# Recommended range: 0 to 5 (default: 0)

# Do not use large values! Large values will cause Motion to skip video frames and

# cause unsmooth mpegs. To smooth mpegs use larger values of post_capture instead.

pre_capture 2

post_capture 5

 

# Gap is the seconds of no motion detection that triggers the end of an event

gap 60

 

output_all off

##################################################

# Image File Output

#turn on-off images

output_normal on

quality 75

output_motion off

ppm off

###################################################

# Video Options

 

# turn on-off video

ffmpeg_cap_new on

 

# mess with this to attempt higher-lower video quality

ffmpeg_bps 500000

ffmpeg_variable_bitrate 0

ffmpeg_video_codec swf

ffmpeg_deinterlace off

 

# 0 turns off timelapse

ffmpeg_timelapse 5

# Valid values: hourly, daily, weekly-sunday, weekly-monday, monthly, manual

ffmpeg_timelapse_mode weekly-sunday

 

ffmpeg_cap_motion off

snapshot_interval 0

############################################################

# Text Display Settings

 

# Text is placed in lower right corner

text_right %m-%d-%Y\n%T-%q

 

# Text is placed in lower left corner

text_left PHILLIP MOXLEY CAMERA %t – RESTRICTED USAGE

 

# This option defines the value of the special event conversion specifier %C

# You can use any conversion specifier in this option except %C. Date and time

# values are from the timestamp of the first image in the current event.

# Default: %Y%m%d%H%M%S

# The idea is that %C can be used filenames and text_left/right for creating

# a unique identifier for each event.

text_event %Y%m%d%H%M%S

text_double off

locate off

text_changes off

############################################################

# Target Directories and filenames For Images And Films

target_dir /home/motionDL

# File path for motion triggered images

jpeg_filename images/%v-%Y%m%d%H%M%S-%q

# File path for motion triggered ffmpeg films (mpeg)

movie_filename movies/%v-%Y%m%d%H%M%S

# File path for snapshots

snapshot_filename snapshots/%v-%Y%m%d%H%M%S-snapshot

# File path for timelapse mpegs

timelapse_filename timelapses/%Y%m%d-timelapse

###########################################################

# Live Webcam Server

 

# on means off and off means on for the live webcam server

webcam_localhost on

# Quality 1-100

webcam_quality 50

webcam_port 8081

webcam_limit 0

webcam_motion on

webcam_maxrate 1

 

############################################################

# HTTP Based Control

 

# on means off and off means on

control_localhost on

# be careful this password will not be encrypted by default

; control_authentication username:password

control_port 8080

control_html_output on

quiet on

 

#These options allow us to write some scripts for extending the functionality

; on_event_start ./home/pi/start.sh

; on_event_end ./home/pi/end.sh

; on_picture_save ./home/pi/picsav.sh; export f=%f

; on_motion_detected ./home/pi/motdet.sh

; on_movie_end ./home/pi/vidcon.sh; export f=%f

; on_area_detected value

; on_movie_start value

; on_camera_lost value

 

Look at the bottom of the putty/command line interface, ctrl+s will not work. Once saved (ctrl+o [enter] ctrl+x) it’s a good idea to restart motion.

 

sudo /etc/init.d/motion restart

 

If you get a message telling you that the dameon is disabled, even though you enabled it in the .conf file, it’s another file that disables it. Just edit the file below and change no to yes and retry the command above.

 

sudo nano /etc/default/motion

 

We need to create the folder that motion will output to and let the motion user have the ability to create the sub-folders. If you are using an ftp is is not advisable to leave the top folder at permissions 777, because it may prevent vsftpd from having access. More on this later on after everything is working correctly. Make sure to walk in front of the camera a few times to get motion to make the directories that you want.

 

sudo mkdir /home/motionDL

sudo chmod 777 /home/motionDL

 

For now we will step aside and complete a few other tasks before filling out the scripts. We want to be able to access the files on Raspberry Pi without having to remove the SD card. This means we can check if things are working correctly. We need to install an ftp server. However, not just any ftp server will do. We want to be able to access the files through an encrypted ftp. There’s a program for this!

 

sudo apt-get install vsftpd

 

Again we get another one of these really long .conf files. It’s not as long as the previous one, but you should read this a couple of times to familiarize yourself with it: http://linux.die.net/man/5/vsftpd.conf.

 

Truly, it may be advisable to separate reading either of these by a good day or so. For the more crazy among us, back-to-back reading is preferable. Just make sure not to mix up commands from either .conf file.

 

sudo cp /etc/vsftpd.conf /etc/vsftpd.conf.originalbackup

sudo rm /etc/vsftpd.conf

sudo nano /etc/vsftpd.conf

 

Paste this in:

 

# Advanced vsftp /etc/vsftpd.conf file

# credit to Phillip Moxley

# modified from original generated file most helpful comments are removed

##############################################################

listen=YES

#listen_ipv6=YES

anonymous_enable=NO

local_enable=YES

write_enable=YES

dirmessage_enable=YES

use_localtime=YES

xferlog_enable=YES

xferlog_file=/var/log/vsftpd.log

connect_from_port_20=YES

idle_session_timeout=600

data_connection_timeout=120

use_localtime=YES

 

# Customization

listen_port=2211

#debug_ssl=YES

#nopriv_user=<DummyUsername>

#chroot_local_user=YES

#chroot_list_enable=YES

#chroot_list_file=/etc/vsftpd.chroot_list

#allow_writeable_chroot=YES

secure_chroot_dir=/var/run/vsftpd/empty

pam_service_name=vsftpd

 

#Security

ssl_enable=YES

ssl_ciphers=HIGH

allow_anon_ssl=NO

force_local_data_ssl=YES

force_local_logins_ssl=YES

ssl_tlsv1=YES

ssl_sslv2=YES

ssl_sslv3=YES

rsa_cert_file=/etc/ssl/certs/vsftpd.pem

rsa_private_key_file=/etc/ssl/certs/vsftpd.pem

 

After editing/saving the .conf file restart vsftpd

 

sudo /etc/init.d/vsftpd restart

 

Right now without anything else you can use Filezilla to login to port 22 using the same certificate, password, and IP address that putty is using. Make sure to use SFTP instead of FTP. If you want to ensure that the SSH cert is being used delete the saved key from the registry. Go to start, type in regedit, navigate to HKEY_CURRENT_USER>Software>SimonTatham>SshHostKeys, and delete the key corresponding to your Raspberry Pi host. After restarting Filezilla it will ask you to confirm the key again when you login. Deleting the key from the registry makes Putty/Filezilla forget about the cert, nothing else. Once you accept it again it will be put back into the registry in the same place. Clicking ‘no’ will let you use the cert key, but not cause it to be put back into the registry.

 

Doing it the easy way above would mean that the ftp user would have access to ALL files on Raspberry Pi. For extra security there is another way of doing the ftp encryption. We will create another user that has access only to the files created by motion and use a different type of connection (so that your SSH tunnel cannot be compromised). Obviously this second option is preferable. However, it makes things so much more complicated. We have to make an SSL cert, a new user, and all sorts of other stuff.

 

To encrypt the traffic for another connection we need to generate an new SSL certificate. This cert will be unsigned because it’s not intended for use with other people. It’s a personal cert generated by you.

 

cd /etc/ssl/certs

sudo openssl req -x509 -nodes -days 7300 -newkey rsa:2048 -keyout /etc/ssl/certs/vsftpd.pem -out /etc/ssl/certs/vsftpd.pem

 

If this command does not ask you a bunch of weird questions, it didn’t work. It doesn’t matter what you type in as long as the country code is only 2 upper case letters and the email address has an @ symbol in the right place. After the cert is generated make sure the permissions are good.

 

sudo chmod 600 /etc/ssl/certs/vsftpd.pem

 

In Filezilla instead of selecting SFTP, change it to FTP and set the port to 2211 (or whatever you changed it to in /etc/vsftpd.conf), and change the encryption to “Require explicit FTP over TLS”. Leaving the username:password the same it should give you a much more complicated certificate trust box. If you would like to check the fingerprint go back to the command line and try this:

 

sudo openssl x509 -noout -in /etc/ssl/certs/vsftpd.pem -fingerprint -sha1

 

Now the FTP and SSH connect differently. The SFTP will still remain accessible, but we can create a different user for connecting over the FTP via SSL. This means that only the files created by motion will be accessible to this user, and only the traffic over the SSL will be ‘listenable’ to said user. Nothing else can be messed with.

 

To create the new user:

 

sudo useradd -d /home/motionDL <username>

sudo passwd <username>

 

This makes the folder motion is putting it’s files into the home folder of the new user. Logging in with this user/password in Filezilla will direct you straight to this folder. However, the user still has access to all of the other files on the Raspberry Pi host. There are also a few other security concerns.

 

Make a dummy user

 

sudo useradd -d /var/run/vsftpd/empty <DummyUsername>

 

Go back into the vsftpd.conf file and edit the comments out of the following lines:

 

sudo nano /etc/vsftpd.conf

 

#This protects against some unauthorized access

nopriv_user=<DummyUsername>

 

#If this is changed to YES, then it will jail all users that are not on the list below

#doing the opposite of what we want

chroot_local_user=NO

 

#This allows users to be ‘trapped’ or ‘jailed’ in their home directory

chroot_list_enable=YES

 

#This is a list of ‘trapped’ users

chroot_list_file=/etc/vsftpd.chroot_list

 

Now edit the ‘trapped’ user list and add the other user you created to control the output of motion (/home/motionDL/). Do not add the DummyUsername to this list.

 

sudo nano /etc/vsftpd.chroot_list

 

Restart vsftpd:

 

sudo /etc/init.d/vsftpd restart

 

The settings we have used should prevent you from logging in via the SSL FTP. This is because vsftpd attempts to block bad configurations. If vsftpd was to allow you to login, you would have the full ability to delete /home/motionDL. So let’s fix this and try again.

 

cd /home/

ls -al

 

The /home/motionDL/ folder should be owned by root root, and have rwxrwxrwx on it so anything can write to it or execute. This is because motion needs the ability to write the folders inside it. If it is owned or grouped with anything other than root run these

 

sudo chown root motionDL

sudo chgrp root motionDL

 

Next vsftpd wants the ftp user to lack the ability to mess with the top level folder because it is a ‘bad’ configuration:

 

sudo chmod 755 motionDL

 

Upon logging in via Filezilla, you will notice that it only displays the output folders of motion. All the captures are there, and nothing else. Keep in mind that motion should be prohibited from creating new folders using the permissions, but also should be able to add files to these folders. However, you can only download these files, and cannot delete them, rename them, or add files.

 

If you would like that ability simply add your ftp user to the motion group and modify the permissions of the folders.

 

sudo usermod -a -G motion <username>

cd /home/motionDL

sudo chmod 775 * #WARNING BE CAREFUL

 

Be careful with that last command if you are not in /home/motionDL/ it can cause some pretty serious damage. If you would like to remove the ability for the ftp user to modify files simply remove it from the group. The idea is to change the permissions for all of the folders that motion outputs, so the above and below commands will not work if motion has not created the folders already.

 

sudo gpasswd -d <username> motion

cd /home/motionDL

sudo chmod 744 * #AGAIN WARNING BE CAREFUL

 

Your ftp user cannot delete /home/motionDL or delete the folders created by motion. However, you can delete, add, or modify anything inside these folders. If this is the way you want it that is fine. It’s also possible to make the files read only.

 

Keep in mind that there are other strategies for making this work. For example, disabling “write_enable=YES” by commenting it out in /etc/vsftpd.conf so that files cannot be removed, or using virtual users https://wiki.archlinux.org/index.php/Very_Secure_FTP_Daemon#Tips_and_tricks.

 

Now we must make the scripts and make them executable. These commands can be run to just make the files for the scripts. It’s okay to leave nothing in them and run motion because nothing will happen if the files are empty. Also, it’s okay to just copy/paste the whole set of commands. They will all run back to back. The last one might require an [enter].

 

cd ~

touch start.sh && sudo chmod +x start.sh && sudo chown motion start.sh && sudo chgrp motion start.sh

touch end.sh && sudo chmod +x end.sh && sudo chown motion end.sh && sudo chgrp motion end.sh

touch picsav.sh && sudo chmod +x picsav.sh && sudo chown motion picsav.sh && sudo chgrp motion picsav.sh

touch motdet.sh && sudo chmod +x motdet.sh && sudo chown motion motdet.sh && sudo chgrp motion motdet.sh

touch vidcon.sh && sudo chmod +x vidcon.sh && sudo chown motion vidcon.sh && sudo chgrp motion vidcon.sh

 

type in either of these to verify that they are there:

 

ls

ls -al

 

The next problem is that we don’t want to have to download all of these files and browse through them in order to see what is in them. It can be tedious to view a couple thousand images, or even several dozen videos. We want to simply inspect one file. There is a couple of ways of doing this but, it depends on how you plan to use your Raspberry Pi – Webcam setup.

 

  1. Live Host (turn off all file saving – enable host and skip ahead)
  2. Busy place (turn off video – leave images and timelapse on)
  3. Non busy place (turn everything on – add script for video concatenation)

The easiest way of inspecting the output is using the timelapse feature. However, it all depends on exactly what you are doing. If you plan on using the Raspberry Pi to host a live camera, then it might be best to disable the video, timelapse, and image capture. Edit the live webcam server settings to work. Do this in /etc/motion/motion.conf. Skip to the part about port forwarding to get the hosted webcam through your router and give it a url.

 

output_normal off

ffmpeg_cap_new off

ffmpeg_timelapse 0

# on means off and off means on for the live webcam server

webcam_localhost on

 

If you plan on using this to watch something that is always active like a restaurant or a busy street then saved video will not be useful at all. However, the stills and the timelapse will be useful.

 

output_normal on

ffmpeg_cap_new off

ffmpeg_timelapse 5

 

If you plan on watching something that is rarely active like a vault or a secret lair then all three are needed.

 

output_normal on

ffmpeg_cap_new on

ffmpeg_timelapse 5

 

This way if it’s a busy camera; it doesn’t get overloaded by the video, if it’s a not-so busy camera; it can be convenient, and if it’s a broadcast camera; there isn’t much to be done in ‘recording’.

 

The trick to the video is that motion does not output mpeg1/2 files through ffmpeg (go figure). In order to output a video that is easy to watch we need mpeg1/2 video files because they can be easily spliced together. We could go grab an older version of ffmpeg and try to cut through all the possible errors of installing deprecated software, but that could be problematic.

 

We are going to write a script to convert the video files into mpeg1/2 and splice them together. This script will kick off each time a new video is produced by motion. The script will convert the video to an mpg and add it to the end of the last one. Thus we will end up with a nice single file to download. There are some problems with this. If somebody dances in front of your camera for an hour then Raspberry Pi will not have the resources to convert the output. Thus the script needs to be told to stop if the previous video has not been processed.

 

Do not use this file if you plan on watching a busy scene.

 

cd ~

nano vidcon.sh

 

#!/bin/bash

#vidcon.sh

#credit Phillip Moxley

#runs on completion of a video event captured in motion

#converts to mpg and appends, stops if previous video has not completed

#All log and debug options removed

 

#How many files are in the folder

cd /home/motionftp/movies

n=0

n=$(ls *.swf | wc -l)

 

#Is a conversion process running right now?

p=0

p=$(ps -ef | grep bin | pgrep vidcon.sh | wc -l)

 

#Convert or abort

if [ $p -eq 2 ]; then

for i in *.swf

do

#convert the file

yes | avconv -i “${i}” -r 20 temp.mpg

#splice the video together

cat temp.mpg >> output.mpg

#remove completed files

rm $i temp.mpg

done

else

cd ~

echo $(date) dammit process is running \ >> loggylog.log

fi

 

The script will execute when a video is over which is governed by the gap variable in /etc/motion/motion.conf. If the video is really long and something else comes along, it will not run anymore ensuring that Raspberry Pi doesn’t get overwhelmed. However, Raspberry Pi was not intended to be constantly re-encoding video. Basically, this script will make one big file for you to watch (that is not timelapsed). This means that if the SD card is not big enough to handle a clown dancing in your vault for 3 hours, then don’t use the script… or get a bigger SD card… or figure something else out.

 

There are other scripts that can run at various times. The settings in the /etc/motion/motion.conf file are explained below

 

on_motion_detect and on_event_start start as soon as motion is detected. The difference is that on_motion_detect will fire off even if the ‘gap’ below has not expired while on_event_start will wait until the gap time is over

 

The number of frames that motion sees before it starts recording governs when on_picture_save starts, and the on_movie_start starts. The difference is that on_picture_save starts at the end of a picture save and passes a %f filename variable to the command. While on_movie_start simply marks the moment a movie begins to save.

 

This bit is pretty self explanatory

minimum_motion_frames 1

 

The number of frames to keep in a video prior to a video starting. Governs how many frames are pre-pended to the video

pre_capture 2

 

Number of frames to capture after motion is no longer detected (default: 0). Governs number of frames that are post-pended to the video.

post_capture 5

 

The gap is the seconds of no motion detection that triggers the end of an event. It is the number of seconds to wait until on_movie_end and on_event_end are triggered. The difference being that on_on_movie_end passes the %f filename variable and is triggered after the video file is closed, while on_event_end happens exactly at event end time.

gap 60

 

You might have noticed that motion is not providing the correct time on the time-stamp in the video and images. This is caused by several issues. One of the issues is that it might be using UTC, not your local time. The other issue is that Raspberry Pi cannot store time without a battery, or a connection to the internet.

 

To solve the first issue, we could simply add UTC to the time-stamp. This still might confuse people.

 

text_right %m-%d-%Y\n%T-UTC-%q

 

However, it might be better to configure Raspberry Pi to output your local time zone. In this way you can replace UTC with the time-zone you are in.

 

sudo raspi-config

 

In the internationalization options pick time-zone, and set it to your time-zone. Also, in Filezilla there is a place in the site manager to adjust the time difference between the server and the client under the advanced tab.

 

WIRELESS NEWORKING (Networking issue No. 2)

 

We have a fully functioning unit now. However, we need to work on the network and connectivity issues (because the clock is off right?). Up until now we have been operating over a wired local wired network or standalone. If we want to work over a wireless network there are some steps. Additionally, if we want to access the machine from some remote location there are even more steps.

 

Be careful in your selection of a wireless dongle because not all dongles will work with all linux distros or Raspberry Pi. Additionally, some dongles will require a powered usb hub because the Pi cannot supply enough energy to run the wireless dongle. Even worse, if your power supply is within spec voltage (but not enough current), it may cause the dongle to be under-powered and simply not function or shut off after an amount of time. To make this issue the mother of all complexity, some dongles must be told to stay on even if power starved.

 

There are two methods of using wireless the ‘router’ method and an ‘adhoc’ method. For here we will assume you have a home router with wifi, and the password for this router. The ‘adhoc’ thing will be set aside for some other time.

 

The easy way:

 

sudo nano /etc/network/interfaces

 

add the following lines:

 

wpa-ssid “XXXXXX-XXXXXX”

wpa-psk “XXXXXXXXX”

 

Where ssid is exactly the name of the router, and psk is the password. This might not work very well, or it could be splendid.

 

The hard way:

 

http://sirlagz.net/2012/08/27/how-to-use-wpa_cli-to-connect-to-a-wireless-network/

http://pingbin.com/2012/12/setup-wifi-raspberry-pi/

 

 

If the network does not automatically connect after following the above link’s steps try enable 0, reassociate, or the help file: http://linux.die.net/man/8/wpa_cli . Also, make sure that the .conf file has your ssid and psk in it.

 

sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

 

SOME STUFF HERE

 

network={

ssid=”XXXXXX-XXXXXX”

psk=”aaabbb111″

OTHER STUFF HERE

}

 

For the Power starved:

 

Add this to the /etc/network/interfaces file… just paste it in anywhere, like at the end.

 

wireless-power off

 

If that doesn’t work there might be some options in some crevice of linux that you can change to force a dongle to not shut off intermittently. However, it is different for each dongle.

 

http://forum.stmlabs.com/showthread.php?tid=9032

 

REMOTE CONNECTING (Networking issue No. 8)

 

Since we have hooked up both SFTP and SSL FTP it is totally acceptable to connect to Raspberry Pi over the internet. First and foremost we need to access the local router. This means knowing the local router IP address, username, and password. First type in the local router IP address into the address bar of a web browser. Next login with the username and password.

 

Regrettably, most routers differ here. Every brand has a similar, but unique looking user interface. Heck there is even a type of linux ‘custom’ operating system for routers. {{LINK}}. Even better it is possible to setup a Raspberry Pi as if it was your router to do awesome things! The objective is to port forward to the Raspberry Pi. The secondary objective is to reserve an address for the Raspberry Pi.

 

Pick a number between 1 and 65536, but there is a long list of ports you want to avoid (e.g. 20, 21, 22, 80, 8080). Take that number and forward it to port 2211 (or whatever you changed it to) and the local IP address of Raspberry Pi. The idea is simple enough once you wrap your head around it.

 

Find your public IP (not to be confused with your local IP). Seach “what is my IP” or something like that . If you really want to be annoying call your ISP and ask. If you port forwarded your random number to the local IP and your listening port then it should work fine.

 

[Public_IP]:[Public_Port] —> [Local_IP_Address_of_Raspberry_Pi]:[2211_or_your_listening_port]

268.301.22.36:4798 —> 10.0.0.109:2211

 

All of this nonsense means that you should be able to type into the address bar of your browser [Public_IP]:[Public_Port], and get the same thing you would get if you typed [Local_IP]:[listening_port]. The only difference is that the public version will work anywhere on the internet, while the local one only works at home.

 

Hang on, because it works just the same for Filezilla and/or Putty. This means you can fully access the thing from anywhere if you know your public IP, public port, username, password, and fingerprints. What? Yeah you might want to write down the fingerprints, unless you are using a laptop that has them saved. If you try to connect remotely and the SSH or SSL fingerprint is not the same its a bad sign. It would be wise not to type in a password because it is probably not private.

 

If you are like most people, your ISP has not given you a static IP address because they want more money for that. This means that at any random time you may not be able to connect to the Raspberry Pi remotely because the ISP changed it. In truth it is more of an arbitrary-random time. It all depends on your ISP. Typically, you can expect it not to change except on a blue moon. However, it’s not your decision or the moons decision.

 

In order to solve this we want to replace your public IP address with a url of some kind. This can be done in most router interfaces, but we’re going to do it with the pi itself instead. This will make the Raspberry Pi report on it’s location to a website at an interval of your choosing. Again, it will report it’s location to a third party if it has any connection to the internet at all. This will happen even if you cannot access Raspberry Pi to tell it to stop. There are other ways of getting around DHCP’s random IP address assignment, but this is the easiest.

 

It starts with:

 

sudo apt-get install ddclient

 

The installation will ask you a bunch of questions, but it’s best to type in nonsense for these. You can edit it later with this command:

 

sudo nano /etc/ddclient/ddclient.conf

 

There are some heavy privacy concerns to weigh on here. Configuring this software will make the device report it’s public ip address to the service you choose on a regular basis. This will happen weather or not it has port forwarding enabled on it’s local network. Meaning that if you cannot access the unit, it will still report it’s public IP address.

 

As a basic common courtesy if it is a free service, please make ‘daemon=XXX’ greater than 3600, which translates into once an hour or less. It may sound like a pain, but good manners will get you more than you expect.

 

dyn.com/dns/ is a website that used to offer free dynamic IP services for free, but now charge $10/year

 

dnsdynamic.org is a website offering the service for free.

 

Do some searches on exactly how to configure ddclient. This might help: http://sourceforge.net/p/ddclient/wiki/usage/

 

Connecting a wireless network from the cli:

http://lcdev.dk/2012/11/18/raspberry-pi-tutorial-connect-to-wifi-or-create-an-encrypted-dhcp-enabled-ad-hoc-network-as-fallback/

 

 

AD-HOC WIRELESS NETWORKING (Networking issue No. 4)

 

Omitted

 

BYPASSING RESTRICTIONS (Networking issue No. 5)

 

This section will not help you. It will only make you ponder things. If you do not have access to the local network router, then you may not be able to remotely access the Raspberry Pi. A few ideas about solving this issue will serve as a refreshment.

 

Tor browser can somewhat mitigate privacy concerns especially related to the dynamic IP host provider you have chosen.

 

Packets can be wrapped in an http wrapper making them appear as regular web traffic if your local network prohibits SSL or SSH packets.

 

It’s illegal, but Backtrac5 can be used to decrypt WPA passwords in somewhere between hours and months. Except, it requires a much more powerful machine than Raspberry Pi.

 

BONUS MATERIAL: AUDIO (Programming issue No. 15)

 

So your Raspberry Pi is keeping your secret fortress under constant private surveillance. However, if someone finds your little spy, then you don’t get to have the images/video that this fool invader steals. Hence, we need to have it upload the files to a remote ftp server somewhere offsite. We need this to also happen privately and encrypted.

 

The less recommended option is to use google docs, or some other free web service like mega. These will start you on your way to uploading to google docs, but I’m not very interested in that… what with current events and all.

 

sudo apt-get install python-gdata

sudo dpkg -i http://googlecl.googlecode.com/files/googlecl_0.9.5-1_all.deb

 

Since we already know how to set up an FTP server with vsftpd we can set one up wherever we like. The crux of the problem lies in the settings on this second server. You see, if this fool that broke into your lair takes this Raspberry Pi with him, he might be a super smart guy and be able to find the password to the backup ftp. As ridiculous as this sounds, it means that the script accessing the remote ftp cannot contain the password. Guess what! The google docs stuff above would require you to save your google password (or the password for the google docs account you created) in a script.

 

On another machine far away from your secret palace all you have to do is repeat most of the steps previously gone over. Except, change the vsftpd server to allow anonymous uploads. Before doing this I recommend reading the manual a good dozen times. http://vsftpd.beasts.org/vsftpd_conf.html

 

Yes, read that until you can almost quote it, because in order to keep everything secure on this server that accepts anonymous uploads, you’re going to have to be crafty. The weakness is that this invader can merely upload a bunch of nonsense files to your backup server before he arrives to steal your Pi. Of course, this is reaching ridiculous proportions.

 

The on_picture_save option in the configuration file passes the full path of the saved image into the command with %f. However, it is somewhat difficult to pass this variable into a script. To make things as easy as possible just add the following command to the /etc/motion/motion.conf file, and remove the part that calls the script up.

 

sudo nano /etc/motion/motion.conf

 

on_picture_save curl -v -k –ftp-ssl -T %f ftp://anonymous@[server_IP_address or URL]:[server_port]/images/

 

Curl should come installed by default. There are several ways of uploading to a remote ftp. However, they all have some kind of problem or another. For example, it appears wput has dropped ssl/tls support. FTP uploading is easy enough to do without ssl, there are many simple tools for uploading via the command line. For example wput can easily upload just like curl. However, wput appears to have dropped support for ssl/tls. Without ssl it might be possible to view the images as they are passed to the remote ftp server.

 

An example of wput without ssl:

 

sudo apt-get install wput

sudo nano /etc/motion/motion.conf

 

on_picture_save wput %f ftp://[serverIP or serverURL]:[Server Port]/images/

 

Another alternative using lftp. Lftp is a fully configurable command line ftp client. It’s nothing less than a command line version of Filezilla. Lftp brings up it’s own sub-command-line interface. The difficulty here is in passing the %f filename path into the command.

 

sudo apt-get install lftp

sudo nano picsav.sh

 

#!/bin/bash

#picsav.sh for whole folder upload. credit: Phillip Moxley

lftp -c ‘set ftp:ssl-force true; \

set ftp:ssl-protect-data true; \

set ftp:ssl-allow true; \

set ftp:ssl-allow-anonymous true; \

set ftp:ssl-auth TLS; \

set ssl:verify-certificate false; \

connect ftp:anonymous@[host_url_or_IP]:[hostport]; \

lcd /home/motionDL/images/; \

cd /images/; \

mput * ‘

 

This script will attempt to upload the entire folder every time an image is saved, so it’s not very useful for the images as it will just overload Raspberry Pi with nonsense. However, you can modify it for use in a cron to upload backups of videos or audio.

 

Speaking of audio, motion does not record audio. Most webcams have mics on them, and it seems a shame to waste the thing.

 

There is a way of using arecord (a linux default) to record audio

http://www.lavrsen.dk/foswiki/bin/view/Motion/SoundAudioRecording

 

However, sox is much better. http://linux.die.net/man/1/sox

 

sudo apt-get install sox

 

There are several things that you might want to do with audio. Once you think about it, it makes perfect sense for motion to leave audio out of the mix. There is just too much for such a simple tool to command.

 

First, you might want to add audio to the movies that motion outputs. However, this would mean that the audio might be cut short or long. The motion detect would govern when the audio starts and stops. There might be pieces that you miss this way, or extra long bits of silence.

 

Second, you can run sox in a way that it records audio in the same manner as motion. This means that sox can capture audio when it occurs. In essence this would be like a ‘audio detector’ in the same way motion is a ‘motion detector. The drawback on this is that the audio will rarely match up to the images/video. There will be bits of audio with no video and some images/video without any audio.

 

Thirdly, another idea is that you might want to stream audio alongside the hosted webcam. Either way, it is highly unlikely that the audio will be uploaded to a remote server if discovered. This is the weakness of the audio. This makes streaming the most highly desired option. As streaming means immediate upload.

 

Just to show we can do it let’s add audio to the video output that we created earlier. Add the following line to the start.sh script. It will start recording whenever an ‘event’ starts. This is for audio to be added to the video only, so it should be in the /movies directory.

 

AUDIODEV=hw:1,0 AUDIODRIVER=alsa rec /home/motionDL/movies/audio.wav &

 

Something that may bug you to no end is that your user ‘motion’ which runs all of the motion related things in the background does not have the ability to run the above command. This is because it is not in the “audio” group. So let’s add it to that group and reboot to make sure it’s all in order.

 

sudo usermod -a -G audio motion

sudo reboot

 

To check that it worked try this and look for the group named audio.

 

cat /etc/group

 

Another way to check would be to run the script under the user ‘motion’ just to check if it runs. However, you might have to use ctrl-c to get out of it.

 

sudo -u motion ./start.sh

 

If you are having trouble with the hardware (or AUDIODEV) part of this whole operation, then try restarting a couple of times. Also, this tool might help a little.

 

sudo apt-get install uvcdynctrl

 

If the start.sh script is putting audio into the /movies folder then we can add a few lines to the vidcon.sh script and it will combine everything for us

 

cd ~

sudo nano vidcon.sh

 

Change the script to look more like this:

 

#!/bin/bash

#vidcon.sh

#credit Phillip Moxley

#runs on completion of a video event captured in motion

#converts to mpg and appends

#stops if previous video has not completed

 

#stop the audio reording

killp=$(ps -ef | grep motion | grep rec | awk ‘{print $2}’)

kill -2 $killp

 

#How many files are in the folder

cd /home/motionftp/movies

n=0

n=$(ls *.swf | wc -l)

 

#Convert or abort

if [ $n -eq 1 ] ; then

#grab the gap seconds from the config file

vidgap=$(cat /etc/motion/motion.conf | grep gap | awk ‘{print $2}’)

 

#trim the excess audio off of the file per the gap

sox -v 8 -r 8000 audio.wav choppedaudio.wav trim 0 0:$vidgap

 

#add audio to video compile command

yes | avconv -i *.swf -i choppedaudio.wav -r 20 temp.mpg

 

#finishes it all up

cat temp.mpg >> output.mpg

rm *.swf temp.mpg recaudio.wav choppedaudio.wav

else

cd ~

echo dammit_too_many $(date) >> /home/motionftp/movies/loggylog.log

fi

 

It might sound like a good idea to mix the audio in with the video. However, it really isn’t the most ideal. It puts some heavy processing loads placed on Raspberry Pi to incessantly re-encode all sorts of video and mix it with audio. It cuts down on everything.

 

The only thing left would be to figure out the whole streaming to a private and/or public server. If you want some clues on how to stream the audio/video this link will help:

http://www1.packtpub.com/article/webcam-and-video-wizardry

 

 

 

 

 

 

 

 

arecord -f S16_LE -r 22050 -D plughw:%t /home/motionftp/audio/%Y%m%d_%H%M

%S_%t.wav

 

b=0; for i in *.swf; do avconv -i “$i” -c:v mpeg2video -r 20 “b.mpg” ;b=b

+1 ; done

 

b=0; for i in *.swf; do avconv -i “$i” -c:v mpeg2video -r 20 “$b.mpg” ;b++

; done

 

for i in *.swf; do

 

x=0; for i in $(ls -t *swf); do counter=$(printf %05d $x); ln -s “$i”

“$counter”.swf; x=$(($x+1)); done

 

 

 

cd /home/pi/motion/movies/ && cat *.swf | ffmpeg -i – -ar 44100 tmp.flv &&

mv tmp.flv VideoOutput`date +%m%d%y`.flv

 

 

 

-something to combine audio pieces

-silence to chop of ends of long silence

 

-a set up prototype upload scripts

-ftp host on kip, mutley, and a card ready for pi2

 

-some more vsftpb reading >> figure out the whole user problem

 

-a cron to stop motion, move files to archives,

-a cron to check disk space used by motion and delete accordingliny– also

send a red flag up.

 

cd /home/motionftp/movies

avconv -i *.swf -r 20 temp.mpg

ps -ef | grep avconv | grep swf

 

 

ps -ef | grep recaudio.wav | awk ‘{print $2}’ | xargs kill -SIGINT

 

 

 

 

 

sudo apt-get install chkconfig

chkconfig <service> off

 

sox output.wav -n stats -s 16 2>&1 | awk ‘/^Max\ level/ {print int($3)}’

#vsftp /etc/vsftpd.conf anon host, credit to Phillip Moxley

##############################################################

# Customization

listen_port=2211

listen_address=192.168.0.6

 

#Anonymous

anonymous_enable=YES

no_anon_password=YES

anon_upload_enable=YES

chown_uploads=YES

#chown_username=kip

#chown_upload_mode 0444

 

#Standard

listen=YES

write_enable=YES

use_localtime=YES

#xferlog_enable=YES

#debug_ssl=YES

#xferlog_file=/var/log/vsftpd.log

 

#Security

ssl_enable=YES

ssl_ciphers=HIGH

ssl_tlsv1=YES

ssl_sslv2=YES

ssl_sslv3=YES

rsa_cert_file=/etc/ssl/certs/vsftpd.pem

rsa_private_key_file=/etc/ssl/certs/vsftpd.pem

#implicit_ssl=YES

 

#Anonymous Security

allow_anon_ssl=YES

force_anon_data_ssl=YES

force_anon_logins_ssl=YES

 

#Access Settings

#dirlist_enable=NO

#download_enable=NO

 

UPDATE!!

This article was written in 2013, a newer version by the same author can be found here:
ComputerDungeonSpycam



Projects & Links

Colonize the Moon
About moon/mars/asteroid colonies and space stuff

Computer Dungeon
About
linux/raspberry-pi/bitcoin and computer stuff

VidUploadServer
A video upload server I coded myself

Recent Posts