jdpdev/birbcam

A bird feeder camera for Raspberry Pi

Version 1.0 Sparrow is official! Download ZIP repository here

Version 2.0 Robin is in beta, introducing BIRBVISION to identify bird species in your pictures. Download ZIP repository here

Check out the Roadmap for future plans.

birbcam

A Raspberry Pi-powered, change-activated camera for watching bird feeders.

Check out Birbserver, a web interface for your Birbcam designed to run on the same RPI.

Birbvision is the prototype machine learning identifier to be introduced in Birbcam v2.0

Set Up

Hardware

Required

• Raspberry Pi - All testing has been done on a Raspberry Pi 4 Model-B with 4GB of RAM. If you're brand new to Raspberry Pi, there are many options for kits that come with the board, the OS pre-installed on a boot SD card, and a case.
• Raspberry Pi Camera
  • Camera Module V2 - A small 8MP, fixed-focus camera. Lens is wide so will have to be mounted close to the feeder. Focus can be adjusted, but it is not really meant to be refocused. Good for testing, but if you stick with it you will want to upgrade to...
  • HQ Camera - 12MP, a much bigger sensor than the V2, and swappable lenses. Standard lenses come with 6mm and 16mm (28mm and 85mm equivalent on a 35mm camera) focal lengths; I find the 16mm to be a very good choice.

Good To Have

• Adafruit HQ Camera Case - Combines the RPI and HQ Camera in a handy package. Has to be 3D printed. The HQ camera has a integrated standard tripod screw mount.
• External USB Storage - Protects the boot SD card from degredation, and provides the convience of being able to plug into another computer to work on your pictures.
• Longer Lenses - Arducam sells lenses for the HQ camera with longer focal lengths, but they require color correction that is not yet available in Birbcam.

Environment

Requires the following Python packages, available on PIP

picamerax
opecv-python
numpy
imutils

config.ini

Important settings are saved in the config.ini file; some can be overridden by CLI arguments when running the app. Defaults are provided, but some will require local configuration.

• [Saving] Directory - Where images taken by the camera are saved. It is suggested that you save to an external drive, rather than the SD card.
• [Saving] LivePictureInterval - Number of seconds between each live picture, used by the server. Set to 0 to disable.
• [Saving] FullPictureInterval - Number of seconds between each full picture. A full picture is not automatically taken after this interval, rather no full pictures will be taken by triggers before the interval has expired.
• [Saving] FullPictureResolution - The resolution of a full picture. See config.ini for more informating relating to camera hardware.
• [Saving] LivePictureResolution - The resolution of a live picture. See config.ini for more informating relating to camera hardware.
• [Detection] Threshold - How strong the difference between the live and reference pictures must be to register as a changed pixel. Higher is less sensitive.
• [Detection] ContourArea - How big a detected, continuous difference region must be to trigger a full picture. This balances out noise from the Threshold setting. Higher is less sensitive.
• [Detection] ExposureInterval - Number of seconds between exposure checks
• [Detection] ExposureLevel - Target exposure level. The ideal value depends on your set up (feeder color, direct/indirect light, etc), but the default of 100 is a good starting place.
• [Detection] ExposureError - Acceptable error +/- ExposureLevel
• [Debug] Enable - Debug mode shows live images from the camera and detector. Can be toggled when running with D key.

Running

Run the app via the command line

python3 birbcam.py

Focus Assist

The first screen that opens is a live view from the camera to help you aim the camera, and set the focus. The number in the top right is a relative, unit-less value that approximates how much of the image is in focus. Max focus corresponds to the largest value.

To select a specific area to focus, you can click and drag a rectangle to zoom. To reset the zoom press R.

To continue, press Q. To exit the app, press X.

Detector Mask

The second screen that opens is a live view from the camera to set the Detection Mask. When the camera is running, only changes to the image within the detection mask are used to trigger the taking of a picture.

To select a specific area to mask, you can click and drag with the mouse. The detection area is within the yellow rectangle.

To continue, press Q. To exit the app, press X.

Camera Watcher

If you are running Debug Mode, the final screen is the observing interface, which you can use to monitor the camera. There are four quadrants in the display:

• Top Left - The live feed from the camera. Changes that could trigger a picture will be highlighted by a green rectangle.
• Top Right - Camera settings and exposure histogram. Camera settings can be changed with the keys marked (x). The histogram plots the luminance of the image and is used to assist exposure setting.
• Bottom Left - The difference between the live and the reference image.
• Bottom Right - The difference image clamped to a threshold value, highlighting significant changes.

The camera starts paused: it will take live pictures, but will not take full pictures until unpaused.

To pause/unpause recording, press P. To exit the app, press Q.

birbcam's Issues

Segment images before passing into Birbvision

The Birbvision model expects the bird to be prominent in the image, but images from the camera are often not bird-centered. So instead of passing one image, segment it into parts that separately get classified.

Assuming the source image is big enough, divide the image into three sections with overlap:

• 50% width on left side
• 50% width on right side
• 50% width centered

Concerns

• Performance
  • Early indication is that classifying on a single image isn't much of a burden

[Config] Picture-taking intervals

Add configuring the intervals between live and full pictures to config.ini

• Live picture interval
• Full picture interval

Run new photos through Birbvision evaluator

Whenever the monitor takes a photo, run it through Birbvision for ML evaluation

This task does not involve doing anything with the evaluation

Concerns:

• How does ML evaluation effect performance?
  • Should evaluation be done on a separate thread?

Add control console

Add a simplified version of the debug console that only shows the control section

Automatic exposure adjustment

Adjust exposure automatically to try to maintain a target level.

• Use the exposure set by user as the target
• Evaluate every configured amount of time

Save Birbvision evaluation result

After evaluating a photo with Birbvision, save the results with the photo

For each photo, save a file with the information:

• Location of full-resolution image
• Location of thumbnail image
• Top X number of predictions with label and confidence

The server can utilize these files to serve images and metadata

Lens shading compensation

Implement picamerax's lens_shading_table and generate a table for the arducam lenses

Check out openflexure_microscope's implementation of generating the table https://github.com/rwb27/openflexure_microscope_software/blob/better_shading_correction/openflexure_microscope/utilities/recalibrate.py

[BUG] App crashes when accessing a large picture log

Describe the bug
App crashes when trying to write to the picture log when the picture log has become too big

To Reproduce
Crash point appears to be when the picture log is a little north of 1 MB, and the crash happens when attempting to write to it.

Raspberry Pi (please complete the following information):

• OS: Buster

Additional context
When this happens, the whole drive the picture log resides on is set to read-only access.

[Config] Output image sizes

Add configuration of output image sizes to config.ini

• Full image size
• Live image size
• Thumbnail image size

Need to integrate with camera modes? https://www.raspberrypi.org/documentation/raspbian/applications/camera.md

Evaluate motion vector processing

PiCamera offers motion estimation and the PiMotionAnalysis class. Evaluate if this data can be used to weed out false positives from the detector from things like swaying of the feeder.

[Config] Minimum contour area

Add configuration of the minimum contour area threshold to config.ini

Utilize ISO when adjusting exposure

Currently, automatic exposure adjustment only uses shutter to match the exposure. ISO adjustment can be used to expand the range of the adjustment, improve image quality, and minimize use of less useful shutter speeds.

Concerns:

• One stop of ISO is a big change, but smaller increments may not be as useful as small shutter increments
• Need to determine logic for deciding whether to change ISO
  • Configure a range of optimal shutter speeds?
• After adjusting ISO the direction of shutter movement may need to be reversed

Do not save photos without a positive identification

Use Birbvision to determine if a bird is in a photo, and do not save photos without birds

[Config] Threshold cut-off

Configure the threshold value