Measurement¶

The goal of this section is to provide a guide on how to measure data with LenslessPiCam and a Raspberry Pi-based lensless camera.

This Medium article discusses the typical measurement process for a lensless camera:

Measure the point spread function (PSF), which involves some calibration. See this article for how to make a low-cost point source generator.
Measure raw data.

All scripts related to measurement can be found in the scripts/measure directory. Some scripts are meant to be run on your local machine, while others are meant to be run on the Raspberry Pi.

Note

Before trying to collect a dataset, you should first go through each section below to ensure that the camera is working correctly and that you are able to capture and display images.

It is highly recommended to connect to your Raspberry Pi through an IDE like VS Code, as it will be much easier to edit configuration files.

Capturing from your Raspberry Pi (on-device capture)¶

The on-device scripts need to work before you can perform remote capture.

LenslessPiCam should be cloned and installed on your Raspberry Pi, see here (scroll down to “Raspberry Pi setup”).

The following script can be used to capture data with the Raspberry Pi camera:

python scripts/on_device_capture.py

Default parameters can be found in configs/capture.yaml. A description of each parameters can be found in this pull request.

Note

Note if using the Legacy Camera on Bullseye OS, you should set legacy=True in your configuration file.

Capturing from your local computer (remote capture)¶

For remote capture, you need to have LenslessPiCam installed on your local machine (see here), and you need to have passwordless access to your Raspberry Pi.

You can remotely capture data with the following script:

python scripts/measure/remote_capture.py \
   rpi.username=USERNAME \
   rpi.hostname=HOSTNAME \
   plot=True

where <HOSTNAME> is the hostname or IP address of your Raspberry Pi. The script will save the captured data (raw_data.png) and plots (including histogram to check for saturation) in demo_lensless.

The default configuration (config/demo.yaml) converts the captured Bayer data to RGB and downsamples the image by a factor of 4 on the Raspberry Pi, and then sends it back to your computer. The capture section, essentially sets the parameters for the on-device capture configation.

The default configuration uses white balancing gains that were determined for our LCD-based lensless camera (capture.awb_gains, camera.red_gain, and camera.blue_gain). You will very likely need to determine these gains for your lensless camera by first capturing the raw Bayer data of a known white object (e.g., a white sheet of paper).

To capture the raw 12-bit Bayer data for the Raspberry Pi HQ camera, you can run the above script with a different configation:

python scripts/measure/remote_capture.py -cn capture_bayer \
   rpi.username=USERNAME \
   rpi.hostname=HOSTNAME

You can then use scripts/measure/analyze_image.py to play around with the red and blue gains to find the best white balance for your camera.

Preparing an external monitor for displaying images (remote display)¶

Our approach for displaying images on a monitor connected to a Raspberry Pi is to view an image in full-screen mode using the feh command line tool, which reads a folder of images and displays them in a slideshow. Our folder only has one image, and we rewrite this image every time we want to display a new image.

You first need to install the feh command line tool on your Raspberry Pi.

sudo apt-get install feh

Then make a folder where we will create and read prepared images.

mkdir ~/LenslessPiCam_display
cp ~/LenslessPiCam/data/original/mnist_3.png ~/LenslessPiCam_display/test.png

We will be overwriting ~/LenslessPiCam_display/test.png whenever we want to display a new image. Then from a Terminal on the Raspberry Pi we can use feh to launch the image viewer.

feh LenslessPiCam_display --scale-down --auto-zoom -R 0.1 -x -F -Y

This command cannot be launched from an SSH session connected to the Raspberry Pi, even if there is a monitor connected to the Raspberry Pi.

Then from your laptop with passwordless access, you can use the following script to display an image on the Raspberry Pi:

python scripts/measure/remote_display.py \
    rpi.username=USERNAME \
    rpi.hostname=HOSTNAME \
    fp=FP

where USERNAME and HOSTNAME are the username and hostname of the RPi, and FP is the path on your local computer of the image you would like to display. The default parameters can be found in config/demo.yaml, specifically the display section, to adjust the screen resolution, positioning, brightness, padding, and rotation.

Measuring an arbitrary dataset¶

Note

Please go through the above sections to ensure that the camera is working correctly and that you are able to capture and display images.

The following script can be used to collect an arbitrary dataset with the proposed camera. The script should be launched from the Raspberry Pi:

python scripts/measure/collect_dataset_on_device.py -cn CONFIG_NAME

The default configuration can be found in configs/collect_dataset.yaml.

The following needs to be done before running the script:

Determine the white balance gains for your camera, e.g. with scripts/measure/analyze_image.py as described above.
As you probably won’t have enough memory on the Raspberry Pi, mount an external hard-drive to save the dataset into.
```
sudo mount /dev/sda1 /mnt
```
Change /dev/sda1 to the correct device name of your external hard-drive.

Define your own configuration file inside the configs directory. You can “inherit” from the default configuration file and overwrite the parameters you want to change:

defaults:
   - collect_dataset
   - _self_

input_dir: /mnt/YOUR_INPUT_DATASET
output_dir: /mnt/YOUR_OUTPUT_DATASET
n_files: 10000
min_level: 140
max_tries: 4

display:
   image_res: [900, 1200]

capture:
   exposure: 0.5
   awb_gains: YOUR_AWB_GAINS

You can schedule the dataset collection with runtime and start_delay.

By default, the script will save downloaded RGB data, as raw Bayer data can quickly take up a lot of space.

See configs/collect_mirflickr_multimask.yaml for an example of measuring a multi-mask dataset with DigiCam.

Note

Play around with the display/capture settings until you get the ones that align well for an ADMM reconstruction, and that require the least amount of re-tries.

Note

As the measurement may run for a while, it is recommended to run the script in a screen session (or similar) to avoid the script being interrupted if the SSH connection is lost.

Note

From the Raspberry Pi, it is recommended to disable screen blanking (the screen from entering power saving mode and turning off) by following these steps.

Troubleshooting¶

The scripts used below assume that you have setup the package on both your local machine and a remote Raspberry Pi with SSH and passwordless access.

Sometimes, we have noticed problems with locale when running the remote capture and display scripts, for example:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
...

This may arise due to incompatible locale settings between your local machine and the Raspberry Pi. There are two possible solutions to this, as proposed in this forum.

Comment SendEnv LANG LC_* in /etc/ssh/ssh_config on your laptop.
Comment AcceptEnv LANG LC_* in /etc/ssh/sshd_config on the Raspberry Pi.