CE 432 Robotics II Fall 2021
ESP32-CAM Introduction
Sean Eaton
smeaton@fortlewis.edu
ESP32-CAM Introduction
Introduction:
For homework 1 we were introduced to using the AI Thinker ESP32-CAM
board. The ESP32-CAM board is a microcontroller with an integrated
antenna and camera connector. It has 10 general purpose input/output
pins along with ground, 5V, and 3.3V pins. It can be programmed using
the Arduino IDE by using a FTDI cable with ground, RX, and TX
connections. The main tasks in this homework consist of verifying
ESP32-CAM functionality by running the Camera Webserver example code.
Before this can happen the Arduino IDE should be set up for ESP32-CAM
development by adding it via the Boards Manager menu. In addition to
this, some soldering work was done to prepare the ESP32-CAM for the
overall robot project in this course in addition to enabling external
antenna functionality. After running the example code we were then
tasked with understanding how the onboard LED can be controlled via the
GPIO pin 4. This LED's purpose is to function as a flashlight and
example code was provided to make it blink, activate via a pushbutton,
and change brightness levels.
Task 1: The Camera Webserver Example and Facial Recognition
Prior to task 1 I was able to successfully setup the Arduino IDE to
enable ESP32-CAM development. I went into the Preferences menu and
added the following URL to the external boards manager:
https://dl.espressif.com/dl/package_esp32_index.json
After doing this I went into the Boards Manager to install the esp32
board package. I then restarted the Arduino IDE so that I was able to
select the AI Thinker ESP32-CAM board in the Tools menu. I was then
able to load the CameraWebServer example, save it to my computer, and
make the necessary fixes to enable the example code to run on my
particular ESP32-CAM board. This consisted of commenting out the
default camera module definition for the WROVER_KIT and uncommenting
the AI_THINKER camera module definition. This section of the code can
be seen in the upper red box in Figure 1 below.
The next section of the code to be modified dealt with the SSID, or
network name, and the network's password. My home WiFi has the name
shrimpNetwork and is able to output a 2G signal and a 5G signal. I
originally tried the 5G connection but realized the ESP32-CAM is
incompatible with it since it was forever waiting for a connection. In
Figure 1 below I have blocked oout my WiFi password below the SSID
name. I have also boxed in the board I have selected down in the bottom
to verify I was able to install the esp32 board package successfully. I
also had issues compiling the code for the CameraWebServer. On Linux I
recieved errors pertaining to Python not being in PATH and pyserial not
being installed. I was able to overcome these issues and I haven't
attempted ESP32-CAM development on a Windows system yet.
Figure 1. CameraWebServer example code screenshot. The relevant changes
to the code are shown along with the currently selected board at the
bottom.
Before I verified the functionality of the CameraWebServer example code
I opted to make the necessary modifications to the ESP32-CAM board to
enable external antenna usage. To do this it was necessary to unsolder
a tiny 0 ohm surface mount resistor from the ESP32-CAM board and
reconnect it in the appropriate way. In Figure 2 below I have boxed in
the specific connection modifed to enable external antenna
functionality. Prior to this the 0 ohm resistor connected the middle
connection to the top connection. When detaching the resistor I applied
too much force when pulling it off and it was lost. Luckily the
resistor functioned more like a wire due to having 0 ohm resistance so
I was able to connect the two pads via a solder blob. I first had
issues using a very thin solder tip which resulted in some solder
sticking to areas it wasn't meant to, such as the metal shielding and
onboard antenna shown below. I removed as much as it can and ended up
switching to a wider solder tip. This tip allowed the solder to adhere
to both pads easier and the extra solder seen on the board didn't
impact functionality for this homework.
Figure 2. My ESP32-CAM board with the external antenna connection
enabled using a solder blob.
In addition to making this adjustment by soldering, I also completed
most of the protoboard soldering for the ESP32-CAM robot. I still need
to make two more connections for the GPIO pins but the majority of the
work has been completed. Using this protoboard I was able to program my
ESP32-CAM board using my connections to header pins, a power supply
board, and the FTDI USB cable. These hardware connections are shown
below in Figure 3. I also added the jumper pins for programing the
board. In one configuration the jumper pins allow the ESP32-CAM to be
programmed by shorting the IO0 pin to ground. The other configuration
disconnects IO0 from ground to enable standard operation.
Figure 3. Hardware connections used to program the ESP32-CAM board.
Now that the ESP32-CAM was able to be programmed I uploaded the
CameraWebServer example code to the board. This process consisted of
verifying that IO0 was shorted to ground, restarting the board, and
then beginning the upload with the Arduino IDE. Within a minute or so
the code was successfully uploaded. I then disconnected IO0 from
ground, restarted the board again, and observed what happened in the
serial monitor. Using the serial monitor I was able to see what IP
address I needed to put into my browser to see the ESP32-CAM web
server. A screenshot of the serial monitor can be seen below in Figure
4. The upper portion is information displayed when restarting the
ESP32-CAM. Below that are some dots indicating the board is trying to
connect to the network. After successful conneciton it sets up the
webserver and displays what IP address is needed to connect to it.
After starting the stream you can see information on what is being
transmitted to the webserver.
Figure 4. Screenshot of the Arduino Serial Monitor after successfully
connecting to the network and enabling the stream.
On the browser side I was able to click "Start Stream" and change the
resolution to UXGA, the highest available resolution. The lighting in
my apartment is not very good since the ESP32-CAM is low and my ceiling
light is above me. Figure 5 below shows Camera WebServer running in my
browser providing a livestream through the ESP32-CAM.
Figure 5. ESP32-CAM Camera WebServer running in the Mozilla Firefox
Browser.
Now that the external antenna, camera webserver example code, and
hardware connections were confirmed to be working the next step was to
enable facial detection and recognition. There are two sections of code
in the 'app_httpd.cpp' file that have to be modified to use the
'dl_lib_free()' function instead of the default 'free()' function. I
made these modifications to the code as seen below in Figure 6 and
Figure 7.
Figure 6. First section of code that needed to be modified.
Figure 7. Second section of code that needed to be modified.
After this the facial detection option in the camera webserver page
should allow the ESP32 to be able to draw a yellow box around your
face, after restarting the stream, indicating that the ESP32-CAM
recognizes the image as containing a face. This functionality only
works when the resolution is lowered to CIF (400x296) or lower. Due to
my lighting and ESP32-CAM positioning, it was a bit difficult to
screenshot the ESP32-CAM as recognizing my face. I was able to capture
the ESP32-CAM successfully capturing my face with the help of another
person, as shown in Figure 8. I had to look into the camera module
directly and whenever I looked over to my computer the facial detection
would intermittently work, it ended up being easier to look into the
camera module and rely on another person to screenshot the results.
Figure 8. Demonstration of ESP32-CAM facial detection functionality.
The ESP32-CAM also has facial recognition functionality that can be
enabled. When flipping the facial recognition switch, faces are
labelled as intruders unless they are classified as enrolled faces. I
enabled facial recognition and I confirmed that my unenrolled face was
labelled as an intruder as expected.
Figure 9. Facial Recognition classifies my face as an intruder.
I enrolled my face using the enroll face button and after the initial
setup of collecting frames for facial recognition I was recognized as
Subject 0.
Figure 10. Facial Recognition classifies my face as Subject 0.
Task 2: Controling the flashlight and GPIO pins
For task 2 the focus was on the on-board LED or flashlight. This LED is
able to omit a very bright light, like the flash on a camera, and is
able to be controlled using the GPIO pin 4. Over the course of this
task we needed to blink the flashlight, control it using a pushbutton,
and change the brightness of the light. The Arduino code needed to
blink the flashlight is very simple and comparable to the Blink example
for standard Arduinos. The main difference is that you shouldn't stare
directly into the ESP32-CAM LED while standard LEDs found in Arduino
kits won't harm your eyes.
In the first example we needed to upload the following code to the
ESP32-CAM and observe the results. I decided to switch to using the
Arduino Beta IDE which is why the IDE screenshot looks a bit different
from the standard Arduino IDE.
Figure 11. Code for the blinking flashlight. Turns the flashlight on
for two seconds and off for two seconds.
The code successfully worked on the ESP32-CAM. I decided to set up the
circuit for the second pushbutton example since I was going to
incorporate it anyway which is why the ESP32-CAM is not using the
protoboard connector anymore.
Figure 11. Flashlight blinking example. It is currently on for 2
seconds.
For the second example the following code was uploading to the
ESP32-CAM so that the flashlight could be controlled using the
pushbutton. It simply checks the state of the pushbutton and turns the
flashlight on or off depending on it.
Figure 12. Code for controlling the flashlight via a pushbutton.
In the figure below I am pushing the button to activate the flashlight.
Figure 13. Controlling the flashlight by pressing the pushbutton down.
The final example showed us how the flashlight brightness can be
controlled using PWM. There are some new functions we haven't been
exposed to such as the ledcSetup, ledcAttachPin, and ledcWrite. It's a
bit different from using PWM on an Arduino board.
Figure 14. Code for the flashlight brightness control example.
I tried to capture the flashlight when it wasn't at full brightness but
it was difficult since the transition happens fairly fast. It should be
evident that the light is not as bright as the previous images.
Figure 15. Flashlight at an intermediary brightness level.
Conclusion:
For HW1 this was a good introduction to how programming the ESP32-CAM
is going to work. I also got to experience how using non-Arduino
microcontrollers with the Arduino IDE works which was interesting. I
also haven't heard of the ESP32-CAM prior to this class and it is very
cool that this low-cost board with camera functionality exists. I look
forward to working with this board more this semester.