CE 432 Robotics II Fall 2021
ESP32-CAM Introduction
Sean Eaton
smeaton@fortlewis.edu

ESP32-CAM Introduction


Introduction:

For homework 1 we were introduced to using the AI Thinker ESP32-CAM board. The ESP32-CAM board is a microcontroller with an integrated antenna and camera connector. It has 10 general purpose input/output pins along with ground, 5V, and 3.3V pins. It can be programmed using the Arduino IDE by using a FTDI cable with ground, RX, and TX connections. The main tasks in this homework consist of verifying ESP32-CAM functionality by running the Camera Webserver example code. Before this can happen the Arduino IDE should be set up for ESP32-CAM development by adding it via the Boards Manager menu. In addition to this, some soldering work was done to prepare the ESP32-CAM for the overall robot project in this course in addition to enabling external antenna functionality. After running the example code we were then tasked with understanding how the onboard LED can be controlled via the GPIO pin 4. This LED's purpose is to function as a flashlight and example code was provided to make it blink, activate via a pushbutton, and change brightness levels.


Task 1: The Camera Webserver Example and Facial Recognition

Prior to task 1 I was able to successfully setup the Arduino IDE to enable ESP32-CAM development. I went into the Preferences menu and added the following URL to the external boards manager:
https://dl.espressif.com/dl/package_esp32_index.json
After doing this I went into the Boards Manager to install the esp32 board package. I then restarted the Arduino IDE so that I was able to select the AI Thinker ESP32-CAM board in the Tools menu. I was then able to load the CameraWebServer example, save it to my computer, and make the necessary fixes to enable the example code to run on my particular ESP32-CAM board. This consisted of commenting out the default camera module definition for the WROVER_KIT and uncommenting the AI_THINKER camera module definition. This section of the code can be seen in the upper red box in Figure 1 below.

The next section of the code to be modified dealt with the SSID, or network name, and the network's password. My home WiFi has the name shrimpNetwork and is able to output a 2G signal and a 5G signal. I originally tried the 5G connection but realized the ESP32-CAM is incompatible with it since it was forever waiting for a connection. In Figure 1 below I have blocked oout my WiFi password below the SSID name. I have also boxed in the board I have selected down in the bottom to verify I was able to install the esp32 board package successfully. I also had issues compiling the code for the CameraWebServer. On Linux I recieved errors pertaining to Python not being in PATH and pyserial not being installed. I was able to overcome these issues and I haven't attempted ESP32-CAM development on a Windows system yet.


Figure 1. CameraWebServer example code screenshot. The relevant changes to the code are shown along with the currently selected board at the bottom.

Before I verified the functionality of the CameraWebServer example code I opted to make the necessary modifications to the ESP32-CAM board to enable external antenna usage. To do this it was necessary to unsolder a tiny 0 ohm surface mount resistor from the ESP32-CAM board and reconnect it in the appropriate way. In Figure 2 below I have boxed in the specific connection modifed to enable external antenna functionality. Prior to this the 0 ohm resistor connected the middle connection to the top connection. When detaching the resistor I applied too much force when pulling it off and it was lost. Luckily the resistor functioned more like a wire due to having 0 ohm resistance so I was able to connect the two pads via a solder blob. I first had issues using a very thin solder tip which resulted in some solder sticking to areas it wasn't meant to, such as the metal shielding and onboard antenna shown below. I removed as much as it can and ended up switching to a wider solder tip. This tip allowed the solder to adhere to both pads easier and the extra solder seen on the board didn't impact functionality for this homework.


Figure 2. My ESP32-CAM board with the external antenna connection enabled using a solder blob.

In addition to making this adjustment by soldering, I also completed most of the protoboard soldering for the ESP32-CAM robot. I still need to make two more connections for the GPIO pins but the majority of the work has been completed. Using this protoboard I was able to program my ESP32-CAM board using my connections to header pins, a power supply board, and the FTDI USB cable. These hardware connections are shown below in Figure 3. I also added the jumper pins for programing the board. In one configuration the jumper pins allow the ESP32-CAM to be programmed by shorting the IO0 pin to ground. The other configuration disconnects IO0 from ground to enable standard operation.


Figure 3. Hardware connections used to program the ESP32-CAM board.

Now that the ESP32-CAM was able to be programmed I uploaded the CameraWebServer example code to the board. This process consisted of verifying that IO0 was shorted to ground, restarting the board, and then beginning the upload with the Arduino IDE. Within a minute or so the code was successfully uploaded. I then disconnected IO0 from ground, restarted the board again, and observed what happened in the serial monitor. Using the serial monitor I was able to see what IP address I needed to put into my browser to see the ESP32-CAM web server. A screenshot of the serial monitor can be seen below in Figure 4. The upper portion is information displayed when restarting the ESP32-CAM. Below that are some dots indicating the board is trying to connect to the network. After successful conneciton it sets up the webserver and displays what IP address is needed to connect to it. After starting the stream you can see information on what is being transmitted to the webserver.


Figure 4. Screenshot of the Arduino Serial Monitor after successfully connecting to the network and enabling the stream.

On the browser side I was able to click "Start Stream" and change the resolution to UXGA, the highest available resolution. The lighting in my apartment is not very good since the ESP32-CAM is low and my ceiling light is above me. Figure 5 below shows Camera WebServer running in my browser providing a livestream through the ESP32-CAM.


Figure 5. ESP32-CAM Camera WebServer running in the Mozilla Firefox Browser.

Now that the external antenna, camera webserver example code, and hardware connections were confirmed to be working the next step was to enable facial detection and recognition. There are two sections of code in the 'app_httpd.cpp' file that have to be modified to use the 'dl_lib_free()' function instead of the default 'free()' function. I made these modifications to the code as seen below in Figure 6 and Figure 7.


Figure 6. First section of code that needed to be modified.


Figure 7. Second section of code that needed to be modified.

After this the facial detection option in the camera webserver page should allow the ESP32 to be able to draw a yellow box around your face, after restarting the stream, indicating that the ESP32-CAM recognizes the image as containing a face. This functionality only works when the resolution is lowered to CIF (400x296) or lower. Due to my lighting and ESP32-CAM positioning, it was a bit difficult to screenshot the ESP32-CAM as recognizing my face. I was able to capture the ESP32-CAM successfully capturing my face with the help of another person, as shown in Figure 8. I had to look into the camera module directly and whenever I looked over to my computer the facial detection would intermittently work, it ended up being easier to look into the camera module and rely on another person to screenshot the results.


Figure 8. Demonstration of ESP32-CAM facial detection functionality.

The ESP32-CAM also has facial recognition functionality that can be enabled. When flipping the facial recognition switch, faces are labelled as intruders unless they are classified as enrolled faces. I enabled facial recognition and I confirmed that my unenrolled face was labelled as an intruder as expected.


Figure 9. Facial Recognition classifies my face as an intruder.

I enrolled my face using the enroll face button and after the initial setup of collecting frames for facial recognition I was recognized as Subject 0.


Figure 10. Facial Recognition classifies my face as Subject 0.


Task 2: Controling the flashlight and GPIO pins

For task 2 the focus was on the on-board LED or flashlight. This LED is able to omit a very bright light, like the flash on a camera, and is able to be controlled using the GPIO pin 4. Over the course of this task we needed to blink the flashlight, control it using a pushbutton, and change the brightness of the light. The Arduino code needed to blink the flashlight is very simple and comparable to the Blink example for standard Arduinos. The main difference is that you shouldn't stare directly into the ESP32-CAM LED while standard LEDs found in Arduino kits won't harm your eyes.

In the first example we needed to upload the following code to the ESP32-CAM and observe the results. I decided to switch to using the Arduino Beta IDE which is why the IDE screenshot looks a bit different from the standard Arduino IDE.


Figure 11. Code for the blinking flashlight. Turns the flashlight on for two seconds and off for two seconds.

The code successfully worked on the ESP32-CAM. I decided to set up the circuit for the second pushbutton example since I was going to incorporate it anyway which is why the ESP32-CAM is not using the protoboard connector anymore.


Figure 11. Flashlight blinking example. It is currently on for 2 seconds.

For the second example the following code was uploading to the ESP32-CAM so that the flashlight could be controlled using the pushbutton. It simply checks the state of the pushbutton and turns the flashlight on or off depending on it.


Figure 12. Code for controlling the flashlight via a pushbutton.

In the figure below I am pushing the button to activate the flashlight.


Figure 13. Controlling the flashlight by pressing the pushbutton down.

The final example showed us how the flashlight brightness can be controlled using PWM. There are some new functions we haven't been exposed to such as the ledcSetup, ledcAttachPin, and ledcWrite. It's a bit different from using PWM on an Arduino board.


Figure 14. Code for the flashlight brightness control example.

I tried to capture the flashlight when it wasn't at full brightness but it was difficult since the transition happens fairly fast. It should be evident that the light is not as bright as the previous images.


Figure 15. Flashlight at an intermediary brightness level.

Conclusion:

For HW1 this was a good introduction to how programming the ESP32-CAM is going to work. I also got to experience how using non-Arduino microcontrollers with the Arduino IDE works which was interesting. I also haven't heard of the ESP32-CAM prior to this class and it is very cool that this low-cost board with camera functionality exists. I look forward to working with this board more this semester.