Camera Detector

Project Description

• Design a wearable that can:

  • Sense sensors in the environment

  • Alert the user of the sensors

  • Alert nearby wearables of what the user is seeing

Initial Wireframe

Progress

• Procurement & Planning

  • For the camera, I went ahead and procured the following: link. This camera is (1) inexpensive but also (2) has a set of near-infrared (NIR) LEDs. These LEDs will be sensed by near-infrared receiving diodes on the glasses, positioned at 90-degree angles from each other so as to detect the camera left, right, and center. The initial idea is to hack into the camera and modulate the LEDs so that I can easily separate the cameras from any surrounding noise. Granted, this would not be representative of all NIR cameras but will make the demo robust.

  • For the glasses, I already have platform called Captivates that I’ve been building for a few months now that I will hack away at so to use for this application. To alert the user, the glasses have two, eye-facing LEDs, one for each eye. This will allow me to alert the user of a camera and also tell the user which direction the camera is in.

  • For the radio connection, I will be developing using an existing 2.4GHz radio within the glasses, using Google’s open source implementation of Thread, a low-power, secure networking architecture.

• Development

  • I went ahead and took apart the camera to see what the circuit looked like inside. The goal is to attach rewire the NIR LED switching circuitry so that it can be quickly toggled with a Teensy for a robust demo.

    

    

    

• On the PCB, I found the transistors for the LEDs so wiring in a microcontroller (MCU) should be fairly trivial. However, before doing that, I wanted to see how the LED signal looks like as is because maybe there are alreading modulating it (I doubt it for such a cheap, non battery powered camera).

• Putting the camera back together and powering it with a 12V DC power supply, I booted it up and found that the NIR LEDs automatically activate if the daylight sensor is covered. So, the next step is to build a transimpedance amplifier with a photodiode sensitive in the NIR domain.

• For constructing of the testing circuit, I just built a textbook transimpedance amplifier shown in the bottom figure and also described here: https://www.digikey.com/en/articles/techzone/2017/jun/how-to-design-stable-transimpedance-amplifiers-automotive-medical-systems

  

• This type of amplifier is needed to convert the current generate by the photodiode when exposed to a voltage output for a microcontroller to sample. For the feedback resistor (Rf), I chose 100k to start which means that the voltage seen will be 100,000x the current outputted by the diode. (Usually, transimpedance amplifier feedback resistors are between 100k to 1Meg). I am going to omit the feedback capacitor for now since that becomes more important for AC applications but for testing (and assuming the signal I will see is DC since the LEDs won’t be inherently modulated), it should be find to omit.

• I ended up using Vishay’s BPV22NF given that I’ve used this diode before and it’s performance is unmatched for its size. In addition, as shown below, this particular diode’s radial sensitivity is around 80% for a 90-degree field-of-view (FOV), perfect for three diodes positioned at 90-degree from eachother. In theory, the three diodes should give my system at least a 270-degree FOV.

  

• I went ahead and breadboarded the transimpedance amplifier and showed it below. For the operational amplifier, I just used an onhand TLV23741 since I am not working with particularly fast signals.

  

• Pointing the camera directly at the photodiode and toggling the NIR diodes on the camera by placing my finger on the daylight sensor, the signal from the transimpedance amplifier is shown below. You can clearly see that the LEDs are NOT being modulated.

  

• Another test I wanted to do was to see how the BPV22NF performs to radial movements of the camera.

  

• So, this is when I reached an inflection point in the project architecture. I was originally going to modulate the NIR signal but I am a bit convinced that I can get this to work without modifying the camera. However, there is one assumption: the system will only operate in “night-time” applications. Meaning that there should not be any other sources in the room giving off a NIR unmodulated signal (e.g. sunlight). This isn’t a crazy assumption since cameras that have NIR illuminating components only turn them on when its dark out (i.e. absence of sunlight). Therefor, I am trading in a robust demo for a more realistic one.

• The Captivates glasses have three NIR photodiode receivers which are used for 3D triangulation using active laser sweeps in the environment (for more information, look here). The photodiodes that are on the glasses are of a different type than the BPV22NF; they’re smaller and less sensitive but should work with additional amplification. However, their location is ideal given that they are 90-degrees from each other. The biggest issue, though, is that the photodiode architecture in the glasses is designed for fast NIR laser sweeps and also the diodes are summed together in hardware. This prevents me from (1) getting a DC NIR signal and (2) getting the individual photodiode signals. Therefor, I need to either rewire the existing diodes and include a secondary circuit or attach a secondary circuit with the BPV22NF diodes. Given that attaching diodes to the outside of the glasses will be less aesthetically pleasing, I am going with the former.

• For rewiring, I need three analog-to-digital conversion (ADC) channels. The glasses have 2 MCUs and the primary MCU is a bit difficult to rewire so I am going to be working with the secondary one (STM32F301K6U6). Given my electrical design of the glasses, ADC-equipped pins 7, 9, and 15 (PA0, PA2, and PB0, respectively) can be rewired with the least PCB modifications (i.e. need to remove thermopile sensor, one pull-up resistor, and external ADC chip). I also need to disable the existing summing amplifier circuit and isolate each photodiode (un-soldering the cathode pins on each diode and adding an insulator between the existing pad and the pin to avoid any shorting).

• Before rewiring, its important to outline the rest of the circuit to make sure there isn’t any other issues I might run into. Each photodiode will go into a separate transimpedance amplifier which will then go into a 1st-order high-pass filter to filter out any modulated NIR signals (e.g. fluorescent lighting). After the filter, each channel will have an additional opamp gain stage that will have an adjustable potentiometer for quick tuning (this pot can be replaced in the last revision for a smaller circuit but will make debugging a whole lot easier). The circuit for each individual photodiode channel is sketched below. The power for the opamps will be routed from the eyeglasses (3.3V).

  • Error: the below should be a low-pass filter (switch the capacitor and resistor)

    

• I built a breadboard version of the circuit to test out if the diodes that are currently in the glasses (VBPW34FASR) are sensitive enough to the cameras.

  

• The circuit did work but I realized that integrating this breadboard into my glasses with result in a rather “frankenstein-like” prototype. So, I went ahead and designed a PCB to manufacture (hopefully it comes in on time). This will allow me to tuck this circuit into one of the battery compartments.

  

  

  

  

While I wait for that board to come in, I want to build a Frankenstein version to make sure everything will work as planned. After going through the schematics for Captivates platform, I realized there has to be several modifications made to get this system to work. That being said, I thought it would be the most ideal to just build a new pair of glasses from the ground up instead of hacking away at assembled PCBs.

The first modification is that the three diodes need to be separated into individual channels. The best way to do that is to not attach the anode to the PCB and instead attach it to a wire that will be routed to a secondary PCB (i.e. the one I designed above).

I then had to solder on some wires to three ADC-capable microcontroller pins so that each diode can be individually sampled. This was a bit of a pain given that the microcontroller is a surface mount integrated chip so I needed to find some locations across the PCB to do the attachments. I also dabbed some hot glue to strain relieve the wires.

Now that all the wires are soldered on, I assembled the pair of glasses.

I went ahead and soldered on a potentiometer to the breadboard that I made before so that I can tune the inverting amplifier. Through testing, I found that a gain of ~65 would be great for a demonstration since that gave me a few feet range while not introducing to much noise.

So now its time to write some code to sample each channel and then operate on that data. I encountered a few issues with this step since it seems STMicroelectronics has a few bugs in their code generating software that’s supposed to aid in the initial configuration of the embedded code project. What cost me a few hours was that the ADC was initialized incorrectly so after some frustration, I found the bug and was able to sample all three channels.

The more annoying issue was getting I2C communication to work between the microcontroller and the LED drivers. I went through an evening of soldering and desoldering components on the boards and rewriting the code to find out that the initialized code was missing a fundamental piece that governs the time-keeping of the device. I was able to solve the problem by creating an entirely new project and migrating the code over.

And…. the system worked!

However, on the day of the demo, I received the PCBs I ordered! They were delayed due to customs. Seeing that I had a video that shows the system working, I went ahead and built the new PCBs, snipped the cables to the breakout board, and soldered it all in. And…. it worked!!!

Given how long making one version has taken, I did not get a chance to make a second version to transmit to. I did, however, want to explore adding a NIR diode to the pair of glasses to wash out the image of the person wearing the glasses. I went ahead and wired up a diode and looked at the video feed. It turns out that modern cameras are awfully robust and it would really take a high-power diode to wash out the video feed. Therefor, I have not attached one since my system can’t handle the power requirement.