Optical Neuron Modeling

Adarsh Jayakumar
Samir Durvasula

Neurophysiology is an extremely complex field with surprisingly few teaching tools. Our goal was to create one such teaching tool, an embedded implementation of a neurophysiological phenomenon. We hope our users not only have fun playing with our product but also leave having learnt something new and extremely cool!


Figure 1: Functional Final Device Our product is a model of the action potentials (the voltages over time) of optical neurons. We utilized three LEDs as our neurons, a piTFT screen as our display, as well as three other LEDs as indicator lights of neuron "spikes". Our neuron behavior was characterized by the integrate and fire model with lateral inhibition due to this model's fantastic teaching value. In terms of integration, we protoboarded our entire circuit and constructed a simple wooden box with a sliding top to demonstrate the functionality of our neurons.


Choosing a Model

In order to best characterize our optical neurons, it was important to choose the most effective physiological model. Each known computational model optimizes for accuracy and allows tenability for different factors. In our project, our goal was to show how optical neurons respond to light as well as how photoreceptors affect the firing rates of neighboring neurons. Given this goal, models such as Hodgkin Huxley or Morris-Lecar were found to be ineffective. These models focus on the conductance of specific ion channels rather than understanding the overall behavior and relationships of neurons. Instead, we decided to use the leaky integrate and fire model of neurons.

Integrate and Fire

At a high level, integrate and fire operates on the principle of a neuron integrating its stimuli over a certain amount of time. After the sum of all of the stimuli over time reaches a threshold, the neuron “spikes” or “fires”. In the case of an optical neuron, if a constant light is held on the neuron for a long enough time, the integral will rise linearly until the threshold is met and the neuron fires. After the spike occurs, the neuron returns to its resting potential. Since neurons should not be constantly firing, a refractory period exists to allow for the neuron to reset.

This model is especially useful for modeling the firing rates of neurons. For optical neurons, if a neuron that reacts to white light is exposed to light for an extended amount of time, the neuron will fire more often. Unfortunately, this model does not implement any time-dependent memory. If a neuron receives a minimal stimulus, these minimal stimuli will still be added. The neuron will then begin to fire arbitrarily, which does not occur in real life. The equation governing integrate and fire dynamics is below, and is derived from the relationship between capacitance and current.

Figure 2: Integrate and Fire governing equation
Figure 3: Cell Physiology Figure 4: Exponential Integration

Leaky Integrate and Fire

Leaky Integrate and Fire is used to add the time-dependent memory that the purely integrate and fire model was lacking. In this model, we assume that the cell membrane does not act as a perfect insulator. The membrane has an inherent resistance and capacitance as seen in Figure 3. As such, integration happens in an RC charging fashion. In this model, a certain threshold must be reached for a stimulus to count towards integration. Otherwise, the stimulus will be leaked out. Unlike integrate and fire neurons, which integrate linearly, leaky integrate and fire neurons integrate exponentially. Figure 4 shows an example of this exponential integration.

Lateral Inhibition

The phenomenon of lateral inhibition has been greatly studied and was found in Horseshoe crab photoreceptors by Haldan Hartline. This finding was particularly meaningful due to the similarities between Horseshoe crab photoreceptors and human photoreceptors. Lateral inhibition is the phenomenon that an excited neuron has the capacity to reduce or inhibit the firing of neighboring neurons. This ensures that maximum firing occurs when a stimulus is isolated to a specific neuron. Lateral inhibition is vital for the visual system in understanding contrast. If there is a small light source, the center neuron was be effectively amplified since the neighboring neurons will be told to respond with a “darker” response. This effect is known to lead to optical illusions where lighter edges are lighter than they are and darker areas are perceived to be darker than they actually are. An example of this can be seen in Figure 5 below

Figure 5: Optical Illusion through Lateral Inhibition

Design & Testing

Libraries and Modules

Two open source libraries were used in our final product. The MCP3008 ADC Python library from Adafruit was used to interface with the MCP3008 ADC over four-pin software SPI. Additionally, he open-source library "pi_ina219" written by Chris Ballard was used to fix conflicting GPIO instantiations by the ADC library and Adafruit.GPIO module.

Several modules were imported in our final product as well. The module "sys" was used to point to the paths where our ADC libraries existed and append them to the in-built Python path. "RPi.GPIO" was imported in order to use the GPIO pins of the Raspberry Pi. The modules "MCP3008" and "Adafruit_MCP3008" worked with the Adafruit ADC library to interface with the MCP3008 ADC chip. "Matplotlib" and "Pyplot" were imported to graph and then save the image of each neuron's action potential. The "Collections" module was used as a high-performance double-sided array. Finally, the standard modules "os" and "math" were used to manipulate operating system functionality and acess basic mathematical functions respectively.

ADC Functionality

As we quickly came to realize, the Raspberry Pi did not come with any analog functionality. As such, we purchased a low-cost 8-channel 10-bit analog to digital converter from Adafruit: the MCP3008.

The MCP3008 has two modes of functionality, hardware SPI and software SPI. Our original intention was to utilize hardware SPI as it is slightly faster and more efficient. However, the hardware SPI pins required by the MCP3008 (SCK, MOSI, MISO, CE0) were already used by the TFT. As such, we implemented a software SPI connection instead. Due to our pin availabilities, the MCP3008 library was modified to use pins 12, 16, 20, and 21 for CLK, CS, MOSI, and MISO respectively. Figure 6 below shows the schematic of our connection between the Raspberry Pi header and MCP3008 chip.

Figure 6: ADC Wiring

The ADC was tested by running the "SimpleTest.py" example code provided by the Adafruit team. This code simply iterates through each ADC channel and prints their values every half second on the command terminal. A sample output of this code can be seen in Figure 7 below. Following this, our ADC was up and ready to go!

Figure 7: ADC SimpleTest Output

Plotting Graphs

In order to plot the graphs of our neuron action potentials, we used the open-source modules "Collections", MatplotLib", and "Pygame". "Collections" was used to create an array that acted as a rolling FIFO. The FIFO included 50 elements that represented the voltages of a singular neuron at any point in time. At the end of each iteration of code, an element was popped off the left end of the array, and a new value was appended to the start of the array (an artificial FIFO). Three FIFOs were created corresponding to each of the three neurons

We utilized the module "Matplotlib" in order to plot these FIFO data streams on our Raspberry Pi. First, we created a constant range of values from 0 to 50 to resemble time. After iterating through our equations and appending a new element to our FIFOs, we use the 'plot' function to create graphs of each array vs. time. Functions such as 'xlabel' and 'ylabel' as well as keywords such as 'b', 'r', and 'g' were used to modify the graph's axes and colors respectively. Finally, the plot was saved as a .png file into the Raspberry Pi's memory.

Our final step involved using "Pygame" to create our multimedia application on top of the SDL libraries. We initialized a Pygame screen with the correct size of our monitor (960x720 for desktop. 320x240 for TFT) and loaded a Pygame image as our saved Matplotlib figure. Finally, we appended this image to our screen every iteration through our code for a smooth animation experience.

Neural Math

The primary mathematics for a neuron is set by the below equation

Figure 8: Differential Equation for Leaky Integrate and Fire

The neuron fires when the voltage V m reaches a certain threshold voltage V thresh . At this point, the neuron spikes. The integrated voltage is directly affected by the stimulating current of the neuron. After a spike occurs, the membrane voltage is set back to resting potential. After the membrane is set to resting potential, its voltage remains constant until the end of the refractory period. As a result, the differential equation can be defined by the time since the end of the refractory period. We can then solve the differential equation to characterize membrane voltage. First, we define the time constant τ as R m C m. We define t as the time since the end of the refractory period. Finally, we define I stim as the current generated by the stimulus. The solution to the differential equation is
Figure 9: Differential Equation Solution for Leaky Integrate and Fire

Implementation in Code

The aforementioned differential equations were implemented in code as follows. We first have several settable parameters. These parameters all effect the final output graph. These settable parameters include tau, the membrane time constant, cap, the membrane capacitance, V thresh , the threshold voltage, and V rest , the resting potential of the neuron. In addition, each of the neurons has a settable excitatory gain and an inhibitory gain. The excitatory gain is multiplied by the stimulus value when the neuron is excited by light and the inhibitory gain is multiplied when it is inhibiting other neurons.

After parameters are set, we run the actual calculations. The same calculations happen in parallel for all neurons. We first set the refractory time, t ref , to 0. This means that the neuron is not currently in the refractory period. If the analog readings of the neuron are less than a threshold, we assume that the stimulus current is automatically leaked. For this reason, the membrane voltage and the integral value are set to the resting potential. If t ref is nonzero, then we are in the refractory period. In this case, we decrement t ref

If the neuron’s value is greater than the threshold, we begin including the stimulus readings into the integral. If t ref is nonzero, we are still in the refractory period, so this value is decremented. In addition, the integral and voltage membrane values are set to the resting potential. The LED used to display neuron spiking is set to output low. Otherwise, if t ref is zero, we are no longer in the refractory period. We increment the value of t refEnd , which is the time since the end of the refractory period. We then add to the integral as in the aforementioned equation. Additionally the stimulus current is changed to include lateral inhibition with the equation below:

Figure 10: Stimulus for Neuron Adjusted for Lateral Inhibition

At each iteration, the integral is updated. In addition, the membrane potentials are updated as in the first equation. If the membrane voltage surpasses the set voltage threshold, a spike is recorded. In this case, t ref is set to 5 to indicate the start of the refractory period. The membrane voltage is set to equal the threshold voltage, and the neuron’s spiking LED is set to output high. If t refEnd is greater than 40, we reset the membrane potential back to the resting potential, as the integration never led to the neuron firing and a spike was not detected. This step is essentially a forced leaking action for the neuron. With that, we have a complete working set of equations to model our neurons!

Box Construction

For constant testing of our optical neurons, we built a box with sliding doors. The sliding doors were used to gradually expose the LEDs to light. With two sliding doors, we were able to choose which neurons were exposed to light as well as cause individual neurons to fire. This allowed for better demonstration of effects such as lateral inhibition.

Figure 11: Side View of Box
The sides of the box were made with 6 ½ x 3 ¾ x ¾ “ blocks of plywood. The bottom of the box was made with a 4 ½ x 6 ½ x 3/16 “ block of wood. The back of the box was made with a 3 ¾ x 4 ½ ” x 3/16” block of wood. All of these components were attached to each other with screws. In order to make the sliding doors, we attached ½ ” x 6 ½ ” x 1” blocks of wood on top of the sides. We created ¼ ” grooves along the length of these blocks. The sliding doors were two 3 ½ x 3 ¼ x 3/16 “ blocks of wood.

Figure 12: Top View of Box
These blocks of wood slide into the grooves on the top of the box to block out light from a hanging lamp. These blocks were designed to block out light from two neurons at a time. We controlled the space between the sliding doors to show how exposure to light affects the neurons. We created a hole in the box to pass the piCobbler ribbon cable through, and attached the raspberry Pi to the outside of the box using Velcro. We also drilled holes for 3 LEDs to be seen on the outside of the box. These LEDs were used to identify spiking of the left, right, and center neurons. The side view of our box can be seen in figure 11 to the right. Figure 12 shows the top view of the box with the sliding doors.

TFT Functionality

The piTFT essentially ran as a plug-and-play system on the Raspberry Pi. In order to get it working, we simply had to export the desktop monitor's framebuffer to our TFT screen and ensure that all environment variables were changed accordingly. Figure 13 below shows our working code on the TFT screen.

Figure 13: Working display of TFT on Box

Startup on Power

Our final step involved creating a simple Bash script so that our program would run on bootup of the Raspberry Pi. This was done by including the python command to run the program in the rc.local file, and setting the command to run in the background.

Figure 14: Final Schematic of Circuit

Results & Conclusions

Final Product

Figure 15: Isolated Red Neuron
Figure 16: Isolated Blue Neuron
Figure 17: Isolated Green Neuron
Overall, we were able to successfully model integrate and fire neurons and observe behavior that resembled the action potentials of real integrate and fire optical neurons. Based on our settings, red neurons had the highest gain, followed by green neurons, and finally blue neurons. The blue neuron was the center neuron, while the red and green neurons were to the side of the center neuron. Figures 15, 16, and 17, show the action potentials of these neurons when they were isolated from the other neurons. This was done by covering light from the other two neurons. We can see that all three neurons quickly reached threshold, spiked, and had a brief refractory period. For all of the neurons, the charging resembled RC charging as we would expect from leaky integrate and fire neurons. In addition, as set by their excitatory gains, the blue neurons converged the slowest, followed by the green neurons, and finally the red neurons.

Figure 18: Red and Center Neurons Illuminated
Figure 19: Green and Center Neurons Illuminated
We then checked the results of pairs of neurons at a time. Specifically, we shined light on only the red and blue neurons, and then only the green and blue neurons. From this, we were able to clearly observe the effects of lateral inhibition on the neuron. As compared to when in isolation, these neurons fired slower. In addition, the green neuron had a larger inhibitory gain than the red neuron. As seen in the figures, the blue neuron fired slower as a result of the inhibitory gain from the green neuron than that of the red neuron. Figure 18 and 19 contains images of pairs of neurons charging and firing.

Finally, we tested the effects of illumination on all of the neurons. We observed all 3 neurons charging and spiking. Lateral inhibition was much stronger in this case, especially for the center neuron. The center neuron spiked much less frequently when all neurons were active. Sometimes, if charging was too slow, the neuron would completely fail to spike as well. This often was a result of the ADC not reading the LED as strongly as previously measured. Figure 20 contains an image of the potentials when all three neurons were illuminated.
Figure 20: Voltage over Time for Neuron
< Figure 20: All Neurons Illuminated

Issues Faced

We faced several issues during the design and testing of this project. Our first issue involved conflicts between Adafruit's MCP3008 ADC and the Raspberry Pi's GPIO instantiation. This conflict essentially involved resulted from a dependency issue of Python3 and Adafruit's GPIO calls. The fix was relatively simple through the use of CHris Ballard's 'pin_ina219' library.

Our second issue involved fixing our strangely slow plotting speed. When first running Matplotlib on the TFT screen, our update times were extremely slow (>2 seconds per new data point). We initially assumed that this was because our entire program was running sequentially, with a new data point being processed only after graphing had finished. However, even after using the "threading" module to separate our graphing and ADC reads, we still were not able to improve efficiency. The saving grace ironically came when we changed our kernel from preemptive real-time Wheezy to standard Jessie. For some reason, this completely fixed our update errors. We can only assume that some portion of the preemptive kernel was messing with our Matplotlib module's savefig function and causing latency issues.

Our final issue involved the seemingly quick sensitivity loss of our LEDs. When connected into our circuit, the LEDs would initially act as perfect photodiodes, with varying analog signals depending on the amount of light shown. However, over a week, their sensitivity would decrease and require them to be replaced with another LED. They would still work perfectly fine as standard LEDs but no longer could work as a photodiode. This was especially confusing as the amount of voltage generated was far less than their threshold voltage and little current was flowing through the LED. Ultimately, we assume that there are some inherent material characteristics that prohibit the LEDs we were using to act photovoltaicly for long periods of time.

Future Work

We first would improve the performance of our system. Our display updates approximately once every half second, which is slow with respect to neuron behavior. In the future, we will update our program to use fixed point calculations instead of floating point to speed computation time. We will also implement better threading so that calculations can truly be done in parallel. In addition, we will change the platform of the program from python to C in order to optimize and be able to use a real time kernel. Finally, matplotlib is overall slow at graphing. We will graph in a way that is more efficient than re-plotting the entire time series every time.

In addition, we will add functionality to the device. Many of the settable parameters are only settable in code. In the future, we will allow them to be settable by potentiometer so that users of the device can see the effects of the parameters on the action potentials. For instance, changing the capacitance and time constant changes the time for the neurons to reach threshold. These changes would add significant value to the device.

Finally, we will implement a way to expand the array in order to see the effects of optical illusions on neurons. We will be able to show grid patterns and see how these patterns affect neuron responses.

Youtube Video

Check out our video to see the optical neurons in action!



We would like to thank Professor Joe Skovira and all of the ECE 5725 TAs for their dedication and guidance in helping us finish our device. We would also like to thank Professor Bruce Land for the inspiration to design this device.

Parts List

Most of the cost of this device came from costs not included in the budget. The Raspberry Pi Kit, including the case, the Raspberry Pi, the piTFT, and the piCobbler ribbon cable do not count towards the budget. In addition, costs of wires and LEDs are negligible, so they are not included in the budget. We also considered the cost of the wood used to build the device to be negligible. The remaining costs are the cost for the solder boards and the cost of the ADC.

Name Description Price
Raspberry Pi Starter Kit (Raspberry PI, Case, TFT, piCobbler, SD Card) Provides overall computing system N/A
Plywood Necessary for Box Construction N/A
3 Red LEDs Acted as our optical neurons N/A
3 Small LEDs Acted as our indicators for neuron spiking N/A
Protoboard Used to reduce size of circuit $2
MCP3008 SPI ADC Converts analog signals from LEDs to digital signals for R-Pi $2.31
Total: $4.31