New Invention

New Invention

SPLIt Near-Infrared charging system for retinal prosthesis

Application

• This invention is used in a vision prosthesis product, particularly the implanted microchip labeled as the "Retinal Implant" to the right
• Individual components of the invention may be useful in other low-light applications
• Each individual pixel is a photodiode and an electrode (or pair of electrodes) - in the case shown to the right the outer hexagonal perimeter is the cathode, the center ring is the anode
• In the circuit the pixel is the diode, the caps are the electrodes, the resistor is the tissue. The light converted to charge is represented by the current source.
• The photodiode collects light, converts to electrical charge, and that charge stimulates the cell tissue through the electrodes
• The lower floor for pixel resolution is determined by the electrode size and spacing, which together influence the magnitude and depth of the signal
• Above that lower threshold, resolution may be further limited by a pixel's ability to convert light into current
• A pixel's size and efficiency determine the percentage of light that gets converted to charge. The less efficient the pixel, the larger it needs to be in order to transmit the same amount of charge

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

Background Physics

• Light -> charge: When light converts to charge, photons are exciting electron/hole pairs in the silicon.
• That charge only becomes useful when it (specifically the minority carrier) reaches a pn junction where it can be collected.
• Before the charge reaches the junction it is diffusing - randomly moving in all directions
• If the charge recombines into the lattice or gets trapped before reaching the junction, then it does not contribute to collected charge
• Infrared light penetrates deep into silicon, so two things must be true in order to collect that energy:
1. The silicon must be thick in order to have more photon to charge conversion
2. The substrate must be lightly doped so the charge can travel further (to reach a junction) without recombining into the lattice

• These are factors unique to this application because in most other sensor applications the charge collected is amplified in any number of processes
• Single photon avalanche diode sensors for example bias the diode at the edge of breakdown so a single photon is enough to trigger an avalanche of current.

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

Prior Art 1 - Prima

Anode1

Cathode

Anode2

• The trenches form the pixel walls, keeping them from talking to each other, but they also act as trap sites

• Stanford is able to achieve 22um pixels by using a manufacturing approach with some key features:
• This process provides not only a narrow trench, but makes possible a n-doped region surrounding the trench (top right).

• As pixel size shrinks, the aspect ratio gets more vertical, making sidewall interactions more critical
• Not only does putting the junction (where the photoelectric energy is captured) around the walls and bottom increase collection area, it protects the electrons/holes from getting trapped at the trench surface

• In a standard lithographic process (below right), the trenches taper with silicon depth, thus the thick silicon (for IR collection) results in wide trenches. Secondly, the option for controlling the doping along the trench is not available, reducing collection efficiency
• In standard sensor circuits, trenches are not needed because the photodiodes operate in a reverse biased configuration and can be isolated with conventional isolation techniques. The forward biased configuration used here would result in parasitic bipolars turning on and would also not solve the deep diffusion issue.

Anode1

Cathode

Anode2

The top solution is Dr. Palanker's solution, used in present research at Stanford, and in the Prima 1, now owned by Science Corporation. The below solution is Science's solution. I doubt they patented that specifically as it's basically a bad version of the Stanford solution.

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

Elegance and Complexity

• The diode as both sense and power source is an elegant solution, but offloads complexity to the glasses
• The glasses must do the following:
• Video capture
• Image processing
• Laser projection
• DMD to control masking pattern
• All of this is very energy hungry and heat dissipation is a major concern
• Once accomplished, the user experience is still pretty bad, akin to viewing the world through a straw - you can only see when looking straight ahead.

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

Prior Art 2 - Inductive Charging

• If pixel power and sense can be decoupled, more options become available
• You can switch to using a conventional image sensor circuit
• Above right, the Harp4k from Iridium uses this approach
• Last I heard they are still in business, they are attempting to sense natural light instead of an IR projected image, but struggling.
• I will guess that they will have the most relevant patents to compete with
• You could do the image sensing externally and transmit to the electrodes
• Below right, the Argus II from Second Sight used this approach
• This approach went out of business. The ribbon cable needed to transmit 600 pixels worth of data. You could see how that's not scalable
• Both of these companies used an inductive charging method, which as you can see significantly complicates the implant

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

New Approach

• 3D silicon technology stacks two or more wafers directly on top of each other
• This is a niche, but available process already used for image sensors (shown above right) to separate sense and image processing circuits

• If the top layer is thin (<5um), and the bottom layer is thick (>25um), then we can use the top for sense and bottom for collection (having lost ~15% of charge to the first 5um)
• This route leads to two families of parts:
1. Continue to project an IR image in addition to the IR flood light, and differentially sense the desired image
2. IR block the image sensor and attempt natural light vision - I think this is the much more exciting option and achieves the goal of system elegance
• The use of the lower substrate may not be limited to power collection
• It will likely make sense to place digital and/or bias circuitry on the lower substrate and leave the top entirely dedicated to sensing

Electrode

Sensor

Sensing Sub

Processing

TSV

TSV

Photodiode

Collection Sub

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

Alternative Approach

• Power COULD be separated along the same plane by placing charging cells on the periphery
• Science has had this idea but hasn't used it because it has some issues that make it not a great approach
1. Requires eye tracking more precise than the present state of the art in order to hit just the power bank with the IR laser, and not hit the image sensors
1. I don't think they've yet considered the possibility of flooding with IR and switching to detecting natural light
2. Any non-image-sensing circuitry is in effect a dark spot for the user. A ring of power-only area becomes a black ring
• Science has a peripheral circuit, but it is for cell-discharge only.
• If they don't have competing claims to this approach, we may as well try to capture this in the claims as well.
• It has the benefit of being able to use a higher percentage of the sense area for image sense

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

System 1: IR Projection

• Continuing to use IR image projection means the external hardware becomes even more complicated, but allows pixels to size down
• External hardware now needs an IR image in addition to an IR flood light
• On-die requirements
• Image sensor - these can be made very small, and sense time can be made very fast OR leave these pixels a little larger and reduce the strength of the projected IR
• This tradeoff is determined by the minimum drive strength requirements (how much current is needed to stimulate the eye tissue), which in turn sets the minimum flood requirements

• Threshold generation - above this threshold a pixel has both flood and image IR content
• Threshold detection - some way to compare the sensor output to the baseline threshold
• Optional clocking/synchronization - continuous sensing is an option, could also clock at the max DMD clock rate (~30kHz), or could do precise clocking by synchronizing with the projected image and putting circuits on a global timer.
• I doubt it makes sense to cover all these avenues individually for this patent. We may cover less by leaving them out, but by including them, I think I'll essentially just be doing the heavy lifting for competitors

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

System 2: Natural Light

• The benefits of switching to natural light are obvious
• Vastly simplified glasses
• Represents significant cost savings even with the more costly 3D stacked die
• More natural user experience - can look around

• Switching to natural light imaging can be used with either the 3D or peripheral approaches but has several challenges
• Sensors need to be larger because the intensity of natural light coming into the eye is EXTREMELY small
• Noise sensitivity will be higher for these picoAmp signal magnitudes
• Cancelling the IR charging light is an issue
• Even if an IR coating is used, blocking 99.99% of light, that remaining 0.01% may still be significant compared to the natural light
• In the 3D solution where the IR block isn't blanketed you'll also have issues with reflected and scattered light
• There's very little room for amplifiers at the pixels
• There's no digital controls for shutter speed and ISO which a normal cameras would have

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

Prior Art: Sensor

• The most viable candidate for low noise, low current image sensing is the classic 4T sense circuit
• Charge collects on the pinned photodiode (PPD) which sits below the surface for very low flicker noise
• The floating diffusion (FD) is set to a reset value and when the transmission gate (TG) is connected, the charge is transferred from the large capacitance PPD to the small capacitance at FD effecting some amount of gain
• The source follower (SF) then buffers that voltage where it waits to be read out to a column amplifier by the column select switch
• The known downsides are the noise due to the use of the source follower and reset switches, and the offsets due to threshold variation of the source follower
• These issues are typically handled by the same method - correlated double sampling (CDS). Sample the output once after reset but before charge has been transferred, then sample it again after charge has been transferred. The difference between the two samples will cancel everything except the source follower noise

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

New: Sensor

• Our specific application of relatively slow image cycles (30Hz) allows us to use very low current circuits which minimizes noise
• The normal 4T uses an NMOS SF to save space on additional wells, but that means it needs a pull-down current since the FD readout always goes down
• By switching to a PMOS SF, and placing a capacitor at the source, the source follower will gradually turn itself off as the capacitor charges
• A small gain as well as CDS is achieved by placing a cap and discharge switch at the drain
• Any current into the source cap also goes through the drain cap, so output voltage scales by C1/C2
• If the drain capacitor is discharged until after the FD reset, then its starting voltage will be zero and any voltage it picks up during transfer from the PPD to FD will be the desired CDS difference voltage
• Some additional noise due to the additional discharge switch
• The big downside is the settling time after TG closes, FD stops moving, and the SF starves itself
• Variation in timing will result in variation in voltage levels
• The longer the sampling time the less variation there will be
• Consequently there will be a dark to light gradient across the die due to propogation delay
• Another downside is the need to reset the source follower. Ordinarily the SF load would do that automatically, now we introduce additional logic to serve the purpose

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

New: Sensor Schematic

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

New: On-die Control

• To take the place of the shutter and ISO controls a camera would have, this system requires a global control mechanism
• The output of each pixel's sensor, shown previously, is compared to a ramped threshold to determine when to connect the electrode to the source of charge
• The source of charge can be either a voltage or current source
• The ramped voltage sweeping through the range of analog sensor inputs determines how long the electrode is driven by the source of charge
• My system uses 2 loops to control the comparator upper and lower thresholds:
• Vstart control loop: PLL-like loop which compares when 10% (can be any low percentage that is measurable) of pixels have been activated to a reference time during the output phase. If the comparator triggers before the reference then threshold must adjust to widen the sense window. In the case of my architecture this means Vstart goes up
• Vend control loop is actually Vramp dV/dt control loop, but works the same way: PLL-like loop which compares when 90% (any high percentage) of pixels have been activated to a reference time during the output phase. If the comparator triggers before the reference time, then the ramp rate decreases
• Conceptually these loops work together so the brightest part of the scene appears white and the darkest appears black regardless of how much of the maximum output voltage range the signal occupies
• A third loop is used to adjust the sense window so it always uses the widest window possible
• Vstart is compared to a desired target value. Every N cycles, if Vstart is below this value the sense time will increase to increase sensitivity.

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

New: Control Diagram

Vref_max

Increase Tsample

Stated as simply as possible:

• Loop 1: Adjust Vramp start voltage to ensure Vsense equals Vref_start at the time when Phase_ref_start triggers
• This controls the light end of the exposure
• Loop 2: Adjust ramp current to ensure Vsense equals Vref_end at the time when Phase_ref_end triggers
• This controls the dark end of the exposure
• Loop 3: Adjust Tsample so that Vstart equals Vref_max whenever possible
• This serves to use as much of the amplifier output range as possible, minimizing the impact of offsets and noise
• When sample time is maxed out Vstart will continue to be determined by Loop1

Sample TimeCounter

Vamp Comp

LOOP 3

Vref_start

Vstart Comp

Pixel Array

Vstart

Phase Detector

Vramp

VDD

Ramp Buffer

Phase_ref_start

Vsense

pix[0]

elec[0]

LOOP 1

. . .

Phase_ref_end

VDD

Phase Detector

pix[n]

Vend Comp

Iramp

elec[n]

Vref_end

LOOP 1

Load Sense

Outbound Circuit Design, James Yasuhara Confidential and Proprietary

Natural Light System

• To demonstrate the control loop I replaced the electrode with a pure capacitor load. Each pulse represents a read out cycle. When the pixel output exceeds the ramped voltage (not shown) a current source is connected to the load - the brighter the pixel, the earlier the ramp starts and the higher the final voltage
• The colored lines represent the outputs of a 16 pixel test case
• First time step magnitude is very low (<0.9V), and the difference between brightest and darkest pixels is small (<400mV)
• Last time step shows full range operation with a 2V peak and nearly 2V difference between brightest and darkest pixels
• After confirming stable operation, I expanded the array to 1200 pixels (30 x 40) for image processing. The image to the right shows this same process as it applies to a full image

Last time step

First time step

Outbound Circuit Design, James Yasuhara Confidential and Proprietary