Artificial vision inspired by natural vision system and neural network. (a) Schematic of human vision and recognition system. The incident light is focused by the lens, detected and preprocessed by the curved retina, and recognized by the neural network in the visual cortex. Inset shows the neural network comprised of neurons and synapses. (b) Ray tracing simulation of human eye model. Reproduced with permission from [7]. Copyright 2020 Nature Publishing Group (NPG). (c) Ray tracing simulation of bio-inspired electronic eye consisting of a plano-convex lens and curved image sensor array. (d) Ray tracing simulation of conventional camera consisting of double Gauss lens and flat image sensor array. Reproduced with permission from [25]. Copyright 2020 NPG. (e) Schematic showing the image acquisition and in-sensor preprocessing performed using the synaptic PD array. (f) Optoelectronic machine vision processor for image filtering tasks [e.g., edge detection (right top) and image embossing (right bottom)] based on analog vector-matrix multiplications. Reproduced with permission from [6]. Copyright 2020 Wiley-VCH. (g) Schematic of an image sensor-based neural network. Inset shows a schematic of in-sensor pattern classification achieved by the image sensor-based neural network. Reproduced with permission from [37]. Copyright 2020 NPG.

Cured image sensor arrays for bio-inspired electronic eyes. (a) Schematic of a bio-inspired electronic eye composed of a single lens and curved image sensor array. (b) Scanning electron microscopy (SEM) image of a curved image sensor array with a pop-up structure. Reproduced with permission from [41]. Copyright 2008 NPG. (c) Optical microscope image of a curved image sensor array with serpentine-shaped interconnections. Reproduced with permission from [44]. Copyright 2011 National Academy of Sciences (NAS). (d) Schematic showing that a silicon image sensor array with truncated icosahedron design deforms into a hemispherical format. Reproduced with a permission from [46]. Copyright 2017 NPG. (e) Optical microscope image of a shape-deformable PD array under biaxial stretching. The inset shows the undeformed device. Reproduced with permission from [47]. Copyright 2021 NPG. (f) Photograph of a high-density curved image sensor array based on 2-D materials and the truncated icosahedron design. (g) Transmission electron microscope image of an ultrathin image sensor array based on the MoS₂/graphene heterostructure. Reproduced with permission from [11]. Copyright 2017 NPG. (h) Photograph of a hemispherical electrochemical eye with high-density perovskite nanowire array. (i) SEM image of vertically aligned perovskite nanowires. Reproduced with permission from [52]. Copyright 2020 NPG.

Electronic eyes for mobile artificial vision applications. Schematic of the optical properties required in mobile artificial vision applications, including (a) focus adjustment, (b) deep depth-of-field, and (c) wide field-of-view. (d) Schematic of focus-adjustable electronic eye with a tunable lens and deformable curved PD array. The shape of lens and PD array is tunable by a microfluidic system. (e) Images acquired by the focus-adjustable electronic eye under four conditions. The objects are located at different distances (z) , and the radii of curvature (RD) of the PD array are adjusted to match those of each focal plane. Reproduced with permission from [44]. Copyright 2011 NAS. (f) Photograph of focus-adjustable electronic eye consisting of an electrically tunable lens and a magnetically deformable curvy PD array. The inset shows the acquired images of the objects located at different distances. Reproduced with permission from [47]. Copyright 2021 NPG. (g) Photograph of an artificial compound eye exhibiting deep depth-of-field and wide field-of-view. (h) Wide field-of view image captured by the artificial compound eye. Reproduced with permission from [56]. Copyright 2013 NPG. (i) SEM image of a microlens array that optically connects omnidirectional incident light to a planar image sensor through waveguides. The inset shows the cross-sectional structure of the microlens array. (j) Imaging setup for capturing the images incident to the 3-D microlens array by optically guiding the light to the planar image sensor. Reproduced with permission from [55]. Copyright 2021 NPG. (k) Ray tracing simulation of aquatic vision-inspired electronic eye. Reproduced with permission from [7]. Copyright 2020 NPG. (l) Photograph of an aquatic vision-inspired electronic eye consisting of a spherical lens and a hemispherical image sensor array. Reproduced with permission from [16]. Copyright 2016 NPG. (m) Wide field-of-view and (n) deep depth-of-field imaging results obtained by the aquatic vision-inspired electronic eye. Reproduced with permission from [7]. Copyright 2020 NPG.

Synaptic PDs for in-sensor image processing. (a) Schematic showing in-sensor preprocessing performed by synaptic PDs to improve image recognition. The inset shows the time-dependent current output of the synaptic PDs compared with the instant current output of the conventional image sensor. (b) Preprocessed images with enhanced contrast compared to the noisy raw images. (c) Image recognition rate of the preprocessed images and noisy raw images at each training epoch. Reproduced with permission from [64]. Copyright 2019 NPG. (d) Images acquired during in-sensor preprocessing. The different numbers of optical pulses are irradiated to the synaptic PD. Reproduced with permission from [65]. Copyright 2021 NPG. (e) Schematic showing the habituation behavior of the sensory nervous system. (f) Preprocessed images acquired under scotopic (top) and photopic (bottom) conditions. Reproduced with permission from [67]. Copyright 2021 Wiley-VCH. (g) Photocurrent generation and decay characteristics for programming the conductance of four pixels of synaptic PD array. (h) Target kernels for image filtering (top) and the corresponding optically programmed kernels based on the conductance of crossbar array (bottom). Reproduced with permission from [6]. Copyright 2020 Wiley-VCH. (i) Microscope image of an image sensor integrated with neural networks. (j) Currents of the image sensor-based neural network in response to the illumination of noisy image of letter “z” during each training epoch. Reproduced with permission from [37]. Copyright 2020 NPG.

Neuromorphic-type electronic eyes based on curved synaptic PD arrays. (a) Schematic showing a neuromorphic-type electronic eye consisting of a plano-convex lens and cNISA. The neuromorphic-type electronic eye derives the preprocessed image from the noisy optical inputs through single lens-based image acquisition and in-sensor preprocessing. The inset shows the photon-triggered synaptic plasticity of the synaptic phototransistor. (b) Schematic of a synaptic phototransistor based on the MoS2-pV3D3 heterostructure. (c) Long-term potentiation of the synaptic phototransistor in response to frequent optical pulses. (d) In-sensor preprocessing for contrast enhancement performed by the neuromorphic-type electronic eye. Reproduced with permission from [25]. Copyright 2020 NPG. (e) Exploded schematic showing the neuromorphictype electronic eye consisting of a single lens and hemispherical synaptic PD array. The preprocessed images obtained after illuminating the pattern of letter “H” for (f) 10 s and (g) 100 s, respectively. Reproduced with permission from [72]. Copyright 2021 Wiley-VCH.