Super-resolution photoacoustic imaging

Super-resolution photoacoustic imaging is a set of techniques used to enhance spatial resolution in photoacoustic imaging. Specifically, these techniques primarily break the optical diffraction limit of the photoacoustic imaging system. It can be achieved in a variety of mechanisms, such as blind structured illumination, multi-speckle illumination, or photo-imprint photoacoustic microscopy in Figure 1.

Photoacoustic (PA) imaging

This particular biomedical imaging modality is a combination of optical imaging, and ultrasound imaging. In other words, a photoacoustic (PA) image can be viewed as an ultrasound image in which its contrast depends on the optical properties, such as optical resolution of biomolecules like hemoglobin, water, melanin, lipids, and collagen. The advantages of photoacoustic imaging are that it gives higher specificity than conventional ultrasound imaging and greater penetration depth than conventional ballistic optical imaging modalities. Photoacoustic imaging works by irradiating the target with a short-pulsed laser, or alternatively an intensity-modulated laser. The target absorbs this optical energy, which is converted into heat; in most cases the amount of heat is a fraction of approximately (1 - fluorescence quantum yield). Heat is further converted into a pressure rise via thermoelastic expansion, and the pressure rise behaves as an ultrasonic wave, the wave of which is called a “photoacoustic wave” that propagates throughout the surroundings of the target. The photoacoustic wave is then detected by ultrasonic transducers, and these signals are used by an image processor and computer to reconstruct an image of the target.

Super-resolution imaging and photoacoustics

Recently, several techniques have broken the diffraction limit of light, enabling the observation of individual cellular structures, sub-cellular structures, and processes at the nanometer level, structures that were previously unresolvable by conventional microscopes due to resolutions finer than the optical diffraction limit (~250 nm in lateral direction at high optical NA). However, as super-resolution optical imaging generally relies on fluorophores, only fluorescence imaging by usage of multiple lasers or chemical manipulation of fluorophores is possible. This results in a set of complex configurations for the imaging system and limited use for fluorescent targets. Photoacoustic tomography, is able to complement these super resolution techniques and achieve much greater field of depth and remove the need for fluorescent molecules.

Super resolution techniques

Photoacoustics relies on optical excitation of targets and detection of acoustic emission. Since the frequencies of electromagnetic waves (light) are significantly higher than that of acoustic waves, the optical excitation generally sets the absolute resolution. This absolute resolution is well known in optics and is called the diffraction limit of light. This diffraction limit shown below represents the minimum distance that can be resolved between two objects (or similarly the minimum separation distance between two objects excitated by a laser).

$d=\lambda /2NA$

Super resolution techniques break this limit by generally using nonlinear techniques such as saturations, switchable quenching, or multiphoton absorption. Photoacoustics directly benefits from these techniques as the excitation can be held to a tighter spot and hence detect acoustic waves to a tighter spot, ultimately improving resolution.

Optical saturation and nonlinear thermal expansion

Photoacoustics primarily relies on the pressure wave generated by the absorbing material. At higher excitation energy levels, the excitation becomes greatly nonlinear, leading to different effects on the pressure wave generated. The two equations below are the resulting pressure waves from either nonlinear thermal expansion or optical saturation.

(1) $p({\overrightarrow {r}}-{\overrightarrow {r_{0}}},z-z_{0})={\frac {1}{\kappa }}\{\beta _{1}T({\overrightarrow {r}}-{\overrightarrow {r_{0}}},z-z_{0})+{\frac {1}{2}}\beta _{2}\left[T({\overrightarrow {r}}-{\overrightarrow {r_{0}}},z-z_{0})\right]^{2}\}$

(2) $p_{0}({\overrightarrow {r_{0}}},z_{0},E_{p})=\sum _{n=1}^{\infty }c_{n}({\overrightarrow {r_{0}}},z_{0})\cdot (E_{p})^{2}$

Equation (1) above is the pressure wave generated by the non-linear thermal expansion. Equation (2) is the resulting wave generated from optical saturation.

The key idea is that the sample is pumped with high levels of energy such that all of its electrons are saturated to the excitated state. In this saturated state, the additional energy will be utilized in different ways such as nonlinear thermal expansion. From these pressure waves, images corresponding to the absorption of different molecules can be reconstructed. In Figure 2a and 2b, two 100 nm gold nanoparticles are resolved with super resolution enhanced photoacoustics and verified by atomic force microscopy, a different modality that can break the diffraction limit resolution since it does not rely on light.

Figure 2: Comparison of different imaging modalities on two 100 nm gold nanoparticle.

This can be extended to whole images with scanning techniques and Figure 3 shows the dramatic difference with the added super resolution technique. With super resolution, the smaller details of the melanoma cell pictured is visible.

Figure 3: Before and after super resolution techniques applied to photoacoustic phase microscopy.

Photoswitchable probes

Reversibly photoswitchable proteins can switch back and forth between two optically separated states, making them useful for high-contrast and high resolution PA imaging. Yao et al. demonstrated differential imaging of the reversibly switchable phytochrome BphP1 in in vivo experiments (terming the technology RS-PAM). The two states of the BphP1 molecule are the Pfr and Pf states, referred to as the ON and OFF states, respectively. When stable, BphP1is at the On state. Upon 780 nm laser pulse train illumination, BphP1 molecules in the on state gradually switch to the off state. As a result, the amplitude of the generated PA signals decrease. The decay rate is proportional to the local excitation intensity. Thus, PA signal in the center of the excitation beam will decay faster compared to the surrounding. Using the difference image between the two states gives a high contrast image of the molecules (Figure 4), and sub-diffraction limited lateral and axial resolutions will be achieved. In the lateral direction, the decay dependence on the local excitation intensity results in a smaller FWHM of the decay PSF. The higher the order of the nonlinear decay, the higher the resolution enhancement will be. The effective lateral PSF for this imaging technique was shown to be $(0.51/{\sqrt {1+bm}})\lambda _{0}/NA$ where b is the power dependence of the switching-off rate on the excitation intensity (for BphP1, b=1), and m is the order of the polynomial fitting. This technique achieves finer resolutions with a factor of ${\sqrt {1+bm}}$ compared to conventional PAM.

In the axial direction, the nonlinear optical effect will determine the resolution (since only in focus molecules contribute in the signal decay), thus, making it independent from the acoustic detection. This allows this technology to have optical sectioning capability. For point targets, the achievable axial resolution is $1.8{\sqrt {2^{1/1+bm}-1}}\times (\lambda _{0}/NA^{2})$ whereas for large targets, it is $1.8{\sqrt {2^{1/bm}-1}}\times (\lambda _{0}/NA^{2})$

Figure 4: PA images of BphP1 expressing U87 cells and HbO2 in scattering media. The differential image effectively removes the background signal, increasing the contrast of the cell area.

As a demonstration (figure 5) this technique showed much finer lateral and axial resolutions compared to conventional PAM. The lateral and axial resolutions were quantified to be ~141 nm and ~400 nm, respectively, which were about 2 and 75 times better than that of conventional PAM.

Figure 5: Comparison of lateral and axial resolution of conventional PAM and RS-PAM in imaging.

Two-photon photoacoustic mechanism

Non-radiative two-photon absorption can be utilized to achieve high 3D resolution in in vivo experiments. In conventional PAM temporal or spatial filtering is needed to eliminate background signal and achieve high axial resolution. However, in 2PAM, only the area within the focal spot generates nonlinear photoacoustic signals (figure 6). Based on the two-photon absorption PSF, the lateral ($\Delta x_{2PAM}$) and axial resolution ($\Delta z_{2PAM}$) of the 2PAM system are determined to be...

(3) $\Delta x_{2PAM}={\begin{cases}{\frac {0.64\lambda _{ex}}{{\sqrt {2}}NA}},&{\text{NA}}\leq {\text{0.7}}\\{\frac {0.65\lambda _{ex}}{{\sqrt {2}}{NA}^{0.91}}},&{\text{NA}}>{\text{0.7}}\end{cases}}$

(4) $\Delta z_{2PAM}={\frac {1.064\lambda _{ex}}{\sqrt {2}}}\left[{\frac {1}{n-{\sqrt {n^{2}-NA^{2}}}}}\right]$

where λ_ex is the excitation wavelength, and n is the medium refractive index. The important property of the axial resolution is its independence from the frequency of the photoacoustic waves. Therefore, lower frequency ultrasound waves can be used for deeper detection.

Figure 6: Comparison between PAM and 2PAM.

Although two-photon excitation can potentially give a smaller PSF, but because one photon absorption is achieved easily by molecules, there will be a prominent 1PA signal from the area around the focal spot, making the detection of 2PA signals a hard problem. The 2PA signal can be separated from the 1PA background signal using a lock-in detection system, as demonstrated by Lee et al... After amplitude modulation of the input laser train (modulation frequency = f), nonlinear absorption of molecules within the focal excitation spot will generate high harmonics of the modulation frequency (2f, 3f….). Photoacoustic signals from the two-photon absorption can be extracted by locking at the second harmonic of the modulation frequency (2f). In this paper, lateral and axial resolutions of 0.51 μm and 2.41 μm in in vivo experiments were achieved, demonstrating the sub-femtoliter resolution (0.49 μm³ with NA=0.8) of the 2PAM imaging system. 1PA images had only 0.71 μm lateral resolution. The two-photon nature of the signals contribute to the 1.4 factor improvement of the 2PAM to 1PAM lateral resolution

Figure 7: Comparison between PAM and 2PAM images of melanin distribution.

Structured illumination

Structured illumination is an imaging technique that when applied to microscopy, can double the spatial resolution of that of conventional fluorescence microscopy using the moiré interference pattern, the coarse pattern that is produced when two finer patterns are overlapped and provides easier viewing than either original pattern. Structured illumination occurs in the interaction of a three-dimensional modulated illumination pattern and high-frequency variations in the sample fluorescence caused by small structures, the interaction of which produces a lower-frequency Moiré pattern that contains non-resolvable structures present in the observed image. When these Moiré patterns are imaged in different positions and subsequently computationally post-processed, then the lower-diffraction sample information can be algorithmically decoded and reconstructed. When information above and below the focal plane are added, then spatial resolution will be enhanced and normally non-resolvable sample structures are more easily resolvable. One advantage of structured illumination is its ability to be used with any conventional fluorophores, and one disadvantage is its image acquisition speed that requires complex imaging and therefore compromises temporal resolution needed for live cell imaging

Blind illumination

Blind structured illumination photoacoustic microscopy (BSIPAM) was employed as a feedback-free imaging method that uses random optical speckle patterns as a structured illumination for enhancing the spatial resolution of PA imaging within scattering media. Unlike structured illumination microscopy, where the spatial resolution enhancement was limited only to 2, BSIPAM has a higher spatial resolution. BSIPAM operates by the key principle: to recover absorber distribution ρ at spatial resolution close to speckle size.

Principle

If there are M different speckle patterns, 𝚽₁,...,𝚽_M, and the assumption holds that speckle patterns and absorber distribution are represented by discrete vectors 𝚽_m, $\rho \in \Re ^{N}$, an expression for measured PA data can be written.

$y_{m}=h*\left[\Phi _{m}\cdot \rho \right]+\epsilon _{m}$

where m = 1,...,M. $h\in \Re ^{N}$ is the PSF of the photoacoustic imaging system in discrete form, $\left[a\cdot b\right](x_{i})=a(x_{i})b(x_{i})$ is the pointwise multiplication, $\left[a*b\right](x_{i})=\sum _{j=1}^{N}a(x_{i}-x_{j})b(x_{j})$ is the discrete deconvolution step, and ε_m is the noise in the data.

The goal was to recover absorber distribution and speckle patterns 𝚽_m from the data from the expression above. As the intensity distributions and speckle patterns are unknown, MN equations with (M+1)N unknown scalar parameters are derived from the expression. Sound sources $p_{m}=\Phi _{m}\cdot \rho$ present in the expression are uniquely determined by deconvolution equations, but deconvolution is ill-conditioned, so authors use block-sparsity to give high-resolution reconstructions.

It is common knowledge that using sparsity gives a super-resolution signal recovery, as sparse-recovery algorithms assume that the signal is a superposition of a few elements from a large set of high-frequency and low-frequency information. But this approach misses the main point of the reconstruction problem: that all products come from same density distribution ρ. A joint sparsity term $\left\Vert p\right\Vert _{2,1}=\sum _{i=1}^{N}{\sqrt {\sum _{m=1}^{M}\left\vert p_{m}(x_{i})\right\vert ^{2}}}$ is implemented as a regularization term on p = (p₁,...,p_M) and a reconstruction algorithm, block-FISTA, is developed to realize this joint-sparsity approach for solving the expression.

Experimental setup

In the experimental setup of BSIPAM (Figure 4), the sample is placed in a water-filled tank with transparent walls, and light from a pulsed Nd:YAG laser (λ = 532 nm) goes through a glass diffuser and focused on the sample with a lens with focal length f = 50mm. The speckle size at the focal plane of the lens is 1.22λf/d, where λ is the wavelength of the light source and d is the aperture diameter. Ultrasonic signals were recorded by an ultrasonic transducer that is connected to an ultrasonic receiver and sampled with a 12-bit digital oscilloscope. Then, photoacoustic signals were collected with a step size of 10 μm over a scan length of 1.0 mm, the time traces were processed by the Fourier transform, and the magnitude was produced between 15 and 85 MHz to determine the net photoacoustic response at each spatial location.

Figure 4: Schematic of the experimental setup of BSIPAM. Figure courtesy of.

Research

BSIPAM was tested with a simple one-dimensional sample where the absorber distribution ρ was a series of 8 lines of 10 μm thickness, the distance between the lines varied from 40 and 150 μm, and the absorbers were illuminated with the speckle pattern with a speckle size of 25 μm (Figure 5). The photoacoustic response distribution was determined by the product of the speckle intensity and absorber distribution. Next, it was assumed the absorber was present in the focal plane of an ultrasonic transducer with a Gaussian point spread function (PSF) h with a FWHM of 100 μm. The photoacoustic response was determined by the convolution of the source distribution with the transducer PSF, and the experiment was repeated M = 100 more times with a new random speckle pattern each time.

Figure 5: The results from running BSIPAM with the simple 1D sample. (a) The sample, (b) The speckle pattern, (c) the photoacoustic source distribution from the product of speckle intensity and absorber distribution, and (d) photoacoustic response from the convolution of PA source distribution and transducer PSF. Figure courtesy of.

Figure 6 shows the results of six line scans of M = 100 total line scans, each of them from a different random speckle pattern.

Figure 6: (a) Results from the photoacoustic PA response measured from line scans over the absorber distribution subjected to six different random illumination patterns. (b) Object and reconstructions of object using 100 speckle patterns where responses are shown. Figure courtesy of.

Based on Figure 6, structured illumination via random speckle illumination influence line scan response—this variation needed for super-resolution imaging. Also, on Figure 6(b), a set of different techniques were applied to compare their photoacoustic responses in reconstructing the absorber distribution. The mean response was obtained by averaging all line scans, the variance response was obtained by applying the square root of the signal variance at each spatial position, the Richardson-Lucy deconvolution (RLD) response was obtained by performing the deconvolution of the mean response with the given Gaussian PSF, and the BSIPAM response was obtained through the block-VISTA algorithm. In comparison to the other responses, the BSIPAM response shows that the smallest feature spacing (40 μm) is resolved, giving a resolution advantage over other approaches.

In another example, BSIPAM was tested with a two-dimensional sample, a star with an absorber of multiple lines. The sample has size 256 μm², the distance between the lines is 17 μm, and the PSF is a two-dimensional Gaussian kernel with a FWHM of 35 μm. The BSIPAM experiment is repeated M = 200 times, each with random speckle patterns, and the speckles each have a size of 9 μm. Figure 7 displays the results. When BSIPAM is applied, the resolution is improved by a factor of 2.4.

Figure 7: The results with a 2D sample and 200 random speckle patterns. (a) The object. The reconstructed object using... (b) the mean PA response, (c) regularized deconvolution, and (d) BSIPAM are displayed. Figure courtesy of.

Speckled Illumination

Multiple optical speckle illumination was used as a source of fluctuations to produce super-resolution PA imaging. Specifically, a second-order analysis of optical speckle-induced PA fluctuations to develop PA images beyond the acoustic diffraction limit

Principle

Multiple optical speckle illumination makes use of the principle of super-resolution optical fluctuation imaging (SOFI). SOFI is the principle that a higher-order statistical analysis of temporal fluctuations caused by fluorescence blinking helps resolve uncorrelated fluorophores in the same diffraction spot.

In principle, it is assumed that the reconstructed PA quantity, A(r), can be written in the form of the equation:

$A(r)=[\mu _{a}(r)\times I(r)]*h(r)$,

where μ_a is the optical absorption distribution, I is the optical intensity pattern, and h is the point spread function (PSF). If the region of interest is illuminated by multiple speckle patterns I_k(r) with the ensemble mean $\left\langle I(r)\right\rangle =I_{0}$, then the expression for the mean PA image can be calculated by averaging the PA images. These PA images can be produced from multiple realizations I_k(r) of the speckle illumination, and therefore the expression is as follows:

$\left\langle A\right\rangle (r)=I_{0}\times [\mu _{a}(r)*h(r)]$.

This shows that the resolution is determined by the spatial frequency in h(r). If it is assumed that the speckle size is much smaller than that in h(r), then the variance image is provided by this expression:

$\sigma ^{2}[A](r)\varpropto \mu _{a}^{2}(r)*h^{2}(r)$.

The squared PSF has a higher frequency content than the PSF itself, and therefore the variance image has a higher resolution than the mean image.

Experimental setup

As shown in Figure 8, a 5 ns pulsed laser beam was focused on a ground glass rotating diffuser, and the light was scattered onto two-dimensional absorbing samples that were embedded in an agarose gel block. The speckle grain size was recorded to be approximately 30 μm due to the sample distance from the diffuser, 5 cm. The absorbing samples were then placed onto the ultrasound transducer array that was connected to an ultrasound scanner, and polyethylene beads of 50-100 μm in diameter were assigned in place of absorbing samples with isotropic emitters. Finally, the diffuser was rotated to produce a series of PA images for 100 different speckle patterns, and the mean and variance images were produced.

Figure 8: Schematic of the experimental setup of multiple optical speckle illumination. Figure courtesy of.

Research

When multiple optical speckle illumination was tested on a sample of a set of randomly distributed 100 μm diameter absorbing beads, the variance image clearly displayed the contributions of each bead, the images displayed approximations of the point spread function (PSF) and its square, and the resolution was enhanced by a factor of 1.4 for the variance image as opposed to the mean image. The variance image appears as the convolution of the sample with the squared PSF, and the results in the below figure (Figure 9) clearly demonstrate the ability of SOFI to produce super-resolution PA imaging with multiple-speckle illumination.

Figure 9: (a) A photograph of the sample of 100 μm diameter beads, (b) The mean PA image of the sample over 100 speckle realizations, (c) The variance image of the sample. Insets in both (b) and (c) are images of a single bead. Figure courtesy of.

Future directions

Super-resolution PA imaging faces potential directions. The algorithm of BSIPAM has the potential of reconstructing structures from signals using other modalities such as photothermal imaging or optical coherence tomography. Multiple speckle illumination can be applied to the fluctuation of the absorption caused by blinking or switchable contrast agents, instead of simply tissue-induced temporal decorrelation of speckle patterns

See also

• Super-resolution imaging
• Photoacoustic imaging
• Super-resolution microscopy
• Diffraction-limited system