1. INTRODUCTION

Fluorescence multiplexing has become a critical tool for multiparameter studies of cell phenotypes. Fluorescence multiplexing, or the simultaneous identification of multiple fluorescent labels in a sample, is often used in flow cytometry for the visualization of many cell-surface and intracellular markers. Multiplexing in flow cytometry enables high throughput analysis of single-cell characteristics such as complex cellular phenotypes and cell size, alongside population measurements [1,2]. The continued development of fluorescent markers and corresponding flow cytometry instrumentation over the last few decades have expanded label identification capabilities from a handful of labels [3,4] to panels consisting of 28 targets [5,6]. Flow cytometry is now regularly used as an exploratory tool to identify unique cell types in a range of tissue environments in various disease states [7–11]. However, spatial tissue information is lost during sample preparation for flow cytometry, so multiplexed immunofluorescence in formalin-fixed paraffin-embedded tissue sections or biopsies has gained traction in recent years [12–14]. As the number of fluorescent labels increases alongside development of corresponding slide scanning instrumentation for spectral discrimination of multiple labels, researchers can identify more advanced cellular phenotypes with regard to their position in the tissue—for example, immune cells’ proximity to a tumor [15–17].

Cell-resolved imaging in live animal models offers the opportunity to follow the dynamic interactions among cells driving normal and disease-state physiological processes and would benefit from the kind of fluorescence multiplexing now used in flow cytometry and immunohistology to enable the simultaneous visualization of a broad variety of cell types. For example, the inflammatory response in mouse models of disease or the immune response to infectious agents involves the recruitment of a complex array of different inflammatory and immune cell types, whose interactions with each other and with surrounding cells and tissue structures influences cellular behaviors. Visualizing such inflammatory or immune responses a few cell types at a time could limit the ability to identify critical cell interactions. Current microscope designs that can effectively separate signals from multiple spectrally overlapping fluorescent markers—typically utilizing spectrally resolved detection—are poorly suited to imaging in scattering samples, limiting their utility for in vivo measurements. Two-photon excited fluorescence (2PEF) microscopy has become the technique of choice for imaging deep into scattering samples and enables visualization of fluorescently labeled features at subcellular resolution. In 2PEF, a tightly focused, infrared wavelength (700–1300 nm), femtosecond duration pulsed laser source provides nonlinear excitation of fluorescent markers that is restricted to a submicrometer focal volume. Images are formed by raster scanning this focal volume and recording the fluorescence intensity point by point. This approach provides high-contrast, high-resolution imaging of labeled structures deep in mouse cortex (${\sim}{1}\;{\rm mm}$), despite optical scattering [18]. However, typical 2PEF systems only have two to four different wavelength channels, limiting measurements to just a few fluorescent markers, constraining the complexity of cellular interactions that can be studied.

A number of methods have been developed to increase spectral resolution and enable 2PEF to distinguish a larger number of fluorescent species. Many commercial (Zeiss Meta/Quasar, Leica SP) and custom systems [19–22] utilize dispersive optics, such as a diffraction grating or prism, to spectrally separate emitted fluorescence for detection on an array of optical detectors. The scattering of fluorescence photons before they are collected by the microscope objective leads to a divergent cone of light at the back aperture of the objective. Dispersive detection methods are angle-dependent, so this divergent light leads to a dramatic loss of spectral resolution—over 100 nm of spectral blurring per degree of divergence with a diffraction grating (Fig. S1 and Supplement 1). On the other hand, blocking this scattered light (e.g., with a confocal pinhole) strongly reduces the signal strength and eliminates much of the benefit of 2PEF for deep imaging [23]. Other approaches to increase spectral resolution in 2PEF have focused on optical designs that simply increase the number of detector channels and/or use multiple excitation laser wavelengths [24–28]. End users may be able to modify commercial instruments to acquire spectral data sets by tuning between multiple laser wavelengths between acquisitions, or manually swapping filter sets. Although adequate for some multiplexed experiments, these approaches are often difficult to implement or are not fully optimized for efficient spectral detection in scattering samples for increasingly multiplexed applications.

Here, we describe a hyperspectral multiphoton microscope (HMM) that temporally multiplexes three excitation wavelengths and tunable spectral detection, on a frame-by-frame basis, to image with high spectral resolution. Emitted fluorescence was spectrally resolved using angle-tuned bandpass filters (ATBFs) with an optical system designed to collect nearly the full cone of divergent light from the objective back aperture, without loss of spectral resolution. A total of 12 successive image scans were acquired over about 35 s to produce a 48-channel hyperspectral data set. We demonstrated hyperspectral imaging in a variety of samples, including multiple colors of fluorescent beads in a gel, multiple colors of fluorescent proteins fused to intracellular proteins in cultured cells, and multiple cell types and tissue structures labeled using both exogenous dyes and fluorescent protein expression in mouse cortex and skin, in vivo.

2. METHODS

A. Hyperspectral Multiphoton Imaging Setup

Imaging was performed on a custom-built laser scanning microscope incorporating three femtosecond laser sources (Ti:Sapphire: Chameleon Vision and Mira, Coherent; Yb:fiber: Satsuma, Amplitude Systèmes) [Fig. 1(a)]. Laser beams were combined and spatially overlapped using two dichroic mirrors with cutoff wavelengths of 875 and 1000 nm (FF875-Di01, Semrock; DMSP1000R, Thorlabs). Beam powers were independently controlled by servo-controlled half-wave plates and a polarizing beam splitter and were monitored using photodiodes. The three laser beams were expanded using independent two-lens telescopes, so the ${1/e}$ diameter just filled the 4-mm clear aperture of the galvanometric scan mirrors (Cambridge Technologies). These telescopes served a dual purpose: to overfill the back aperture of the objective and to allow for independent tuning of the axial focal location for each excitation laser to ensure parfocality. Two objectives were used for imaging: a ${25\times}$, 1.05 numerical aperture (NA) water immersion objective for bead samples and in vivo imaging (Olympus, XLPLN25XWMP2), and a ${63\times}$, 1.2 NA water immersion objective for live cell imaging (Zeiss). To overlap the excitation volumes of the three lasers, we ensured all three beams were colinear for 3 m of propagation, and we varied the divergence of the shortest and longest wavelength lasers so that the focal planes of all three lasers overlapped. Axial alignment of the focal planes was typically within 1 µm with the 1.05 NA objective.

 

Fig. 1. The HMM multiplexed three excitation lasers and 16 spectral emission bands to provide a 48-channel image. (a) Schematic diagram of the HMM. Three different wavelength femtosecond laser beams passed through mechanical shutters, were combined with long-pass (LP) and short-pass (SP) dichroics and directed to galvonometric scan mirrors (SMs). Fluorescence emission was collected via the microscope objective and routed using LP dichroics to four separate, broad color channels (Chan A–D). ATBFs were placed before each detector to select small passbands of emission light. (b) Plot of the transmission through the ATBFs for the angles used in hyperspectral imaging. The vertical dashed lines represent the cutoff wavelengths for the LP secondary dichroics and the 700-nm SP blocking filter. A hyperspectral image was acquired by taking a four-channel image using one excitation laser for the first filter position (1), then repeating this image across the other two excitation lasers, and then repeating this entire process across the other filter angles (2–4).

Download Full Size | PDF

Fluorescence was epidetected from the sample and directed to custom detection optics with a dichroic mirror (700dcxru, Chroma). Fluorescence from the sample was first sent through a 720-nm short-pass filter to block excitation light (FF01-720/SP, Semrock) and then divided into four broad color channels using three long-pass dichroic mirrors with cutoffs at 495, 552, and 624 nm (FF495-Di03, FF552-Di02, FF624-Di01, Semrock). In each channel, an ATBF (TBP01-490/15, TBP01-550/15, TBP01-620/14, TBP01-700/13, VersaChrome, Semrock) was attached to a high-torque motor (S9352HV, Futaba) via a custom aluminum frame. Signals were detected with GaAsP photomultiplier tubes (PMTs, H10770PB-40SEL, Hamamatsu) with gain control and conditioned with a custom preamplifier and a four-channel lowpass filter set to a 1-MHz cutoff frequency (3944 Four Channel, Krohn-Hite). Signal acquisition and scan mirror signals were controlled with a PCI-6610 DAQ board (National Instruments). Wave plate rotation, shutters, and filter motors were controlled with a PCI-6602 DAQ board (National Instruments). All images were acquired with ScanImage software (Vidrio) in MATLAB (MathWorks), with custom scripts for hyperspectral acquisition.

B. Scattering-Insensitive Detection Optics Design

When imaging in scattering samples, the 2PEF exits the back aperture of a microscope objective in a divergent cone. Detection optics were designed and ray-traced in Zemax (Zemax, LLC) to capture as much of this divergent light as possible and deliver it to the active area of the PMT, while simultaneously minimizing the spread in incidence angles on the ATBF to minimize spectral broadening and maximize filter transmission (Fig. S2, Supplement 1). Optics were designed around the 13.5° half-angle of light that exited the back aperture of the 1.05 NA objective (Olympus) when highly diffuse light entered the objective. This approximated the distribution of light from a highly scattering sample. A four-lens optical detection system was designed, with the first lens (80-mm focal length, AC508-080-A, Thorlabs) shared by all detection channels and sitting 75 mm from the back aperture of the objective. A second lens (175-mm focal length, LA1399A.1, Thorlabs) was positioned 175 mm from the first lens, with fixed dichroics that split the emitted fluorescence into four channels between these two lenses. After this second lens, the light passed through the ATBF, with a maximal angular divergence of a 7.7° half-angle. Finally, a pair of lenses (75-mm focal length at 126 mm from second lens and 16-mm focal length at 33 mm further behind; LA1145-A and ACL25416U-A, Thorlabs) condensed the light onto the 5-mm active area of the PMT.

C. Channel Transmittance Measurements

The transmittance of the ATBFs in our setup was measured as a function of angle (Fig. S3, Supplement 1). To acquire these spectra, the PMT for each channel was replaced by an integrating sphere (IS200-4, Thorlabs) with an optical fiber attached to a side port. The collected light was spectrally resolved on a commercial spectrometer (HR4000, Ocean Optics). A halogen lamp with a diffusing plate was placed beyond the focal length of the microscope objective to provide diffuse, spectrally broad illumination. The transmitted spectrum was first measured with the ATBF removed, and then with the filter in place for 1° increments of the filter angle over 60°. The transmittance of the ATBF was calculated and smoothed over a 1.5-nm window.

D. Image Acquisition

Before acquiring a hyperspectral image, it was necessary to first optimize the gain of the four PMTs and the power of the three lasers to achieve high signal-to-noise detection of all fluorescent species while avoiding PMT saturation across all angles of the ATBFs and all excitation sources. Images were acquired from all four PMTs simultaneously. An example imaging sequence was acquired as follows: all four ATBFs were rotated to their first imaging angle (the farthest blueshifted within their channel), the shutter for the first laser opened, a four-channel image was acquired, the shutter closed, and the next laser shutter opened, and a second four-channel image was acquired; this was repeated so that images with all three excitation lasers were acquired. Then, the ATBFs were rotated to the next imaging angle, and the process repeated, until images were acquired across all desired filter angles for all three lasers. In most of the imaging shown in this paper, four distinct angles of the ATBFs [Table S1, of Supplement 1, and Fig. 1(b)] were used, which provided coverage of the visible spectrum (420–700 nm) with ${\sim}{20\,\,\rm nm}$ spectral resolution. To create a 3D image stack, the imaging depth would then be adjusted and the whole process repeated for the new image plane.

E. Spectral Calibration

Because PMT gain settings were optimized for each image, it was necessary to calibrate the signal across PMT channels if determining real spectral profiles was necessary (Fig. S4, Supplement 1). The spectrum of an incandescent flashlight with color-balancing filters (FGT165, Thorlabs) was multiplied by the measured transmission profile for the ATBFs for each acquisition angle (Fig. S3, Supplement 1), providing a “predicted” PMT signal for the incandescent spectrum. After each hyperspectral image acquisition and before changing PMT gains, the flashlight was directed toward a white piece of card stock placed directly below the objective, and the angle of the flashlight was manually adjusted to avoid PMT saturation. The PMT signal was then measured at the ATBF angles used for hyperspectral imaging, as well as at 1° increments over $\pm 4^\circ$ around those angles. For each ATBF angle, these data were used to find a calibration factor that fit the measured PMT signal from the flashlight across those nine angles to the predicted PMT signal. We thus generated 16 such calibration factors (one for each PMT at each of the four ATBF angles commonly used), each averaged across the nine, closely spaced angles where measurements were made. These calibration factors were used to scale the spectral end members to compare with actual emission spectra.

To test the accuracy of this calibration procedure, it was used to measure the spectrum of fluorescent dyes with variable PMT gain and laser power settings (Fig. S5, Supplement 1). Pools of 25 µM fluorescein and 25 µM sulforhodamine 101 (S359, Thermo Fisher Scientific) dye were placed on glass slides, covered with a glass coverslip, and sealed. Images were acquired across 16 emission detection channels with the 900-nm laser source, for three different settings of PMT gain and laser power. These gain/power combinations were chosen to ensure a detectable signal while avoiding saturation, and were representative of settings that would be chosen during image acquisition. After each acquisition, the spectral calibration procedure was performed, and the extracted spectra compared to published spectra and to the predicted spectra for our system, with good agreement all around (Fig. S5, Supplement 1).

F. General Image Analysis

In images where data were acquired with bidirectional scans, we made small shifts in every other row to minimize mismatch between forward and backward scans. A two-dimensional (2D) median filter with a radius of 1 pixel was applied to each image using FIJI [29]. For analysis methods relying on literature spectra of fluorescent species, images were scaled by the appropriate calibration factors.

Due to the chromatic aberration induced by the excitation optics across the three laser wavelengths used for excitation, identical structures imaged with different excitation sources were separated spatially, especially toward the edges of large-area scans [typically less than ${\sim}{2}\;{\unicode{x00B5}{\rm m}}$ for a field of view (FOV) of ${\sim}{300}\;{\unicode{x00B5}{\rm m}}$]. Spectral channels containing similar structures across all lasers were identified, and images for the 800- and 1030-nm excitation were registered to the 900-nm image using a 2D affine transformation (affine2d function in MATLAB). The calculated transform was then applied to all images corresponding to that laser.

Using a nonnegative least squares approach (lsqnonneg function in MATLAB), the hyperspectral data for each voxel in the image were fit as a linear sum across multiple spectral end members. Each fluorescent species or other nonlinear signal [e.g., second harmonic generation (SHG)] in the image was represented by one of these spectral end members. For each set of imaging experiments, the derivation of these end members was unique and is discussed in detail for each experiment. The normalized squared 2-norm of the residual was used to evaluate the goodness of this fit and to identify image regions that the chosen spectral end members did not represent well. For relevant data sets, regions of interest (ROIs) were chosen to evaluate the goodness of fit (Supplement 1).

G. Multicolor Fluorescent Bead Sample

Ten colors of dye-loaded, 15-µm diameter polystyrene beads (F8837 Blue, F8838 Blue-green, F21010 Green, F8844 Yellow-green, F21011 Yellow, F8841 Orange, F21012 Red-orange, F8842 Red, F21013 Carmine, F8839 Crimson, Thermo Fisher Scientific) were embedded in 2% (wt/vol) agarose gel. The bead solutions were concentrated to achieve a density high enough to visualize all bead colors in a single FOV. Briefly, ${\sim}{0.5}\;{\rm mL}$ of bead solution was centrifuged, and the supernatant removed. Additional colors were sequentially added and centrifuged until a pellet with all colors was produced. A small amount of hot agarose was added to the bead pellet, allowed to briefly solidify, and then scooped onto the lid of a cell culture dish for imaging. Samples that contained each bead color individually were similarly prepared. The multicolor bead sample was imaged first, with PMT gains and laser powers set to visualize all beads without saturation. Without changing any imaging parameters, the single-color bead samples were imaged, and the flashlight calibration described above was performed. Image slices were acquired at 2-µm steps, with eight slices for the mixed sample and five slices for the single-color bead samples.

Bead data were preprocessed using the standard workflow. All single-color images were maximum-projected, and both single-color images and mixed-bead images were registered using the affine transformation. Spectral end members were determined by averaging across five different manually determined ROIs for each single-color sample. A single slice of the mixed-bead image was unmixed, and each of the unmixed images assigned a false color to create the color composite. Once data were unmixed, new spectral end members were chosen from the mixed-bead sample, using the unmixed image to identify beads of each color (Fig. S6, Supplement 1). Unmixed images [Fig. 2(c)] were again assigned a false color to create the color composite [Fig. 2(d)]. For the 48-channel image array [Fig. 2(b)] and extracted bead spectra [Fig. 2(f)], data were scaled according to the calibration procedure. The bead spectra were also measured with a spectrometer for comparison with the spectra published by the manufacturer [Fig. 2(a)] and the extracted bead spectra from the HMM [Fig. 2(f), Fig. S7 of Supplement 1].

 

Fig. 2. Hyperspectral image of 10 colors of 15-µm diameter fluorescent beads embedded in agarose gel. (a) Fluorescence emission spectra of the 10 bead colors, as reported by the manufacturer (Thermo Fisher); (b) 48-channel image array of the bead sample; images acquired at a given laser wavelength are shown in rows, and at a given filter angle in columns (denoted by the passband center wavelength). The colored bands indicate the four broad color channels, Chan A–D. (c) Unmixed images of individual bead colors; (d) false-color composite image of spectrally unmixed bead sample; some beads appear to range in size from 15 to 20 µm in diameter. (e) Intensity values of beads for all 10 unmixed images across Lines 1 and 2 in d; (f) calibrated bead spectra across the three excitation lasers measured using the HMM. Spectra were normalized to the brightest dye (Carmine, 800-nm excitation) for relative comparison of bead brightness. Scale bars, 50 µm.

Download Full Size | PDF

To quantify the quality of unmixing, the number of pixels that overlapped between multiple unmixed images was determined. A Rènyi entropy filter (FIJI) was applied to each unmixed image to mask out background pixels and set signal pixels to 1. The number of pixels present in more than one image was measured by summing these thresholded images and identifying pixels with a value higher than 1. The number of pixels present in all image pairs were counted to evaluate significant crossover between different colors of the unmixed image (Table S2). A 2D correlation coefficient was also computed between all unmixed images (Table S3, Supplement 1). The signal-to-noise ratio (SNR) was calculated by selecting an ROI including background pixels to represent background, and multiple ROIs inside the same color beads to represent the bead signal. The SNR was calculated as the ratio of the mean bead signal to the standard deviation of the background signal (Table S4, Supplement 1). To visually evaluate overlap, pixel intensity values were measured across two, single-pixel-wide lines in the unmixed 10-color image [Fig. 2(d)] and the intensity values plotted as a function of distance [Fig. 2(e)].

To evaluate the suitability of our data for pixel-clustering and segmentation, beads were clustered in 10-dimensional space. Each of the 10 unmixed images was thresholded using the Rènyi entropy function in FIJI and maximum projected to mask out the background. A range filter (rangefilt function in MATLAB) was applied with a neighborhood of nine pixels to include texture information in addition to spectral information in clustering, as beads should contain pixels belonging to only one cluster. ${K}$-means clustering was performed using 10 clusters (kmeans function in MATLAB) on all nonbackground pixels. Each cluster was seeded with the average intensity of a particular color bead in its respective unmixed image and zeros in other images. An image was then created that classified all pixels into the 10 clusters identified by ${K}$-means [Fig. S8(a), Supplement 1] and plotted the 10-dimensional profile of 5000 random points belonging to each cluster [Fig. S8(b), Supplement 1]. Because beads cannot physically overlap in the sample, each pixel should belong to only one bead color.

Images collected in each PMT channel (Chan A–D) across all ATBF angles for the 800-nm laser source were summed to simulate a four-channel microscope image [Fig. S9(a), Supplement 1]. The same ROIs used to select spectral end members for 48-channel unmixing were used to extract four-channel end members in the simulated four-channel data set. The four-channel mixed bead image was unmixed and used to generate a composite [Fig. S9(b), Supplement 1], with each of the unmixed images assigned the same false colors as the 48-channel composite [Fig. S9(c), Supplement 1]. The same bead, one for each color, was extracted from both the four-channel and 48-channel composites for direct comparison [Fig. S9(d), Supplement 1].

H. Cultured Cells Labeled with Multiple, Differently Colored Fluorescent Protein Fusion Constructs

To demonstrate multicolor imaging in live cell culture, we expressed fusion proteins that linked different color fluorescent proteins to various intracellular proteins (Tables S5 and S6), using transduced cell lines and additional transient transfection (Table S7, see Supplement 1). We needed cells to exhibit similar, high expression levels of all fluorescent proteins, as dim fluorescent markers are often difficult to image with high signal to noise in the presence of bright markers, even with hyperspectral detection. Simultaneous transient transfection with all vectors would lead to poor expression of some labels, making it difficult to locate cells that expressed all labels. Thus, we generated stable HeLa cell lines expressing one or two pairs of the fusion constructs using lentiviral methods (Table S6). This was done by creating polycistronic plasmids that coded for the production of two different fluorescent proteins, each fused to a different intracellular protein, with a self-cleaving peptide sequence between the two fusion proteins (Figs. S10 and S11 of Supplement 1) [30]. These plasmids were packaged into lentiviral vectors (Fig. S12, Supplement 1). HeLa cells were then transduced with one of these polycistronic constructs, and cells that stably expressed both fusion proteins, mNeon-Keratin/mTFP1-CEBP237, were selected. These cells were then transduced with a second lentiviral vector and cells stably expressing four fusion proteins (mNeon-Keratin/mTFP1-CEBPA237, and sfGFP-Caveolin/mKO2-TOMM20) were selected. Starting with these cells, we then transfected with additional plasmids coding for single or paired fusion proteins to label additional structures before imaging.

 

Fig. 3. Hyperspectral images of live HeLa cells labeled with up to seven different fluorescent protein fusion constructs. (a) Fluorescence emission spectra of seven different fusion constructs, indicating both the fluorescent protein and the fused cellular protein. Note that mAmetrine-LaminB1 is denoted in black in the emission plot and white in the images for optimal contrast. (b) Spectrally unmixed, false-color composite image of cells expressing three fusion proteins with highly overlapping spectral emission [bold curves in (a)]; (c)–(e) spectrally unmixed, false-color composite images of groups of HeLa cells expressing up to seven constructs. Insets (i) and (ii) show cells with multiple nuclear labels. Inset (iii) shows a cell with the signal from mTurquoise-VASP digitally removed to more clearly show the four labels near the cell nucleus, while (iii’) shows the same cell with tdTomato-Calreticulin, also removed for further clarity. See Table S7 of Supplement 1 for cell transfection details for each image panel. Spectra sources, mTurquoise and dKatushka (Addgene, see Supplement 1), mTFP1 [35], sfGFP [36], mNeon [37], mAmetrine [38], tdTomato (Thermo Fisher SpectraViewer). Scale bars. (b) 5 µm; (c)–(e) 25 µm.

Download Full Size | PDF

I. Imaging and Analysis of Cultured Cells

Multicolor cells were plated in 60-mm cell culture dishes, and media were replaced with phosphate-buffered saline (PBS) for imaging. All cell images were acquired with a ${63\times}$, 1.2 NA water immersion objective (Zeiss). Groups of multicolor cells (Table S7) were imaged with fixed laser power and PMT gain settings. Cells expressing a single fusion protein were then imaged with the same settings to enable selection of spectral end members for unmixing. Images were median-filtered, registered, and maximum-projected across multiple slices. After projection, spectral end members were extracted from single-color images (Fig. S13, Supplement 1) and used for nonnegative least squares unmixing of mixed-color samples (Fig. 3; Figs. S14–S17 of Supplement 1). Because each cell expressed an unknown combination of fluorescent labels, residual images and a priori knowledge of structure morphology were used in an iterative manner to identify the labeled structures present (and thus the relevant spectral end members) in the cells in each image.

J. Multicolor Labeling of Brain Cells Using Transgenic Mice and Exogenous Dyes

All animal procedures were approved by the Cornell Institutional Animal Care and Use Committee and were performed under the guidance of the Cornell Center for Animal Resources and Education. Mice expressing yellow fluorescent protein (YFP) in pyramidal neurons (Thy1-YFP-H) were crossed with mice expressing green fluorescent protein (GFP) in microglia (Cx3cr1-GFP) to achieve homozygous expression of both genes. Optical access to the cortex was achieved through a glass-covered cranial window. Animals were anesthetized using isoflurane (1.5%–2% in oxygen) and placed on heating pads with feedback control for body temperature maintenance at 37 ˚C (40-90-8D, FHC, Inc.). Mice were given atropine sulfate subcutaneously (0.005 mg/100 g mouse weight; 54925-063-10, Med-Pharmex Inc.) to prevent lung secretions. Mice were also given dexamethasone subcutaneously (0.025 mg/100 g mouse weight; 07-808-8194, Phoenix Pharm Inc.) and ketoprofen subcutaneously (0.5 mg/100 g mouse weight; Zoetis Inc.) to preemptively reduce pain and inflammation. Bupivacaine (0.1 mL, 0.125%; Hospira Inc.) was subcutaneously administered at the incision site to provide a local nerve block. A 6-mm diameter bilateral hole over the cerebral cortex was created with a dental drill (HP4-917-21, Fordom). Approximately 45 min before removing a portion of the skull, 50 µL of 1 mM Alexa Fluor 633 hydrazide (A30634, Life Technologies) in saline was injected retro-orbitally to label elastin in arterioles [31]. Once the skull piece was removed carefully with forceps, 500 nL of 20 µM SR101 (S359, Thermo Fisher) in saline was pressure-injected through a pulled glass pipette ${\sim}{300}\;{\unicode{x00B5}{\rm m}}$ deep into the cortex, while saline was used on the brain surface to prevent drying of the tissue. After injection, the cranial opening was covered with fresh saline and an 8-mm glass coverslip was glued to the skull using cyanoacrylate adhesive (Loctite) and dental cement (Co-Oral-Ite Dental). The animal was immediately moved for hyperspectral imaging. Prior to imaging, 50 µL of 10% weight/volume (w/v) Cascade Blue dextran (D1976, Thermo Fisher) was injected retro-orbitally to label the vasculature. An hourly dose of atropine sulfate and 5% w/v glucose was administered subcutaneously while imaging. After imaging, some mice were allowed to recover from anesthesia for future imaging sessions. In these animals, recovery was allowed for three weeks and then mice were imaged again. Three hours prior to subsequent imaging sessions, mice were briefly anesthetized and injected with 100 µL of 5 mM SR101 and 30 µL of 1 mM Alexa Fluor 633 hydrazide retro-orbitally to allow time for the dye to clear from the vasculature and to label their respective structures. The mouse was allowed to wake up for a 3 h waiting period, and then reanesthetized for retro-orbital injection of Cascade Blue dextran (as above) and imaging.

Images were median-filtered, registered, and maximum-projected across multiple slices. After projection, spectral end members were extracted from within the image and used for nonnegative least squares unmixing of that image (Fig. 4, Figs. S18–S22 of Supplement 1). Residual images and a priori knowledge of structure morphology was used in an iterative manner to select end members until most structures were accurately defined. To enable visualization of detailed features in some images, logarithmic functions were applied to the image in FIJI.

 

Fig. 4. Hyperspectral images of multiple exogenous dyes and transgenic labels in live mouse cortex. (a) Fluorescence emission spectra of five different fluorescent labels (Thermo Fisher SpectraViewer), indicating both the fluorescent label and the corresponding labeled structure; (b) spectrally unmixed, false-color composite image of five labels 25 µm below the cortical surface. Large blood vessels (cyan) are identified as arterioles due to the elastin lining (magenta) along the vessel edge. Astrocyte end feet (red) also line the vessel wall, with dendrites (yellow) and microglia (green) scattered throughout the FOV. The density of labeling obscures many structural subtleties, and digital removal of the YFP-dendrite and SR101-astrocyte colors reveals small morphological features of microglia processes and elastin along the arteriole (right). (c)–(d) Spectrally unmixed, false-color composite image of four labels 100 and 200 µm below the cortex surface, respectively. The vascular dye decreases in signal rapidly with depth as the 800-nm laser source used for its excitation becomes power-limited. Scale bars, 50 µm.

Download Full Size | PDF

To demonstrate image unmixing with fewer spectral channels, five of the 48 channels were selected, each containing the brightest signal from one of five labels used in the in vivo cortical imaging. Spectral end members from the same ROIs as the 48-channel data set were extracted and used for unmixing [Fig. S23(a), Supplement 1]. In addition to the five-channel data set, another data set was selected by choosing additional channels that could be collected simultaneously from the other PMTs, not adding additional acquisition time. This 12-channel data set was unmixed in the same manner [Fig. S23(b), Supplement 1].

We also created mice that expressed variable amounts of different color fluorescent proteins in cortical neurons, using viral transfection techniques and relying on the stochasticity of viral transfection (Figs. S24–S26; see Supplement 1).

K. Imaging Invasion of Immune and Inflammatory Cells in Skin after Inflammatory Stimulus

We took a multipronged strategy to label different immune cells and stromal components. Three different types of fluorescently labeled antigen-specific CD4 T cells were adoptively transferred into transgenic hosts with tissue specific expression of GFP. Additional subsets of immune cells and stroma were labeled by acute intravascular (IV) and intradermal injections with fluorescently labeled antibodies and small molecule detection reagents. First, naïve CD4 T cells were isolated from OT-II T cell receptor (specific for ovalbumin 323-339 complexed to ${\rm I} - {{\rm A}^b}$) expressing mice (strain 004194, Jackson Laboratory) that lacked expression of a fluorescent protein, or expressed either tdTomato [by breeding OT-II mice with CD4 CRE mice (022071, Jackson Laboratory) and Stop-floxed tdTomato mice (007909, Jackson Laboratory)], or ECFP [by breeding of OI-II mice with CAG-ECFP mice (004218, Jackson Laboratory)]. Next, naïve fluorescent protein expressing CD4 cells were differentiated in vitro into TH2 cells, while the unlabeled naïve CD4 T cells were differentiated into TH1 cells. Briefly, naïve OT-II CD4 T cells were enriched from the spleen and lymph nodes by complement-mediated lysis of MHC class II (clone M5/114, produced in-house), ${\rm CD8}\alpha$ (clone 3.115, produced in-house), and CD24 (clone J11D, produced in-house) expressing cells. These CD4 cells were further enriched for naïve cells by positive selection for CD62L (clone MEL-14, BD Biosciences) expression with paramagnetic beads. Antigen-presenting cells (APCs) were prepared from spleens of syngeneic mice by depleting T cells via complement-mediated lysis of CD90 expressing cells (clone JIJ, produced in-house). To drive TH1 differentiation, naïve CD4 T cells were cocultured with irradiated APCs, OVA peptide (chicken ovalbumin 323-339AA, 1 µM, Biopeptide Company), anti-IL4 antibody (clone 11B11, 40 µg/mL, Bio X Cell), rmIL-12 (20 ng/mL, Peprotech), and rhIL2 (10 U/mL, NIH). TH2 differentiation was achieved by coculturing naïve CD4 T cells with irradiated APCs in the presence of OVA peptide (1 µM), anti-${\rm IFN}\gamma$ (clone XMG1.2, 50 µg/mL, Bio X Cell), rmIL-4 (50 ng/mL, Peprotech), and rhIL-2 (10 U/mL, NIH) (see Ref. [32] for detailed methods). After five days of cell culture, unlabeled TH1 cells were stained with Cell Tracker Deep Red (Thermo Fisher). To visualize TH1 and TH2 CD4 T cells in vivo, ${3} \times {{10}^6}$ Deep Red TH1, ECFP TH2, and tdTomato TH2 cells were transferred by tail vein injection into MacGreen (strain 018549, Jackson Laboratory) mice after 5 days of in vitro differentiation. MacGreen mice express GFP under the control of the csf1r promoter, thus labeling Langerhans cells in the epidermis as well as macrophages and inflammatory monocytes in the dermis. Immediately after cell transfer, the pinna of the ear was immunized with 10 µL of OVA peptide emulsified in complete Freund’s adjuvant (CFA). Ears were imaged 3 days after the OVA injection. 500 ng of each of the following antibodies were coinjected intradermally at the imaging site 2–3 h before the start of imaging: anti-CD8-BV650 (clone H35017.2, BD Biosciences) to label CD8 T cells, anti-CD11c-PE-CF594 (clone N418, BD Biosciences) to label dendritic cells, and anti-GR1-Alexa 594 (clone RB6-8C5, BD Biosciences) to label neutrophils. To decrease background staining due to Fc capture of fluorescently labeled antibodies, anti-CD16/CD32 (clone 2.4G2, produced in-house) were coinjected with the labeled antibodies. Just prior to imaging, elastin, endothelial cells, and lymphatic vessels were labeled, respectively, with an intravenous injection of 5 µg of hydrazine-Alexa 633 (Thermo Fisher), 25 µg anti-CD31-BV421 (clone 390, BD Biosciences), and 25 µg anti-LYVE1-Alexa647 (clone 223322, ${R}$ and ${D}$ Systems), all dissolved in saline.

The ear of the mouse was then imaged at the site of the OVA injection on the HMM. Briefly, the mouse was anesthetized using isoflurane, and received hourly subcutaneous injections of atropine and glucose, as described above. Body temperature was maintained at $37^{\circ}\text{C}$ with a heating blanket. The ear was mounted on a custom-built mouse ear imaging platform, following the approach outlined in Ref. [33]. Power for the three lasers and PMT gains were set for the 16 emission channels to ensure detectable signal across all 48 channels of the HMM without saturation. An imaging field was identified that contained as many differently labeled cell types and tissue structures as could be found. A 3D image stack was taken containing 4-µm spaced slices across a depth of 100 µm, beginning 20 µm beneath the surface of the ear. The mouse was euthanized at the conclusion of the imaging experiment.

Using custom-written MATLAB code, background subtraction was performed for each channel, where the background signal was taken as the mean of the dimmest 10% of the pixels in each channel. The images were then median-filtered using a 1-pixel radius median filter. Spectral end members were found by manually selecting ROIs for cell-shaped objects or tissue structures that appeared to contain single labels, using the full 3D image stack. Using these pure label ROIs, end-member spectra for each label were extracted. To identify the cells and tissue structures imaged, we calibrated the spectral end members with the flashlight procedure described above and compared these to literature spectra that had been scaled by the measured spectral transmission of the ATBFs at the angles used in the HMM image acquisition [Figs. 5(a) and 5(d)]. The extracted 48-element end-member spectra were then used to carry out nonnegative least squares unmixing of the full image stack to generate pure label images [Figs. 5(b) and 5(c); Fig. S27 of Supplement 1]. Maximum projections across 3 to 10 slices were generated for different labels at different depths in the image stack to generate single label images.

 

Fig. 5. Hyperspectral images of multiple cell types and tissue structures in a multicolor fluorescently labeled mouse ear after a dermal inflammatory injury. (a) Literature emission spectra of the six exogenous labels that were identified (with the corresponding marker and the labeled structure) from the region with the most different label types present (Thermo Fisher SpectraViewer, BioLegend Spectra Analyzer); (b) 3D rendering of the stack of image slices taken across a depth of 100 µm at 4-µm intervals beginning at 20 µm below the ear surface. The different cell types and tissue structures are false-colored as indicated in (a). (c) Spectrally unmixed grayscale single-label images obtained from maximum projections across three slices for the eight distinct spectral end members identified in the image stack. The projections were centered at different depths in the overall image stack: blood vessels, 40 µm; hair, 16 µm (10 slices); collagen, 28 µm; macrophages, 36 µm; T helper-2 cells, 92 µm; dendritic cells, 64 µm; elastin, 16 µm; lymphatic vessels, 92 µm; (d) plots of the normalized spectral end members for each label, extracted from pure pixels in the raw image data. Scale bar in panel (c), 50 µm.

Download Full Size | PDF

3. RESULTS

A. HMM

The HMM was a custom-built setup with a four-channel detection design [Fig. 1(a)]. In place of standard interference bandpass filters, we placed ATBFs in front of each PMT for improved spectral discrimination (Fig. S3, Table S1 of Supplement 1). The ATBFs shift transmission to shorter wavelengths as the incidence angle increases, as all interference filters do, but maintain better transparency, bandwidth, and shape of the transmission profile at high incidence angles, as compared to standard filters. In addition, we routed three different-wavelength, femtosecond laser sources into the system to provide a range of excitation conditions for a variety of fluorescent labels. Detection optics were ray-traced to collect as much of the divergent cone of light coming from the back aperture of the objective as possible (Fig. S2, Supplement 1). Hyperspectral images were acquired by alternating laser excitation source and ATBF angle on a frame-by-frame basis [Fig. 1(b)]. To compensate for differences in PMT gain, we imaged a calibration light source and calculated appropriate image scaling factors for the PMT gain settings (Fig. S4, Supplement 1). The accuracy of this calibration technique was confirmed by reconstructing known fluorophore spectra (Fig. S5, Supplement 1).

Each fluorophore in a sample exhibits a unique combination of excitation and emission intensities, defined as the spectral end member for that fluorophore. Images containing multiple fluorophores were unmixed using nonnegative least squares to fit each pixel’s 48-channel hyperspectral signal to a linear combination of spectral end members. End members were based on literature curves for fluorophore excitation and emission or were extracted from single-color samples or from the mixed-color samples themselves, depending on the specific sample features. Finally, false-color composite images were generated that color-code each structure by its fluorophore identity.

B. Fluorescent Bead Samples

We demonstrated the ability of the HMM to differentiate highly overlapped fluorescent labels by imaging 10 colors of fluorescent polystyrene beads embedded in agarose gel [Fig. 2(a)]. A 48-channel hyperspectral image was acquired for the bead mixture [Fig. 2(b)] and for samples containing only one bead color (data not shown). We first extracted spectral end members from the single-color samples and used these to unmix the image of the mixed-bead sample [Fig. S6(a), Supplement 1]. We then selected beads in the mixed-bead image that represented the 10 different colors (now easily distinguishable after the first iteration of unmixing) and created a refined set of spectral end members that were used to, again, unmix the 48-channel data from the mixed bead sample, yielding an image with 10 separate colors [Figs. 2(c) and 2(d); Fig. S6(b) of Supplement 1]. This iterative approach allowed an unambiguous identification of beads representing each of the 10 colors in the mixed bead sample so we could define spectral end members that were minimally affected by any drift in imaging parameters (e.g., laser powers) over the ${\sim}{4}\;{\rm h}$ required to take image stacks of each single-color and the mixed-bead sample. This iterative approach resulted in a modestly smaller residual after unmixing [Figs. S6(c) and S6(d), Supplement 1]. Single-pixel wide intensity profiles show high contrast and good signal to noise across all 10 unmixed colors [Fig. 2(e)]. Some cross talk between the blue and blue-green colors as well as the red-orange and red colors was apparent.

To assess how well-separated the 10 colors in the final unmixed image were, we thresholded the image for each color using the Rènyi entropy method (FIJI) and found that less than 8.6% of total image pixels were above threshold for more than one color, with most of the overlap occurring between the blue and blue-green (4.8%), red-orange and red (1.7%), and carmine and crimson (0.3%) colors (Table S2, Supplement 1). Further, we correlated the intensity across the 10 color unmixed image pixel-by-pixel and found that all color combinations had a correlation coefficient of less than 0.1, except for three bead pairs: blue versus blue-green (0.5), yellow versus orange (0.16), and red-orange versus red (0.2) (Table S3, Supplement 1). Several factors likely contributed to overlap among these unmixed images, including the short wavelength limit of our detection cutting off the emission of the blue beads (${\sim}{72}\%$ of the area under the emission curve cut-off, leading to a low SNR for the blue beads and similarity between the end members for the blue and blue-green beads), the crossover between channels B and C occurring right between the peaks for the yellow and orange emitting beads, and the similar shape of the spectral end members (although different brightness) for carmine and crimson. In addition, we measured actual bead spectra with a spectrometer and found that many of the emission profiles were redshifted and broader when compared to the spectra provided by the manufacturer, likely because the spectra provided by the manufacturer were measured for dyes in aqueous solution at low concentration (Fig. S7, Supplement 1). After calibration, the spectral end members we measured with the hyperspectral microscope [Fig. 2(f)] closely matched the spectrometer-measured spectral shapes, but varied widely in brightness across different bead colors as well as across different excitation laser wavelengths, highlighting the importance of multiple excitation sources for clean unmixing. Both the yellow/orange and red-orange/red pairs exhibited significant spectral overlap in measurements of the real emission spectra, likely leading to the difficulty in separation we observed in our unmixing analysis. We estimated the signal-to-background and signal-to-noise ratios by selecting individual beads and clear background areas for each color channel. We found that signal to background ranged from 15 to 2400 across the 10 unmixed colors, while the SNR varied from 13 to 440 (Table S4, Supplement 1). To assess how well our 10-color image data could be used for clustering and image segmentation, we used K-means clustering on the unmixed data, assuming 10 clusters and masking away the background. We found that the beads clustered very well, with some confusion between the blue and blue-green beads (Fig. S8, Supplement 1). Finally, we estimated what a more standard, 4-channel single excitation source image of the mixed bead sample would look like by summing the data across each ATBF angle for channels A–D for the 800-nm excitation (Fig. S9, Supplement 1). This calculated 4-channel data set was unable to visualize three bead colors (yellow-green, red-orange, and red) and exhibited overlap between bead colors that were well separated in the 48-channel image (e.g., yellow versus orange, carmine versus crimson), frequently leading to misidentification of beads (e.g., a ${\sim}{170}\%$ and ${\sim}{380}\%$ false-positive rate for blue and orange beads, respectively).

C. Cells Labeled with Multiple Fusion Proteins

In order to test the capability of the HMM in cultured cells, we generated HeLa cells with multiple fusion proteins—fluorescent proteins fused to different subcellular proteins [Fig. 3(a), Table S5 of Supplement 1]. In cells transfected with a single plasmid, fusion proteins could be readily imaged on our system (Fig. S13, Supplement 1). However, we found that using standard transient transfection techniques with multiple fusion proteins created cells with such varying expression levels of the different fusion proteins that it was difficult to separate the colors due to large intensity differences, even with hyperspectral detection. Therefore, we used a polycistronic expression system under the control of one promoter that enabled more even expression of a pair of fusion constructs that were separated by 2A self-cleaving peptide sites (Fig. S10, Table S6 of Supplement 1) [34]. We then generated cell lines using lentiviral transduction of one or two of these paired fusion constructs. We were not able to efficiently two-photon excite mKO2 (fused to TOMM20), one of the fusion constructs expressed in a cell line, so the cell lines provided two or three colors total. We then transiently transfected other plasmids that drove expression of additional single-fusion constructs or paired-fusion constructs in these cells. Using this approach, we were able to readily find fields of view containing single cells with up to seven colors, with sufficient signal to noise for each color to be unmixed from the 48-channel data (Table S7, Supplement 1).

To evaluate the ability of the HMM to differentiate highly spatially and spectrally overlapped fluorescent labels, we imaged live cells stably expressing mTFP1-CEBPA/mNeon-Keratin (labeling nucleus and keratin fibers, respectively) and transiently transfected to express sfGFP-Caveolin (cell membrane invaginations). We clearly separated mTFP1, sfGFP, and mNeon labels in the final image, although they exhibit emission peaks separated by only 18 and 7 nm, respectively [Fig. 3(b); Fig. S14 of Supplement 1). Caveolin appeared as small dots and keratin as fine meshwork, as expected. To establish the capability of the HMM to separate more colors in live cells, we imaged a FOV containing cells stably expressing mTFP1-CEBPA237/mNeon-Keratin and sfGFP-Caveolin/mKO2-TOMM20, additionally transfected with mTurquoise-VASP (focal adhesions), mAmetrine-LaminB1 (nuclear membrane), tdTomato-Calreticulin (endoplasmic reticulum), and dKatushka-CENPB (nucleus). Each of the seven colors was well separated despite significant spatial and spectral overlap [Fig. 3(c); Fig. S15 of Supplement 1]. We found the lateral spatial resolution of the unmixed images to be ${\sim}{0.5}\;{\unicode{x00B5}{\rm m}}$ by measuring the full width at half-maximum of small keratin extensions in these cells [Fig. 3(c), inset]. We then imaged another group of cells expressing the same combination of labels [Fig. 3(d); Fig. S16 of Supplement 1], although tdTomato-Calreticulin was not visible in this FOV. We found two cells, each expressing a different combination of four labels [Fig. 3(d), insets i and ii]. We saw that mAmetrine-LaminB1 outlined the nucleus, which was labeled throughout by dKatushka-CENPB. sfGFP-Caveolin was distributed throughout the cell and was contained in small extensions near the cell edge. mTurquoise-VASP appeared to be expressed throughout the cell membrane, creating a “haze” in the final projected image. In another group of cells [Fig. 3(e); Fig. S17 of Supplement 1], we found a cell in which we could visualize seven different labels simultaneously. We digitally removed the mTurquoise-VASP [Fig. 3(e), inset iii] and the tdTomato-Calreticulin colors [Fig. 3(e), inset iii’] to provide visual clarity for each label. We saw that mTFP1-CEBPA tended to label most of the nuclear body, while dKatushka-CENPB primarily labeled punctate structures within the nucleus. We also observed that mNeon-Keratin tended to form dense rings around the nucleus and a fine meshwork throughout the rest of the cell body.

D. In vivo Mouse Cortex Labeled with Exogenous Dyes and Fluorescent Proteins

We demonstrated the ability of the HMM to differentiate multiple spectral labels in the living cerebral cortex of mice. We used both fluorescent protein expression as well as exogenous dyes to label multiple cell types and tissue structures simultaneously. We crossed mice expressing YFP in pyramidal neurons and GFP in microglia. We applied sulforhodamine 101 (SR101), which has been shown to label astrocytes and oligodendrocytes, both by pressure injection directly into the cortex and by IV injection [39]. We labeled the elastin along arterioles and the blood plasma through IV injection of Alexa Fluor 633 hydrazide [31] and Cascade Blue-conjugated dextran, respectively [Fig. 4(a)]. Animals were imaged either immediately after performing a craniotomy or after ${\sim}{3}$ weeks of recovery.

We were able to visualize all tagged structures at and beneath the cortical surface [Fig. 4(b); Fig. S18 of Supplement 1]. Several expected anatomical features were observed, including the presence of astrocytes and their end feet along the outside of surface arterioles, along with an ${\sim}$1-µm-wide band of elastin adjacent to the vessel lumen. The imaged structures were dense, and digital removal of the SR101 and YFP colors enabled visualization of microglial processes along an arteriole [Fig. 4(b), right]. Due to the large dynamic range of the hyperspectral image, setting the contrast across each unmixed image for clear visualization of all structures simultaneously did not convey the amount of detail present. Taking the logarithm of the intensity values helped to elucidate these structures (Fig. S18, Supplement 1). We compared an image of microglia from a standard 4-channel microscope and our 48-channel hyperspectral microscope for both linear and logarithmic intensity values and found that details were comparable (Fig. S19, Supplement 1). We also demonstrated spectral unmixing at depths of up to 200 µm in the mouse cortex (Figs. 4(c) and 4(d), and Fig. S20 and S21 of Supplement 1), although the Cascade Blue signal dropped off quickly with depth due to power limitations of the 800-nm laser source. We further separated a number of intrinsic and exogenous signals in the dura-mater, including SHG from collagen (seen from both 900- and 1030-nm excitation, yielding a hyperspectral SHG end member), GFP-expressing patrolling monocytes, likely present due to postsurgical inflammation, and several unidentified autofluorescent species (Fig. S22, Supplement 1). One concern is the potential for variation in the spectral signature of fluorescent labels with imaging depth, due to broadening of the observed emission spectrum due to scattering-induced changes in the incidence angles on the ATBFs. To evaluate the severity of these effects, we took images of IV FITC-dextran at different depths in the cortex of a live, wild-type mouse at finely spaced emission wavelength settings to determine a depth-dependent emission profile of this dye. We found minimal changes in spectral shape at depths of up to 0.5 mm (Fig. S28, Supplement 1). As the number of spectral channels greatly exceeded the number of fluorescent labels, we also demonstrated unmixing with a reduced number of channels (Fig. S23, Supplement 1). We selected five channels corresponding to the brightest signal from one of the labeled structures for unmixing, and also demonstrated unmixing with 12 channels, including the other channels that could be acquired simultaneously alongside the five selected channels. We found that unmixed signal quality remained similar to the 48-channel unmixed images, but reduced imaging speed to approximately 11 s from the 35 s required for a 48-channel frame. SNR was measured for the same areas across the 48-, 12-, and 5-channel images. Cascade Blue and Alexa 633 SNRs remained similar despite changes in channel number, but SNR was reduced by approximately 68%, 25%, and 42% for the 12-channel GFP, YFP, and SR101 unmixed images, respectively, and 60%, 28%, and 42% for the 5-channel GFP, YFP, and SR101 unmixed images, respectively. In most cases, these changes were not visually apparent, and all SNRs remained above 40.

Additionally, we used the HMM to distinguish the hues of color expressed by neurons in a Brainbow-like sample [40]. We simultaneously injected multiple adeno-associated viral (AAV) particles that each coded for the expression of a different color fluorescent protein. Different neurons took up different amounts of each AAV and thus expressed different ratios of each color, giving cells a unique hue (Fig. S24, Supplement 1). We used synthetically derived end members from literature spectra and two-photon cross sections for this unmixing. Although the unmixing was broadly successful, residuals were higher than for other samples with image-derived end members, as expected (Figs. S25–S26, Supplement 1).

E. In vivo Imaging of Immune Cell Invasion after Induction of Dermal Inflammation

We further demonstrated the potential applications of the HMM by visualizing the immune response to immunization with a protein antigen (OVA) and CFA in the skin. We used a combination of strategies to label multiple tissue structures and cell types, including transgenic mice, adoptive cell transfer, and injection of fluorescently labeled antibodies and small molecules, resulting in a mouse with 10 different cell types or tissue structures fluorescently labeled [Fig. 5(a)]. HMM image stacks were acquired at different locations near the inoculation site. We were not able to find a location where all 10 labeled cells or tissue structures were visible within a single FOV, but we did identify an imaging field where six of the ten labels were visible, along with two endogenous signals, autofluorescence and SHG [Fig. 5(b); Fig. S27 of Supplement 1). Macrophages and monocytes (csf1r-GFP mouse) were found at a high density near the epidermis [Fig. 5(c)]. Elastin (Alexa 633 hydrazide) was also clearly visible towards the epidermis. We identified blood vessels (BV 421 anti-CD31), which were dimly visible (likely because much of the emission of BV 421 is at a shorter wavelength than the bluest channel on the HMM). Moving deeper into the skin, we spotted dendritic cells (PE-CF594 anti-CD11c) and a large lymphatic vessel (Alexa 647 anti-LYVE-1). Despite significant overlap of the emission spectra of their labels [Fig. 5(d)], we were able to clearly distinguish lymphatic vessels and elastin. At a depth of ${\sim}{100}\;{\unicode{x00B5}{\rm m}}$, we were able to identify a single brightly labeled Th2 cell (adoptive transfer of OVA-sensitized T-cells expressing tdTomato). The emission spectra for the Th2 cell and dendritic cell labels also had a high degree of overlap [Figs. 5(a) and 5(d)], but the HMM was able to clearly separate them [Fig. 5(c)]. In addition to exogenous labels, we were able to identify and extract spectra for endogenous signals such as SHG from collagen (spectral end member with peaks at 450 and 515 nm with 900- and 1030-nm excitation, respectively) and autofluorescence from hair shafts (visible only with 800-nm excitation, but spectrally broad).

4. DISCUSSION

Our HMM produced high-contrast images of multiple fluorescent labels with submicrometer spatial resolution and the capability to image hundreds of micrometers into scattering samples. Utilizing multiple, different wavelength excitation sources enabled the use of a wide variety of fluorescent labels (28 different fluorescent dyes, proteins, and beads used in this paper, not counting autofluorescent species and SHG), and multiplexed detection provided fine spectral resolution for differentiation between labels with similar emission characteristics. We demonstrated unmixing of 10 colors of fluorescent beads with highly overlapped emission spectra. We also imaged seven fluorescent protein fusion constructs in live cells, opening the door to studies of complex intracellular interactions involving cell membrane, organelle, and nuclear proteins. We further imaged five labeled structures simultaneously in a live mouse cortex, distinguished different neurons based on spectral hue in a Brainbow-like sample, and imaged multiple classes of immune and inflammatory cells at a site of skin immunization. These in vivo demonstrations establish the capability of the HMM to image the interactions of multiple cell types in animal models for normal and disease state physiology studies, using the full breadth of fluorescent labeling strategies available to the research community.

Our HMM design compares favorably with a number of other approaches. Many multiplexed confocal fluorescence methods have been developed to address fluorescent label overlap [41–44], although they have not been translated to in vivo studies, largely due to the poor performance of confocal microscopy in the presence of optical scattering. Some multiphoton designs have utilized the wavelength tuning capabilities of modern femtosecond laser systems to separate different fluorescent labels by their excitation characteristics [25–28]. However, laser tuning requires long delays during image acquisition, on the order of seconds per frame, depending on the source. In addition, the small steps in excitation wavelength (on the order of 2–10 nm) do not generate sufficient variations in emission intensity to provide much of an advantage over a few, discrete excitation wavelengths. Our use of 800-, 900-, and 1030-nm excitation provides large differences in spectral emission signatures and enables excitation of labels that emit from blue to red. Recently available femtosecond lasers that can tune 100 nm in 25 ms, coupled with, for example, electro-optic tuning of beam divergence to maintain a common focal plane, could enable similar imaging to be accomplished with a single excitation source (at least for excitation wavelengths within the tuning range of the laser).

Other techniques utilize multiplexed detection, such as spectrometers that are prism or grating-based [20,45], including most spectral detectors on commercial instruments. These are poorly suited for imaging in scattering samples, as these systems must utilize a descanned and confocal detection path to maintain spectral resolution and thus cannot make use of highly scattered emitted fluorescence for image formation. Lansford et al. utilized a liquid crystal tunable filter (LCTF) in place of a standard glass interference filter to increase spectral resolution [46]. However, LCTFs exhibit low passband transmission, poor out-of-band blocking, and a single unit costs as much as a high-end microscope objective, with multiple LCTFs required for multichannel detection. Ricard et al. utilized five different wavelength detection channels (using standard filters) over two different excitation wavelengths and demonstrated unmixing of six labels in a mouse glioblastoma model [27]. However, our system provides a higher level of multiplexing with ATBF detection, enabling the differentiation of closely overlapped labels.

Currently, some commercially available instruments provide semicomparable capabilities to the HMM. These instruments have the ability to vary both the center wavelength and the bandwidth of each of four detection channels. Such flexible tuning of detection channels can be achieved through the use of combinations of angle-tuned short- and long-pass filters, or with translated, continuously variable dichroics [47]. With four tunable detection channels, such an instrument is well suited for rapidly changing detection bands for a particular experiment’s dye configuration. With appropriate excitation lasers and scripting, end users may be able to replicate the data acquired in this paper, although the speed at which the channels can be spectrally tuned remains unclear (our setup aimed to minimize the time for angle adjustments of the ATBFs). We expect that other multiplex-specific systems will continue to be developed to take full advantage of the wide range of fluorescent labels currently available.

Features other than emission spectra may be utilized for the differentiation of similar fluorescent species. In fluorescence lifetime imaging microscopy (FLIM), image contrast is generated by differences in exponential fluorescence lifetime decay rates, independent of spectral characteristics [48]. This imaging modality has found particular use in the identification of autofluorescent species and their varying redox states, which are often spectrally similar but exhibit large differences in fluorescence lifetimes [49–51]; this technique may also be applied to standard fluorescent labels [52–54]. The fluorescence lifetime of a fluorophore can depend significantly on local environmental conditions, such as pH, molecular conformation, and temperature [55–57], and thus adds additional information and complexity to the unmixing data set. Interestingly, studies in confocal microscopy and macroscopic fluorescence imaging have demonstrated utility in combining spectral detection with FLIM [42,58], and future iterations of the HMM would likely benefit from fewer spectral channels and the addition of lifetime measurements for particularly overlapped labels or the simultaneous identification of varying autofluorescent species.

Although the HMM enabled visualization of multiple fluorescent labels in vivo, there were some significant limitations that future instrument iterations could address. The frame-by-frame acquisition approach limited imaging speed to ${\sim}{35}\;{\rm s}$ for a 48-channel image, which is too slow to catch fast cell dynamics such as calcium spiking, blood flow, or rapid cell migration, but fast enough for time-lapse imaging of cell dynamics over the scale of minutes or longer. As with any linear unmixing problem, theoretically only ${\rm N}$ channels must be acquired to identify ${\rm N}$ fluorophores. Therefore, for many of the samples in this paper, analysis of the full 48-channel data set was not required to separate the fluorophores present. Several examples of subsampled data sets were analyzed and show favorable unmixing results compared to the larger data set (Fig. S23, Supplement 1). For future in vivo experiments with this instrument, multiplexed samples could first be imaged across all 48 channels to identify the channels with high SNR and unique features for the dyes in that particular sample. Any channels with little or no signal could be left out of further images, significantly reducing sample acquisition time and data set size, and the remaining channel order optimized to minimize scan time using a basic algorithm (i.e., as many channels as possible acquired in a parallel manner). In addition, the spectral width of channels was not optimized here and depended on what ATBFs were commercially available. In hyperspectral imaging, SNR is generally improved by reducing the channel number, using spectrally broader channels, and rejecting channels with low SNR in the unmixing (provided a sufficient number of channels for unmixing is retained) [59]. A book chapter by Zimmermann provides a detailed discussion of the influence of factors such as channel number, channel bandwidth and separation, reference spectra, and SNR on unmixing precision in fluorescence microscopy [59]. To further increase speed, new instrument iterations could also include two-color two-photon excitation using synchronized pulses to excite fluorophores not otherwise usable [60–64], or temporal multiplexing of excitation wavelengths alongside time-gated signal detection to multiplex across excitation wavelengths in a pixel-by-pixel manner, dramatically increasing acquisition speed to detect faster cell dynamics and avoiding concerns about sample motion between different excitation lasers.

In addition to scan speed limitations, we were concerned that imaging the same area with three excitation lasers risked sample damage. However, we imaged live cells typically sensitive to high levels of radiation (dendrites and microglia) with no signs of cell toxicity or damage. We also noticed that shorter laser wavelengths failed to penetrate deep into tissue, leading to the inability to excite certain dyes throughout the entire imaging tissue volume [for example, Cascade Blue dextran and the 800-nm source, Figs. 4(c) and 4(d)].

As demonstrated in this paper, in addition to instrumentation capable of spectral detection, effective multiplexing in vivo requires the development of robust labeling methods and access to the imaging site. Researchers have long realized the need for complex, in vivo studies in rodent disease models and have developed a wide range of mouse models that enable the study of Alzheimer’s disease, stroke, and cancer, among many others. Within these models, surgical techniques have been developed to provide optical access for imaging in various organ systems, including the brain [65], spinal cord [66], heart [67–69], gut [70,71], kidney [72], lung [73], etc. In addition, new fluorescent labeling strategies provide unprecedented specificity in identifying structures and cell types. Confetti-based multicolor methods for cell identification within a population have been demonstrated in kidney [74] and gut [75]; a similar method termed multiaddressable genome-integrative color (MAGIC) markers has been demonstrated in embryonic neural progenitors in mouse and chick [76]; Brainbow labeling has been shown in a variety of organs [40,77–79]; and cobbled-together multicolor labeling strategies utilizing fluorescent protein expression as well as exogenous labels have been developed for multicellular interaction studies in cancer [27] and immunology [80–84]. Efforts to generate mouse lines with spectrally distinct labels for a wide range of cells (immune cells, vasculature, neurons, cancer cells, etc.) would enable a multitude of in vivo studies of multicellular interactions when combined with the surgical approaches that provide long-term optical access to different organs and imaging instruments that enable clean separation of different fluorescent labels even in scattering tissue, such as the HMM described here.