Quantum ground-state cooling of two librational modes of a nanorotor

MainThe quantum harmonic oscillator is among the most fundamental systems of physics and has been experimentally realized on a variety of mechanical platforms such as cantilevers1 and bulk acousto-optic resonators2. Optically levitated nanoparticles provide a realization particularly close to the ideal harmonic oscillator. When trapped by Gaussian laser beams in high vacuum, they can achieve exceptionally high mechanical quality factors3. Their motion has been cooled to the quantum ground state in one4,5,6,7 and two linear degrees of freedom8,9 and recently also around one rotation axis10.For more than a decade, angular degrees of freedom have attracted increasing interest11,12,13 because rotational dynamics introduces distinctive features to optomechanics. When driven to gigahertz rotation rates14,15, nanorotors can probe material stress limits and enable ultrasensitive torque measurements16, paving the way for tests of vacuum friction17,18, nanoscale magnetism and the search for non-Newtonian forces near surfaces19. In addition, a rapidly spinning nanorod has been used as a nanomechanical clock hand, acting as a local pressure sensor with micrometre spatial resolution20.As rotational motion follows nonlinear dynamics in a periodic, compact phase space, it enables distinct mesoscopic quantum phenomena21, such as tunnelling in persistent quantum tennis racket flips22, coherent coupling between spins and mechanical angular momentum23,24,25,26, or rotational matter–wave interference, where the rotational wave functions of an initially aligned nanorotor can divide, expand and rephase without the need for beamsplitters or mirrors27,28. This opens alternative pathways for the preparation of massive Schrödinger cat states, which would be sensitive to models of wave-function collapse29 or dark matter30,31,32.Although rotational quantum revivals have already been studied in molecular systems33,34, observing similar quantum effects with more massive objects requires trapping, cooling and initializing them with alignment uncertainties close to the quantum zero-point fluctuation. This can be achieved by confining rotational motion in a two-dimensional (2D) harmonic potential, giving rise to librational oscillations through the interaction between an optical tweezer and the anisotropic polarizability of the particle. It has been proposed that such librational motion can be cooled by the coherent scattering of light into a high-finesse cavity35,36. Recent experiments have demonstrated the cooling of a SiO2 particle to millikelvin temperatures in one37 as well as of up to three38,39,40,41 librational degrees of freedom, culminating in the coherent scattering cooling to a high-purity quantum ground state for a single librational mode10.Here we demonstrate the cooling of two librational degrees of freedom individually and show that both modes can be cooled simultaneously, such that the nanorotor’s alignment is defined close to its zero-point uncertainty, a necessary condition for future experiments on rotational quantum interference and quantum-enhanced torque sensing.Extending ground-state cooling from one to two librational modes requires implementing several key experimental advances. First, it is necessary to avoid hybridization between the two librational oscillation modes. We achieve this by coupling them to two orthogonal modes of a high-finesse optical cavity40,42, which also enables the unambiguous identification of individual mechanical modes. This mechanism is specific to librational motion and does not exist for translational degrees of freedom, which necessarily couple to the same optical mode8. Precise control of the cavity birefringence allows the cavity modes to be aligned along predefined laboratory axes as well as tuning the birefringence-induced frequency splitting between them.Second, although a high-finesse cavity enables efficient cooling, it also enhances the impact of laser phase noise, leading to excessive optical heating of mechanical motion. To overcome this limitation, we extend feedback-based phase-noise reduction schemes10,43 to a multifrequency implementation, which is crucial for achieving 2D ground-state cooling.Finally, we implement an improved laser-induced nanoparticle loading mechanism. It is similar to laser-induced acoustic desorption, which has been used in physical chemistry44 and optomechanics before45,46,47. However, by reducing the thickness of the desorption layer from tens of micrometres to tens of nanometres, we lower the required laser pulse energy by about two orders of magnitude. This makes the method cleaner and more suitable for vacuum environments. As a result, the total experimental cycle time—from nanorotor launch and characterization in prevacuum to ground-state cooling in high vacuum—is reduced to less than an hour. This capability enables the ground-state cooling of several different nanorotors, including dimers, trimers and clusters of silica nanospheres, on the same day.Experimental setupOur experimental platform is shown in Fig. 1a and discussed in more detail in the Methods and Extended Data Fig. 1. The nanorotors are assembled from two or more silica spheres with a nominal mean diameter of d = 119 ± 4 nm (specified by microparticles GmbH) for cooling a single librational mode (one dimensional (1D)) and 156 ± 5 nm for cooling two librational modes (2D). Single spheres, dumbbells, trimers or clusters are launched at low pressure (~6 mbar) using laser-induced desorption (see the ‘Robust and repeatable trap loading and cooling’ section) and trapped in the optical tweezer light (wavelength λ = 1,550 nm) that propagates along the z axis, linearly polarized along the x axis.Fig. 1: Nanorotor trapped in an optical tweezer.Full size imagea, A silica nanorotor is trapped in an optical tweezer formed by light propagating along the z axis and polarized along the x axis, and focused using a high-numerical-aperture lens. The y-polarized backward-scattered light is collected and monitored on heterodyne detector 1. The coherently scattered tweezer light of the nanorotor populates an optical cavity, which is formed by two mirrors and oriented along the x direction. The orthogonally polarized cavity modes are split using a PBS and monitored in heterodyne detectors 2 and 3. Inset: librational modes α, β and γ in the defined reference frame. b, PSD Shet of heterodyne detection of the backward-scattered light (1, green) and cavity modes polarized along the y (2, orange) and z (3, violet) axes show all the degrees of freedom. The spectra are taken at a tweezer–cavity detuning of Δ/2π ≈ 800 kHz, normalized to the shot-noise level Sshot noise and shown here with respect to the heterodyne frequency ωhet. Librations α and β are visible in different cavity modes, which are frequency separated by ~8.2 kHz. Note that the birefringence splitting aligns the cavity modes such that the higher-frequency cavity mode couples to the higher-frequency librational α mode and the lower-frequency cavity mode to the lower-frequency β mode.We describe each nanorotor as an asymmetric rigid body with distinct moments of inertia and susceptibilities χa 0), the interaction Hamiltonian in equation (2) predicts cooling via coherent scattering49,50,51. At resonance, the interaction rate between the particle and cavity light field increases. If the detuning to the tweezer matches the level spacing of the mechanical harmonic oscillator, the probability of anti-Stokes scattering is maximized. Each anti-Stokes scattering event reduces the oscillator quantum number by one, n → n − 1 and increasing the photon energy from ℏωl to ℏ(ωl + Ωμ). Starting from a thermal state, the net effect of these individual transitions is a decrease in the mean phonon occupation 〈n〉, that is, cooling of the mechanical oscillator, in close analogy to early experiments with optical cooling in other nanomechanical systems52. By contrast, Stokes scattering raises the oscillator quantum number and reduces the photon energy, thereby heating the nanorotor motion. However, such photons are off-resonant with respect to the cavity and, therefore, suppressed. On average, the resulting imbalance transfers mechanical energy to the optical field, raising the energy of the photons that leave the cavity49,53.The minimum phonon occupation nμ is reached when the cooling rate balances the heating rate. The net cooling rate is determined by the difference between the Stokes \({A}_{\mu }^{+}\) and anti-Stokes \({A}_{\mu }^{-}\) scattering rates. Heating arises from collisions with background gas and photon recoil, which together contribute Γμ, as well as Stokes scattering. We reduce the influence of gas collisions by pumping the chamber to about 3 × 10−8 mbar. The equilibrium phonon occupation of the librational modes μ ∈ {α, β}, including a laser phase-noise contribution nϕ(Ωμ) (ref. 54), is then given by$${n}_{\mu }=\frac{{\varGamma }_{\mu }+{A}_{\mu }^{+}}{{A}_{\mu }^{-}-{A}_{\mu }^{+}}+{n}_{\phi }({\varOmega }_{\mu }).$$ (3) The phase-noise contribution is approximately nϕ(Ωμ) ≈ Sϕ(Ωμ)ncav/κ and determined by the spectral density of the laser phase noise Sϕ(Ωμ) and the intracavity photon number ncav (refs. 54,55). To reduce the effect of phase noise, we actively stabilize the laser phase using feedback derived from an unbalanced Mach–Zehnder interferometer10,43 (Extended Data Fig. 1b). In this way, we are able to reduce the noise background by more than two orders of magnitude at the eigenfrequencies of both α and β oscillations (Methods and Extended Data Fig. 2b,c).1D ground-state coolingFor the SiO2 cluster shown in Fig. 1b, we record the cooling of α libration as a function of detuning Δ around Ωα. In Fig. 2a, we plot the PSD of this motion at positive and negative frequencies to perform sideband thermometry with the Stokes (red) and anti-Stokes (blue) sidebands. We fit the peaks to a Lorentzian profile to extract their frequencies, linewidths and background noise levels.Fig. 2: 1D ground-state cooling.Full size imagea, PSD of the Stokes (red) and anti-Stokes (blue) scattering process for Δ1/2π = 986 kHz and Δ2/2π = 1,042 kHz. The imbalance of the peak heights is used to extract the occupation n. b, Stokes (red) and anti-Stokes (blue) scattering rates for different occupations n show their proportionality to n + 1 and n. c–e, Fitted mechanical frequency (c), linewidth (d) and obtained occupation (black points; e) are shown as a function of the tweezer–cavity detuning Δ. Error bars in b–e denote the 1σ uncertainties from individual fits and are smaller than the marker size for most data points. Green regions denote the 1σ uncertainty of the global fit, which accounts for uncertainties in linewidth, coupling, offset frequency, heating rate and phase-noise occupation. The grey-shaded region marks a ground-state population with a probability greater than 50%.The Stokes and anti-Stokes scattering rates, which are proportional to the respective peak areas, scale with the occupation number n as A+ ∝ n + 1 and A− ∝ n. The occupation number n can, therefore, be extracted from the sideband amplitudes (Methods).Already at a cavity detuning of Δ1/2π = 986 kHz, the imbalance between the red and blue sidebands is substantial, signalling a phonon occupation of n = 0.94 ± 0.04. Shifting the detuning to Δ2/2π = 1,042 kHz leads to an occupation of α libration as low as n = 0.21 ± 0.03, which corresponds to a probability of populating the harmonic oscillator’s ground state of 83 ± 2 %.Figure 2b shows the normalized peak areas as a function of the occupation number. As expected for a quantum harmonic oscillator, the probabilities for Stokes and anti-Stokes scattering scale with the occupation as n + 1 and n, respectively. The stability and repeatability of the experiment are evidenced by plotting the difference (grey) of the normalized sidebands for different cavity detunings and, hence, occupation numbers. The difference remains consistently close to unity.Scanning the cavity detuning around mechanical resonance changes the optomechanical coupling and, therefore, both eigenfrequency (Fig. 2c) and linewidth (Fig. 2d) of librational motion. Forces due to the intracavity light field act like a frequency-tunable optical spring and damper52. At the same time, the detuning strongly affects the cooling and final occupation number (Fig. 2e).Fitting the curves shown in Fig. 2c,d allows us to extract the optomechanical coupling gα (Methods). Since the geometry of the nanorotor determines the coupling strength, we can calculate the particle-specific moment of inertia about the z axis as Ib = 3.3 ± 0.4 × 10−32 kg m2. In the coldest state (nα = 0.21 ± 0.03), we determine the standard deviation of the librational amplitude as σα = 17.4 ± 0.9 μrad, corresponding to an effective temperature of Tα = 28 ± 2 μK (Methods).Ground-state cooling of two librational modesTo tightly align the particle with the polarization axis, we extend coherent scattering cooling now to both modes, α and β. A dumbbell formed from two silica spheres with d = 156 nm is trapped and oscillates at Ωα/2π = 1,035 kHz and Ωβ/2π = 978 kHz. As the two frequencies differ only by about twice the cavity decay rate κ, the two modes can be cooled simultaneously if the tweezer–cavity detuning is properly chosen and the phase-noise reduction is activated at both frequencies (Extended Data Fig. 2c).We measure the occupation of both librational modes as a function of detuning Δ via sideband thermometry (Extended Data Fig. 3a,b). Because the librational modes α and β couple to orthogonal cavity modes, we can treat their dynamics separately. By setting the detuning close to the librational frequencies Ωα or Ωβ, we can cool the α or β motion individually into their quantum ground states, namely, nα = 0.65 ± 0.19 and nβ = 0.54 ± 0.32, respectively (Fig. 3a). At Δ2/2π = 984 kHz, we achieve the lowest combined phonon number with nα = 1.02 ± 0.08 and nβ = 0.73 ± 0.22. This corresponds to effective librational temperatures of Tα = 73 ± 4 μK and Tβ = 57 ± 28 μK. Again, we fit the occupations as a function of detuning (Fig. 3b). The evaluation of optomechanical coupling from the oscillator linewidth (Methods and Extended Data Fig. 3c) reveals that the nanorotor’s alignment along the x axis is defined with an uncertainty of σα = 18 ± 1 μrad and σβ = 17 ± 3 μrad, which is close to the quantum zero-point fluctuations of about 13 μrad.Fig. 3: Ground-state cooling of α and β librations.Full size imagea, PSDs of detector 1 signal (green points) show cooling to the ground state of α (top) and β (bottom), and close to the simultaneous ground state of α and β (middle), for three different detunings of the cavity’s y mode (Δ1, Δ2, Δ3) = 2π × (1,043, 984, 968) kHz. The cavity transfer function is indicated by the purple gradient at the bottom of each plot, with the central frequency and its 1σ uncertainty shown as a line marker. We extract the mode occupations from the imbalance between the fitted Stokes (red) and anti-Stokes scattering (blue). b, Extracted phonon occupations of β libration (grey triangles) and α libration (black circles) as a function of tweezer–cavity detuning Δ. Theoretical predictions for occupations of the α and β modes are shown as light and dark green regions, respectively. The green-shaded regions and the error bars denote 1σ uncertainties. The grey-shaded region marks a ground-state population with a probability greater than 50%.Such cooling near the quantum limit is an important prerequisite for future experiments on rotational interference and quantum sensing21. The aligned state corresponds to a coherent superposition of angular momentum states with a mean of \(j\simeq \sqrt{{k}_{{\rm{B}}}TI}/\hslash \approx 6\times 1{0}^{4}\). If we were to release the rotor non-adiabatically from its orientational ground state, it would evolve into a superposition of rotational quantum states with classically mutually exclusive angular momenta27. This is expected to lead to rotational dispersion and quantum revivals due to the constructive interference of the rotational wave packets after a time Trev = 2πI/ℏ. For the nano-dumbbell in our experiment, the revival time is 50 min. Therefore, observing revivals at a realistically observable timescale requires smaller particles or a scheme to resolve fractional revivals28.Robust and repeatable trap loading and coolingAdvanced experiments in levitated optomechanics require sources that can load and cool nanoparticles with a high repetition rate and reliability, but are simultaneously able to handle different particle types and geometries. We demonstrate here the repeatable loading and ground-state cooling of half a dozen different nanorotors, formed from silica nanospheres with d = 119 nm. We limit this study to the ground-state cooling of α libration to save measurement time. The particles are coated on a glass slide covered by a 50-nm-thick silicon film and placed above the cavity (Fig. 4a). A green laser pulse with a duration of about 6 ns and an energy of 100 μJ focused down to a waist diameter of about 100 μm hits the backside of this sample and ejects the particles into dry nitrogen at a base pressure of 6 mbar (Extended Data Fig. 4 shows the experimental details). This can release single spheres, dumbbells, trimers or bigger clusters. The process is similar to laser-induced acoustic desorption (LIAD)46, but because the absorption layer is thin enough to be fully evaporated, we refer to the method as laser-induced desorption (LID). As corroborated by scanning electron microscopy, the particles already aggregate in solution, but we also observe indications of occasional dimer growth in the trap through sequential capture of two spheres.Fig. 4: Repeatable ground-state cooling.Full size imagea, Experimental sketch of loading nanorotors in vacuum with laser-induced desorption. A pulsed nanosecond laser is focused onto a sample coated with SiO2 nanoparticles; scanning electron microscopy of such samples prepared with 119-nm particles reveals dumbbells and trimers (inset) alongside single spheres and agglomerated clusters. The ejected nanoparticles are trapped in the optical tweezer. With a split detection scheme in forward scattering (4, blue), the particle geometry is analysed by extracting the mechanical damping rates. b, Measured damping rates γx and γy for cluster (i), three dumbbells (ii), (iii) and (v), and two linear trimers (iv) and (vi), as well as simulated values for different geometries. c, Pressure trace during loading and ground-state cooling (highlighted by green solid lines). Particles that were ejected are highlighted as grey dotted lines. d, Measured occupation numbers of α libration. The error bars denote 1σ uncertainties. The grey-shaded region marks a ground-state population with a probability greater than 50%.To characterize the geometry of the trapped rotor, we track its motion along the x and y axes by monitoring the transmitted tweezer light in a split detection scheme (Methods). Although the oscillator damping in the residual gas is the same along all axes for isotropic nanoparticles, the ratio of the damping rates γy/γx was simulated to be 1.258 for dumbbells and 1.378 for linear trimers14 (Fig. 4b).After shape assessment at prevacuum, we activate the cavity and evacuate the system to high vacuum. As the particle shape approaches cylindrical symmetry, the system becomes increasingly susceptible to heating and mechanical instabilities due to resonances between γ libration and the translational z mode40. To stabilize γ, we introduce a slight ellipticity to the tweezer polarization during pump down. Once high vacuum is reached, we return to linear x polarization; at this point, we maintain control even over axially symmetric dumbbells, although their γ motion then remains largely free.We have repeated the procedure of trapping, shape assessment, evacuation to high vacuum and cavity cooling for a series of nanoparticles over a period of ~28 h. Six nanorotors, marked (i)–(vi) in Fig. 4b–d, were successfully cooled (near) to their librational quantum ground state. They comprise dumbbells, trimers and clusters. To illustrate the scale of this experiment, we plot the trap pressure as a function of time (Fig. 4c). Every observation of librational ground-state cooling of a fresh particle is marked by a green line, whereas the grey dashed lines mark events of intentional or accidental particle loss (Extended Data Fig. 5 shows details about particle loss). The fastest cycle from the ejection of one particle to the ground-state cooling of a new one took 58 min, primarily limited by the duration of evacuation. The final occupation numbers for all six successful events are shown in Fig. 4d.ConclusionUsing our bimodal high-finesse cavity, we have demonstrated repeated ground-state cooling of the α and β libration modes of differently shaped nanorotors via coherent scattering. We find occupation numbers down to nα ≃ 0.21 for a nanocluster composed of spheres with d = 119 nm and nα ≃ 1.02 and nβ ≃ 0.73 when optimizing the cavity detuning for the simultaneous cooling of both librational degrees of freedom of a dumbbell with d = 156 nm. This process aligns the nano-dumbbell with an angular uncertainty close to its 2D quantum zero-point fluctuations.Combined with our capability for fast loading and cooling of dumbbells, trimers and larger clusters, this is a stepping stone towards previously inaccessible tests of quantum mechanics and quantum-enhanced rotational torque sensing. However, there is a general trade-off between cooling efficiency and quantum readiness: larger particles are easier to manipulate and cool due to their higher polarizability and cavity coupling, whereas lighter particles exhibit faster wave-function expansion, facilitating both linear and rotational interferometry.The nano-dumbbell used in our 2D cooling experiment, with a mass around 4 × 109 atomic mass units (u), would exhibit a rotational quantum-state revival time of Trev = 2πI/ℏ ≃ 1 h, which is prohibitively long even when considering fractional revivals56. Rotational matter–wave interferometry will, therefore, require particles with smaller moments of inertia Irot27. For a dumbbell made of two 20-nm silica spheres, with a total mass of 1 × 107 u, the revival time is 150 ms, corresponding to roughly 40 cm of free fall. This timescale is compatible with a laboratory-scale experiment, provided that particles with comparable moments of inertia can be prepared, or the same particle can be reused. Such an experiment is an important goal, as it could boost the macroscopicity value57 by orders of magnitude beyond the current state of the art58.The mass scale of 1 × 107 u is also intriguing because it encompasses relevant nanobiological materials. The tobacco mosaic virus stands out as a natural nanorotor with a length of 300 nm, a diameter of 18 nm and a mass of 4 × 107 u. Such thermolabile materials will require soft loading and cooling methods, most probably in the dark59. Cooled to the ground state, a trapped tobacco mosaic virus would feature a resonant torque sensitivity of about 3 × 10−29 N m Hz−1/2 (ref. 60).MethodsOptical setupThe optical setup is shown in Extended Data Fig. 1. Light emitted by an infrared fibre laser (NKT Photonics Koheras Adjustik E15) passes through the fibre electro-optic modulator EOM 2. We split off a small fraction of the light to lock the cavity (Extended Data Fig. 1a). The rest is amplified to a power of 6 W (NKT Photonics Boostik HP) and then divided into three parts: one for phase-noise detection, one serving as the LO in heterodyne detection (Extended Data Fig. 1c) and up to 3 W for the optical tweezer.The tweezer mode is cleaned by a polarization-maintaining fibre, and its polarization is set by wave plates to be linear along the cavity axis. This orientation minimizes Rayleigh scattering into the cavity when the nanorotors are perfectly aligned. The laser light fills the aspherical tweezer lens, which has a diameter of 25.4 mm, a numerical aperture of numerical aperture of 0.81 and an effective focal length of 13.2 mm (Thorlabs, custom design). For a cluster assembled using 119-nm nanospheres (Fig. 1b), we determine a trap power of P = 2.7 W and trapping waists of wx = 1.17 μm and wy = 0.98 μm.We detect the trapped nanoparticle by collecting its backscattered light (Extended Data Fig. 1c). Its two polarization components are split by the PBS and detected separately. The vertical component provides the most information about the particle’s rotation, particularly about the rotation around the tweezer propagation axis z. This signal is only weakly sensitive to Rayleigh scattering of the aligned rotor and scattering at surfaces along the beam path. Therefore, this component is used to monitor cooling to the librational ground state. The horizontal contribution is isolated using a fibre circulator, which provides intrinsic alignment of the backscattering signal and is, therefore, used during trap alignment. To reduce the Rayleigh scattering peak, we filter the electrical signal using a crystal oscillator.The trapped nanoparticle is centred at an antinode of the cooling cavity mode. The resonator is formed using mirrors with intensity reflectivity of R ≥ 0.99999 (FiveNine Optics) and radius of curvature of 5 cm, yielding a finesse of \({\mathcal{F}}\approx300,000\) at a free spectral range of 9.72 GHz, corresponding to a linewidth of κ/2π = 32.4 kHz and a central waist of wcav = 94 μm. By careful design of the cavity mirror mount (Extended Data Fig. 4), we achieve an alignment of the cavity modes both along and orthogonal to the direction of tweezer propagation. The birefringence splitting between the two modes can be tuned in the range of 2π × (0−30 kHz) by applying pressure onto the mirrors via screws.We lock the laser to the cavity using the Pound–Drever–Hall scheme (Extended Data Fig. 1a). EOM 1 (iXblue, PHT MPZ-LN-10-00-P-P-FA-FA) generates the locking sidebands and is used together with acousto-optic modulator AOM 1 (G&H, T-M200-0.1C2J-3F2P) to shift the locking frequency by one free spectral range of the cavity. This minimizes interference between the locking and the cooling light in detection.To detect the particle motion in all directions, we use a heterodyne scheme, which mixes the scattered light with an LO. This enhances the signal interferometrically and shifts the signal to a spectral range of lower noise. The LO is blueshifted by 4.99814 MHz with respect to the tweezer beam using two polarization-maintaining fibre modulators (G&H, T-M200-0.1C2J-3F2P): AOM 2 at 197.5 MHz and AOM 3 at −202.49 MHz (Extended Data Fig. 1c).The scattered light transmitted by the cavity mirror is divided into its horizontal and vertical polarization components. They are individually combined with the LO beam using a 50:50 fibre beamsplitter (Thorlabs PN1550R5A2). Each polarization output is then detected by a balanced photodiode (Thorlabs PDB425C-AC). In both backplane detections, we use variable-ratio fibre beamsplitters (KS Photonics) to balance the outputs, which are also detected by balanced photodiodes (Thorlabs PDB440C-AC) (Extended Data Fig. 1c).After the optical trap, we collimate the tweezer light using a low-numerical-aperture aspheric lens (Thorlabs C560TME-C) and isolate the particle signal using a split detection scheme (Extended Data Fig. 1d). We use a D-shaped mirror to split the optical beam into two halves that are detected by balanced photodiodes (Thorlabs PDB440C-AC). This detection is built for both x and y axes.Phase-noise reductionIn the presence of the cavity, laser phase noise can heat the mechanical motion55. The cavity delays the release of scattered light, effectively creating an unbalanced interferometer in heterodyne detection between scattered light and the LO. The laser phase noise appears in cavity transmission as an increased noise background around the cavity mode resonance. In Fig. 1b, this is shown at a frequency of around ~800 kHz and fitted with a Lorentzian to extract the exact frequency and to determine the birefringence splitting. We also use the fitted frequency to determine the actual tweezer–cavity detuning and its error during the detuning scan (Fig. 2e).Strong cooling of the librational modes without the active suppression of phase noise leads to noise squashing61 (Extended Data Fig. 2c, top), which distorts the motional sideband and generates a dip in the phase-noise background. This prevents accurate sideband thermometry. We, therefore, implement a phase-noise reduction scheme, using an unbalanced Mach–Zehnder interferometer43 (Extended Data Fig. 1b). The short arm contains a polarization-maintaining fibre attenuator to equalize the optical power in both arms. The long arm consists of a 100-m single-mode fibre (SMF-28), enclosed in a chamber at prevacuum. This arm also includes a fibre stretcher to stabilize slow path-length fluctuations (>10 ms), and it combines a manual fibre polarization controller and a fibre PBS to correct for polarization changes. Light from both arms is recombined using a 50:50 fibre coupler and directed to a balanced detector (Thorlabs PDB450C-AC). After filtering, the interferometer output is fed back into EOM 2, which controls the phase of the tweezer light.With active feedback, the noise level is reduced by more than 30 dB both at a single frequency (Extended Data Fig. 2b) and two frequencies (Extended Data Fig. 2c, bottom). The reduction is also visible in cavity transmission, restoring the expected shape of the motional sidebands (Extended Data Fig. 2, middle).Mode identificationTo assign the peaks shown in Fig. 1b to translational and librational modes, we first use the fact that the translational frequencies for nanoparticles much smaller than the optical wavelength hardly depend on the particle shape. We, therefore, use individual spherical nanoparticles to identify the frequencies associated with the z, x and y modes, where the x and y frequencies change depending on the tweezer polarization, whereas the z frequency stays invariant. When switching to anisotropic nanoparticles, three additional frequency peaks appear. Due to the prolate geometry of our nanorotors (mostly dimers and linear trimers), we have one peak at smaller frequencies (γ) and two peaks at larger frequencies. As described in the ‘Experimental setup’ section, we use the polarization-sensitive detection of the cavity transmission to discriminate between α and β.Theoretical descriptionThe nanorotor is an asymmetric rigid body (Ic Ωβ across all nanorotors trapped in our setup, which is compatible with γ ≃ π/2 and motivates this choice in our modelling.We define the librational mode variables bα = αzpf(α + ipα/IαΩα) and bβ = βzpf(β − π/2 + pβ/IβΩβ), with zero-point fluctuation amplitudes \({\alpha }_{{\rm{zpf}}}=\sqrt{\hslash /2{I}_{b}{\varOmega }_{\alpha }}\) and \({\beta }_{{\rm{zpf}}}=\sqrt{\hslash /2{I}_{a}{\varOmega }_{\beta }}\), to obtain the quantized interaction Hamiltonian of equation (1), where we introduced the coupling constants$$\begin{array}{l}{g}_{\alpha }={\alpha }_{{\rm{zpf}}}{k}_{\alpha },\\ {g}_{\beta }={\beta }_{{\rm{zpf}}}{k}_{\beta }.\end{array}$$ (9) In summary, this leads to the total libration cavity Hamiltonian in equation (2). A standard calculation then yields the optomechanical damping rates and the resulting steady-state occupation in equation (3)42.Optomechanical couplingThe optomechanical coupling determines the interaction between the particle and cavity mode and, therefore, the cooling performance. By solving the equations of motion, with cooling providing additional damping, we obtain an effective motional linewidth of$${\gamma }_{\mu }^{\mathrm{eff}}(\omega )={\gamma }_{\mu }+\frac{4| {g}_{\mu }{| }^{2}{\varOmega }_{\mu }{\Delta }_{{\rm{c}}}\kappa }{\left[{\left(\frac{\kappa }{2}\right)}^{2}+{(\omega +{\Delta }_{{\rm{c}}})}^{2}\right]\left[{\left(\frac{\kappa }{2}\right)}^{2}+{(\omega -{\Delta }_{{\rm{c}}})}^{2}\right]},$$ (10) which depends on the coupling strength. In the regime of strong cooling, when the cavity resonance is close to the mechanical frequency, energy loss through the cavity determines the damping and the cavity-induced linewidth dominates over the thermal linewidth γμ. We use this expression to fit the linewidths extracted from cavity-detuning scans for 1D (Fig. 2d) and 2D (Extended Data Fig. 3c) cooling with a constant coupling. We verify the extracted coupling by additionally fitting the observed optical spring effect (Fig. 2c):$${\varOmega }_{\mu }^{\mathrm{eff}}(\omega )=\sqrt{{\varOmega }_{\mu }^{2}-\frac{4\,| {g}_{\mu }{| }^{2}\,{\varOmega }_{\mu }\,{\Delta }_{{\rm{c}}}\left[{\left(\frac{\kappa }{2}\right)}^{2}-{\omega }^{2}+{\Delta }_{{\rm{c}}}^{2}\right]}{\left[{\left(\frac{\kappa }{2}\right)}^{2}+{(\omega +{\Delta }_{{\rm{c}}})}^{2}\right]\left[{\left(\frac{\kappa }{2}\right)}^{2}+{(\omega -{\Delta }_{{\rm{c}}})}^{2}\right]}}.$$ (11) Since the optomechanical coupling is determined by the rotor geometry, we can determine the moment of inertia for each mode. Combining equations (5) and (8) with the zero-point fluctuation, we calculate as follows:$${I}_{b}=\frac{| {g}_{\alpha }{| }^{2}}{{\varOmega }_{\alpha }^{3}}\frac{| {E}_{{\rm{tw}}}(0){| }^{2}}{| {E}_{c}(0){| }^{2}8\hslash },\,{I}_{a}=\frac{| {g}_{\beta }{| }^{2}}{{\varOmega }_{\beta }^{3}}\frac{| {E}_{{\rm{tw}}}(0){| }^{2}}{| {E}_{c}(0){| }^{2}8\hslash }.$$ (12) Noise contributionsFor quantum-limited measurements, the signal must be isolated from noise. The noise contributions in backscattering detection are shown in Extended Data Fig. 2a. The raw spectrum contains dark noise (photodetector and oscilloscope), shot noise and phase noise of the LO. The latter originates from the frequency generators that drive LO AOMs 2 and 3 (Extended Data Fig. 1c). In postprocessing, we, therefore, subtract the background levels as extracted from the Lorentzian fits. Additionally, the detector sensitivity shows a weak frequency dependence, which differs for the Stokes and anti-Stokes peaks. The sensitivity is calibrated by acquiring the spectra of dark noise and LO’s shot noise. Since shot noise is white, any residual frequency dependence must be due to the detector response. We, therefore, divide the background-corrected signals by the difference between shot noise and dark noise.Occupation numberThe areas of the Stokes (AS) and anti-Stokes (AaS) peaks scale with the mean occupation number n of the harmonic oscillator as AS = C(n + 1) and AaS = Cn, respectively, where C is a proportionality constant. The occupation number can, therefore, be extracted from the ratio of the Stokes and anti-Stokes peak areas62. In practice, the precision of area measurements is limited by the available integration time. When recording a detuning scan within a fixed total acquisition time, increasing the number of detuning points necessarily reduces the integration time per point, which would, in turn, degrade the precision of the occupation number estimates. Since the difference in the sideband areas satisfies AS − AaS = C, independent of the occupation number n, we determine C by averaging the differences AS − AaS over all spectra in a given scan. Figure 2b and Extended Data Fig. 3a,b show the resulting normalized peak areas AS/C and AaS/C, whose difference is supposed to be unity by construction. The occupation number at each detuning is then obtained from n = (AS + AaS − C)/2C. With this procedure, the statistical uncertainty of each extracted n is comparable with the uncertainty obtained by spending the entire integration time on a single detuning point. In other words, pooling the area differences across the full scan allows us to estimate n with high precision and still resolve its detuning dependence.Knowing n, we estimate the mode temperature T by assuming the Bose–Einstein distribution for a quantum harmonic oscillator in thermal equilibrium:$$T=\frac{\hslash {\varOmega }_{\mu }}{{k}_{{\rm{B}}}}{\left(\mathrm{ln}\left[1+\frac{1}{n}\right]\right)}^{-1}.$$ (13) From the same thermal distribution, we also extract the ground-state population probability as$${p}_{0}=1-\exp \left(-\frac{\hslash {\varOmega }_{\mu }}{{k}_{{\rm{B}}}T}\right)=\frac{1}{1+n}.$$ (14) Heating ratesIn the absence of external heating, cooling is governed by the cavity-enhanced imbalance between anti-Stokes and Stokes scattering. For both processes, we define the weak-coupling damping and heating rate as$${A}_{\mu }^{\pm }=\frac{| {g}_{\mu }{| }^{2}\kappa }{{(\kappa /2)}^{2}+{(\Delta \pm {\varOmega }_{\mu })}^{2}},$$ (15) which yields, together with equation (3), a minimum occupation number of \({n}_{\min }={\kappa }^{2}/16{\varOmega }_{\mu }^{2}\). It depends only on the cavity linewidth and mechanical frequency. For librational frequencies of ~2π × 1 MHz, this implies a theoretical lower bound of nα ≈ nβ ≈ 6.2 × 10−5, far below our measured values. The system must, therefore, be limited by other sources, such as recoil heating, gas collisions or phase noise.The recoil limit depends on both cavity and tweezer parameters. For our linearly polarized tweezer, we estimate Γrecoil = 3.2 kHz (ref. 42), which limits cooling to nrecoil = 0.064. Phase noise and collisional contributions, however, vary with the particle geometry, as this determines the librational frequency and collisional cross-section. The phase-noise occupation can be obtained using equation (3).We analyse heating and decoherence for the ground-state-cooled nanocluster (Fig. 2); the frequency dependence of the occupation reveals that the phase-noise contribution of nϕ(Ωα) = 0+0.01 is negligible. Additionally, the fit displays a total heating rate of Γα = 6.8 ± 0.7 kHz, originating from both recoil and thermal noise. Since the former is pressure independent, the thermal part follows by subtraction from the total heating rate \({\varGamma }_{\alpha }^{{\rm{thermal}}}=3.6\pm 0.8\,{\rm{kHz}}\). For this cluster particle, recoil and thermal heating contribute approximately equally.The same noise analysis can be performed for the trapped nano-dumbbell, where we treat both librational modes separately (Fig. 3). For β libration, the fit finds the phase noise to dominate with an occupation of nϕ(Ωβ) = 0.38 ± 0.17, whereas the α mode is again only barely affected by it, with nϕ(Ωα) = 0+0.07. From the fitted total heating rates in both dimensions, namely, Γβ = 20 ± 4 kHz and Γα = 18 ± 2 kHz, we estimate the thermal heating rates as \({\varGamma }_{\beta }^{{\rm{thermal}}}=16\pm 4\,{\rm{kHz}}\) and \({\varGamma }_{\alpha }^{{\rm{thermal}}}=14\pm 2\,{\rm{kHz}}\), respectively. We conclude that collisional heating dominates the α mode, whereas β libration is also limited by phase noise.