<?xml version='1.0' encoding='utf-8'?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:media="http://search.yahoo.com/mrss/" version="2.0"><channel><title>Wityworks</title><link>https://wityworks.com/blog/</link><description>Research notes, paper reviews, and essays on AI, math, and life.</description><language>en</language><lastBuildDate>Fri, 21 Aug 2026 22:28:03 +0900</lastBuildDate><item><title>Waves on a Sphere, Spherical Harmonics</title><link>https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics</link><guid isPermaLink="true">https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics</guid><pubDate>Fri, 21 Aug 2026 00:00:00 +0900</pubDate><category>Math &amp; Stats Notes</category><description>Spherical harmonics are the natural language for directional variation on a sphere. Rooted in Laplace's study of gravitational potentials and central to atomic orbitals, they are like sine and cosine Fourier modes on a line or circle, representing view-dependent color in 3D Gaussian Splatting.</description><content:encoded><![CDATA[<figure><img src="https://wityworks.com/static/img/blog/spherical-harmonics-cover.png" alt="Waves on a Sphere, Spherical Harmonics"></figure><p>Spherical harmonics are the natural language for directional variation on a sphere. Rooted in Laplace&#x27;s study of gravitational potentials and central to atomic orbitals, they are like sine and cosine Fourier modes on a line or circle, representing view-dependent color in 3D Gaussian Splatting.</p>
<h2 id="table-of-contents">Table of contents</h2>
<ul><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#1-prologue-why-spherical-harmonics-today">1. Prologue: Why spherical harmonics today?</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#2-historical-background-of-spherical-harmonics">2. Historical background of spherical harmonics</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#3-mathematical-definition-of-spherical-harmonics">3. Mathematical definition of spherical harmonics</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#31-laplace-equation-in-spherical-coordinates">3.1 Laplace equation in spherical coordinates</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#32-separation-of-variables-amp-associated-legendre-polynomials">3.2 Separation of variables &amp; associated Legendre polynomials</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#33-key-mathematical-properties">3.3 Key mathematical properties</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#4-modern-applications-3dgs-ref-nerf-and-ide">4. Modern applications: 3DGS, Ref-NeRF, and IDE</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#41-view-dependent-color-and-radiance-in-3d-gaussian-splatting-3dgs">4.1 View-dependent color and radiance in 3D Gaussian Splatting (3DGS)</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#42-ref-nerf-and-integrated-directional-encoding-ide">4.2 Ref-NeRF and Integrated Directional Encoding (IDE)</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#5-quantum-mechanical-correlation-bohr-model-amp-wave-mechanics">5. Quantum mechanical correlation: Bohr model &amp; wave mechanics</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#51-why-spherical-harmonics-appear-in-the-hydrogen-atom">5.1 Why spherical harmonics appear in the hydrogen atom</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#52-schrdinger-equation-orbital-structure-and-spherical-harmonics">5.2 Schrödinger equation, orbital structure, and spherical harmonics</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#6-epilogue-amp-conclusion">6. Epilogue &amp; conclusion</a></li><li><a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#references">References</a></li></ul>
<hr>
<h2 id="1-prologue-why-spherical-harmonics-today">1. Prologue: Why spherical harmonics today?</h2>
<p>Spherical Harmonics (SH) represent the two-dimensional angular equivalent of the classic 1D Fourier series. While the standard Fourier series decomposes signals on periodic 1D lines or 2D Cartesian grids, Spherical Harmonics act as fundamental vibration modes on the continuous surface of a sphere ($S^2$).</p>
<p>In recent years, Spherical Harmonics have experienced a resurgence in neural 3D, Neural Radiance Fields (NeRF) <a id="ref-mildenhall2020-1" href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-mildenhall2020"></a>[Mildenhall et al., 2020] and 3D Gaussian Splatting (3DGS) <a id="ref-kerbl2023-1" href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-kerbl2023"></a>[Kerbl et al., 2023]. Because real-world objects reflect light non-isotropically based on view direction, SH functions provide a lightweight, closed-form polynomial basis to store and evaluate view-dependent color and high-frequency specular highlights in real time.</p>
<hr>
<h2 id="2-historical-background-of-spherical-harmonics">2. Historical background of spherical harmonics</h2>
<figure>
  <img src="https://wityworks.com/static/img/blog/spherical-harmonics-pioneers.png" alt="Line art illustration of Laplace, Legendre, Thomson, and Tait">
  <figcaption>Historical figures of Spherical Harmonics: Laplace, Legendre, Kelvin, and Tait.</figcaption>
</figure>
<p>Spherical Harmonics were first introduced in the late 18th century by Pierre-Simon Laplace (1782) and Adrien-Marie Legendre (1785) to analyze gravitational potentials and celestial mechanics without relying on computationally heavy Cartesian series expansions. Throughout the 19th century, the framework expanded to describe physical phenomena like heat conduction, wave propagation, and electrostatics in spherical geometries—giving rise to a fundamental class of mathematical formulations known as <strong>boundary value problems</strong>.</p>
<p>In differential equations and classical physics, boundary value problems refer to differential equations coupled with a set of constraints—known as boundary conditions—that the solution must satisfy at the physical borders of a specified domain. Unlike initial value problems, which evolve a system forward in time from a starting state, a boundary value problem seeks a static or stationary state governed by fixed constraints on the perimeter. In the context of Spherical Harmonics, the underlying domain is typically a continuous spherical surface ($S^2$) or a 3D spherical volume. The boundary conditions mandate that physical properties—such as temperature, electric field, or wave amplitude—remain well-behaved, continuous, and single-valued across the boundary.</p>
<p>To solve these problems in spherical geometry, mathematicians apply the technique of <strong>separation of variables</strong>, decomposing complex 3D boundary constraints into simpler radial and angular differential equations. The spherical harmonics ($Y_l^m$) emerge naturally as the fundamental angular spatial modes that satisfy these spherical boundary conditions. Because any well-behaved continuous function defined on a spherical boundary can be represented as a linear combination of these harmonic modes, spherical harmonics provide an exact analytical framework for constructing full solutions to spatial field equations under spherical constraints.</p>
<p>The term itself was officially introduced by Lord Kelvin (William Thomson) and Peter Guthrie Tait in their landmark 1867 work, <em>Treatise on Natural Philosophy</em> <a id="ref-kelvin1867-2" href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-kelvin1867"></a>[Thomson &amp; Tait, 1867]. They coined the name <strong>&quot;spherical harmonics&quot;</strong> to highlight the underlying mathematical property of these functions: they are the restriction of harmonic functions (solutions to Laplace&#x27;s equation) onto a spherical surface, behaving as the 2D spherical analogs of the trigonometric harmonic oscillations found in 1D Fourier analysis.</p>
<hr>
<h2 id="3-mathematical-definition-of-spherical-harmonics">3. Mathematical definition of spherical harmonics</h2>
<h3 id="31-laplace-equation-in-spherical-coordinates">3.1 Laplace equation in spherical coordinates</h3>
<p>In vector calculus and mathematical physics, <strong>Laplace&#x27;s equation</strong> ($\nabla^2 f = 0$) governs source-free, static field configurations—such as gravitational potentials in empty space, electrostatic fields outside charges, and steady-state heat distributions. Solutions to Laplace&#x27;s equation are known as harmonic functions, which possess strong smoothness and the mean-value property—stating that the value of the function at any point equals the average of its values over any sphere centered at that point.</p>
<p>To solve Laplace&#x27;s equation in spherically symmetric domains, we transform the standard Cartesian Laplacian operator $\nabla^2 = \frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2} + \frac{\partial^2}{\partial z^2}$ into spherical coordinates $(r, \theta, \phi)$ using the coordinate transformation $x = r \sin\theta \cos\phi$, $y = r \sin\theta \sin\phi$, and $z = r \cos\theta$. Applying the multivariable chain rule yields the spherical Laplacian:</p>
<p>$$\nabla^2 f = \frac{1}{r^2} \frac{\partial}{\partial r}\left(r^2 \frac{\partial f}{\partial r}\right) + \frac{1}{r^2 \sin\theta} \frac{\partial}{\partial \theta}\left(\sin\theta \frac{\partial f}{\partial \theta}\right) + \frac{1}{r^2 \sin^2\theta} \frac{\partial^2 f}{\partial \phi^2} = 0 \tag{1}$$</p>
<p>Equation (1) naturally decomposes the spatial field into a radial variation scale ($r$) and an angular surface geometry $(\theta, \phi)$ defined on the unit sphere $S^2$.</p>
<figure>
  <img src="https://wityworks.com/static/img/blog/spherical-harmonics-spherical-coordinates.png" alt="Spherical coordinate system showing the radial distance r, polar angle theta, and azimuthal angle phi on a unit sphere">
  <figcaption>Spherical coordinates separate radial distance $r$ from the angular coordinates $(\theta, \phi)$ on the unit sphere.</figcaption>
</figure>
<h3 id="32-separation-of-variables-associated-legendre-polynomials">3.2 Separation of variables &amp; associated Legendre polynomials</h3>
<p>To isolate the angular behavior, we apply the technique of <strong>separation of variables</strong>, assuming a factorized solution $f(r, \theta, \phi) = R(r) Y(\theta, \phi)$. Substituting this decomposition into Equation (1) and multiplying by $r^2 / (R Y)$ separates the equation into independent radial and angular Ordinary Differential Equations (ODEs) bound by a separation constant $l(l+1)$:</p>
<p>$$\frac{1}{R} \frac{d}{dr}\left(r^2 \frac{dR}{dr}\right) = -\frac{1}{Y} \left[ \frac{1}{\sin\theta} \frac{\partial}{\partial \theta}\left(\sin\theta \frac{\partial Y}{\partial \theta}\right) + \frac{1}{\sin^2\theta} \frac{\partial^2 Y}{\partial \phi^2} \right] = l(l+1) \tag{2}$$</p>
<p>Further factorizing the angular component $Y(\theta, \phi) = \Theta(\theta) \Phi(\phi)$ with a second separation constant $m^2$ splits the spherical surface into longitudinal ($\Phi$) and latitudinal ($\Theta$) components:</p>
<ul><li><strong>Azimuthal equation ($\Phi$)</strong>: $\frac{d^2\Phi}{d\phi^2} + m^2 \Phi = 0 \implies \Phi(\phi) = e^{im\phi}$</li><li><strong>Associated Legendre equation ($\Theta$)</strong>: Enforcing boundary periodicity and smoothness restricts degree $l \ge 0$ and order $-l \le m \le l$ to integers.</li></ul>
<p>Combining the normalized solutions yields the standard complex Spherical Harmonics $Y_l^m(\theta, \phi)$:</p>
<p>$$Y_l^m(\theta, \phi) = N_l^m P_l^m(\cos\theta) e^{im\phi} \tag{3}$$</p>
<ul><li><strong>Normalizing constant ($N_l^m$)</strong>:</li></ul>
<p>$$N_l^m = \sqrt{\frac{(2l+1)}{4\pi} \frac{(l-m)!}{(l+m)!}} \tag{4}$$</p>
<ul><li><strong>Associated Legendre polynomials ($P_l^m$)</strong>:</li></ul>
<p>Defined for $m \ge 0$ via Rodrigues&#x27; formula using $x = \cos\theta$, incorporating the standard Condon–Shortley phase $(-1)^m$:</p>
<p>$$P_l^m(x) = (-1)^m (1-x^2)^{m/2} \frac{d^m}{dx^m} P_l(x) \quad (m \ge 0) \tag{5}$$</p>
<p>$$P_l(x) = \frac{1}{2^l l!} \frac{d^l}{dx^l} (x^2 - 1)^l \tag{6}$$</p>
<p>For negative order $-l \le m &lt; 0$, $P_l^m(x)$ is defined via $P_l^{-m}(x) = (-1)^m \frac{(l-m)!}{(l+m)!} P_l^m(x)$.</p>
<h3 id="33-key-mathematical-properties">3.3 Key mathematical properties</h3>
<ul><li><strong>Orthonormal basis</strong>: Spherical Harmonics form a complete orthonormal basis over the unit sphere $S^2$, where superscript $*$ denotes the complex conjugate required for complex-valued functions:</li></ul>
<p>$$\int_{0}^{2\pi} \int_{0}^{\pi} Y\_l^m(\theta, \phi) \left( Y\_{l&#x27;}^{m&#x27;}(\theta, \phi) \right)^* \sin\theta ~ d\theta ~ d\phi = \delta\_{ll&#x27;} \delta\_{mm&#x27;} \tag{7}$$</p>
<ul><li><strong>Rotation equivariance</strong>: Under a 3D rotation $\mathbf{R}$, the original SH coefficients $\mathbf{c}\_{l, m}$ transform linearly within degree $l$ into new coefficients $\mathbf{c}&#x27;\_{l, m&#x27;}$ using Wigner D-matrices $\mathbf{D}\_{m&#x27;, m}^{(l)}(\mathbf{R})$:</li></ul>
<p>$$\mathbf{c}&#x27;\_{l, m&#x27;} = \sum\_{m=-l}^{l} \mathbf{D}^{(l)}\_{m&#x27;, m}(\mathbf{R}) ~ \mathbf{c}\_{l, m} \tag{8}$$</p>
<ul><li><strong>Laplacian eigenvalue &amp; frequency</strong>: Spherical Harmonics are eigenfunctions of the Laplace–Beltrami operator $\Delta\_{S^2}$ on the unit sphere:</li></ul>
<p>$$\Delta\_{S^2} Y\_l^m = -l(l+1) Y\_l^m \tag{9}$$</p>
<p>Degree $l$ relates directly to spatial angular frequency, with an approximate wavelength $\approx \frac{180^\circ}{l}$.</p>
<figure>
  <img src="https://wityworks.com/static/img/blog/spherical-harmonics-low-order-modes.png" alt="Comparison of low-order spherical harmonic modes, progressing from a uniform sphere to increasingly structured angular patterns">
  <figcaption>Visual representations of the first few real spherical harmonics. Light regions represent positive values and dark regions negative values. For each angular direction $(\theta, \phi)$, the surface&#x27;s radial distance from the origin encodes the magnitude $|Y_l^m(\theta, \phi)|$. Increasing degree $l$ produces increasingly fine angular structure.</figcaption>
</figure>
<hr>
<h2 id="4-modern-applications-3dgs-ref-nerf-and-ide">4. Modern applications: 3DGS, Ref-NeRF, and IDE</h2>
<h3 id="41-view-dependent-color-and-radiance-in-3d-gaussian-splatting-3dgs">4.1 View-dependent color and radiance in 3D Gaussian Splatting (3DGS)</h3>
<p>In 3D Gaussian Splatting (3DGS) [<a id="ref-kerbl2023-41" href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-kerbl2023"></a>Kerbl et al., 2023], each 3D Gaussian primitive stores SH coefficients to model view-dependent RGB radiance:</p>
<p>$$\mathbf{c}(\mathbf{v}) = \sum_{l=0}^{l_{\max}} \sum_{m=-l}^{l} \mathbf{c}_{l, m} Y_l^m(\mathbf{v}) \tag{9}$$</p>
<p>For standard real-time rendering, degree $l_{\max} = 3$ is used, corresponding to $(3+1)^2 = 16$ coefficients per color channel (48 parameters per primitive).</p>
<h3 id="42-ref-nerf-and-integrated-directional-encoding-ide">4.2 Ref-NeRF and Integrated Directional Encoding (IDE)</h3>
<p>To account for specular reflections and surface roughness without aliasing artifacts, Ref-NeRF models a distribution of reflected directions with a von Mises–Fisher (vMF) distribution [<a id="ref-verbin2022-42" href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-verbin2022"></a>Verbin et al., 2022]. Here, $\mathbf{\mu}_r \in S^2$ is the unit reflection direction and the mean direction of the distribution, $\mathbf{x} \in S^2$ is a unit direction sampled on the sphere, and $\kappa \ge 0$ is the concentration parameter: lower $\kappa$ represents a broader lobe and thus a rougher surface. The factor $C_3(\kappa)$ normalizes the density over the 3D unit sphere; the subscript $3$ is the ambient dimension because $S^2$ consists of unit vectors in $\mathbb{R}^3$, as specified by the <a href="https://en.wikipedia.org/wiki/Von_Mises%E2%80%93Fisher_distribution#Definition"></a>general vMF definition.</p>
<p>$$f(\mathbf{x}; \mathbf{\mu}_r, \kappa) = C_3(\kappa) \exp(\kappa \mathbf{\mu}_r^T \mathbf{x}) \tag{10}$$</p>
<p>Taking the expectation of SH basis functions over the vMF distribution yields Integrated Directional Encoding (IDE), which closed-form attenuates higher frequencies:</p>
<p>$$\mathbb{E}_{\boldsymbol{\omega} \sim \textnormal{vMF}(\mathbf{\mu}_r, \kappa)} \left[ Y_l^m(\boldsymbol{\omega}) \right] = A_l(\kappa) Y_l^m(\mathbf{\mu}_r) \tag{11}$$</p>
<p>$$A_l(\kappa) \approx \exp\left( -\frac{l(l+1)}{2\kappa} \right) \tag{12}$$</p>
<p>Here, $\boldsymbol{\omega} \in S^2$ is the random unit direction drawn from the vMF distribution, and $A_l(\kappa)$ is the roughness-dependent attenuation factor for SH degree $l$.</p>
<p><strong>Why does Equation (11) hold? (Proof sketch.)</strong> First rotate the coordinates so that the mean direction $\mathbf{\mu}_r$ becomes the north pole. Write a direction on the sphere using its polar angle $\theta \in [0, \pi]$ and azimuthal angle $\phi \in [0, 2\pi)$. In these coordinates, the vMF density in Equation (10) depends on $\theta$ but not on $\phi$. A spherical harmonic of degree $l$ and order $m$ has azimuthal dependence proportional to $e^{i m \phi}$. Consequently, its integral over $\phi$ is zero whenever $m \ne 0$, because $\int_0^{2\pi} e^{i m \phi}\,d\phi = 0$. This is consistent with the fact that $Y_l^m$ evaluated at the north pole is zero for $m \ne 0$.</p>
<p>For an arbitrary mean direction $\mathbf{\mu}\_r$, rotate the north-pole result back to $\mathbf{\mu}\_r$. Since the density depends only on the angle between $\boldsymbol{\omega}$ and $\mathbf{\mu}\_r$, this averaging operation commutes with every 3D rotation. It therefore preserves each SH degree and applies the same scalar factor $A_l(\kappa)$ to every order $m$ within that degree. More generally, for a directional function $g$ that can be expanded in spherical harmonics as $g(\boldsymbol{\omega}) = \sum\_{l=0}^{\infty}\sum\_{m=-l}^{l} c\_{l,m}Y\_l^m(\boldsymbol{\omega})$, where $c\_{l,m}$ are its SH coefficients, vMF averaging gives</p>
<p>$$\mathbb{E}\_{\boldsymbol{\omega} \sim \textnormal{vMF}(\mathbf{\mu}\_r, \kappa)}[g(\boldsymbol{\omega})] = \sum\_{l=0}^{\infty}\sum_{m=-l}^{l} A\_l(\kappa)c\_{l,m}Y\_l^m(\mathbf{\mu}_r). \tag{13}$$</p>
<p>Thus, no coefficient is transferred to a different degree or order: each coefficient is only attenuated according to its degree. This derivation is an application of the Funk–Hecke theorem; Ref-NeRF gives the full derivation, including the Legendre-polynomial integral for $A_l(\kappa)$, in <a href="https://dorverbin.github.io/refnerf/refnerf.pdf"></a>Appendix A, “Integrated Directional Encoding Proofs”.</p>
<figure>
  <img src="https://wityworks.com/static/img/blog/spherical-harmonics-ide-frequency-attenuation.png" alt="Integrated Directional Encoding comparison showing that a broad low-kappa von Mises-Fisher distribution attenuates high-degree spherical harmonic frequencies more strongly">
  <figcaption>IDE applies a roughness-dependent angular low-pass filter: lower $\kappa$ (rougher surfaces) attenuates higher-degree spherical harmonic frequencies more strongly.</figcaption>
</figure>
<hr>
<h2 id="5-quantum-mechanical-correlation-bohr-model-wave-mechanics">5. Quantum mechanical correlation: Bohr model &amp; wave mechanics</h2>
<h3 id="51-why-spherical-harmonics-appear-in-the-hydrogen-atom">5.1 Why spherical harmonics appear in the hydrogen atom</h3>
<p>In a hydrogen atom, a negatively charged electron is attracted to a positively charged nucleus. In the simplest model, the strength of this attraction depends only on the distance from the nucleus, not on the direction. Physicists call this distance-dependent energy the Coulomb potential. Because rotating the atom does not change that energy, the quantum description of the electron separates naturally into a radial part, which describes distance, and an angular part, which describes direction. The angular part must be an eigenfunction of the spherical Laplacian, and those eigenfunctions are precisely the spherical harmonics $Y_l^m$. Thus, the degree $l$ and order $m$ of a spherical harmonic label an electron orbital&#x27;s angular structure and orientation; the next section makes this separation explicit.</p>
<figure>
  <img src="https://wityworks.com/static/img/blog/spherical-harmonics-bohr.png" alt="Illustration of the Bohr atomic model with electrons on discrete circular orbits around a central nucleus">
  <figcaption>Bohr’s historical model represents electrons in discrete circular orbits around a nucleus. The atom is deliberately shown excited—an apt prelude to leaving Bohr’s fixed orbits for the richer angular structure of wave mechanics in the following section. ;-)</figcaption>
</figure>
<h3 id="52-schrdinger-equation-orbital-structure-and-spherical-harmonics">5.2 Schrödinger equation, orbital structure, and spherical harmonics</h3>
<p>The Schrödinger equation is the basic equation of quantum mechanics: it determines a quantum wavefunction from the energy landscape of a physical system. For hydrogen, this energy landscape is written as $V(r)$, where $r$ is the distance from the nucleus; as Section 5.1 explained, it has no preferred direction. This symmetry lets the three-dimensional spatial-curvature operator, called the Laplacian and written as $\nabla^2$, split into a distance-dependent part and a direction-dependent part:</p>
<p>$$\nabla^2 = \frac{1}{r^2}\frac{\partial}{\partial r}\left(r^2\frac{\partial}{\partial r}\right) + \frac{1}{r^2}\Delta\_{S^2}. \tag{14}$$</p>
<p>The first term describes how the wavefunction changes with distance. The second contains the spherical Laplacian $\Delta\_{S^2}$ from Equation (9), which describes how it changes with direction on a sphere. A spherical harmonic $Y_l^m$ is an eigenfunction of this angular operator: applying the operator changes only its scale, not its angular pattern. This lets the three-dimensional Schrödinger equation separate into one radial equation for each angular mode. A wavefunction with a definite energy can consequently be written as</p>
<p>$$\psi(r, \theta, \phi) = R\_{n,l}(r)Y\_l^m(\theta, \phi). \tag{15}$$</p>
<p>Here, $\psi$ is the spatial wavefunction; $R\_{n,l}$ is its radial part; and $Y\_l^m$ is its angular part. The principal quantum number $n$ is a positive integer that labels the radial energy level, while the nonnegative integer $l$ and integer $m$ label the spherical harmonic, with $0 \le l \le n-1$ and $-l \le m \le l$.</p>
<p>This factorization gives spherical harmonics their physical meaning. The degree $l$ controls the angular complexity and the orbital angular momentum: $l=0,1,2,3$ correspond to the familiar $s,p,d,f$ orbital families. The order $m$ distinguishes the $2l+1$ orientation states within one family. In a perfectly spherically symmetric atom, these states have the same energy; a magnetic field can distinguish them, which is why $m$ is called the magnetic quantum number.</p>
<p>More broadly, this is a useful change of viewpoint: spherical harmonics are not merely convenient functions for drawing orbitals. They are the natural coordinates for any phenomenon with rotational symmetry. The same decomposition that separates a quantum wavefunction into angular modes also organizes lighting on a sphere, directional signals, and neural rendering features into angular frequencies.</p>
<h2 id="6-epilogue-conclusion">6. Epilogue &amp; conclusion</h2>
<p>Spherical Harmonics illustrate the deep interdisciplinary connection across mathematical physics, quantum mechanics, and modern computer science. Originally developed to calculate celestial gravity in the 18th century, SH functions became foundational to 20th-century quantum physics by defining electron orbital geometries. Today, they power real-time 3D neural rendering engines like 3D Gaussian Splatting [<a id="ref-kerbl2023-6" href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-kerbl2023"></a>Kerbl et al., 2023] and Ref-NeRF [<a id="ref-verbin2022-6" href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-verbin2022"></a>Verbin et al., 2022], solidifying their role as an essential tool for spatial computing.</p>
<hr>
<p><em>This article is based on lecture notes from the Neural Graphics module of the Fall 2025 course, &quot;Multimodal Generative AI Theories and Applications,&quot; at Seoul National University.</em></p>
<h2 id="references">References</h2>
<ul><li><a id="ref-kerbl2023"></a>Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. <em>ACM Transactions on Graphics (TOG)</em>, 42(4):1–14, 2023. [<a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-kerbl2023-1"></a>1, <a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-kerbl2023-41"></a>4.1, <a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-kerbl2023-6"></a>6]</li><li><a id="ref-mildenhall2020"></a>Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In <em>European Conference on Computer Vision (ECCV)</em>, pages 405–421. Springer, 2020. [<a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-mildenhall2020-1"></a>1]</li><li><a id="ref-verbin2022"></a>Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T. Barron, and Pratul P. Srinivasan. Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields. In <em>IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)</em>, pages 5491–5500, 2022. [<a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-verbin2022-42"></a>4.2, <a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-verbin2022-6"></a>6]</li><li><a id="ref-kelvin1867"></a>William Thomson (Lord Kelvin) and Peter Guthrie Tait. Treatise on Natural Philosophy, Part I. Clarendon Press, Oxford, 1867. [<a href="https://wityworks.com/blog/?post=math-stats-notes/spherical-harmonics#ref-kelvin1867-2"></a>2]</li></ul>]]></content:encoded><media:content url="https://wityworks.com/static/img/blog/spherical-harmonics-cover.png" medium="image" /></item><item><title>Why Attention Is Normalized with Softmax</title><link>https://wityworks.com/blog/?post=ml-concepts/why-attention-is-normalized-with-softmax</link><guid isPermaLink="true">https://wityworks.com/blog/?post=ml-concepts/why-attention-is-normalized-with-softmax</guid><pubDate>Sat, 15 Aug 2026 00:00:00 +0900</pubDate><category>ML Concepts</category><description>Revisiting softmax's role in turning query-key dot products into a probability distribution, alongside the temperature parameter.</description><content:encoded><![CDATA[<p>Attention scores start life as raw dot products between query and key vectors. The goal is to produce a weighted mixture of value vectors — formally,</p>
<p>$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^\top}{\sqrt{d_k}}\right)V$$</p>
<p>where $Q$, $K$, $V$ are matrices whose rows are queries, keys, and values respectively. The softmax output is itself a matrix of the same shape as $QK^\top$, so the product with $V$ is ordinary matrix multiplication. For those weights to define a valid mixture at each position, each row must be non-negative and sum to one: a probability distribution over positions.</p>
<p>The unnormalized score for position $i$ is $s_i = q^\top k_i / \sqrt{d_k}$, where $d_k$ is the key dimension. The scaling is not cosmetic: if $q_j, k_{ij} \sim \mathcal{N}(0,1)$, then $q^\top k_i = \sum_{j=1}^{d_k} q_j k_{ij}$ has variance $d_k$, so its standard deviation grows as $\sqrt{d_k}$. Dividing by $\sqrt{d_k}$ restores unit variance regardless of head size — without it, dot products in high dimensions blow up and push softmax into a near-one-hot regime.</p>
<p>Softmax with temperature $\tau$ (here $\tau = \sqrt{d_k}$ folded in) is:</p>
<p>$$ \mathrm{softmax}(x_i) = \frac{\exp(x_i / \tau)}{\sum_{j} \exp(x_j / \tau)} $$</p>
<p>As $\tau \to 0$ this sharpens toward a one-hot argmax; as $\tau \to \infty$ it flattens toward a uniform average over all positions. The $1/\sqrt{d_k}$ scaling in standard attention is exactly this temperature knob, chosen to keep the pre-softmax logits at a consistent scale regardless of head dimension. When one logit dominates, softmax output approaches 1 there and 0 everywhere else; the Jacobian of softmax becomes near-zero, so gradients through the attention weights effectively vanish during backprop.</p>
<h2 id="why-softmax-specifically">Why softmax specifically</h2>
<p>Alternatives like sparsemax or per-element sigmoid attention exist, but softmax remains the default. A few properties explain why, compared to a plain normalized sum $s_i / \sum_j s_j$:</p>
<ul><li>Differentiable everywhere, unlike a hard argmax.</li><li>Shift-invariant: $\mathrm{softmax}(x + c) = \mathrm{softmax}(x)$ for any constant $c$, which is what makes the usual &quot;subtract the max before exponentiating&quot; trick numerically safe.</li><li>Monotonic in each $x_i$, so larger scores always get larger weight — no sign issues the way a raw normalized sum would have with negative dot products.</li></ul>]]></content:encoded></item></channel></rss>
