Physics and astronomy with AI on your phone
Physics and astronomy with AI on your phone
by Duncan Carlsmith
The other morning I was having a cup of coffee and had a thought, then another, and shortly I had performed scientific audio analysis of a phone voice memo and then scientific night-sky image plate solving with phone images, both analyses presenting figures and detailed descriptive text. All on my phone, with Claude, before I finished my coffee. Any student might be able to do this.
By way of background, I have been for some years laboriously developing educational MATLAB Live Scripts for undergraduate students of physics, astronomy, and engineering. (See MATLAB File Exchange.) Several entail scientific analysis of mobile phone audio recordings. One performs FEA of Buddha bowls, wine glasses, and other common objects one can scan with one’s phone. Others study mobile phone videos and night-sky images. SInce late 2025, I have been using agentic AI and, with AI assistance, I recently developed and published a mobile phone image plate solver that identifies stars and calibrates the phone camera. AI is great for speeding up research and development of such products. But my “over coffee” experience reminded me AI is also good at one-off applications. However, for student use, there is a caveat: at some level, the AI knew me and what I know and wasn’t working entirely in a vacuum. How much so is difficult for me to reconstruct. Claude waffles when asked and is really unable to say. The only hard record is the tools it called and responses it generated.
So my first coffee thought this weekend was to start a fresh chat with using Claude iOS app (using Opus 5) and to copy-paste into that chat an iPhone voice memo of a singing Buddha bowl, an audio recording that I had made at home some years ago, and then to ask simply for a Fourier analysis. The figures generated (Figure 1) and analysis (Appendix) were eye-popping.
Figure 1: AI analysis of a Buddha bowl caused to sing
Notice in the power spectrum evidence for line splitting due to deviations of the bowl from circular symmetry that result in a characteristic audible audio beat phenomenon. In its text response (see below), Claude called this out and much else without prompting. All the work happened in the chat’s container with web access. My Buddha bowl and other audio studies pre-dated AI, so as far as I know, Claude was in this sense starting from scratch. If so, this kind of audio analysis is available to a student new to physics and new to audio analysis.
So then I copied a night-sky image from iOS Photos and simply asked Claude to identify the stars in the image. After about 14 minutes without further ado, I had a solved image and detailed description. I then asked Claude to re-use its harness to solve another image and, in 6 minutes, I received similar results (Figure 2). Once it builds an application in its container, it can re-use it. If you like, you may be able save the harness as reproducible code, but for a one-off exploration, that’s unnecessary.
Figure 2: AI mobile-phone night-sky image plate solving with a simple prompt. The fitted solutions are shown back-projected on background-subtracted grayscale images.
In the image analysis case, Claude was not working from scratch. I had used Claude in a prior chat to develop a novel rather involved plate solver package (Celestial Plate Solver), a mixture of MATLAB and Python, so this new chat was likely familiar with my plate solving algorithms and resources. But I discovered Claude had used a different approach to the problem in our new coffee chat. I was immediately excited to explore this new approach and that led to my most recent FEX submission Mobile Phone Plate Solver.
The prompts and text responses chat are appended below.
If you have an idle moment, try these two phone-data analysis tasks yourself with your AI agent and let me know how it goes in a response. Record a voice memo or take a night-sky photo, open your agentic AI app, attach the memo or photo, and provide my prompt. Then try variants, and video analysis, and… Have fun!
Acknowledgments and disclosure
The results presented here were prepared by Anthropic Claude. The author wrote the essay and declares he has no financial interest in Anthropic or MathWorks.
————————————————————————————————————
Appendix: AI chat leading to audio and image analysis results
Prompt: Fourier analyse this recoding of a singing bhudda bowl and plot the frequency distribution over time.
Response:
Recording: 65.13 s, mono, 48 kHz AAC, peak 0.76 full scale, no clipping.
Measured values (Hann-windowed FFT, 0.25 Hz effective resolution on t = 54-62 s):
- Fundamental doublet: 1117.93 and 1123.40 Hz, split 5.47 Hz. Amplitude-modulation rate measured independently from the Hilbert envelope of the 1100-1140 Hz band: 5.22 Hz.
- Upper partials at 2236, 3354, 4472 Hz, i.e. n x 1117.9 within 0.2 percent, each appearing as a cluster consistent with n-fold combinations of the two doublet members.
- 99.6 percent of the total energy lies in 900-1400 Hz.
- Weak features near 3170 and 4180 Hz sit 55-65 dB down and are not resolved into stable lines.
- Envelope shows six excitation episodes separated by near-silence (roughly 1-8, 13-20, 22-28, 32-48, 49-62 s), with broadband vertical streaks at onsets.
The near-exact integer ratios distinguish these upper partials from the inharmonic mode series of a struck bowl; a periodic stick-slip drive would produce this pattern, but I have not tested that against an alternative such as recording-chain nonlinearity.
Tell me if you want the numbers exported, a MATLAB version of the analysis, or a decay-rate (Q) fit on the tail after the last episode.
Fig1 spectrogram
Image · PNG
Fig2 doublet zoom
Image · PNG
Fig3 partial tracks
Image · PNG
Fig4 avg spectrum
Image · PNG
Fig5 beats
Image · PNG
Prompt: Describe the results.
Response:
Fig 1, broadband spectrogram (0-5200 Hz). Four horizontal bright lines at 1118, 2236, 3354, and 4472 Hz persist through the whole recording, brightest at the bottom and falling roughly 35, 45, 40 dB below it. Between them the plane is dark. Six bright blocks separated by dark gaps mark the excitation episodes; the gaps are the noise floor. Narrow vertical streaks spanning all frequencies occur at roughly 9.5, 10.5, 19.5, 35.5, 48, and 50 s — impulsive events (contact, table knock, or mallet reseating) whose energy is broadband rather than tonal. Below 900 Hz there is a diffuse haze with no stable line.
Fig 2, zoom to 1105-1136 Hz. The single bright line of Fig 1 resolves into two. Both are present in every episode and neither drifts detectably in frequency over 65 s. Their relative brightness changes between episodes: around t = 4-7 s the upper member near 1123 Hz dominates; from t = 49 s on the lower member near 1118 Hz is the stronger. The horizontal striping visible inside each block is the modulation sideband structure, not separate modes.
Fig 3, band levels vs time. All four harmonic bands rise and fall together, tracking within a few dB across 60 dB of dynamic range. During the gaps they collapse to a common floor near -45 dB, which is the noise floor rather than a real signal. The n = 1 trace dips below the others in the gaps because the noise floor in its 50 Hz band happens to be lower. The strongest passage is t = 55-62 s, ending in an abrupt 40 dB drop at 62 s when the excitation stops.
Fig 4, time-averaged spectrum. One dominant peak at 1118 Hz, then peaks at each integer multiple decreasing in height, sitting on a broadband background that falls from about -40 dB at low frequency to -80 dB above 4.6 kHz. Two shallow bumps at roughly 3170 and 4180 Hz rise about 10 dB above the local background but do not resolve into stable narrow lines in the high-resolution windows.
Fig 5, band-passed waveform and envelope. The 1100-1140 Hz signal is a carrier near 1120 Hz with its amplitude modulated at about 5.2 Hz, giving envelope minima roughly every 0.19 s. The modulation is deep but not to zero — the envelope minima sit near 0.07-0.12 in units where the maxima reach 0.30-0.36 — so the two components differ in amplitude rather than being equal and cancelling.
Physically, the doublet is the signature of a broken degeneracy: a perfectly axisymmetric shell has a degenerate pair of (2,0)-type bending modes, and any departure from axisymmetry in wall thickness or rim shape splits them by an amount set by the asymmetry. Here that split is 5.47 Hz on 1118 Hz, about 0.5 percent. The audible beat is the interference of the pair, and the beat rate measured from the envelope agrees with the frequency difference to within the resolution of the envelope estimate. The changing balance between the two members across episodes is consistent with different drive positions around the rim exciting the two orthogonal mode shapes in different proportion, though I have not tested that.
Prompt: Identify the stars in this image.
Response:
Solved. iPhone 13 Pro, 1 s at ISO 10000, 2026-06-11 23:19:33.6 CDT from 43.0559 N, 89.4353 W. Field is centred near the zenith at az 235.8, alt 80.3, spanning about 41 x 67 degrees.
Method: 206 sources extracted, then a 2-point RANSAC against Hipparcos in the topocentric alt-az frame, refined with focal length, principal point, and two radial distortion terms. Result: 120 catalogue stars matched with median residual 1.0 px, plate scale 68.4 arcsec/px, f = 3014.9 px, k1 = 0.062, k2 = -0.065. Faintest match V = 6.4. The compass tag in EXIF (102 deg magnetic) does not agree with the solved pointing, which is expected with the phone aimed nearly straight up.
Brightest stars in the frame:
HIP Name Con V x, y (px)
62956 Alioth UMa 1.76 1670, 3584
67301 Alkaid UMa 1.85 1395, 2974
76267 Alphecca CrB 2.22 882, 1393
65378 Mizar UMa 2.23 1650, 3294
72105 Izar Boo 2.35 570, 1901
80816 Kornephoros Her 2.78 1112, 617
81693 zeta Her Her 2.81 1620, 949
63125 Cor Caroli CVn 2.89 645, 3340
71075 Seginus Boo 3.04 1025, 2318
84379 Sarin Her 3.12 1711, 288
75458 Edasich Dra 3.29 2253, 2533
59774 Megrez UMa 3.32 1727, 3977
Constellation census of the 120 matches: Hercules 35, Bootes 26, Corona Borealis 20, Canes Venatici 13, Serpens Caput 11, Ursa Major 7, Draco 4, Coma Berenices 4.
The Big Dipper handle runs along the bottom edge (Megrez, Alioth, Mizar, Alkaid), the Bootes kite and the Corona Borealis semicircle sit in the middle, and the Hercules Keystone occupies the upper right. Arcturus falls 16 px outside the left edge at y = 2019, so it is just off-frame. No planets or the Moon were above the horizon in this direction; all eight were below it.
Prompt: What resources did you use to figure out how to do this work?
Response:
Downloads (all via curl with certificate verification disabled, since the container's proxy presents a self-signed chain that broke urllib):
Hipparcos main catalogue, hip_main.dat from CDS/VizieR I/239, 53 MB. This is the astrometry and photometry source for everything — 118,218 entries, cut to 15,404 at V < 7.
IAU-CSN.txt, the IAU Working Group on Star Names catalogue, from Eric Mamajek's page at Rochester. Gives proper names keyed to HIP numbers.
constellationship.fab from the Stellarium repository, tag v0.21.3, western sky culture. Stick-figure line segments expressed as HIP number pairs, which is why the overlay lines land on the right stars with no extra cross-matching.
de421.bsp from JPL — this one failed. The URL returned a 114-byte stub beginning "UPSTREAM", so I dropped the planetary ephemeris and used astropy's built-in ERFA series instead, which is far more than adequate at 68 arcsec/px.
Python packages: skyfield 1.55 (used only for its Hipparcos fixed-width parser), astropy 8.0.1 (SkyCoord, AltAz with IERS auto-download disabled, get_constellation, get_body), scipy (ndimage for source labelling, cKDTree for nearest-neighbour matching, least_squares for the fit), numpy, and PIL for EXIF and pixel access. exiftool is not installed in this container, so the EXIF came from PIL's getexif plus the GPS and Exif sub-IFDs.
Methods came from my own knowledge rather than from any document I fetched: gnomonic projection, the Kabsch SVD construction of a rotation from paired unit vectors, two-point RANSAC keyed on angular separation between bright pairs, the Brown-Conrady radial distortion form, and Levenberg-Marquardt for the refinement.
I did not use astrometry.net, any index files, your CelestialPlateSolver code, or a web search. The only prior input was the EXIF time and site coordinates, which fixed the alt-az frame; the identification itself came from pattern matching and would stand even if the timestamp were wrong.
Prompt: You should have all the tools in your container. Solve this image with the same methods. I want to know how long this step takes and compare to the time to solve the previous image when you started from scratch.
Solved. IMG_1078: 2026-06-07 01:35:53 CDT, ISO 5000, same site. Field centre az 238.56, alt 56.30, looking southwest at Bootes with Hercules toward the upper left. 222 of 404 in-frame Hipparcos stars matched, median residual 1.08 px, scale 68.22 arcsec/px, FOV 41.1 x 67.4 deg, f = 3023.7 px, k1 = 0.0488, k2 = -0.0420 — consistent with the values from IMG_1095 (68.28 arcsec/px, k1 = 0.0616), as expected for the same lens.
Brightest stars: Arcturus (V -0.05, at 1945, 2710), Alphecca 2.22, Izar 2.35, Zubeneschamali 2.61, Unukalhai 2.63, Muphrid 2.68, Kornephoros 2.78, zeta Her 2.81, Sarin 3.12. Constellation census: Her 93, Boo 47, CrB 31, Ser 29, Vir 11, Lib 9, Dra 2. Trees fill the bottom corners and a lit cloud sits at bottom centre; those regions produced detections but no false matches.
Timing.
IMG_1095, from scratch IMG_1078, reusing the pipeline
Elapsed from upload to final figure about 14 min (14:14 to 14:28 by file mtimes) about 6 min
Scripted compute only 123.3 s (measured by re-running the same pipeline on it just now) 129.6 s
The compute is essentially identical, 123 vs 130 s, and the small difference is just source count: IMG_1078 yielded 699 detections against 206, so the RANSAC pair loop had more work. That loop dominates completely — 106 s and 109 s of the two totals, 86 percent. Everything else is trivial: EXIF 0.1 s, extraction 2.2 s, catalogue transform 1-4 s, the distortion refinement 0.01 s, and 14 s in get_constellation, which is a slow per-star Python loop.
So the real saving was not compute but the exploratory overhead: the first pass spent roughly 11 of its 14 minutes on catalogue downloads, the failed ephemeris fetch and workaround, discovering the constellation-line file, and iterating on the figure. The second pass ran one script.
One correction worth noting, found while checking the new solve. Deneb came out at pixel (850, 319), apparently inside the frame, but it is 65.5 deg from the field centre against a 37.4 deg half-diagonal — the radial distortion polynomial is non-monotonic beyond r ≈ 0.85 and folds far-off-axis stars back into the frame. I added a guard on undistorted radius. It removed 2 spurious matches from IMG_1078 (224 to 222) and 203 phantom in-frame entries; IMG_1095's 120 matches were unaffected, though its in-frame count drops from 620 to 417.
Prompt: Create in my new ClaudePlateSolver folder a package with this pipeline and the requisite files and a folder of scripts for installation of dependencies and a Readme.md explanation of how the user can use this package to solve their own image. Do you have any questions? Do not overwrite any file in my system or launch MATLAB without my approval.
…
Tags
