Beyond Video
A photograph captures light. A video captures light over time, with sound. But experience is more than light and sound. You feel the sand between your toes, smell the ocean, sense the warmth of sun on skin, feel your heart race with excitement or calm with peace.
What if all of that could be captured? Not just what you saw and heard but what you felt—the full sensation of being somewhere, doing something, living a moment?
This is the ultimate promise of media technology: not just recording events but recording experiences. Not just showing what happened but enabling others to feel what it was like.
The technology is nowhere near this yet. But the components are developing. Spatial video captures 3D visual environments. Biometric sensors track physical states. VR creates immersive environments. Haptics provide physical feedback. AI can analyze and reconstruct experiences from partial data.
This chapter explores the trajectory from current recording technology toward full-sensation capture: what's possible now, what's developing, and what it might mean to record and replay experience itself.
2026 Snapshot — Immersive Recording Today
Spatial Video
What it is: Video that captures depth—3D rather than flat. Viewable in VR headsets with sense of presence.
Current technology: Apple Vision Pro can capture spatial video with dual cameras. Other spatial cameras exist for professional use.
Quality: Good but not perfect. Resolution improving. 3D effect compelling in current hardware.
Limitations: Fixed perspective (wherever camera was). No interaction. Not truly "being there."
360° Video
What it is: Video capturing all directions simultaneously. Viewer chooses where to look.
Current technology: Consumer 360 cameras ($300-1000); professional rigs; integrated in some phones.
Quality: Consumer cameras are low resolution; professional is better. "Stitching" artifacts where cameras merge.
Limitations: Still fixed position. Can look around but not move around. Not truly immersive.
Volumetric Capture
What it is: Capturing 3D space that can be viewed from any angle. Not just spherical video but true 3D.
Current technology: Studio-based (Microsoft Mixed Reality Capture, etc.). Requires significant equipment.
Quality: Can be very high in controlled environments. Limited capture volume.
Limitations: Expensive; requires studio; can't capture outdoor or spontaneous events easily.
Biometric Recording
What it is: Capturing physiological data—heart rate, skin conductance, brain activity, etc.
Current technology: Wearables (Apple Watch, Oura Ring, etc.) capture some biometrics. Research devices capture more.
Quality: Heart rate reliable; other metrics variable.
Limitations: External only; can't capture subjective experience; correlation with emotion imperfect.
Haptic Recording and Playback
What it is: Capturing and reproducing tactile sensations.
Current technology: Very limited. Some VR controllers provide basic haptic feedback. Research on more sophisticated haptics.
Quality: Current consumer haptics are crude vibration. Nothing close to realistic touch.
Limitations: Fundamental challenges in reproducing touch sensations.
Notable Players
Spatial Video
Apple: Vision Pro spatial video capture and playback. Integrated into Apple ecosystem.
Meta: VR headsets with limited capture capability; primarily playback focused.
Canon, Sony, RED: Professional spatial/3D camera systems.
Insta360, GoPro: Consumer 360 and some spatial cameras.
Volumetric Capture
Microsoft: Mixed Reality Capture studios. Azure Remote Rendering.
Volograms, 8i, Tetavi: Volumetric capture for entertainment.
Various startups: NeRF and related technologies for 3D reconstruction.
Haptics
HaptX: High-fidelity haptic gloves. Enterprise focused.
bHaptics: Consumer-accessible haptic vests, gloves.
Research labs: MIT, Stanford, various universities advancing haptic science.
VR/AR Platforms
Meta Quest: Largest VR platform for playback.
Apple Vision Pro: Spatial computing for immersive media.
Pico, HTC: Additional VR platforms.
The Technology Roadmap
Visual Capture
Current: Spatial video from phones/headsets; 360 video; studio volumetric.
Near-term: Better resolution; more accessible volumetric; AI enhancement of captures.
Future goal: Capture any space in full 3D with consumer device. "Walk through" captured memories.
Key technologies: NeRF (Neural Radiance Fields), Gaussian splatting, light field capture, improved sensors.
Audio Capture
Current: Spatial audio is well-developed. Binaural recording, Dolby Atmos, object-based audio.
Near-term: Improved spatial audio in consumer devices. AI audio enhancement.
Future goal: Perfect recreation of acoustic environment.
Status: Audio is ahead of video in immersive capture. Less to solve.
Haptic Capture and Playback
Current: Very limited. Research stage.
Near-term: Better VR controllers; limited haptic feedback devices.
Future goal: Full-body haptic feedback reproducing touch, pressure, temperature.
Key challenges: Miniaturization; actuation mechanisms; sensory complexity; wearing something.
Chemical Senses (Smell, Taste)
Current: Almost no capability. Some research prototypes.
Near-term: Very limited. Scent dispensers exist but are crude.
Future goal: Reproduce smell and taste to accompany experience.
Key challenges: Chemical delivery; safety; individual variation; complexity of olfaction.
Vestibular (Balance, Motion)
Current: VR creates mismatch (visual motion without physical). Motion sickness.
Near-term: Motion platforms (enterprise); visual techniques to reduce mismatch.
Future goal: Create sensation of motion without physical movement.
Key challenges: Directly stimulating vestibular system; physical platforms don't scale.
Neural Recording/Stimulation
Current: Research stage. BCIs can read some brain signals (Chapter 6).
Future goal: Directly record and playback neural experience.
Key challenges: Understanding neural coding; safe stimulation; ethical concerns; science fiction timeline.
Applications
Personal Memory
Current: People record video constantly. Photos and videos are the primary way memories are preserved.
Near-term: Spatial video of family moments. More immersive memory preservation.
Future: "Relive" memories with full sensation. Return to any moment of your life.
Implications: Memory becomes external, perfect, reviewable. Nostalgia industrialized. What does remembering mean?
Entertainment
Current: VR games and experiences. 360 video documentaries. Immersive theater.
Near-term: Higher quality immersive content. Sports and events in VR. Spatial storytelling.
Future: Entertainment you experience, not just watch. Movies where you're present.
Implications: New art forms. Attention capture intensified. Physical and digital worlds merge.
Education and Training
Current: VR training for surgery, flight, dangerous situations. Some educational VR.
Near-term: More training applications. Medical, military, industrial.
Future: Learn by experiencing. History you witness. Science you feel.
Implications: More effective learning. Access to experiences otherwise impossible.
Therapy and Mental Health
Current: VR for exposure therapy, relaxation, pain management.
Near-term: More therapeutic applications. Trauma processing. Anxiety treatment.
Future: Therapeutic experiences designed for healing. Revisiting memories safely.
Implications: New treatment modalities. But also: potential for harm if misused.
Presence and Communication
Current: Video calls. VR meetings (limited adoption).
Near-term: Spatial presence—feeling like you're in same room. Telepresence improving.
Future: Meet anyone anywhere with full presence. Distance eliminated.
Implications: Travel reduction. New forms of connection. Physical presence premium?
Philosophical Questions
What Is Experience?
Experience involves sensory input, emotional response, cognitive processing, embodied state, temporal flow, memory formation.
Can all of this be captured? Replayed? Transferred?
Philosophical traditions differ. Some say experience is fundamentally subjective—it can never truly be shared. Others suggest experience could, in principle, be recorded if it were understood well enough.
The technology will test these theories. If perfect recording proves impossible, something will be learned about the nature of experience.
Whose Experience?
If someone replays another person's captured experience, whose experience is being had?
A viewer might see what the original person saw, feel what that person felt—but brings a different mind, different associations, different context. The experience becomes something new.
This isn't a problem necessarily—it's how empathy works. But it raises questions about authenticity, about understanding, about whether experience can truly be shared.
Can Experience Be Edited?
If experience can be recorded, can it be edited? Changed? Improved?
Remove the pain from a memory. Add pleasure that wasn't there. Create experiences that never happened but feel real.
This might seem beneficial—therapeutic even. But it also enables manipulation, false memories, manufactured emotions.
What Becomes Valuable?
If any experience can be recorded and shared, what's valuable?
Scarcity matters: experiences that can't be recorded or replicated. Human presence, perhaps—the knowledge that another person is actually there.
Authenticity matters: experiences that actually happened, not simulations.
Or maybe scarcity and authenticity become obsolete values. Maybe quality is all that matters, regardless of source.
The Path Forward
Near-Term Likely (2026-2032)
Spatial video normalizes: More devices capture and display spatial video. Personal memory preservation evolves.
VR entertainment matures: Higher quality, more content. Still niche but established.
Haptics improve incrementally: Better VR controllers and accessories. Still crude.
Training applications expand: VR training standard in more industries.
Plausible (2032-2040)
Volumetric capture goes mainstream: Capture 3D spaces with consumer devices. Walk through memories.
Haptics become meaningful: Devices that provide substantial tactile feedback. Not perfect but useful.
Immersive entertainment is mainstream: Significant portion of media consumption is immersive. New art forms mature.
Smell/taste experiments: Some capability to add chemical senses. Gimmicky but present.
Wild Trajectory (2040+)
Near-complete experience capture: Most sensory dimensions recordable. Replay feels close to real.
Experience sharing is normal: People share experiences, not just media. Gift someone a sunset. Share a first kiss.
Memory manipulation: Edit memories. Enhance experiences. Create false memories. Therapy and danger.
Neural interface integration: Direct brain recording/playback becomes possible. Experience becomes fully shareable.
Risks and Guardrails
Addiction and Disengagement
Risk: Recorded experiences more pleasant than reality. People prefer replay to living.
Guardrails: Design for enhancement, not replacement; awareness and education; social norms; balance mechanisms.
False Memory and Manipulation
Risk: Edited or fabricated experiences presented as real. Memory becomes unreliable. Manipulation.
Guardrails: Provenance tracking for captured experiences; authentication; legal frameworks; awareness.
Privacy and Consent
Risk: Experiences recorded without consent. Others' experiences captured in your recordings.
Guardrails: Consent requirements; recording indicators; data protection; privacy regulation.
Psychological Harm
Risk: Reliving trauma; trapped in negative experiences; inability to process or move forward.
Guardrails: Therapeutic frameworks; professional involvement; design considerations; not pure consumer product.
Experience Inequality
Risk: Rich experiences available only to wealthy. New digital divide in experience access.
Guardrails: Democratizing access; public experiences; not allowing capture/replay to become purely commercial.
The Deeper Questions
Is Mediated Experience Enough?
Humans have always sought to share experience—through story, art, ritual, media. Each technology extends this.
But there's always something lost in mediation. The direct experience differs from the recording. Will that gap close, or is there something irreducibly present in presence?
The technology will test the limits. Perhaps it will be found that experience can be fully captured. Perhaps something will be found that cannot be captured—and that something will reveal what experience really is.
Does This Make Life Better?
The promise is profound: access to any experience, relive any moment, share what it's like to be you.
But more access to experience does not necessarily mean better life. What people seek in experience—meaning, growth, connection—may require more than sensation.
The technology is a tool. Whether it makes life better depends on how it is used—what experiences are sought, how mediated and direct experience are balanced, whether depth or just quantity is pursued.
What Remains Human?
If experience can be captured, replayed, shared, simulated—what is unique about being a human experiencing?
Perhaps nothing—perhaps humans are information processes that could, in principle, be recorded and reproduced.
Perhaps something—perhaps there's a quality to being that can't be captured, a presence that requires presence.
These are ancient questions. The technology makes them practical.
Conclusion
Humanity is on a journey from recording light to recording experience. Photography captured a moment's appearance. Film captured motion and sound. VR creates presence. Full-sensation capture would enable truly sharing what it's like.
The technology is nowhere near the destination. Current technology captures visual and audio well, haptics and chemical senses barely at all, subjective experience not at all. The technical challenges are immense; some may be insurmountable.
But the trajectory is clear. Each generation of media technology captures more of experience. Spatial video today; volumetric capture soon; haptics improving; neural interfaces eventually.
The implications are profound. Memory becomes external and perfect. Experience becomes shareable in ways never before possible. The line between experiencing and observing blurs.
The risks are equally profound. Addiction to replayed experience. Manipulation through false memories. Privacy violations in recording others' experiences. The technology must be governed wisely.
But the most important implication may be what this pursuit teaches about experience itself. What can be captured reveals something about the nature of sensation, consciousness, memory, and presence.
People record because they want to preserve, share, and understand experience. The technology serves human needs that are timeless. What changes is the fidelity of the capture—and the questions that higher fidelity forces humanity to answer.
Endnotes — Chapter 42
- Apple Vision Pro spatial video captures stereoscopic video at 2200x2200 pixels per eye; viewable in VR with 3D depth effect.
- NeRF (Neural Radiance Fields) developed 2020; enables 3D scene reconstruction from 2D images; rapid advancement in quality and efficiency.
- Gaussian splatting (2023) provides alternative to NeRF with faster rendering; enabling more practical volumetric capture and playback.
- Microsoft Mixed Reality Capture studios enable high-quality volumetric capture; used for entertainment and enterprise applications.
- Spatial audio well-developed: Dolby Atmos, Apple Spatial Audio, Sony 360 Reality Audio provide immersive audio in consumer products.
- HaptX gloves provide high-fidelity haptic feedback; primarily enterprise/research use; consumer haptics much more limited.
- VR training effectiveness documented in various studies; medical, military, industrial applications show improved outcomes vs. traditional training.
- VR therapy applications include exposure therapy for phobias and PTSD; pain management; anxiety treatment; research is ongoing.
- "Smell-o-vision" and scent delivery systems have been attempted repeatedly since 1960s; remain crude and largely novelty.
- Direct neural recording/stimulation for experience capture remains science fiction; current BCI technology far from capturing subjective experience.