Recent XR / AI Development

Here's a quick glance at some research, prototyping and projects I've been working on lately.

World Cup in a Day - SensAI Hack Siggraph

I mentored the SensAI hack at Siggraph this year, I started tinkering with the tools and learned how to splat and use it in Unity. Using Meshy.AI to generate players from prompt, texture, rig and animate them. I stole some animations from Mixamo for soccer. I used the World Labs API to generate a World Cup Stadium from photo, and then I used SensAI plugin to get it into Unity and displaying on the Meta Quest 3. I then used Claude Code to generate behavior.

Making some Friends

I did a deep dive on vibe coding with Claude Code in March. I built several NPCs using Meshy.AI all with unique personalities. Each has a local LLM running on Meta Quest 3, a C++ server acts as bridge between the app and LLM on device, no wifi or internet. It also has local Speech to Text and Text to Speech. Each NPC seeks you out learns your name and then uses it when talking to you. They all have a command level architecture and can also follow orders like: sit, stop, follow,, catch, shut down, among many others. They also have needs, like The Sims, and they seek out objects and interactions in the environment to fulfill them. All of the character meshes and animations were made with Meshy.AI and Mixamo.

Meta Horizon Worlds - Desktop First Steps

In late 2025 I spent a day tinkering in Horizon Worlds desktop editor. I received a demo at GDC of some of their GenAI content creation and was curious to try it for myself. This is a tutorial level where you build out a graveyard and a simple shooting mechanic. I put in a couple Gen AI characters I made in the toolkit based on the muppet below to see what sort of results it would have. On 2 prompts I received very different results, but still promising about what might be possible as it advances. It's published for VR and mobile if you want to try it.

Pika - Video Experiments

I started with an image of myself, using ChatGPT 4, made it into a muppet. Then I used some video and the muppet image to prompt Pika to make some cool effects!

Meta Quest - MR Interactive Elements

This demo uses Passthrough cameras on Quest3, and makes use of Meta hand tracking to interact with softbodies and other physical objects in a mixed reality environment

Meta Quest Mixed Reality Shaders

I spent some time building out different shaders, and experimenting with Meta's "scene understanding" which allows for virtual light and shadows to be projected onto real world elements. Also, cutting out the hands and having them occlude properly against virtual objects, and cast virtual shadows, no virtual hands needed!

Meta Quest - XStadium-esque MR

I built this project for some colleagues to showcase what 8K playback could be like in a mixed reality setting. The goal was to emulate a stadium experience at home.

Spacebar Arcade

A friend of mine from SVVR, Sam Mateosian, has had this passion project to make a virtual bar with bar games and pinball for almost 2 years. I jumped in and helped with some small things like making the hand tracking physical, modeling/texturing some whiskey bottles and experimenting with mixed reality.

Check it out here: https://spacebar.games/

Unreal 5 Engine Hands Test

This was a quick pipeline test to see if I could develop for Meta Quest and hand tracking using the Unreal 5 Engine

Fortnite Editor (UEFN)

I fired up the UEFN to see if could make a simple mod to Fortnite and publish it. Here is my first Island. Next step is to add the gameplay and publish.

XREAL - Air Pro 2 Weather App

Spent some time with both the XREAL Air Pro 2 and the Air Pro Ultra. I built a simple weather app that pull real locational weather data, and lets you place stickies around your home using your hands.