My First VR Project: A Documented Experience

We all must start somewhere.

This beginner project for my VR High Immersion Nanodegree was an interesting and humbling exercise in the development process, and the hard lessons gained by realizing what doesn’t work early.

As a software product manager, I’m often the loudest voice shouting “fail fast” or “iterate quickly”. But I don’t apply the same mantras to my own projects. This course project, while highly instructive about the nuts and bolts of VR development in Unity, was an inspiring reminder of how to get your projects made without wasted work.

The Sample App: Puzzler

Puzzler is a simple VR game for 3 DoF headsets. Unlike high immersion HMDs such as Oculus Rift or HTC Vive, there is no sophisticated control system. You have a single button which activates whatever object is underneath the visual reticle in the center of your view.

You use this control system to enter a dungeon and play a Simon Says-like game where you must repeat a 5-note pattern of colorful orbs. After getting the pattern correctly, you escape the dungeon and complete the game.

While creating the app is the stated outcome, the real outcomes involve an application of good development technique.


Of course it’s important to define your audience. Even if all you do is accept that there is a group of people other than yourself who will use your app, defining your audience forces your thinking in an empathetic direction. Asking questions such as “Will people understand this is a button?” or “Is it clear what you’re expected to do here?” will challenge some of your preconceptions and make for a better VR experience.

But in this project we were encouraged to go beyond this surface-level and make personas. This means defining different types of users by visualizing an actual person and viewing the app through their experience.

So instead of saying that this is for casual VR fans, I defined the persona of Jeff. Jeff doesn’t have a powerful computer or a PS4, but he’s still excited about VR. He owns an Oculus Go and is eager to try anything and everything out there. He’s motivated to discover novel and immersive experiences and he shares the best ones with his group of friends. He is fairly tech-savvy, but has a low tolerance for simulator sickness.

Defining a person to this amount of detail really helps direct your design and development process. Asking “Would Jeff find this puzzle challenging?” or “Is this something that Jeff would want to show his friends?” is the line of questioning that makes a difference between a good and great experience.


Instead of getting our hands dirty with Unity right away, we were encouraged to make sketches of our vision for the app. Taking a pencil to paper seems nearly primitive these days, but it remains the quickest way to ideate. The primary task is not to decide what to do, but to articulate your vision about the final work so they can be evaluated critically.

In my case, I was actually more impressed by my sketch than I thought I would be. Putting this idea on paper let me think about it in more detail and realize that I liked the way it looked. But I have been involved with other sketching sessions where I tried 10 different approaches to the same idea and ended up hating all but the last one.

The more time that can be devoted to the ideation process, the better the final work becomes. But it is difficult to restrain the urge to get involved with the high-tech tools. In fact, ideation is perhaps the purest form of creation because there is nothing holding you back.


VR presents some interesting challenges because of how visceral it is. As opposed to 2D media, VR experiences plunge the user out of their comfort zone and fully within the control of the creator. A bad VR experience has the potential to be an order of magnitude more unpleasant than a bad app.

But there is a more subtle form of experience going on, one of immersion, scale and environment. Someone might not immediately be able to put words to what feels “off” about a VR experience, but it will be obvious as soon as the headset comes on. Getting the experience right can only be achieved by putting the headset on early and often, catching problems before more problems are built on top of them. We were encouraged to test out everything from whether the scale of common objects felt correct, to the way audio faded between waypoints. This sort of attention to detail really makes the experience feel coherent and whole.


The above two techniques are something I’m familiar with, at least on paper, but I learned the most through the experience of documenting my work. It was inspiring to fill up pages with evidence of my work and to see visually how the project evolved. Comparing the first renders of my environment compared to the final work really makes the results of the other two techniques obvious.

I also appreciated the chance to articulate my thought process and current challenges. It gave me the opportunity to critique my own thoughts and understand what I was trying to do, where I thought there was room for improvement, and what I thought about the work I was doing.

The process

Visual inspiration

I was looking for a dark and murky image similar to what I’ve seen in some of the Final Fantasy games. Here was one piece of artwork which I took as inspiration.

From this, I drew the following sketch of the background and entrance. I let myself sketch what I wanted to see without worrying too much about whether any of this was technically feasible.

Assembling the dungeon

The course suggested a pretty small space which didn’t fit with my initial image of the room the game would take place in. I wanted there to be a feeling of being in a massive space and surrounded by environment. I might not be able to pull off the 3D modelling, but I figure at least I could get the scale right.

Initially I put the door off to the right so that there was nothing distracting attention from the orbs. I don’t like how the orbs overlap the doorway in the sample app.

Lighting and decoration

In order to make the terrain feel immersive, I raised up some hills and mountains. I mostly wanted to avoid seeing a flat horizon when you left the dungeon. I also got some snow texture and rock objects to decorate the outside. Finally, I added lighting to the interior to make everything feel warm and dark.

Adding the UI

I’m not a fan of non-diagetic UI in VR, but it was the simplest path forward here so at least I wanted to make the theme match the surroundings. I created some graphics in Photoshop and then applied it to the panel and buttons for my UI elements


Getting the motion right

After setting up the waypoints, I had a chance to see how it felt navigating through this environment. But unfortunately, my early plan to have the exit off to the right ended up making me feel very sick. I also didn’t like how I had to rotate my view after hitting Restart in order to begin again.

So begrudgingly, I reverted to a version with the doorway in front of the camera. I was able to organize the orbs so that they weren’t directly overlapping the doorway. This felt a lot better, but it wasn’t as cool as I hoped.

User testing outcomes and iteration

User testing was very valuable to figure out which aspects of the motion and environment were not working quickly, before I had built too much work around a mistake. I was also able to notice a lot of problems (such as rocks floating in the air that I did not notice without stereoscopic vision) just by putting my headset on early and often.

I was also able to quickly notice when something that I did made the framerate dip. After adding the terrain and lighting, I noticed that simply looking around became a painful experience. I was able to improve the framerate by setting the lighting to be baked and the objects to static, but I’m sure that there is a lot more optimization that will become obvious after proceeding further in the course.

Breakdown of the final piece

The completed experience can be seen below.


While I’m still happy with the completed app, I am most happy with the framework I have learned which can I be applied to future projects. I feel empowered to tackle my more ambitious projects using this method of ideation, iteration, and user testing.

Thanks to my documentation process, the problems which remain with this project are well-known and I am in a great position to fix them at a later date after my skills improve.