Background

When Apple first launched their augmented reality tools for iOS, known as ARKit, we got excited about the possibilities for applications that could provide both entertainment and productivity by seamlessly blending the real world with the digital one through the lens of your iPhone.

We love playing games, and building games is a great way to explore new technology capabilities. So, we teamed up with our friends at Koedal to create FlySwatAR, a game that has players running around the building searching out pesky flying insects and swatting them to earn points, collect Amber, and level up. An augmented reality first person swatter.

The game begins with some training for the player to master their swatting skills. As the player levels up, the pace and difficulty increases, more flies attack, some with special powers. The game continues in difficulty until the player reaches the final level of the season or runs out of time before exterminating the plague of unrelenting flies. An in-game store gives the player options to trade amber for advanced swatters like the popular Bug Bomb and Fly Freeze canister launcher.

Storyboarding

A good game starts with a good storyline. We started the project with a seed of an idea and grew it into a rich story with multiple characters, play modes, levels, inventive swatters, and rewards. As with most projects we do at T Minus 5, we begin with storyboards and sketching to flesh out ideas.

Sketching on paper offers designers a creative and quick means to express concepts and ideas that are cost effective for the project budget. Clients can enjoy the simplicity of paper or digital photo assets to review, markup, and provide feedback. Sketching in whiteboard sessions is also something we make use of in the early stages. Clients often feel comfortable to engage in the brainstorming during these activities without having to learn new digital tools.

Visual Design

As the story and early sketches take shape, we started work on the visual design, the look and feel of the game. Style choices and text copy define the voice and personality of any design. We wanted the game to be inviting and approachable by any age group, and convey a sense of organic construction...irregular shapes, crooked lines, forms found in nature, built in a world flies created.

Choosing the right font was our first task. We loved the sketchy style of the hand drawn storyboard screens. They were inherently organic as our designers refused to use rulers, compasses, stencils, or protractors. Hands and pencils only. We searched for just the right font that would meet the requirements of a world that conveyed the organic structures of nature. It was cosmic alignment or synchronicity or karma or something that led us to Butterfly Ball.

For the color palette, we called upon childhood memories of insect parts, and goo, and guts and all the icky things under our feet, to select the right hues and then pumped up the saturation and brightness to make them stand out from the real world being augmented in front of you...more cartoony if that is even a word.

In keeping with the visual direction, the 2D art designed for the game controls UI that overlay the world being augmented used the same stylistic model of our paper sketches and font. The game was built using all native iOS SDKs from Apple, and for the game controls, we used UIKit along with CoreAnimation, and other technologies. The resulting experience felt right as the 2D game control UI popped into and out of view of the players 3D world view during play.

Audio

One technical element of the game was the use of 3D spacial audio for the character’s buzzing. Flies can be small and too fast for the eyes to follow sometimes so spacial audio provided an additional helpful directional tracking tool, your two ears and your brain’s decoding algorithms. Turns out the spacial buzzing proved to be as annoying to the player as real world pests. The remaining audio assets were kept simple to support the organic, low-tech style of the game but also funny so swatting insects made you laugh instead of sounding gross and crunchy.

3D Animation & Haptics

FlySwatAR utilized 3D animated models for the insect characters invading the players real world. These characters were designed to move and evade swatters with realistic behaviors using ARKit world tracking and surface detection, GameKit AI, and custom software logic we built for collision detection and advanced aerobatics.

To help the player track character movement, we added SceneKit particle emitters to leave contrails marking the insects’ flight path. These dissipated over time and changed color depending on the character’s special abilities and the game play taking place.

3D models were also created for advanced swatters like the Bug Bomb. These swatters used AR world tracking, custom animations and controls to give the player enhanced swatting capabilities to defeat the more tenacious and speedy opponents.

Another technical feature of the game involved using the Taptic Engine Apple introduced for iPhone to provide haptic force feedback to the hand when making contact with flies. We built collision detection and custom logic to control the haptics so swatting the phone toward the enemy characters to splat them to the floor provided an affirming feel of a job well done.