On the making of a graphic novel app

Phallaina
Try the demo→

Phallaina is a singular app that breaks from the canonical graphic novel format by delivering a narrative scrolling experience. Visually, it spans the physical length of 1600 iPad screens, is over an hour of reading and includes an immersive soundtrack.

iOS device
Role
Solo Software Engineer, Interaction Designer
Activities
UX & Interaction Design, Technical direction. C++, OpenGL, iOS & Android native app development.

The traditional paginated book structure is directly tied to the medium itself, inherited from the original codex that brought a superior practicality over ancient scrolls. Since digital apps are liberated from such physical limitations, it opens up possibilities of different ways for the readers to interact with a story.

 

Released in January 2016, the app was well received from the general public: 4.8+ stars rating on the European Apple Store and Google Play. It has been featured in numerous articles and received prestigious digital awards including The Webby Awards, FWA, and Peabody Future of Media Awards. The app is available for both iOS and Android.

Designing a creative workflow

My role was to lead both the technical and interaction design aspects. How do you turn a 1600 screen story idea into an actual thing? At such a large scale content production has to be seamlessly integrated into the creative workflow. When we started prototyping the concept in 2014, there was no authoring tool for us to make a 60fps native app directly from the drawings, animations and sound/assets. Unity was an option, but this meant introducing a learning effort for the whole team.

After Effects as an authoring tool for Phallaina After Effects as an authoring tool for Phallaina

By observing how Marietta Ren, the illustrator, and Julien Baret, the animator, were used to working, I decided to rely on the tools they were already comfortable with: Adobe Photoshop and After Effects. From there my goal was to move the effort on the technology side and industrialize the authoring workflow through new tools, formats and a custom rendering engine.

Phallain app workflow

Contrary to websites or content apps, there was no need for regular updates. No CMS, admin interface or “manually coded” animations. All images are generated automatically from the creative suite to produce a standalone object, like a physical book.

New tools and process

Once the author started drawing on a Wacom Cintiq & Photoshop setup, we were able to compose the scenes on After Effects: putting together different parallax layers, sprites, text positions, motion paths and interactions with the camera movement. This is where I had to dig into the AE composition structure and produce a plugin that automatically translates the scene information into our very own format. To extend this, a set of python scripts were automating the rest of the export by assembling every single asset into textures files directly usable by our rendering engine.

An example of how the story was cut in multiples tiles. An example of how the story was cut in multiples tiles.
A texture sprite board for dynamic elements and paralax. A texture sprite board for dynamic elements and paralax.

My goal was to preserve the iterative aspect of the creative process. This is all about having the shortest path between the creation and the result.

For this to happen, we need a direct preview and feedback of the final rendering. My solution was to create Phallaina Studio, a MacOS native app, that allowed us to read and play the After Effect compositions through our custom player.

Phallaina Studio Phallaina Studio, an in-house tool running on Mac OSX

Phallaina Studio was a critical tool for the sound engineer to synch the compositions with FMod, a sound engine from the gaming industry, and create an audio background that adapts dynamically to the reader progression through the app.

Interaction design

With a seamless content production flow and a working player, we were ready to wrap this into an actual app for the user to navigate and interact with the story. From the user experience perspective, we discovered that this is less about finding a specific part but keeping track of the reading progress and being able to freely explore any scene.

The timeline UI, to navigate the whole story. The timeline UI, to navigate the whole story.

Instead of trying to fit a preview of 1600 screens, we designed a simple linear timeline representing the chapter structure, the reading progression and current position. Through design, user testing and coding iterations, we ended up with a fine-tuned timeline interaction that extends the experience and specific mood of Phallaina.

A custom rendering engine

Behind the 2D reading experience, there is actually a camera moving horizontally through a 3D environment. I wanted to ensure a constant 60fps flow while loading the/assets on the fly, and not be limited with scene description formats or the rendering effects ideas we could come up with. At the end of the day, what we needed was a specialized 3D framework in black and white. This looked super fun to code, so I decided to write the entire thing from scratch in C++ and OpenGL, for it to support both the iOS and Android platforms. I have ported a demo to the web via Emscripten and WebAssembly.

The different software components of Phallaina The different software components of Phallaina

The end result was a compact and efficient 2D/3D engine that supports scene loaders, asynchronous/assets management, distance field text rendering, scene graph, motion paths, GLSL shaders and basic special effects.

The speech dialogues had to be completely dynamic and translated in English, French or any other future language. Fine typographic rendering is a tricky thing to implement in OpenGL, but a more challenging one was actually the speech balloons. To obtain a shape with a hand drawing feel to it, I designed an algorithm that generates a dynamic polygon outline around the text that stretches a brush texture with natural pressure variations.

How the speech balloons are rendered. How the speech balloons are rendered.

As some parts of the Phallaina player could be useful for the creative community, I am thinking about releasing the core engine as an open-source project in the future.

Phallain source code

Teamwork

Phallaina can be seen as a design response to the opportunity of exploring new graphic novel reading experiences. Behind such a simple and minimalist result, we made tremendous efforts as a team to create a product greater than the sum of it parts.

Phallaina is a SmallBang & France Télévisions Production

Author and Illustrator
Marietta Ren
Animator
Julien Baret
Sound Engineer
Côme Jalibert
Software Engineer & UX Designer
Christophe da Silva
Graphic Designer
David Benmussa
Assistant Director
Martin Bessin

 

Japan Media Arts Festival
Jury Selection 2017
Future of Story Telling
Mobile 2017

 

Phallain app