Interaction Beyond 2D Environments with Cooliris

Friday, January 23, 2009

When comparing game play scenes from “The Legend of Zelda“: released in 1986 with those from “The Legend of Zelda: Twilight Princess“:, release in 2006, the immediately obvious difference is dimensionality. The classic game takes place on a 2D grid, the modern game unfolds in 3D with particles systems, reflections, shaded textures, and more.

At Helioid, we strongly believe that the future of the world wide web is experiencing a similar transition from minimal 2D interaction towards immersive 3D interaction. The 1986 style of constrained navigation offered by current browsers is outmoded and outdated. The popularity of the Wii and iPhone demonstrate that if users are given improved alternatives to the classic styles of interaction they will make use of them. Immersive 3D environments are an improved alternative for web browsing.

In addition to our work at Helioid, Kenneth and I are both actively involved in the 3D immersion working group “IoImmersive”: At IoImmersive we work on developing a roadmap that will push interaction beyond 2D environments by integrating more dimensionality into user interfaces. Many of Mac OS X Leopard’s “features”:, such as “Quick Look” and “Stacks”, involve overlay effects. Windows Vista’s “Flip 3D”: uses “visual depth” to display open files and programs. The closer one looks the more movement towards a break from the straight 2D environment there is. IoImmersive proposes that 3D integration will go to the core of user interfaces and peripherals. From the browser to the operating system, the mobile phone to whatever replaces the mouse, we are headed towards immersion with virtual environments.

Cooliris’ browser plug-ins are an inspiring push towards immersive 3D environments. “Features”: include 3D navigation, integration with various rich media and product search engines, as well as the ability to store favorites. It is lacking in many areas, but, in its nascent stages, the Cooliris browser provides a solid framework upon which great things can be built.

The “vision”: presented by Cooliris board member Randy Komisar is just as encouraging as their products: “What would the world be like if rather than having to browse - if rather than having to pull up things in applications - what if we were able to navigate and find and share directly through the rich media that is now dominating the web.” Indeed, what would happen?

We would retrieve and arrange query results in a 3D environment. We would automatically reorganize items in this environment in response to user interaction. We would data mine users’ actions to find improved methods of presenting information to the user. We would significantly increase the efficiency and abilities of users.

CoolIris 3D

We would use newly proposed “3D standards based markup”: and additional 3D handlers (e.g. ‘onrotate’, etc). We would integrate current “opensource”: 3D suites and SVG “editors”: We would finally have an excuse to use “spatial operating environments”: for interaction.

Still, the above ideas barely scratch the surface of what we would do. With an intuitive 3D environment and standards based programming frameworks, the primacy of the operating system would fade. The hugely important component missing from web browsing is an integrated graphical user interface. Each web page has its own rules and implicit philosophy of user interaction. Ones mind is forced to readjust to these different rules and work within them. In general, there is absolutely no sense of continuity.

Shifting to a completely immersive browsing system would set information on the internet (web pages, emails, feeds, etc) within a larger ecosystem. Relations between objects could be made abundantly clear through visual cues. For example, I could open an email message and it would be centered in front of me with the pages linked to by the email, summaries of the key concepts given in the email, and collections of information on the sender and recipients set back to the sides (and perhaps above). If any of these related items interest me I can seamlessly bring them to primacy and the email recedes slightly.

The undercurrents of modern innovation hold a revolutionary concept in information interaction. Helioid is dedicated to promoting this revolution.