Grids And Gaming: Developing
MMOGs (Massively Multiplayer
Online Games) on the Grid
Johnny L. Hopkins (
Department of Software and Information Systems
College of Information Technology
November 8, 2005
ITCS 5010: Grid Computing
The University of North Carolina at Charlotte
Gaming on the Grid
What is a game?
Different game genres
Basic steps to designing and developing a
Curent MMP Engines
Languages for MMP development
How grids can help the game development process
The “butterfly grid”
Resources for Gaming
What is a Game?
• What are YOUR definitions of a good
video game?
• Definition: “A great game is a series of
interesting and meaningful choices made
by the player in pursuit of a clear and
compelling goal” (Rabin 2005).
Different Game Genres
Casual (Texas Hold ‘Em Poker,,, You Don’t
Know Jack)
Educational (Where in the World is Carmen Sandiego)
Fighting/First-person shooters (Halo II)
Puzzle (Puzzle Pirates, Bejeweled, Sudoku)
Racing (Grand Tourismo)
Rhythm (Dance Dance Revolution)
Sports (Madden 2005)
Role-Playing (EverQuest, World of Warcraft)
Strategy (Civilization IV)
• Acronyms
– MMORPG: Massively Multiplayer Online Role
Playing Game
– MMORTS: Massively Multiplayer Online RealTime Strategy Game
– MMOG: Massively Multiplayer Online Game
Social aspect
“Escape” from reality
Simultaneous interaction
“Social hijacking”
Ability to “mod” (Unreal, Quake, Half-Life,
• In all cases, these are video games
designed to be played over a computer
network and/or the Internet (e.g. LAN
• EverQuest (Sony Entertainment) is the
most popular MMORPG among players
• Are often some of the harder games to
design and develop
Issues with MMOGs/MMORPGs
• Long development cycle (two to three
• Versioning issues
• Basic client-server development
• Security: Hacking, In-game cheating,
Illegal plugins, etc.)
Basic Architecture of MMOGs
Hsiao, Tsun-Yu and Yuan, Shyan-Ming, “Practical Middleware for Massively
Multiplayer Online Games,” IEEE Internet Computing, vol. 9, no. 5, 2005,
pp. 47-54
Basic Steps
Create character
Save the character on the server-side
Login with character
Enable chat and “shout”
Create and enable 3D navigation
Create a database (data store) to hold info
on players, stats, worlds, guilds, etc.)
Different MMORPG architectures
Peer-to-Peer (P2P)
Grid (That’s us!!!!! )
Distributed Systems
MMORPG Architectures:
• Strong points: Mainframe heritage
– Time tested: Mature and widespread
– Single point of control: Allows for monitoring
– Strong security: Servers are in data centers,
hard to compromise systems (Yeah, RIGHT!!!)
– Modular scalability: Easy to scale and extend
by adding servers to server farm)
– Most MMPs use this architecture (currently)
MMORPG Architectures:
• Pitfalls:
– Dependency on a single physical channel to
data center
– Having C/S architecture for MMPs (Massive
Multiplayer) require large capital expense
– O(n^2) scalability problems (Think back to
your data structures/algorithms class. Why is
this bad for MMPs
MMORPG Architectures: Peer-toPeer (P2P)
• Strong points
Highly scalable: Good performance
O(n^2) solution: Bandwidth grows with additional players
Massive processing capabilities: Contribute idle resources
Massive storage capability: 100s TBytes of storage at no
additional costs
Low hardware expenses: Very few centralized resources,
everything is done on the network
Low bandwidth requirements: No central data center, dynamic
connections between each node
Dynamic topology, no single point of failure: Resilient to attacks
and catastrophes
Improved latency: Ave round-trip in dynamically formed P2P
network is significantly less than round trip to a fixed C/S
MMORPG Architectures: Peer-toPeer
• Pitfalls
– Decentralization
– No guarantees of “legitimate” data sources
– No way to establish data authenticity
– No way to control the behavior of P2P
MMORPG Architectures: Grids
• Scalable, high-performance network: Combines
the power of existing data centers and servers
into a “super server.”
• Massive storage capabilities: Servers AND data
centers can provide terabytes of storage at no
• Low bandwidth constraints: Generally favorably
located near Internet backbones, gives ample
• Low hardware expenses: Most processing and
storage takes place on servers connected to the
grid, little additional resources needed.
MMORPG Architectures: Grids
• Pitfalls
– No possibly of direct management of servers
because an open grid consists of
decentralized servers not owned by a single
– Different machines running different OSes
– Hard to monitor in real-time or make changes
to network
MMORPG Architectures:
Distributed Computing
• Close “cousin” to P2P networks
• Focuses on contributing computer power
• Shows promise for large-scale
computational projects (SETI@Home)
• Finds corrupted data and malfunctioning
nodes by repeating the same transmission
on different nodes
MMORPG Architectures:
Distributed Computing
• Pitfall: It is not a real-time system, so you
can’t use this architecture for MMORPGs
Different MMP Engines
Game Gardnens
BigWorld Technology
ICE (Internet Communications Engine)
Different MMP Engines
• Game Gardens (Three Rings Design,
– Foundational MMP engine for Puzzle Pirates
– Java-based
– Client/Server architecture
– FREE!!!!!!
Different MMP Engines
• BigWorld MMO Engine
– Developed in Australia
– Made up of client, server, tools
– Licensed by comapanies in Asia
– Interfaces with Python, XML, etc
– Aimed at serious developers
Different MMP Engines
• Kaneva (Kaneva, Game Platform,
– First downloadable engine for online games
– Integrated system (AI, persistent worlds, etc.)
– Licensing: Free to $50, 000 for a sinlge use
Different MMP Engines
• Populace (
– Designed for use with 3D Game Studio (Need
3D Game Studio for this to work)
– Developed for a MMORPG game, but network
engine was abstracted out of the game and
marketed as an API
– Only $149.99 (approx. $150)
– Major limitation: Only usable on Windows
Different MMP Engines
• Quazal (In-Game Multiplayer Engine, NetZ,
– Based on duplicated objects: Game objects
are duplicated to different stations on a
– When a change is made to master node, it
gets “sent” to other nodes
Different MMP Engines
• Massiv (
– Open-source engine built on top of C++ and
standard libraries
– Uses client/server architecture
– Extended object model
– Started as a school project, no new work is
being done on the engine (possible thesis
topic for someone, maybe………..)
Different MMP Engines
• Internet Communications Engine (ICE,
– Object-state versioning
– Distribution of software updates
– Efficiency
– Multilanguage support (C, C++, C#, Java,
Python, Visual Basic, PHP)
Different MMP Engines
• RakNet (
– Uses UDP connectivity
– Rapid development of online games and
adding multiplayer capabilities for singleplayer games
– Open-source but has a commercial license
Different MMP Engines
• ReplicaNet (
– C++ based
– Allows developer to focus on game logic without
resorting to network programming
– Shared objects
– Has a freeware version (no source code), but a
license costs around US$5000 (Some of the network
APIs I researched for this presentation requires a
game development shop to shell out at least
US$100,000 or more)
Different MMP Engines
• Zona (
– Commercial MMP Engine
– Complete solution:
Different MMP Engines
• Zona (cont)
– Three frameworks:
• Zona Application Framework (SDK)
• Zona Gaming Framework
• Zona Billing Framework
Languages for MMP Development
Visual Basic.NET
Java (Java 3D)
Python (PyGame)
Improving game development with
• Considering the needs of these
– Developers:
Use tools (C, C++, Python, Lua, etc.)
Need to use rendering engines
Write games for PlayStation2, Xbox, etc.
Improving game development with
• Considering the needs of these
– Publishers
• Finance games
• Make money
Improving game development with
• Considering the needs of these
– Service provider
• ISPs
• Decentralization
The “Butterfly” Grid
• Standards-based
• Chapter 13 in The Grid 2 by Foster and
• Provided the following at the time:
– High-performance
– Scalability
– Reliability
The “Butterfly” Grid: Architecture
• Builds on the core grid services provided by
• Major Tiers:
Object Management System (OMS)
Network Protocol Stack (NPS)
Gateway servers
Daemon controllers
Game servers
Grid Services (Think: Assignment 2)
The “Butterfly” Grid: Globus
• Staging and maintenance of a code base
in distributed environment
• Scheduling, monitoring, termination of
application process instances in a
distributed environment
• Monitoring framework for real-time
The “Butterfly” Grid: Specification
• Virtual organizations are defined by a
game configuration specification (GCS), a
XML-based spec structure with these
specific tags:
– <GRID>: Base node
– <MACHINE>: Gives names and addresses to
physical machines on the grid
– <ACCOUNT>: Authentication record for
– <SOFTWARE>: Defines a version of a game
The “Butterfly” Grid: Specification
• GCS tags (cont.):
– <WORLD>: A world is a game
– <INSTANCE>: Physical instantiation of game with
these tags within <INSTANCE>:
<NETWORK>: Multicast information
<SERVER>: Specifies running server process
<CONTENT>: Scripts, datafiles, executables, shared libraries
<GATEWAY>: Instantiates gateway process on a physical
• <DATASTORE>: How the instantiated game will connect to a
data store (identifiers, user names, passwords)
The “Butterfly” Grid: Deployment as
a GRAM Job
• Had to be mapped to Globus Toolkit
• Parse GCS to produce three types of
– Game configuration file
– Resource spec language (RSL) files
– Production of glue scripts: executes RSL on
machines and capture job submission ID
The “Butterfly” Grid: Current Status
• has changed its name to
Emergent Game Technologies
• Merged with NDL
• Formal announcement of product line:
March 2006
• They have changed their software to
create a new server engine
• Have they left grid computing?............
• International Game Developers Association
• Books:
– Introduction to Game Development edited by Steve
Rabin (Charles River Media)
– Massively Multiplayer Game Development 2 edited by
Thor Alexander
• General Game Development
GameDeveloper Magazine:
• Massively multiplayer games are
increasing in popularity
• Many different architectures and engines
to fit these architectures are available
• Butterfly grid was one solution to scale for

Grids And Entertainment