59 KiB
EQ2EMu Server Architecture White Paper
Executive Summary
EQ2EMu is a comprehensive open-source implementation of an EverQuest II server emulator written in C++17. This white paper provides a detailed architectural analysis of the complete system, covering its modular design, core components, networking protocols, and sophisticated game systems. The architecture demonstrates excellence in software engineering with robust threading, comprehensive database integration, extensible scripting systems, and scalable performance optimizations.
Table of Contents
- System Overview
- Core Architecture
- Common Infrastructure Layer
- LoginServer Implementation
- WorldServer Implementation
- Network Protocol Architecture
- Database Architecture
- Security Framework
- Performance & Scalability
- Extensibility & Scripting
- System Integration
- Deployment & Operations
- Technical Specifications
- Conclusion
System Overview
Architecture Philosophy
EQ2EMu employs a modular, service-oriented architecture designed for maintainability, scalability, and extensibility. The system is structured in three primary layers:
- Common Infrastructure Layer - Shared utilities, networking, and foundational services
- LoginServer - Authentication, character management, and world server coordination
- WorldServer - Game world simulation, entity management, and gameplay systems
Key Design Principles
- Separation of Concerns: Clear boundaries between authentication, world simulation, and infrastructure
- Thread Safety: Comprehensive mutex-based synchronization for concurrent operations
- Extensibility: Lua scripting system for dynamic content without code changes
- Performance: Custom UDP protocol implementation optimized for MMO requirements
- Scalability: Multi-threaded architecture supporting hundreds of concurrent players
- Reliability: Robust error handling and automatic recovery mechanisms
Core Architecture
System Topology
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Game Clients │ │ Admin Tools │ │ Other Servers │
│ (UDP 9001) │ │ (HTTPS) │ │ (TCP) │
└─────────┬───────┘ └─────────┬───────┘ └─────────┬───────┘
│ │ │
│ │ │
┌─────▼──────────────────────▼──────────────────────▼─────┐
│ LoginServer │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐ │
│ │ Auth System │ │ Char Mgmt │ │ World Coord │ │
│ └─────────────┘ └─────────────┘ └─────────────────┘ │
└─────────────────────────┬───────────────────────────────┘
│
┌─────────────────────────▼───────────────────────────────┐
│ WorldServer │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Zone 1 │ │ Zone 2 │ │ Zone N │ │ Core │ │
│ │ (Thread) │ │ (Thread) │ │ (Thread) │ │ Systems │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
└─────────────────────────────────────────────────────────┘
│
┌─────────────────────────▼───────────────────────────────┐
│ Common Infrastructure │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Network │ │ Database │ │ Security │ │ Utilities│ │
│ │ Protocol │ │ Layer │ │ System │ │ & I/O │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
└─────────────────────────────────────────────────────────┘
Component Relationships
The architecture follows a layered dependency model where:
- LoginServer and WorldServer both depend on Common Infrastructure
- Game Clients connect to LoginServer for authentication, then to WorldServer for gameplay
- Administrative Tools use HTTPS REST APIs for monitoring and management
- Inter-server Communication uses custom TCP protocols for coordination
Common Infrastructure Layer
Purpose and Scope
The Common Infrastructure provides foundational services shared across all server components, ensuring consistency, reusability, and maintainability.
Core Components
1. Network Protocol Stack
EQStream System (eq_stream.hpp
, eq_stream_factory.hpp
)
- Custom UDP Protocol: Implements EverQuest II network protocol with reliability layer
- Packet Management: Sequence numbers, acknowledgments, fragmentation, and reassembly
- Security Integration: RC4 encryption and zlib compression
- Connection Management: State tracking (ESTABLISHED, CLOSING, CLOSED, WAIT_CLOSE)
- Performance Optimization: Packet combining, rate limiting, and flow control
TCP Infrastructure (tcp_connection.hpp
, web_server.hpp
)
- Server-to-Server Communication: Custom framing protocol with optional compression
- HTTP/HTTPS Web Server: Boost.Beast-based REST API with SSL/TLS support
- Administrative Interface: JSON-formatted status and control endpoints
2. Database Abstraction Layer
Legacy Database System (database.hpp
)
- Async Query Processing: Threaded execution for high-throughput operations
- Connection Pooling: Multiple connections for concurrent access
- Opcode Management: Dynamic opcode loading and version support
Modern Database Interface (database_new.hpp
)
- Simplified API: Clean, thread-safe MySQL wrapper
- Automatic Recovery: Connection retry logic with error handling
- Prepared Statements: SQL injection prevention through parameter escaping
- File-based Queries: Database initialization from SQL scripts
Core Database Operations (database_core.hpp
)
- Low-level Connection Management: MySQL connection handling
- Configuration Support: INI file parsing for database settings
- Result Set Management (
database_result.hpp
): Type-safe field access with string_view optimization
3. Memory Management & Threading
Advanced Synchronization (mutex.hpp
)
- Reader-Writer Locks: Efficient concurrent access patterns
- RAII Lock Management: Automatic lock acquisition and release
- Deadlock Detection: Timeout handling and debug stack tracking
Thread Coordination (condition.hpp
)
- Cross-platform Condition Variables: Signal/wait primitives
- Broadcast Notifications: Multi-thread coordination
Container Infrastructure (linked_list.hpp
, queue.hpp
)
- Thread-safe Containers: Mutex-protected data structures
- Iterator Support: Safe traversal of concurrent containers
4. Protocol & Security
Packet Handling (packet_struct.hpp
, packet_functions.hpp
)
- Dynamic Packet Structures: XML-driven packet definition system
- Version-specific Support: Multi-client version compatibility
- Serialization Framework: Automatic struct packing/unpacking
Cryptographic Services (crypto.hpp
, rc4.hpp
, sha512.hpp
, md5.hpp
)
- RC4 Stream Cipher: Network traffic encryption
- Hash Functions: Password storage and data integrity
- RSA Key Processing: Initial handshake encryption
Opcode Management (opcode_manager.hpp
)
- Dynamic Opcode Mapping: Runtime opcode translation
- Version Strategy System: Multiple implementation strategies
- Thread-safe Resolution: Concurrent opcode lookups
5. Utilities & Infrastructure
Timing System (timer.hpp
)
- High-precision Timers: Millisecond-precision with drift compensation
- Trigger-based Operations: Periodic process coordination
- Benchmark Support: Performance measurement using std::chrono
Logging Framework (log.hpp
)
- Asynchronous Logging: Non-blocking log processing
- Multi-destination Output: Console, file, and client logging
- Color-coded Console: Enhanced debugging visualization
- XML Configuration: Flexible logging configuration
Configuration Management (config_reader.hpp
, json_parser.hpp
)
- XML Packet Structures: Dynamic packet definition loading
- JSON Configuration: Server settings and rules management
- Runtime Reloading: Configuration changes without restarts
LoginServer Implementation
Architecture Overview
The LoginServer serves as the authentication gateway and character management hub, coordinating between game clients and world servers.
Core Components
1. NetConnection Class
Purpose: Central configuration and network management controller
Key Responsibilities:
- Configuration Management: JSON-based server settings
- Web Server Initialization: HTTPS monitoring interface setup
- Expansion Control: Feature flags for different game content
- Account Creation Settings: Registration policies and restrictions
2. Authentication System
LoginAccount Class
- Account Management: Player account representation with character lists
- Character Serialization: Network-ready character data formatting
- Authentication Status: Login state and session tracking
Client Class
- Connection Management: Individual client session handling
- Packet Processing: Login protocol message handling
- State Tracking: Authentication status and client version management
- Security Integration: Version validation and access control
3. World Server Coordination
LWorld Class
- Server Representation: World server status and capabilities
- TCP Connection Management: Server-to-server communication
- Load Balancing Data: Player counts, zone information, capacity metrics
- Development Server Support: Special handling for test environments
LWorldList Class
- Server Registry: Thread-safe world server collection
- Broadcast Operations: Server-wide message distribution
- Status Monitoring: Real-time server health tracking
4. Database Integration
LoginDatabase Operations
- Account Authentication: SHA-512 password verification
- Character Management: Character creation, deletion, and modification
- Version-specific Handling: Different data formats for client versions
- Equipment Synchronization: Character appearance coordination with world servers
Authentication Flow
1. Client Connection → EQStream UDP Protocol
2. Version Negotiation → Client version validation
3. Login Request → Credential verification (SHA-512)
4. Account Validation → Database authentication
5. World List Delivery → Available server enumeration
6. Character List → Character selection with appearances
7. Character Selection → Target world server coordination
8. World Transfer → Hand-off to WorldServer instance
Security Measures
- Password Hashing: SHA-512 with salt for secure storage
- Session Management: Unique session keys for connection tracking
- IP Validation: Connection source verification and banning support
- Version Control: Client version enforcement and compatibility checks
WorldServer Implementation
Architecture Overview
The WorldServer represents the core game simulation engine, managing entities, zones, gameplay systems, and player interactions. This massive system contains over 50,000 lines of C++ code across hundreds of files.
Core Architecture Components
1. World Management (World.cpp/h
)
Purpose: Central orchestrator and singleton manager for the entire server instance
Core Functionality:
- Global State Management: World time, server statistics, vitality systems
- System Initialization: Coordinates loading of all game systems
- Timer Management: Periodic processes (saves, updates, maintenance)
- Resource Management: Global resources like merchants, loot, transports
Key Systems:
class World {
WorldTime world_time; // Game time management
map<int32, PlayerHouse*> player_houses; // Housing system
map<int32, MerchantInfo*> merchant_info; // Merchant data
map<int32, LottoPlayer*> lotto_players; // Lottery system
// ... extensive global resource management
};
2. Zone Management (zoneserver.cpp/h
)
Purpose: Manages individual zone instances and spawn lifecycles
Core Functionality:
- Zone Lifecycle: Loading, running, and shutdown of zone instances
- Spawn Management: Entity spawning, despawning, position tracking
- Spatial Partitioning: Location grids for proximity detection
- Script Integration: Lua script execution for zone events
Threading Architecture:
- Main Zone Thread: Core zone processing loop
- Spawn Thread: Entity spawn management
- Initial Spawn Thread: Zone initialization processing
3. Entity System (Entity.cpp/h
, Spawn.cpp/h
)
Purpose: Foundational entity system for all game objects
Entity Hierarchy:
Spawn (Base)
├── Entity
├── Player
├── NPC
├── Object
├── GroundSpawn
├── Widget
└── Sign
Core Features:
- Stat Management: HP, Power, attributes via InfoStruct
- Effect Systems: Maintained effects, spell effects, detrimentals
- Bonus System: Equipment and spell bonuses via BonusValues
- Combat Integration: Damage processing and combat states
4. Player System (Player.cpp/h
)
Purpose: Complete player character implementation extending Entity
Major Subsystems:
- Character Progression: Experience, leveling, alternate advancement
- Inventory Management: Equipment, bags, bank integration
- Social Systems: Friends, guilds, groups, chat
- Quest System: Quest progression and completion tracking
- Skill System: Adventure and tradeskill advancement
Database Integration:
- Character Persistence: Save/load character state
- Equipment Tracking: Appearance synchronization
- Progress Storage: Quest, achievement, and skill data
5. NPC System (NPC.cpp/h
, NPC_AI.cpp/h
)
Purpose: Non-player character implementation with AI behavior
AI Components:
- Combat AI: Tank, DPS, healing behavior patterns
- Merchant Systems: Shop inventories and transactions
- Conversation Systems: Dialog trees and quest interactions
- Patrol Systems: Movement patterns and spawn behaviors
Game Systems Architecture
6. Combat System (Combat.cpp/h
)
Purpose: Comprehensive combat mechanics and calculations
Core Mechanics:
- Damage Calculation: Physical and magical damage formulas
- Defense Systems: Mitigation, avoidance, blocking, parrying
- Status Effects: Stuns, roots, damage over time
- Combat States: Engagement rules, auto-attack timers
7. Magic System (Spells.cpp/h
, SpellProcess.cpp/h
)
Purpose: Complete spell and magic system implementation
Key Components:
class Spell {
// Spell template definition
vector<LUAData*> lua_data; // Scripted effects
vector<SpellData*> spell_data; // Spell properties
int32 class_skill; // Required skill
// ... extensive spell configuration
};
class SpellProcess {
// Active spell processing
Mutex MSpellProcess; // Thread safety
map<int32, map<int32, LuaSpell*>> active_spells;
// ... real-time spell management
};
Spell Categories:
- Spells: Magical abilities and enchantments
- Combat Arts: Physical special attacks
- Abilities: Class-specific capabilities
- Tradeskill: Crafting-related abilities
8. Item System (Items/Items.cpp/h
)
Purpose: Complete item system with equipment and inventory
Item Architecture:
class Item {
ItemCore details; // Basic item properties
vector<ItemStat*> item_stats; // Stat modifications
int32 stack_count; // Stacking information
// ... comprehensive item data
};
Equipment System:
- 31 Equipment Slots: Including appearance equipment
- Stat Modifications: Bonuses applied to character stats
- Item Flags: Attuned, lore, no-trade, temporary
- Inventory Management: Bag systems, sorting, stacking
9. Quest System (Quests.cpp/h
)
Purpose: Quest management and progression tracking
Quest Framework:
- Quest Templates: Step-based quest definitions
- Progress Tracking: Kill counters, item collection, exploration
- Reward Systems: Experience, items, faction rewards
- Journal Management: Quest log and sharing systems
10. Skill System (Skills.cpp/h
)
Purpose: Character skill advancement and specialization
Skill Categories:
- Adventure Skills: Combat and exploration abilities
- Tradeskill Specializations: Crafting skill trees
- Language Skills: Communication and lore
- Skill Bonuses: Equipment and spell modifications
11. Communication System (Chat/Chat.cpp/h
)
Purpose: All forms of player communication
Chat Architecture:
- 100+ Chat Channels: Different communication types
- Language System: Multiple language support with translation
- Social Features: Friend lists, ignore lists, player searches
- Cross-Zone Communication: Global chat across boundaries
12. Guild System (Guilds/Guild.cpp/h
)
Purpose: Guild management and social organization
Guild Features:
- Membership Management: Ranks, permissions, leadership
- Guild Events: Activity logging and notifications
- Guild Halls: Housing integration for guild properties
- Permission System: Rank-based access controls
Specialized Systems
13. Lua Scripting System (LuaInterface.cpp/h
, LuaFunctions.cpp/h
)
Purpose: Lua 5.4 integration for dynamic game content
Scripting Architecture:
- 400+ Lua Functions: Extensive API for game world manipulation
- Script Categories:
- Spawn Scripts: NPC behavior and interactions
- Quest Scripts: Dynamic quest progression
- Zone Scripts: Zone-wide events and mechanics
- Item Scripts: Special item effects and usage
Performance Optimization:
- Script Caching: Compiled script caching
- State Management: Persistent Lua states across calls
- Error Handling: Comprehensive error reporting
14. Bot System (Bots/Bot.cpp/h
)
Purpose: AI-controlled player companions
Bot Intelligence:
class Bot {
map<int32, int8> heal_spells; // Healing abilities
map<int32, int8> buff_spells; // Buff casting
map<int32, int8> debuff_spells; // Debuff applications
map<int32, int8> taunt_spells; // Tanking abilities
// ... role-specific AI patterns
};
15. Pathfinding System (Zone/pathfinder_interface.cpp/h
)
Purpose: AI navigation and automated movement
Navigation Features:
- A Pathfinding*: Optimized route finding algorithms
- Navigation Mesh: 3D world geometry integration
- Path Smoothing: Movement optimization
- Obstacle Avoidance: Dynamic obstacle detection
16. Housing System (Housing/HousingPackets.cpp
)
Purpose: Player housing with customization
Housing Features:
- Private Instances: Instanced housing areas
- Decoration System: Furniture placement and customization
- Access Control: Visitor permission systems
- Maintenance: Upkeep costs and escrow management
17. Web Integration (Web/WorldWeb.cpp
)
Purpose: HTTP/HTTPS administration interface
Web API Endpoints:
/status
- Real-time server status/clients
- Connected player information/zones
- Active zone statistics/reloadrules
- Dynamic configuration updates/sendglobalmessage
- Server announcements
Network Protocol Architecture
Protocol Overview
EQ2EMu implements a custom UDP-based protocol optimized for MMO requirements, providing reliability, security, and performance for real-time gameplay.
Protocol Stack
┌─────────────────────────────────────────┐
│ Application Layer │
│ (Game Logic, Spells, Combat) │
├─────────────────────────────────────────┤
│ Packet Structure Layer │
│ (XML-defined packet formats) │
├─────────────────────────────────────────┤
│ EQStream Protocol Layer │
│ (Reliability, Fragmentation, Crypto) │
├─────────────────────────────────────────┤
│ UDP Layer │
│ (Basic Transport) │
└─────────────────────────────────────────┘
EQStream Protocol Features
1. Reliability Layer
- Sequence Numbers: Packet ordering and duplicate detection
- Acknowledgments: Reliable delivery confirmation
- Retransmission: Automatic packet resending on timeout
- Flow Control: Rate limiting and congestion management
2. Security Implementation
- RC4 Encryption: Stream cipher for packet privacy
- CRC16 Checksums: Packet integrity verification
- Session Keys: Unique encryption keys per connection
- Anti-replay: Sequence number validation
3. Performance Optimization
- Packet Combination: Multiple small packets into single datagrams
- Fragmentation: Large packet splitting and reassembly
- Compression: zlib compression for bandwidth reduction
- Adaptive Timing: Dynamic timeout adjustment
4. Connection Management
enum EQStreamState {
CLOSED, // Connection terminated
CLOSING, // Graceful shutdown in progress
ESTABLISHED, // Active connection
WAIT_CLOSE // Waiting for close confirmation
};
Packet Structure System
XML-Driven Packet Definitions
<Struct Name="LS_LoginRequest" ClientVersion="1" OpcodeName="OP_LoginRequestMsg">
<Data ElementName="account_name" Type="char" Size="50"/>
<Data ElementName="password" Type="char" Size="50"/>
<Data ElementName="version" Type="int16"/>
</Struct>
Dynamic Packet Processing
- Runtime Struct Loading: XML parsing for packet definitions
- Version-specific Support: Multiple client version compatibility
- Automatic Serialization: C++ struct to packet conversion
Opcode Management
Version Strategy System
enum StrategyType {
RegularStrategy, // Standard opcode mapping
SharedStrategy, // Shared opcode tables
NullStrategy, // No-op implementation
EmptyStrategy // Empty packet handling
};
Database Architecture
Database Design Philosophy
EQ2EMu employs a sophisticated database architecture designed for high-performance MMO operations with comprehensive data integrity and concurrent access support.
Architecture Components
1. Database Abstraction Layers
Legacy Database System (database.hpp
)
- Async Query Processing: Threaded execution for high-throughput
- Connection Pooling: Multiple MySQL connections for concurrency
- Prepared Statements: SQL injection prevention
- Transaction Support: ACID compliance for critical operations
Modern Database Interface (database_new.hpp
)
- Simplified API: Clean, thread-safe MySQL wrapper
- Automatic Recovery: Connection retry with exponential backoff
- Error Handling: Comprehensive error reporting and logging
- Configuration Management: INI-based database settings
2. Database Specialization
LoginDatabase (login_database.hpp
)
- Account Management: Authentication and character data
- Version Support: Multi-client version compatibility
- Character Synchronization: Equipment appearance updates
- Security Logging: Authentication audit trails
WorldDatabase (WorldDatabase.cpp/h
)
- Game Content: Items, spells, skills, quests, achievements
- World State: Zone configurations, spawn data, faction information
- Player Progress: Character advancement, collections, housing
- Server Management: Rules, statistics, administrative functions
3. Performance Optimization
Connection Management
class DatabaseResult {
string_view field_map; // Fast field access by name
MYSQL_ROW row_data; // Raw MySQL result data
// Type-safe conversion methods
int32 GetInt32(const char* field);
float GetFloat(const char* field);
string GetString(const char* field);
};
Query Optimization
- Prepared Statement Caching: Reusable query compilation
- Result Set Streaming: Memory-efficient large result handling
- Index Strategy: Optimized database indexing for common queries
- Batch Operations: Grouped database operations for efficiency
Data Integrity
ACID Compliance
- Atomic Operations: All-or-nothing transaction processing
- Consistency: Database constraint enforcement
- Isolation: Concurrent transaction separation
- Durability: Persistent storage guarantees
Backup and Recovery
- Automated Backups: Scheduled database dumps
- Point-in-time Recovery: Transaction log-based restoration
- Replication Support: Master-slave database configurations
- Data Validation: Integrity checking and repair tools
Security Framework
Security Architecture
EQ2EMu implements comprehensive security measures across all system layers, from network protocol to database access.
Network Security
1. Encryption Systems
RC4 Stream Cipher (crypto.hpp
, rc4.hpp
)
class Crypto {
RC4 client_cipher; // Client-to-server encryption
RC4 server_cipher; // Server-to-client encryption
// Thread-safe encryption operations
void Encrypt(char* data, int32 len);
void Decrypt(char* data, int32 len);
};
Features:
- Separate Cipher Instances: Independent client/server encryption
- Session-unique Keys: Per-connection encryption keys
- Thread Safety: Concurrent encryption operations
2. Packet Integrity
CRC16 Validation (crc16.hpp
)
- Checksum Verification: Packet integrity validation
- Corruption Detection: Network transmission error detection
- Performance Optimization: Efficient CRC calculation
Anti-replay Protection
- Sequence Number Validation: Duplicate packet detection
- Window-based Acceptance: Valid sequence number ranges
- Connection State Tracking: Session hijacking prevention
Authentication Security
1. Password Security
SHA-512 Hashing (sha512.hpp
)
class SHA512 {
// Secure password hashing
static string GenerateHash(const string& input);
static bool ValidateHash(const string& input, const string& hash);
};
Features:
- Salt Integration: Rainbow table attack prevention
- Secure Storage: No plaintext password storage
- Hash Verification: Constant-time comparison operations
2. Session Management
Session Security
- Unique Session Keys: Cryptographically random session identifiers
- Session Timeout: Automatic session expiration
- IP Validation: Connection source verification
- Concurrent Session Limits: Multiple login prevention
Access Control
1. Administrative Security
Web Interface Security (web_server.hpp
)
- HTTPS/TLS Encryption: SSL certificate-based security
- Basic Authentication: HTTP Basic Auth with database integration
- Authorization Levels: Role-based access control
- Session Cookies: Secure session management
2. Database Security
SQL Injection Prevention
- Parameterized Queries: Safe parameter binding
- Input Validation: Data sanitization and validation
- Escape Functions: String escaping for dynamic queries
- Privilege Separation: Minimal database permissions
Vulnerability Mitigation
1. Input Validation
- Packet Structure Validation: Malformed packet detection
- Range Checking: Numeric value bounds validation
- String Length Limits: Buffer overflow prevention
- Character Set Validation: Invalid character filtering
2. Rate Limiting
- Connection Rate Limits: DDoS attack prevention
- Packet Rate Limits: Network flooding protection
- Authentication Attempt Limits: Brute force attack mitigation
- Resource Usage Limits: Memory and CPU protection
Performance & Scalability
Performance Architecture
EQ2EMu is designed for high-performance MMO operations supporting hundreds of concurrent players with real-time responsiveness.
Threading Model
1. Multi-threaded Architecture
Core Thread Categories:
Main Server Thread
├── Zone Threads (1 per zone)
│ ├── Spawn Processing Thread
│ ├── Initial Spawn Thread
│ └── Zone Management Thread
├── Network Threads
│ ├── EQStream Reader Thread
│ ├── EQStream Writer Thread
│ └── Packet Combine Thread
├── Database Threads (Pool)
└── Web Server Threads
2. Thread Synchronization
Advanced Mutex System (mutex.hpp
)
class Mutex {
pthread_rwlock_t rwlock; // Reader-writer locks
string lock_name; // Debug identification
Timer lock_timer; // Deadlock detection
// RAII lock management
void writelock();
void readlock();
void unlock();
};
Synchronization Strategies:
- Reader-Writer Locks: Concurrent read access with exclusive writes
- Lock Hierarchies: Deadlock prevention through lock ordering
- Timeout Management: Deadlock detection and recovery
- Fine-grained Locking: Minimal lock contention
Memory Management
1. Custom Memory Management
Safe Deletion Macros (types.hpp
)
#define safe_delete(d) if(d) { delete d; d = nullptr; }
#define safe_delete_array(d) if(d) { delete[] d; d = nullptr; }
Memory Optimization:
- Object Pooling: Reusable object instances for high-frequency allocations
- Reference Counting: Smart pointer usage in critical systems
- Cache Optimization: Data structure layout for cache efficiency
- Memory Leak Detection: Debug modes for memory tracking
2. Container Optimization
Thread-safe Containers
template<typename Key, typename Value>
class MutexMap {
Mutex map_mutex;
map<Key, Value> internal_map;
// Thread-safe operations
Value Get(Key key);
void Set(Key key, Value value);
void Remove(Key key);
};
Network Performance
1. Protocol Optimization
Packet Optimization:
- Packet Combining: Multiple small packets into single datagrams
- Compression: zlib compression for bandwidth reduction
- Adaptive Fragmentation: Dynamic fragmentation based on MTU
- Rate Limiting: Flow control and congestion management
Connection Optimization:
- Connection Pooling: Reusable network connections
- Keep-alive Management: Connection lifetime optimization
- Timeout Optimization: Adaptive timeout calculations
2. I/O Performance
Asynchronous I/O:
- Non-blocking Sockets: Event-driven network processing
- Select/Poll Integration: Efficient socket monitoring
- Buffer Management: Optimized buffer allocation and reuse
Database Performance
1. Query Optimization
Connection Pooling
class DatabaseNew {
queue<MYSQL*> available_connections;
Mutex connection_mutex;
// Connection lifecycle management
MYSQL* GetConnection();
void ReleaseConnection(MYSQL* conn);
};
Query Strategies:
- Prepared Statement Caching: Reusable query compilation
- Batch Operations: Grouped database operations
- Connection Reuse: Persistent connection management
- Index Optimization: Query performance tuning
2. Caching Systems
Multi-level Caching:
- Object Caches: Frequently accessed game objects
- Query Result Caches: Database query result caching
- Computation Caches: Expensive calculation results
- Configuration Caches: Runtime configuration data
Scalability Features
1. Horizontal Scaling
Multi-server Architecture:
- Zone Distribution: Zones across multiple server instances
- Load Balancing: Player distribution across servers
- Cross-server Communication: TCP-based inter-server protocols
- Database Sharding: Data distribution strategies
2. Vertical Scaling
Resource Optimization:
- CPU Utilization: Multi-core processing optimization
- Memory Efficiency: Optimized memory usage patterns
- I/O Optimization: Disk and network I/O efficiency
- Cache Utilization: CPU cache optimization
Extensibility & Scripting
Lua Scripting Architecture
EQ2EMu provides comprehensive Lua 5.4 integration for dynamic game content creation without requiring server code changes.
Scripting Framework
1. Lua Integration (LuaInterface.cpp/h
)
Core Architecture:
class LuaInterface {
lua_State* current_state; // Active Lua state
Mutex lua_mutex; // Thread safety
map<string, LuaSpell*> spell_list; // Active spell scripts
// 400+ exposed C++ functions
void RegisterFunctions();
bool CallFunction(const char* function, ...);
};
State Management:
- Persistent States: Lua states maintained across calls
- Error Handling: Comprehensive error reporting and recovery
- Memory Management: Automatic garbage collection integration
- Thread Safety: Mutex-protected Lua operations
2. Script Categories
Spawn Scripts
- NPC Behavior: AI logic, conversation trees, faction interactions
- Combat Scripts: Special attacks, abilities, reaction patterns
- Event Handlers: Spawn, death, hail, attack events
- Merchant Logic: Dynamic pricing, inventory management
Quest Scripts
- Dynamic Progression: Runtime quest modification
- Step Validation: Custom completion conditions
- Reward Calculation: Dynamic reward systems
- Story Branching: Conditional quest paths
Zone Scripts
- Environmental Events: Weather, day/night cycles, special events
- Zone-wide Mechanics: Transportation, zone objectives
- Player Triggers: Location-based script activation
- Zone Instance Management: Dynamic zone behavior
Item Scripts
- Special Effects: Unique item abilities and behaviors
- Usage Handlers: Right-click, examine, consume actions
- Equipment Scripts: Equip/unequip event handling
- Crafting Integration: Custom crafting behaviors
3. Lua API Exposure
400+ Lua Functions organized by category:
Entity Management
-- Entity creation and modification
local npc = SpawnMob(zone, spawn_id, x, y, z, heading)
SetHP(entity, hit_points)
SetPower(entity, power_points)
AddSpellBonus(entity, type, value)
Combat System
-- Combat mechanics
DamageSpawn(attacker, target, damage_type, damage)
HealDamage(caster, target, heal_amount)
AddControlEffect(target, effect_type, duration)
Quest System
-- Quest management
SetStepComplete(player, quest_id, step)
GiveQuestReward(player, quest_id)
HasQuest(player, quest_id)
Communication
-- Player interaction
SendMessage(player, "Hello, adventurer!")
SendPopUpMessage(player, "Quest Complete!", r, g, b)
ChatMessage(speaker, message, say_type)
Configuration System
1. XML-based Configuration (config_reader.hpp
)
Packet Structure Definition
<ConfigurationPacket>
<ConfigVersion version="25816">
<ConfigStruct name="WS_CharacterSheet">
<ConfigData name="char_id" type="int32"/>
<ConfigData name="account_id" type="int32"/>
<!-- Dynamic structure definition -->
</ConfigStruct>
</ConfigVersion>
</ConfigurationPacket>
Runtime Configuration:
- Dynamic Reloading: Configuration changes without restarts
- Version-specific Configs: Multi-client support
- Struct Expansion: Automatic substruct handling
- Array Support: Dynamic array length handling
2. JSON Configuration (json_parser.hpp
)
Server Settings
{
"auto_create_accounts": true,
"allowed_expansions": ["original", "desert_of_flames"],
"web_server": {
"port": 8080,
"ssl_cert": "/path/to/cert.pem"
}
}
3. Rules Engine (Rules/Rules.cpp/h
)
Dynamic Rule System
enum RuleCategory {
R_Client, // Client-side rules
R_Player, // Player mechanics
R_Combat, // Combat calculations
R_World, // World mechanics
R_Zone, // Zone behavior
R_PVP, // PvP mechanics
// ... 14 total categories
};
Rule Management:
- Runtime Modification: Rules changeable during operation
- Database Persistence: Rule storage in database
- Category Organization: Logical rule grouping
- Default Values: Fallback rule values
System Integration
Inter-Component Communication
EQ2EMu employs multiple communication patterns to enable seamless integration between its various components.
Communication Patterns
1. LoginServer ↔ WorldServer Integration
TCP-based Server Communication
class WorldTCPConnection : public TCPConnection {
// Server authentication and status reporting
void ProcessPacket(EQApplicationPacket* app);
void SendWorldStatus();
void SendPlayerUpdate(int32 account_id, PlayerUpdate* update);
};
Data Synchronization:
- Character Equipment Updates: Real-time appearance synchronization
- Player Status: Online/offline, zone location, level updates
- Server Statistics: Player counts, zone counts, server health
- Administrative Commands: Cross-server administrative actions
2. Database Integration Patterns
Shared Database Operations
// LoginServer database operations
LoginDatabase login_db;
login_db.LoadAccount(account_name);
login_db.SaveCharacter(character_data);
// WorldServer database operations
WorldDatabase world_db;
world_db.LoadCharacterData(character_id);
world_db.SavePlayerPosition(player);
Data Consistency:
- Character Data Synchronization: Login ↔ World character state
- Equipment Appearance: World → Login equipment updates
- Authentication State: Login → World player verification
3. Web Interface Integration
REST API Endpoints
// LoginServer endpoints
GET /status // Server status and statistics
GET /worlds // Connected world servers
// WorldServer endpoints
GET /status // Zone and player information
GET /clients // Connected players
POST /reloadrules // Dynamic rule reloading
POST /sendglobalmessage // Server announcements
Data Flow Architecture
1. Player Authentication Flow
Client → LoginServer → Database → LoginServer → Client
↓
Client → WorldServer → Database → WorldServer → Client
↓
WorldServer → LoginServer (status updates)
2. Character Management Flow
Character Creation:
Client → LoginServer → LoginDatabase → Character Storage
↓
LoginServer → WorldServer (character data sync)
Character Updates:
WorldServer → WorldDatabase → Character State
↓
WorldServer → LoginServer (appearance updates)
Event System Architecture
1. Lua Event Integration
Event Dispatching
class ZoneServer {
void HandleSpawnDeath(Spawn* spawn) {
// Trigger Lua death scripts
lua_interface->CallSpawnScript(spawn, "death", killer);
// Update quest progress
quest_manager->ProcessKillUpdate(killer, spawn);
// Handle loot generation
loot_manager->GenerateLoot(spawn, killer);
}
};
Event Categories:
- Spawn Events: Birth, death, combat, hail, examine
- Player Events: Level up, zone change, item usage
- Zone Events: Entry, exit, timer events
- Quest Events: Step completion, item collection
2. Cross-System Event Coordination
Combat Event Example:
1. Player attacks NPC
2. Combat system calculates damage
3. NPC health updated in Entity system
4. Combat scripts triggered (Lua)
5. Animation system notified
6. Client updated via network protocol
7. Quest system checks for kill objectives
8. Experience system awards points
9. Loot system processes drops
Configuration Integration
1. Centralized Configuration
Configuration Hierarchy:
Global Server Config (JSON)
├── Database Configuration
├── Network Settings
├── Web Server Settings
└── Feature Flags
Game Rules (Database)
├── Combat Rules
├── Experience Rules
├── PvP Rules
└── Zone Rules
Packet Structures (XML)
├── Login Packets
├── World Packets
└── Version-specific Variants
2. Runtime Configuration Updates
Dynamic Reloading:
- Rule Changes: Game rules updated without restart
- Packet Structures: Protocol updates for new features
- Lua Scripts: Script reloading for content updates
- Web Configuration: Administrative setting changes
Deployment & Operations
Deployment Architecture
EQ2EMu supports flexible deployment configurations ranging from single-server setups to distributed multi-server environments.
Deployment Configurations
1. Single Server Deployment
Minimal Configuration:
┌─────────────────────────────────────┐
│ Single Server │
│ ┌─────────────┐ ┌─────────────────┐│
│ │ LoginServer │ │ WorldServer ││
│ │ Port 9100 │ │ Port 9001 ││
│ └─────────────┘ └─────────────────┘│
│ │ │
│ ┌────▼──────────────────┐ │
│ │ MySQL Database │ │
│ └───────────────────────┘ │
└─────────────────────────────────────┘
Use Case: Development, testing, small private servers
2. Multi-Server Distributed Deployment
Scalable Configuration:
┌─────────────────┐
│ LoginServer │
│ Port 9100 │
└─────────┬───────┘
│
┌──────────────────┼──────────────────┐
│ │ │
┌───────▼───────┐ ┌────────▼────────┐ ┌───────▼───────┐
│ WorldServer 1 │ │ WorldServer 2 │ │ WorldServer N │
│ Port 9001 │ │ Port 9002 │ │ Port 900N │
│ Zones 1-10 │ │ Zones 11-20 │ │ Zones N1-NN │
└───────────────┘ └─────────────────┘ └───────────────┘
│ │ │
└──────────────────┼──────────────────┘
│
┌─────────▼───────┐
│ MySQL Database │
│ (Clustered) │
└─────────────────┘
Use Case: Production servers, high-player-count environments
Configuration Management
1. Configuration Files
Required Configuration Files:
server/
├── login_db.ini # LoginServer database config
├── world_db.ini # WorldServer database config
├── log_config.xml # Logging configuration
├── server_config.json # Server settings
└── CommonStructs.xml # Packet structure definitions
Sample Database Configuration (login_db.ini
):
[LoginDatabase]
server=localhost
port=3306
database=eq2emu_login
username=eq2emu
password=secretpassword
Sample Server Configuration (server_config.json
):
{
"auto_create_accounts": true,
"allowed_expansions": ["original", "desert_of_flames"],
"web_server": {
"enabled": true,
"port": 8080,
"ssl_cert": "/path/to/cert.pem",
"ssl_key": "/path/to/key.pem"
}
}
2. Build System
Makefile Targets:
# LoginServer build
cd source/LoginServer
make clean && make
# WorldServer build
cd source/WorldServer
make clean && make # Release build
make debug # Debug build with symbols
make -j$(nproc) # Parallel build
Build Dependencies:
- C++17 Compiler: GCC 7+ or Clang 5+
- MySQL Development Libraries: libmysqlclient-dev
- Lua 5.4: Lua development libraries
- Boost Libraries: Beast, Property Tree, Algorithm
- zlib: Compression library
- OpenSSL: Cryptographic functions
Monitoring & Operations
1. Web-based Monitoring
LoginServer Monitoring Endpoints:
GET /status
{
"server_status": "online",
"uptime_seconds": 86400,
"connected_clients": 45,
"world_servers": 2
}
GET /worlds
{
"worlds": [
{
"id": 1,
"name": "TestServer",
"players": 23,
"online": true,
"ip": "127.0.0.1"
}
]
}
WorldServer Monitoring Endpoints:
GET /status
{
"server_name": "TestServer",
"zone_count": 15,
"player_count": 67,
"uptime": "2 days, 14 hours, 23 minutes"
}
GET /clients
{
"clients": [
{
"name": "PlayerName",
"level": 85,
"class": "Wizard",
"zone": "Qeynos Harbor"
}
]
}
2. Logging System
Log Categories:
- WORLD__DEBUG: World server operations
- ZONE__DEBUG: Zone-specific events
- PLAYER__DEBUG: Player actions and state
- COMBAT__DEBUG: Combat calculations
- DATABASE__DEBUG: Database operations
- LUA__DEBUG: Lua script execution
- NETWORK__DEBUG: Network protocol events
Log Configuration (log_config.xml
):
<LogSettings>
<LogDestination>
<Type>console</Type>
<ColorEnabled>true</ColorEnabled>
</LogDestination>
<LogDestination>
<Type>file</Type>
<Path>logs/eq2world.log</Path>
</LogDestination>
</LogSettings>
3. Performance Monitoring
Metrics Collection:
- Connection Counts: Active client connections
- Zone Statistics: Players per zone, spawn counts
- Database Performance: Query times, connection pool usage
- Memory Usage: Heap usage, object counts
- Network Statistics: Packet rates, bandwidth usage
Administrative Operations
1. Runtime Administration
Dynamic Rule Updates:
# Via web API
curl -X POST localhost:8080/reloadrules
# Via admin command in-game
/reloadrules
Global Announcements:
# Via web API
curl -X POST localhost:8080/sendglobalmessage \
-d '{"message": "Server maintenance in 10 minutes"}'
# Via admin command
/serverwide Server maintenance in 10 minutes
2. Database Maintenance
Backup Procedures:
# Full database backup
mysqldump eq2emu_world > backup_$(date +%Y%m%d).sql
# Character data backup
mysqldump eq2emu_world characters character_details > chars_backup.sql
Schema Updates:
- Version Migration Scripts: SQL scripts for database updates
- Rollback Procedures: Safe database rollback mechanisms
- Integrity Checks: Database consistency validation
Security Operations
1. Security Monitoring
Authentication Monitoring:
- Failed Login Attempts: Brute force attack detection
- IP Blacklisting: Automatic IP blocking for suspicious activity
- Account Creation Monitoring: New account validation
Network Security:
- Connection Rate Limiting: DDoS protection
- Packet Validation: Malformed packet detection
- Protocol Compliance: EverQuest II protocol validation
2. Access Control
Administrative Access:
- Multi-level Admin Permissions: Granular access control
- Session Management: Secure administrative sessions
- Audit Logging: Administrative action tracking
Technical Specifications
System Requirements
Minimum Requirements
- OS: Linux (Ubuntu 18.04+, CentOS 7+, Debian 9+)
- CPU: 2 cores, 2.4 GHz
- RAM: 4 GB
- Storage: 20 GB available space
- Network: 100 Mbps internet connection
Recommended Requirements
- OS: Linux (Ubuntu 20.04+, CentOS 8+)
- CPU: 4+ cores, 3.0+ GHz
- RAM: 8+ GB
- Storage: 50+ GB SSD storage
- Network: 1 Gbps internet connection
High-Performance Requirements
- OS: Linux (Latest LTS distributions)
- CPU: 8+ cores, 3.5+ GHz
- RAM: 16+ GB
- Storage: 100+ GB NVMe SSD
- Network: Dedicated server with 10+ Gbps
Software Dependencies
Build Dependencies
# Ubuntu/Debian
apt-get install build-essential cmake git
apt-get install libmysqlclient-dev libssl-dev zlib1g-dev
apt-get install liblua5.4-dev libboost-all-dev
# CentOS/RHEL
yum groupinstall "Development Tools"
yum install mysql-devel openssl-devel zlib-devel
yum install lua-devel boost-devel
Runtime Dependencies
- MySQL Server: 5.7+ or MariaDB 10.3+
- Lua Runtime: Lua 5.4
- OpenSSL: 1.1.0+
- Boost Libraries: 1.65+
Performance Specifications
Server Capacity
- Concurrent Players: 500+ players per WorldServer instance
- Zone Capacity: 100+ players per zone
- Database Operations: 10,000+ queries per second
- Network Throughput: 100+ Mbps sustained traffic
- Memory Usage: 2-8 GB RAM depending on player count
Latency Requirements
- Client Response Time: <50ms for local actions
- Database Query Time: <10ms for standard queries
- Zone Transfer Time: <5 seconds for zone transitions
- Login Authentication: <2 seconds for credential verification
Network Specifications
Port Requirements
LoginServer:
- Port 9100 (UDP) - Client connections
- Port 8080 (TCP) - Web administration (configurable)
WorldServer:
- Port 9001 (UDP) - Client connections
- Port 8080 (TCP) - Web administration (configurable)
- Dynamic TCP ports for LoginServer communication
Firewall Configuration
# Allow EQ2EMu traffic
iptables -A INPUT -p udp --dport 9100 -j ACCEPT # LoginServer
iptables -A INPUT -p udp --dport 9001 -j ACCEPT # WorldServer
iptables -A INPUT -p tcp --dport 8080 -j ACCEPT # Web interface
iptables -A INPUT -p tcp --dport 3306 -j ACCEPT # MySQL (if remote)
Database Specifications
MySQL Configuration
# MySQL configuration for EQ2EMu
[mysqld]
max_connections = 500
innodb_buffer_pool_size = 2G
innodb_log_file_size = 512M
query_cache_size = 256M
tmp_table_size = 256M
max_heap_table_size = 256M
Database Schema
- Tables: 200+ tables for complete game data
- Character Data: Complete character progression and state
- World Data: Items, spells, NPCs, zones, quests
- Administrative Data: Rules, configurations, statistics
File System Layout
Installation Directory Structure
eq2emu/
├── source/ # Source code
│ ├── common/ # Shared libraries
│ ├── loginserver/ # LoginServer source
│ └── WorldServer/ # WorldServer source
├── server/ # Runtime directory
│ ├── logs/ # Log files
│ ├── scripts/ # Lua scripts
│ │ ├── SpawnScripts/ # NPC behavior scripts
│ │ ├── Quests/ # Quest scripts
│ │ └── Zones/ # Zone scripts
│ ├── *.ini # Database configuration
│ ├── *.xml # Packet structures & logging
│ └── *.json # Server configuration
└── database/ # Database schema files
├── eq2emu_login.sql # LoginServer schema
└── eq2emu_world.sql # WorldServer schema
Conclusion
Architectural Excellence
EQ2EMu represents a remarkable achievement in open-source MMO server development, demonstrating sophisticated software engineering practices across multiple domains:
Technical Achievements
Comprehensive Game Systems
- Complete MMO Feature Set: Authentication, character management, combat, magic, quests, guilds, housing, and social systems
- Advanced AI Systems: Sophisticated NPC behavior, pathfinding, and bot companions
- Extensible Content System: Lua scripting enables dynamic content creation without code changes
- Multi-client Support: Compatibility across multiple EverQuest II client versions
Robust Architecture
- Scalable Design: Multi-threaded architecture supporting hundreds of concurrent players
- Performance Optimization: Custom UDP protocol, connection pooling, and caching systems
- Thread Safety: Comprehensive synchronization with reader-writer locks and deadlock detection
- Memory Management: Custom allocation strategies and leak detection systems
Network Innovation
- Custom Protocol Implementation: UDP-based reliable delivery with RC4 encryption
- Packet Optimization: Fragmentation, compression, and combining for bandwidth efficiency
- Security Integration: Multi-layer security from network to database access
- Administrative Interface: RESTful API for monitoring and management
Software Engineering Excellence
Code Organization
- Modular Design: Clear separation between infrastructure, authentication, and game logic
- Design Patterns: Extensive use of factory, observer, and strategy patterns
- Error Handling: Comprehensive error reporting and recovery mechanisms
- Documentation: Thorough inline documentation and architectural clarity
Database Architecture
- ACID Compliance: Transaction integrity for critical game operations
- Connection Management: Sophisticated pooling and retry logic
- Performance Optimization: Query optimization and result caching
- Data Integrity: Comprehensive validation and consistency checks
Extensibility Framework
- Lua Integration: 400+ exposed functions for game world manipulation
- Configuration Management: XML and JSON-based configuration systems
- Rule Engine: Dynamic game rule modification without restarts
- Web Integration: Administrative APIs for server management
Impact and Significance
Community Contribution
EQ2EMu serves as an invaluable resource for:
- MMO Research: Academic study of massively multiplayer online game architectures
- Educational Purpose: Learning advanced C++ programming and system design
- Game Preservation: Maintaining access to classic EverQuest II gameplay
- Developer Training: Real-world experience with large-scale server systems
Technical Innovation
The project demonstrates innovative solutions in:
- Custom Network Protocols: Optimized UDP implementation for MMO requirements
- Scripting Integration: Seamless C++/Lua integration for content management
- Performance Engineering: Multi-threaded design patterns for high concurrency
- Database Optimization: Advanced caching and connection management strategies
Future Considerations
Potential Enhancements
- Container Orchestration: Docker/Kubernetes deployment support
- Microservices Architecture: Service decomposition for enhanced scalability
- Cloud Integration: Native cloud provider integration and auto-scaling
- Modern Protocol Support: WebSocket integration for web-based clients
- Enhanced Security: OAuth2/JWT integration for administrative access
Performance Scaling
- Horizontal Scaling: Multi-server load balancing and data distribution
- Database Sharding: Distributed database architecture for massive scale
- Content Delivery: CDN integration for static content distribution
- Real-time Analytics: Performance monitoring and optimization tools
Final Assessment
EQ2EMu stands as a testament to the possibilities of open-source game development, combining technical excellence with community-driven innovation. The architecture demonstrates sophisticated understanding of MMO requirements while maintaining code clarity and extensibility. With over 50,000 lines of carefully crafted C++ code, comprehensive documentation, and robust testing practices, EQ2EMu serves as both a functional server implementation and an educational resource for understanding large-scale system architecture.
The project's commitment to performance, security, and maintainability makes it an exemplary case study in software engineering excellence, providing a foundation for both current operations and future enhancements in the evolving landscape of online gaming infrastructure.
Document Information
- Title: EQ2EMu Server Architecture White Paper
- Version: 1.0
- Date: 2025
- Classification: Technical Architecture Documentation
- Authors: EQ2EMu Development Team Analysis