add documentation

2025-09-01 13:19:28 -05:00 · 2025-09-01 13:19:28 -05:00 · 325885b06c
commit 325885b06c
parent c29ab9cb0a
2 changed files with 2273 additions and 0 deletions
--- a/EQSTREAM.md
+++ b/EQSTREAM.md
--- a/LOGIN.md
+++ b/LOGIN.md
@ -0,0 +1,817 @@
+# EverQuest II Login Server - Complete Technical Documentation
+
+## Table of Contents
+1. [Overview](#overview)
+2. [Architecture](#architecture)
+3. [Network Protocol Stack](#network-protocol-stack)
+4. [Encryption and Security](#encryption-and-security)
+5. [Packet Structure](#packet-structure)
+6. [Complete Login Flow](#complete-login-flow)
+7. [Character Management](#character-management)
+8. [World Server Communication](#world-server-communication)
+9. [Error Handling](#error-handling)
+10. [Database Operations](#database-operations)
+
+## Overview
+
+The EverQuest II Login Server is responsible for:
+- Client authentication and authorization
+- World server discovery and status management
+- Character listing and management
+- Secure handoff to world servers
+- Account management and creation
+
+### Key Components
+- **NetConnection**: Main network manager
+- **Client**: Individual client connection handler
+- **ClientList**: Thread-safe client container
+- **LWorld**: World server representation
+- **LWorldList**: World server manager
+- **LoginDatabase**: Database interface
+- **EQStream**: Network stream handler with encryption
+
+## Architecture
+
+```
+┌─────────────┐       UDP         ┌──────────────┐
+│  EQ2 Client ├─────────────────►│ Login Server │
+└─────────────┘                  └──────┬───────┘
+                                        │ TCP
+                                        ▼
+                              ┌──────────────────┐
+                              │ World Servers    │
+                              └──────────────────┘
+```
+
+### Port Configuration
+- **Default Login Port**: 5999 (UDP for game clients)
+- **Web API Port**: Configurable (TCP/HTTPS)
+- **World Server Communication**: TCP (internal)
+- **World Server Game Port**: Dynamic per world (UDP for game clients)
+
+## Network Protocol Stack
+
+### Layer Structure
+```
+Application Layer:    [Game Packets]
+Presentation Layer:   [PacketStruct Serialization]
+Session Layer:        [EQStream with RC4/CRC]
+Transport Layer:      [UDP Datagram Protocol]
+Network Layer:        [IP Protocol]
+```
+
+**Note**: EQStream implements its own reliability layer on top of UDP, providing:
+- Sequence numbers for ordering
+- Acknowledgments for reliability
+- Retransmission for lost packets
+- This gives the benefits of TCP while maintaining lower latency
+
+### EQStream Protocol
+
+The EQStream protocol provides:
+1. **Sequencing**: Packet ordering and acknowledgment
+2. **Fragmentation**: Large packet splitting
+3. **Compression**: zlib compression for efficiency
+4. **Encryption**: RC4 stream cipher
+5. **Integrity**: CRC32 checksums
+
+#### Packet Types
+```cpp
+enum EQStreamOp {
+    OP_SessionRequest    = 0x0001,  // Initial connection
+    OP_SessionResponse   = 0x0002,  // Server accepts connection
+    OP_Combined         = 0x0003,  // Multiple packets in one
+    OP_SessionDisconnect = 0x0005,  // Connection termination
+    OP_KeepAlive        = 0x0006,  // Connection heartbeat
+    OP_SessionStatRequest = 0x0007, // Statistics request
+    OP_SessionStatResponse = 0x0008,// Statistics response
+    OP_Packet           = 0x0009,  // Application data
+    OP_Fragment         = 0x000d,  // Fragmented packet piece
+    OP_Ack              = 0x0015,  // Acknowledgment
+    OP_AckFuture        = 0x0016,  // Future acknowledgment
+    OP_AckPast          = 0x0017   // Past acknowledgment
+};
+```
+
+## Encryption and Security
+
+### RC4 Encryption
+
+The login server uses RC4 stream cipher for packet encryption:
+
+1. **Key Exchange**:
+   ```cpp
+   // Initial session setup
+   struct SessionRequest {
+       uint32_t unknown;       // Protocol version
+       uint32_t session_id;    // Client session ID
+       uint32_t max_length;    // Max packet size
+   };
+   
+   struct SessionResponse {
+       uint32_t session_id;    // Server session ID
+       uint32_t key;          // RC4 key seed
+       uint8_t  crc_length;   // CRC bytes (2)
+       uint8_t  compression;  // Compression flag
+       uint32_t unknown;      // Reserved
+       uint32_t max_length;   // Max packet size
+   };
+   ```
+
+2. **RC4 Key Generation**:
+   ```cpp
+   // Key is derived from session response
+   void GenerateRC4Key(uint32_t key_seed) {
+       // Initialize RC4 state with key_seed
+       RC4_KEY encrypt_key, decrypt_key;
+       unsigned char key_bytes[4];
+       memcpy(key_bytes, &key_seed, 4);
+       
+       // Setup encryption/decryption keys
+       RC4_set_key(&encrypt_key, 4, key_bytes);
+       RC4_set_key(&decrypt_key, 4, key_bytes);
+   }
+   ```
+
+3. **Packet Encryption**:
+   ```cpp
+   void EncryptPacket(uint8_t* data, size_t length) {
+       // Skip protocol header (2 bytes)
+       RC4(&encrypt_key, length - 2, data + 2, data + 2);
+   }
+   ```
+
+### CRC32 Integrity Check
+
+Every packet includes a CRC32 checksum:
+
+```cpp
+struct EQProtocolPacket {
+    uint16_t opcode;        // Operation code
+    uint8_t  data[];        // Packet data
+    uint32_t crc32;         // CRC32 of opcode + data
+};
+
+uint32_t CalculateCRC(const uint8_t* data, size_t length) {
+    // Standard CRC32 calculation
+    return crc32(0, data, length);
+}
+```
+
+### CRC Verification Process:
+1. Extract CRC from packet end
+2. Calculate CRC of packet data
+3. Compare calculated vs received CRC
+4. Reject packet if mismatch
+
+## Packet Structure
+
+### Base Packet Header
+```cpp
+struct EQ2Packet {
+    uint16_t opcode;        // Operation identifier
+    uint16_t sequence;      // Packet sequence number
+    uint32_t size;          // Payload size
+    uint8_t  compressed;    // Compression flag
+    uint8_t  data[];        // Actual payload
+};
+```
+
+### Application Packet Structure
+```cpp
+struct EQApplicationPacket {
+    uint16_t opcode;        // Game operation code
+    uint32_t version;       // Client version
+    uint8_t  data[];        // Serialized data
+};
+```
+
+## Complete Login Flow
+
+### Phase 1: Connection Establishment
+
+1. **Client Initiates UDP Communication**:
+   ```
+   Client → Server: UDP Datagram with OP_SessionRequest
+   (No TCP handshake - connectionless protocol)
+   ```
+
+2. **EQStream Session**:
+   ```
+   Client → Server: OP_SessionRequest
+   {
+       protocol_version: 0x0003
+       session_id: random()
+       max_packet_size: 512
+   }
+   
+   Server → Client: OP_SessionResponse
+   {
+       session_id: server_session
+       rc4_key: generated_key
+       crc_length: 2
+       compression: true
+       max_packet_size: 512
+   }
+   ```
+
+3. **Encryption Initialization**:
+   - Both sides initialize RC4 with the shared key
+   - All subsequent packets are encrypted
+
+### Phase 2: Authentication
+
+1. **Login Request**:
+   ```cpp
+   Client → Server: OP_LoginRequestMsg
+   {
+       version: client_version      // e.g., 1208
+       username: "account_name"      // EQ2_16BitString
+       password: "hashed_password"   // EQ2_16BitString
+       unknown3: cl_eqversion        // From eq2_defaults.ini
+   }
+   ```
+
+2. **Version Validation**:
+   ```cpp
+   // Server checks version compatibility
+   if (!EQOpcodeManager.count(GetOpcodeVersion(version))) {
+       // Send incompatible version error
+       SendLoginDeniedBadVersion();
+       return;
+   }
+   ```
+
+3. **Account Verification**:
+   ```cpp
+   // Database lookup
+   LoginAccount* acct = database.LoadAccount(username, password);
+   
+   // Check for duplicate login
+   Client* existing = client_list.FindByLSID(acct->getLoginAccountID());
+   if (existing) {
+       existing->getConnection()->SendDisconnect();
+   }
+   
+   // Update account info
+   database.UpdateAccountIPAddress(acct->id, client_ip);
+   database.UpdateAccountClientDataVersion(acct->id, version);
+   ```
+
+4. **Login Response**:
+   ```cpp
+   Server → Client: OP_LoginReplyMsg
+   {
+       account_id: account_id
+       login_response: 0            // 0 = success
+       sub_level: 0xFFFFFFFF       // Subscription level
+       race_flag: 0x1FFFFF         // Available races
+       class_flag: 0x7FFFFFE       // Available classes
+       expansion_flag: 0x7CFF      // Enabled expansions
+       cities_flag: 0xFF           // Starting cities
+       enabled_races: 0xFFFF       // Race availability
+   }
+   ```
+
+### Phase 3: World List Request
+
+1. **Client Requests World List**:
+   ```cpp
+   Client → Server: OP_AllWSDescRequestMsg
+   {}  // Empty request
+   ```
+
+2. **Server Builds World List**:
+   ```cpp
+   Server → Client: OP_AllWSDescReplyMsg
+   {
+       num_worlds: count
+       worlds[]: {
+           world_id: id
+           world_name: "ServerName"
+           world_status: 1          // 0=down, 1=up, 2=locked
+           num_players: current
+           max_players: maximum
+           language: "en"
+           recommended: false
+       }
+   }
+   ```
+
+3. **World Status Updates**:
+   - Server sends periodic updates (every 10 seconds)
+   - Updates include player counts and status changes
+
+### Phase 4: Character List
+
+1. **Character Loading**:
+   ```cpp
+   // Server loads characters from database
+   database.LoadCharacters(GetLoginAccount(), GetVersion());
+   ```
+
+2. **Character List Response**:
+   ```cpp
+   Server → Client: LS_CharSelectList
+   {
+       account_id: account_id
+       num_characters: count
+       characters[]: {
+           char_id: database_id
+           server_id: world_id
+           name: "CharacterName"
+           race: race_id
+           class: class_id
+           level: current_level
+           zone: "ZoneName"
+           gender: 0/1
+           deity: deity_id
+           body_size: size
+           body_age: age
+           soga_race_type: soga_id
+           x: position_x
+           y: position_y
+           z: position_z
+       }
+   }
+   ```
+
+### Phase 5: Character Creation
+
+1. **Creation Request**:
+   ```cpp
+   Client → Server: OP_CreateCharacterRequestMsg
+   {
+       server_id: target_world
+       name: "NewCharacter"
+       race: selected_race
+       class: selected_class
+       gender: 0/1
+       deity: selected_deity
+       body_size: size_value
+       body_age: age_value
+       // Appearance data...
+   }
+   ```
+
+2. **Forward to World Server**:
+   ```cpp
+   Login → World: ServerOP_CharacterCreate
+   {
+       version: client_version
+       account_id: account_id
+       [original_request_data]
+   }
+   ```
+
+3. **World Server Validation**:
+   - Check name availability
+   - Validate race/class combination
+   - Check character limits
+
+4. **Creation Response**:
+   ```cpp
+   World → Login: ServerOP_CharacterCreateResponse
+   {
+       success: true/false
+       char_id: new_character_id   // If successful
+       reason: error_code          // If failed
+   }
+   
+   Login → Client: OP_CreateCharacterReplyMsg
+   {
+       response: CREATESUCCESS_REPLY or error_code
+       name: "NewCharacter"
+       account_id: account_id
+   }
+   ```
+
+### Phase 6: World Entry
+
+1. **Play Request**:
+   ```cpp
+   Client → Server: OP_PlayCharacterRequestMsg
+   {
+       char_id: selected_character
+       server_id: world_server      // Version > 283
+       name: "CharacterName"        // Version <= 283
+   }
+   ```
+
+2. **World Server Handoff**:
+   ```cpp
+   Login → World: ServerOP_UsertoWorldReq
+   {
+       char_id: character_id
+       lsaccountid: account_id
+       worldid: server_id
+       ip_address: client_ip
+   }
+   ```
+
+3. **World Server Response**:
+   ```cpp
+   World → Login: ServerOP_UsertoWorldResp
+   {
+       worldid: server_id
+       response: 1                  // 1 = success
+       ip_address: world_ip
+       port: world_port
+       access_key: session_key      // For authentication
+   }
+   ```
+
+4. **Client Redirect**:
+   ```cpp
+   Login → Client: OP_PlayCharacterReplyMsg
+   {
+       response: 1                  // Success
+       server: "world.ip.address"
+       port: world_port
+       account_id: account_id
+       access_code: session_key
+   }
+   ```
+
+5. **Client Connects to World**:
+   - Client disconnects from login server
+   - Connects to world server using provided IP:port
+   - Authenticates using access_key
+
+## Character Management
+
+### Character Deletion
+
+1. **Delete Request**:
+   ```cpp
+   Client → Server: OP_DeleteCharacterRequestMsg
+   {
+       char_id: character_id
+       server_id: world_id
+       name: "CharacterName"
+   }
+   ```
+
+2. **Verification**:
+   ```cpp
+   // Verify ownership
+   bool valid = database.VerifyDelete(account_id, char_id, name);
+   ```
+
+3. **World Server Notification**:
+   ```cpp
+   Login → World: ServerOP_DeleteCharacter
+   {
+       char_id: character_id
+       account_id: account_id
+   }
+   ```
+
+4. **Delete Response**:
+   ```cpp
+   Server → Client: OP_DeleteCharacterReplyMsg
+   {
+       response: 1                  // 1 = success
+       char_id: deleted_id
+       server_id: world_id
+       name: "CharacterName"
+   }
+   ```
+
+## World Server Communication
+
+### Inter-Server Protocol
+
+1. **World Server Registration**:
+   ```cpp
+   World → Login: ServerOP_LSInfo
+   {
+       name: "WorldName"
+       address: "public_ip"
+       port: game_port
+       admin_port: admin_port
+       status: 1                    // 0=down, 1=up, 2=locked
+       players: current_count
+       max_players: maximum
+   }
+   ```
+
+2. **Heartbeat**:
+   ```cpp
+   World → Login: ServerOP_LSStatus
+   {
+       status: current_status
+       players: current_count
+       zones: active_zones
+   }
+   // Sent every 30 seconds
+   ```
+
+3. **Player Updates**:
+   ```cpp
+   World → Login: ServerOP_UsertoWorldResp
+   {
+       lsaccountid: account_id
+       worldid: server_id
+       response: status
+       // 0 = rejected
+       // 1 = accepted
+       // -1 = world full
+       // -2 = character not found
+       // -3 = world locked
+   }
+   ```
+
+## Error Handling
+
+### Login Error Codes
+```cpp
+enum LoginResponseCodes {
+    LOGIN_SUCCESS = 0,
+    LOGIN_BADPASS = 1,              // Invalid username/password
+    LOGIN_BADVERSION = 6,           // Client version mismatch
+    LOGIN_SUSPENDED = 7,            // Account suspended
+    LOGIN_BANNED = 9,               // Account banned
+    LOGIN_WORLDFULL = 10,           // World at capacity
+    LOGIN_DISCONNECT = 100          // Generic disconnect
+};
+```
+
+### Play Error Codes
+```cpp
+enum PlayResponseCodes {
+    PLAY_SUCCESS = 1,
+    PLAY_ERROR_PROBLEM = 2,         // Generic error
+    PLAY_ERROR_WORLDFULL = 3,       // World full
+    PLAY_ERROR_LOCKED = 4,          // World locked
+    PLAY_ERROR_BANNED = 5,          // Character banned
+    PLAY_ERROR_SUSPENDED = 6,       // Character suspended
+    PLAY_ERROR_SERVER_TIMEOUT = 7,  // Timeout waiting for world
+    PLAY_ERROR_CHAR_NOT_LOADED = 8  // Character data issue
+};
+```
+
+### Character Creation Error Codes
+```cpp
+enum CreateResponseCodes {
+    CREATESUCCESS_REPLY = 1,
+    INVALIDRACE_REPLY = 2,          // Invalid race selection
+    INVALIDGENDER_REPLY = 3,        // Invalid gender
+    BADNAMELENGTH_REPLY = 9,        // Name too short/long
+    NAMEINVALID_REPLY = 10,         // Invalid characters
+    NAMEFILTER_REPLY = 11,          // Profanity filter
+    NAMETAKEN_REPLY = 12,           // Name already exists
+    OVERLOADEDSERVER_REPLY = 13,    // Server overloaded
+    UNKNOWNERROR_REPLY = 20         // Generic error
+};
+```
+
+## Database Operations
+
+### Account Management
+
+1. **Account Loading**:
+   ```sql
+   SELECT id, name, password, suspended, banned 
+   FROM login_accounts 
+   WHERE name = ? AND password = SHA2(?, 256)
+   ```
+
+2. **Account Creation** (if enabled):
+   ```sql
+   INSERT INTO login_accounts (name, password, created) 
+   VALUES (?, SHA2(?, 256), NOW())
+   ```
+
+3. **IP Address Update**:
+   ```sql
+   UPDATE login_accounts 
+   SET last_ip = ?, last_login = NOW() 
+   WHERE id = ?
+   ```
+
+### Character Operations
+
+1. **Character List**:
+   ```sql
+   SELECT c.id, c.name, c.server_id, c.level, c.race, 
+          c.class, c.zone, c.gender, c.deity
+   FROM characters c
+   WHERE c.account_id = ? AND c.deleted = 0
+   ORDER BY c.server_id, c.name
+   ```
+
+2. **Character Creation**:
+   ```sql
+   INSERT INTO characters 
+   (account_id, server_id, name, race, class, gender, 
+    deity, body_size, body_age, created)
+   VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, NOW())
+   ```
+
+3. **Character Deletion**:
+   ```sql
+   UPDATE characters 
+   SET deleted = 1, deleted_date = NOW() 
+   WHERE id = ? AND account_id = ?
+   ```
+
+### World Server Tracking
+
+1. **World Registration**:
+   ```sql
+   INSERT INTO login_worldservers 
+   (name, ip_address, port, status) 
+   VALUES (?, ?, ?, ?)
+   ON DUPLICATE KEY UPDATE 
+   ip_address = VALUES(ip_address),
+   port = VALUES(port),
+   status = VALUES(status)
+   ```
+
+2. **Status Updates**:
+   ```sql
+   UPDATE login_worldservers 
+   SET status = ?, players = ?, last_update = NOW() 
+   WHERE id = ?
+   ```
+
+## Security Considerations
+
+### Password Handling
+- Passwords are hashed using SHA256 before transmission
+- Never store plaintext passwords
+- Use prepared statements to prevent SQL injection
+
+### Session Management
+- Generate unique session keys for world handoff
+- Session keys expire after use or timeout
+- IP address validation between login and world entry
+
+### Rate Limiting
+- Limit login attempts per IP
+- Throttle character creation requests
+- Monitor for abnormal packet patterns
+
+### Packet Validation
+- Verify packet size limits
+- Check sequence numbers for replay attacks
+- Validate all input data ranges
+
+## Performance Optimizations
+
+### Connection Pooling
+- Maintain persistent database connections
+- Reuse world server connections
+- Implement connection timeout and retry logic
+
+### Caching
+- Cache world server status for 10 seconds
+- Cache character lists until modified
+- Cache opcode mappings per version
+
+### Threading Model
+- Main thread handles network I/O
+- Database operations on thread pool
+- World server communication async
+- Client processing single-threaded per client
+
+## Monitoring and Logging
+
+### Key Metrics
+- Active client connections
+- Login success/failure rates
+- World server availability
+- Database query performance
+- Packet processing latency
+
+### Log Levels
+```cpp
+enum LogLevel {
+    LOGIN__INFO,     // General information
+    LOGIN__DEBUG,    // Detailed debugging
+    LOGIN__ERROR,    // Error conditions
+    LOGIN__WARNING,  // Warning conditions
+    WORLD__INFO,     // World server info
+    WORLD__ERROR,    // World server errors
+    OPCODE__DEBUG,   // Packet opcodes
+    INIT__INFO,      // Initialization
+    INIT__ERROR      // Init failures
+};
+```
+
+### Critical Events to Log
+- Failed login attempts
+- Account creation
+- Character creation/deletion
+- World server registration/deregistration
+- Network errors and timeouts
+- Database connection issues
+
+## Configuration
+
+### Main Configuration (config.json)
+```json
+{
+    "loginconfig": {
+        "serverport": 5999,
+        "serverip": "",
+        "accountcreation": 1,
+        "expansionflag": 32463,
+        "citiesflag": 255,
+        "defaultsubscriptionlevel": -1,
+        "enabledraces": 65535,
+        "webloginaddress": "0.0.0.0",
+        "webloginport": 8080,
+        "webcertfile": "",
+        "webkeyfile": "",
+        "webkeypassword": "",
+        "webhardcodeuser": "admin",
+        "webhardcodepassword": "password"
+    }
+}
+```
+
+### Database Configuration
+```json
+{
+    "database": {
+        "host": "localhost",
+        "port": 3306,
+        "username": "eq2login",
+        "password": "password",
+        "database": "eq2login",
+        "max_connections": 10,
+        "connection_timeout": 5
+    }
+}
+```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **"Version Mismatch" Error**:
+   - Client version not in opcode database
+   - Solution: Update opcodes.sql
+
+2. **"Cannot Connect to Login Server"**:
+   - Firewall blocking port 5999
+   - Server not running
+   - Network configuration issue
+
+3. **"World Server Unavailable"**:
+   - World server not registered
+   - World server crashed
+   - Network issue between login and world
+
+4. **Character Creation Fails**:
+   - Name already taken
+   - Invalid race/class combination
+   - World server communication timeout
+
+5. **Cannot Enter World**:
+   - Session key mismatch
+   - World server full
+   - Character data corruption
+
+### Debug Commands
+
+The login server supports console commands (Linux):
+- `l` - List all connected world servers
+- `v` - Display version information
+- `h` - Show help menu
+- `q` - Quit server gracefully
+
+## Protocol Evolution
+
+### Version Differences
+
+**Version <= 283**:
+- Server ID not included in play request
+- Character name used for lookup
+
+**Version <= 561**:
+- Auto-enter world after character creation
+- Different packet structure alignment
+
+**Version >= 546**:
+- Enhanced error logging format
+- Larger packet size support
+
+**Version >= 1208**:
+- New login request structure
+- Additional client version field
+
+## Conclusion
+
+The EverQuest II login server implements a sophisticated authentication and routing system with multiple layers of security and error handling. The combination of RC4 encryption, CRC32 integrity checking, and session-based authentication provides a robust framework for managing player connections and character data.
+
+Key design principles:
+- **Security First**: All communications encrypted
+- **Scalability**: Support for multiple world servers
+- **Reliability**: Comprehensive error handling
+- **Performance**: Efficient caching and threading
+- **Maintainability**: Clear separation of concerns
+
+The login server acts as the gateway to the game world, ensuring only authenticated players can access their characters while maintaining the integrity and security of the game environment.