eq2go/internal/languages/README.md

# Languages Package

The languages package provides comprehensive multilingual character communication system for EverQuest II server emulation. It manages language learning, chat processing, and player language knowledge with thread-safe operations and efficient lookups.

## Features

- **Master Language Registry**: Global list of all available languages with ID-based and name-based lookups
- **Player Language Management**: Individual player language collections with learning/forgetting mechanics
- **Chat Processing**: Multilingual message processing with scrambling for unknown languages
- **Database Integration**: Language persistence with player-specific language knowledge
- **Thread Safety**: All operations use proper Go concurrency patterns with mutexes
- **Performance**: Optimized hash table lookups with benchmark results ~15ns per operation
- **EQ2 Compatibility**: Supports all standard EverQuest II racial languages

## Core Components

### Language Types

```go
// Core language representation
type Language struct {
    id         int32        // Unique language identifier
    name       string       // Language name
    saveNeeded bool         // Whether this language needs database saving
    mutex      sync.RWMutex // Thread safety
}

// Master language registry (system-wide)
type MasterLanguagesList struct {
    languages map[int32]*Language  // Languages indexed by ID
    nameIndex map[string]*Language // Languages indexed by name
    mutex     sync.RWMutex         // Thread safety
}

// Player-specific language collection
type PlayerLanguagesList struct {
    languages map[int32]*Language  // Player's languages indexed by ID
    nameIndex map[string]*Language // Player's languages indexed by name
    mutex     sync.RWMutex         // Thread safety
}
```

### Management System

```go
// High-level language system manager
type Manager struct {
    masterLanguagesList *MasterLanguagesList
    database            Database
    logger              Logger
    // Statistics tracking
    languageLookups      int64
    playersWithLanguages int64
    languageUsageCount   map[int32]int64
}
```

### Integration Interfaces

```go
// Entity interface for basic entity operations
type Entity interface {
    GetID() int32
    GetName() string
    IsPlayer() bool
    IsNPC() bool
    IsBot() bool
}

// Player interface extending Entity for player-specific operations
type Player interface {
    Entity
    GetCharacterID() int32
    SendMessage(message string)
}

// LanguageAware interface for entities with language capabilities
type LanguageAware interface {
    GetKnownLanguages() *PlayerLanguagesList
    KnowsLanguage(languageID int32) bool
    GetPrimaryLanguage() int32
    SetPrimaryLanguage(languageID int32)
    CanUnderstand(languageID int32) bool
    LearnLanguage(languageID int32) error
    ForgetLanguage(languageID int32) error
}
```

## Standard EQ2 Languages

The package includes all standard EverQuest II racial languages:

```go
const (
    LanguageIDCommon   = 0  // Common tongue (default)
    LanguageIDElvish   = 1  // Elvish
    LanguageIDDwarven  = 2  // Dwarven
    LanguageIDHalfling = 3  // Halfling
    LanguageIDGnomish  = 4  // Gnomish
    LanguageIDIksar    = 5  // Iksar
    LanguageIDTrollish = 6  // Trollish
    LanguageIDOgrish   = 7  // Ogrish
    LanguageIDFae      = 8  // Fae
    LanguageIDArasai   = 9  // Arasai
    LanguageIDSarnak   = 10 // Sarnak
    LanguageIDFroglok  = 11 // Froglok
)
```

## Usage Examples

### Basic Setup

```go
// Create language system
database := NewMockDatabase() // or your database implementation
logger := NewMockLogger()     // or your logger implementation
manager := NewManager(database, logger)

// Add standard EQ2 languages
languages := map[int32]string{
    LanguageIDCommon: "Common",
    LanguageIDElvish: "Elvish",
    LanguageIDDwarven: "Dwarven",
    // ... add other languages
}

for id, name := range languages {
    lang := NewLanguage()
    lang.SetID(id)
    lang.SetName(name)
    manager.AddLanguage(lang)
}
```

### Player Integration

```go
// Player struct implementing LanguageAware
type Player struct {
    id              int32
    name            string
    languageAdapter *PlayerLanguageAdapter
}

func NewPlayer(id int32, name string, languageManager *Manager, logger Logger) *Player {
    return &Player{
        id:              id,
        name:            name,
        languageAdapter: NewPlayerLanguageAdapter(languageManager, logger),
    }
}

// Implement LanguageAware interface by delegating to adapter
func (p *Player) GetKnownLanguages() *PlayerLanguagesList {
    return p.languageAdapter.GetKnownLanguages()
}

func (p *Player) KnowsLanguage(languageID int32) bool {
    return p.languageAdapter.KnowsLanguage(languageID)
}

func (p *Player) LearnLanguage(languageID int32) error {
    return p.languageAdapter.LearnLanguage(languageID)
}

// ... implement other LanguageAware methods
```

### Chat Processing

```go
// Create chat processor
processor := NewChatLanguageProcessor(manager, logger)

// Process message from speaker
message := "Mae govannen!" // Elvish greeting
processed, err := processor.ProcessMessage(speaker, message, LanguageIDElvish)
if err != nil {
    // Handle error (speaker doesn't know language, etc.)
}

// Filter message for listener based on their language knowledge
filtered := processor.FilterMessage(listener, processed, LanguageIDElvish)
// If listener doesn't know Elvish, message would be scrambled

// Language scrambling for unknown languages
scrambled := processor.GetLanguageSkramble(message, 0.0) // 0% comprehension
// Returns scrambled version: "Nhl nbchuulu!"
```

### Database Operations

```go
// Player learning languages with database persistence
player := NewPlayer(123, "TestPlayer", manager, logger)

// Learn a language
err := player.LearnLanguage(LanguageIDElvish)
if err != nil {
    log.Printf("Failed to learn language: %v", err)
}

// Save to database
err = player.languageAdapter.SavePlayerLanguages(database, player.GetCharacterID())
if err != nil {
    log.Printf("Failed to save languages: %v", err)
}

// Load from database (e.g., on player login)
newPlayer := NewPlayer(123, "TestPlayer", manager, logger)
err = newPlayer.languageAdapter.LoadPlayerLanguages(database, player.GetCharacterID())
if err != nil {
    log.Printf("Failed to load languages: %v", err)
}
```

### Statistics and Management

```go
// Get system statistics
stats := manager.GetStatistics()
fmt.Printf("Total Languages: %d\\n", stats.TotalLanguages)
fmt.Printf("Players with Languages: %d\\n", stats.PlayersWithLanguages)
fmt.Printf("Language Lookups: %d\\n", stats.LanguageLookups)

// Process management commands
result, err := manager.ProcessCommand("stats", []string{})
if err != nil {
    log.Printf("Command failed: %v", err)
} else {
    fmt.Println(result)
}

// Validate all languages
issues := manager.ValidateAllLanguages()
if len(issues) > 0 {
    log.Printf("Language validation issues: %v", issues)
}
```

## Database Interface

Implement the `Database` interface to provide persistence:

```go
type Database interface {
    LoadAllLanguages() ([]*Language, error)
    SaveLanguage(language *Language) error
    DeleteLanguage(languageID int32) error
    LoadPlayerLanguages(playerID int32) ([]*Language, error)
    SavePlayerLanguage(playerID int32, languageID int32) error
    DeletePlayerLanguage(playerID int32, languageID int32) error
}
```

## Performance

Benchmark results on AMD Ryzen 7 8845HS:

- **Language Lookup**: ~15ns per operation
- **Player Language Adapter**: ~8ns per operation
- **Integrated Operations**: ~16ns per operation

The system uses hash tables for O(1) lookups and is optimized for high-frequency operations.

## Thread Safety

All operations are thread-safe using:
- `sync.RWMutex` for read-heavy operations (language lookups)
- `sync.Mutex` for write operations and statistics
- Atomic operations where appropriate

## System Limits

```go
const (
    MaxLanguagesPerPlayer = 100  // Reasonable limit per player
    MaxTotalLanguages     = 1000 // System-wide language limit
    MaxLanguageNameLength = 50   // Maximum language name length
)
```

## Command Interface

The manager supports administrative commands:

- `stats` - Show language system statistics
- `list` - List all languages
- `info <id|name>` - Show language information
- `validate` - Validate all languages
- `reload` - Reload from database
- `add <id> <name>` - Add new language
- `remove <id>` - Remove language
- `search <term>` - Search languages by name

## Testing

Comprehensive test suite includes:

- Unit tests for all core components
- Integration tests with mock implementations
- Performance benchmarks
- Entity integration examples
- Thread safety validation

Run tests:
```bash
go test ./internal/languages -v
go test ./internal/languages -bench=.
```

## Integration Notes

1. **Entity Integration**: Players should embed `PlayerLanguageAdapter` and implement `LanguageAware`
2. **Database**: Implement the `Database` interface for persistence
3. **Logging**: Implement the `Logger` interface for system logging
4. **Chat System**: Use `ChatLanguageProcessor` for multilingual message handling
5. **Race Integration**: Initialize players with their racial languages at character creation

The languages package follows the same architectural patterns as other EQ2Go systems and integrates seamlessly with the entity, player, and chat systems.