eq2go/MODERNIZE.md

Package Modernization Instructions

Goal

Transform legacy packages to use focused, performance-optimized bespoke master lists with simplified database operations. REMEMBER THAT THIS IS FOR A TOTAL REWRITE, NOT A VARIATION. If any step does not apply to a given package, it is okay to skip it and to focus on modernizing other areas of the package.

Steps

1. Create Bespoke MasterList Implementation

Replace generic implementations with specialized, high-performance master lists:

// Create domain-specific master list optimized for the use case
type MasterList struct {
    // Core storage
    items map[K]*Type // ID -> Type
    mutex sync.RWMutex

    // Specialized indices for O(1) lookups
    byCategory map[string][]*Type   // Category -> items
    byProperty map[string][]*Type   // Property -> items

    // Cached metadata with invalidation
    categories []string
    metaStale  bool
}

2. Consolidate Package Structure

Remove:

• Legacy wrapper functions (LoadAll, SaveAll, etc.)
• Duplicate database files (database.go, database_legacy.go)
• Generic MasterList dependencies
• README.md (use doc.go instead)
• Separate "active_record.go" files

Consolidate into:

• {type}.go - Main type with embedded database operations
• types.go - Supporting types only (no main type)
• master.go - Bespoke MasterList optimized for domain
• player.go - Player-specific logic (if applicable)
• doc.go - Primary documentation
• {type}_test.go - Focused tests
• benchmark_test.go - Comprehensive performance tests

3. Refactor Main Type

Transform the main type to include database operations:

// Before: Separate type and database operations
type Achievement struct {
    ID    uint32
    Title string
}

func LoadAchievement(db *database.Database, id uint32) (*Achievement, error)
func SaveAchievement(db *database.Database, a *Achievement) error

// After: Embedded database operations
type Achievement struct {
    ID    uint32
    Title string

    db    *database.Database
    isNew bool
}

func New(db *database.Database) *Achievement
func Load(db *database.Database, id uint32) (*Achievement, error)
func (a *Achievement) Save() error
func (a *Achievement) Delete() error
func (a *Achievement) Reload() error

4. Create Bespoke MasterList

Replace generic implementations with specialized, optimized master lists:

// Before: Generic base with limited optimization
type MasterList struct {
    *common.MasterList[uint32, *Achievement]
}

// After: Bespoke implementation optimized for domain
type MasterList struct {
    // Core storage
    achievements map[uint32]*Achievement
    mutex        sync.RWMutex

    // Domain-specific indices for O(1) lookups
    byCategory  map[string][]*Achievement
    byExpansion map[string][]*Achievement

    // Cached metadata with lazy loading
    categories  []string
    expansions  []string
    metaStale   bool
}

func (m *MasterList) AddAchievement(a *Achievement) bool {
    m.mutex.Lock()
    defer m.mutex.Unlock()

    // Check existence
    if _, exists := m.achievements[a.AchievementID]; exists {
        return false
    }

    // Add to core storage
    m.achievements[a.AchievementID] = a

    // Update specialized indices
    m.byCategory[a.Category] = append(m.byCategory[a.Category], a)
    m.byExpansion[a.Expansion] = append(m.byExpansion[a.Expansion], a)

    // Invalidate metadata cache
    m.metaStale = true
    return true
}

// O(1) category lookup
func (m *MasterList) GetByCategory(category string) []*Achievement {
    m.mutex.RLock()
    defer m.mutex.RUnlock()
    return m.byCategory[category]
}

5. Optimize for Domain Use Cases

Focus on the specific operations your domain needs:

// Implement GetID() for consistency
func (a *Achievement) GetID() uint32 {
    return a.AchievementID
}

// Add domain-specific optimized methods
func (m *MasterList) GetByCategoryAndExpansion(category, expansion string) []*Achievement {
    // Use set intersection for efficient combined queries
    categoryItems := m.byCategory[category]
    expansionItems := m.byExpansion[expansion]

    // Use smaller set for iteration efficiency
    if len(categoryItems) > len(expansionItems) {
        categoryItems, expansionItems = expansionItems, categoryItems
    }

    // Set intersection using map lookup
    expansionSet := make(map[*Achievement]struct{}, len(expansionItems))
    for _, item := range expansionItems {
        expansionSet[item] = struct{}{}
    }

    var result []*Achievement
    for _, item := range categoryItems {
        if _, exists := expansionSet[item]; exists {
            result = append(result, item)
        }
    }
    return result
}

6. Simplify API

Remove:

• Generic MasterList dependencies
• Legacy type variants (Achievement vs AchievementRecord)
• Conversion methods (ToLegacy, FromLegacy)
• Duplicate CRUD operations
• Complex wrapper functions
• Slow O(n) filter operations

Keep:

• Single type definition
• Direct database methods on type
• Domain-specific optimized operations
• O(1) indexed lookups where possible
• Lazy caching for expensive operations

7. Update Documentation

Create concise doc.go:

// Package achievements provides [brief description].
//
// Basic Usage:
//
//	achievement := achievements.New(db)
//	achievement.Title = "Dragon Slayer"
//	achievement.Save()
//
//	loaded, _ := achievements.Load(db, 1001)
//	loaded.Delete()
//
// Bespoke Master List (optimized for performance):
//
//	masterList := achievements.NewMasterList()
//	masterList.AddAchievement(achievement)
//
//	// O(1) category lookup
//	combatAchievements := masterList.GetByCategory("Combat")
//
//	// O(1) expansion lookup
//	classicAchievements := masterList.GetByExpansion("Classic")
//
//	// Optimized set intersection
//	combined := masterList.GetByCategoryAndExpansion("Combat", "Classic")
package achievements

8. Testing

Create focused tests:

• Test core type operations (New, Save, Load, Delete)
• Test MasterList basic operations
• Remove legacy compatibility tests
• Keep tests simple and direct

Key Principles

1. Single Source of Truth: One type definition, not multiple variants
2. Embedded Operations: Database methods on the type itself
3. Bespoke Master Lists: Domain-specific optimized implementations
4. Performance First: O(1) lookups, lazy caching, efficient algorithms
5. Thread Safety: Proper RWMutex usage with minimal lock contention
6. No Legacy Baggage: Remove all "Legacy" types and converters
7. Documentation in Code: Use doc.go, not README.md
8. Comprehensive Benchmarking: Measure and optimize all operations

Migration Checklist

• Identify main type and supporting types
• Consolidate database operations into main type
• Add db field and methods to main type
• Create bespoke MasterList optimized for domain
• Add specialized indices for O(1) lookups
• Implement lazy caching for expensive operations
• Add set intersection algorithms for combined queries
• Implement GetID() for consistency
• Remove all generic MasterList dependencies
• Remove all legacy types and converters
• Update doc.go with performance-focused examples
• Simplify tests to cover core functionality
• Add concurrency tests for thread safety
• Create comprehensive benchmarks (benchmark_test.go)
• Verify performance meets targets (sub-microsecond operations)
• Run go test -race to verify thread safety
• Run go fmt, go test, and go test -bench=.

Expected Results

• 80% less code in most packages
• Single type instead of multiple variants
• Thread-safe operations with optimized locking
• O(1) performance for common lookup operations
• Sub-microsecond response times for most operations
• Specialized algorithms for domain-specific queries
• Consistent API across all packages
• Better maintainability with focused, understandable code
• Performance predictability through comprehensive benchmarking

Benchmarking

After modernization, create comprehensive benchmarks to measure performance improvements and ensure no regressions:

Required Benchmarks

Core Operations:

func BenchmarkTypeCreation(b *testing.B)         // New type creation
func BenchmarkTypeOperations(b *testing.B)       // CRUD operations
func BenchmarkMasterListOperations(b *testing.B) // Collection operations

Performance Critical Paths:

func BenchmarkCoreAlgorithm(b *testing.B)      // Main business logic
func BenchmarkConcurrentAccess(b *testing.B)   // Thread safety
func BenchmarkMemoryAllocation(b *testing.B)   // Memory patterns

Comparison Benchmarks:

func BenchmarkComparisonWithOldSystem(b *testing.B) // Before/after metrics

Benchmark Structure

Use sub-benchmarks for detailed measurements:

func BenchmarkMasterListOperations(b *testing.B) {
    ml := NewMasterList()
    // Setup data...

    b.Run("Add", func(b *testing.B) {
        for i := 0; i < b.N; i++ {
            ml.Add(createTestItem(i))
        }
    })

    b.Run("Get", func(b *testing.B) {
        b.RunParallel(func(pb *testing.PB) {
            for pb.Next() {
                _ = ml.Get(randomID())
            }
        })
    })
}

Mock Implementations

Create lightweight mocks for interfaces:

type mockPlayer struct {
    level    int16
    location int32
}

func (p *mockPlayer) GetLevel() int16 { return p.level }
func (p *mockPlayer) GetLocation() int32 { return p.location }

Performance Expectations

Target performance after bespoke modernization:

• Creation operations: <100ns per operation
• ID lookup operations: <50ns per operation (O(1) map access)
• Indexed lookup operations: <100ns per operation (O(1) specialized indices)
• Combined queries: <5µs per operation (optimized set intersection)
• Cached metadata: <100ns per operation (lazy loading)
• Memory allocations: Zero allocations for read operations
• Concurrent access: Linear scaling with cores, minimal lock contention
• Specialized algorithms: Domain-optimized performance characteristics

Running Benchmarks

# Run all benchmarks
go test -bench=. ./internal/package_name

# Detailed benchmarks with memory stats
go test -bench=. -benchmem ./internal/package_name

# Compare performance over time
go test -bench=. -count=5 ./internal/package_name

# CPU profiling for optimization
go test -bench=BenchmarkCoreAlgorithm -cpuprofile=cpu.prof ./internal/package_name

Example Commands

# Remove legacy files and generic dependencies
rm README.md database_legacy.go active_record.go

# Remove generic MasterList imports
grep -r "eq2emu/internal/common" . --include="*.go" | cut -d: -f1 | sort -u | xargs sed -i '/eq2emu\/internal\/common/d'

# Rename if needed
mv active_record.go achievement.go

# Test the changes
go fmt ./...
go test ./...
go test -race ./...  # Verify thread safety
go test -bench=. ./...
go build ./...

# Performance verification
go test -bench=BenchmarkMasterListOperations -benchtime=5s ./internal/package_name
go test -bench=. -benchmem ./internal/package_name