# XML Schema Comparison: xml-schema.lua vs xml-lint.lua

## Executive Summary

This document compares the XML processing capabilities of `xml-schema.lua` and `xml-lint.lua` to identify code duplication, unique features, and optimization opportunities. Based on the analysis, we recommend consolidating validation logic into `xml-lint.lua` while keeping `xml-schema.lua` as a lightweight wrapper for specialized use cases.

## 1. Functional Overlap Analysis

### 1.1 Duplicate Functionality

| Function | xml-schema.lua | xml-lint.lua | Notes |
|----------|----------------|--------------|-------|
| **XSD Validation** | `validateXSD()` (lines 13-87) | `validateXSD()` (lines 335-337) | xml-lint.lua delegates to xml-schema |
| **DTD Validation** | `validateDTD()` (lines 90-177) | `validateDTD()` (lines 340-342) | xml-lint.lua delegates to xml-schema |
| **Error Handling** | Uses xmlUtil.errorWrapper | Uses xmlUtil.errorWrapper + custom error collection | xml-lint has additional error handling layer |

### 1.2 Implementation Differences

#### XSD Validation

- **xml-schema.lua**: Full implementation with detailed error handling, memory management, and cleanup
- **xml-lint.lua**: Simple wrapper that calls xml-schema's validateXSD through the lint() function

#### DTD Validation

- **xml-schema.lua**: Comprehensive implementation with cXML support, DOCTYPE replacement, and version synchronization
- **xml-lint.lua**: Wrapper through lint() function

## 2. Unique Features in xml-lint.lua

### 2.1 Advanced XML Processing

- **RelaxNG Validation** (lines 367-458): Complete RelaxNG schema validation
- **Schematron Validation** (lines 461-468): Placeholder for future implementation
- **XPath Queries** (lines 356-364): XPath expression evaluation
- **XInclude Processing** (lines 286-297): XInclude directive handling
- **XML Formatting** (lines 349-353): Pretty-print XML output
- **HTML Detection** (lines 570-582): Auto-detect HTML content

### 2.2 Parsing Options

```lua
-- Advanced libxml2 options (lines 166-244)
local advancedOptions = {
    load_dtd = true,           -- Load external DTD
    dtd_attribute = true,      -- Add default DTD attributes
    substitute_entity = true,   -- Substitute entities
    clean_namespace = true,    -- Clean redundant namespaces
    remove_blank_node = true,  -- Remove blank nodes
    huge = true,               -- Allow huge files
    recover = true,            -- Recover from errors
    sax = true,                -- SAX parsing mode
    stream = true,             -- Stream processing
    max_amplification = 1000   -- Security limit
}
```

### 2.3 Utility Functions

- **XML Information**:
  - `getXMLEncoding()` (lines 524-544)
  - `getXMLVersion()` (lines 548-567)
  - `isHTML()` (lines 570-582)
- **Cleaning and Processing**:
  - `cleanXML()` (lines 505-521)
  - `processXInclude()` (lines 500-502)
  - `autoLint()` (lines 585-597)

### 2.4 Error Handling Enhancements

- Custom error collection with stack traces (lines 25-59)
- Error type categorization (GENERIC, WARNING, VALIDATION)
- Test mode error formatting with colors
- Centralized error cleanup through `cleanup()` function

## 3. Unique Features in xml-schema.lua

### 3.1 Specialized DTD Processing

- **cXML Support**: Automatic DOCTYPE replacement for cXML documents
- **Version Synchronization**: Auto-update XML version to match DTD
- **Pattern-based Processing**: Uses peg patterns for XML manipulation

```lua
-- Version extraction patterns (lines 94-99)
local entityPattern = notEntity * entity * quote * peg.capture(notQuote) * quote
local versionPattern = notCxml * cxml * notVersion * version * quote * peg.capture(notQuote) * quote
```

### 3.2 Memory Management

- Explicit libxml2 cleanup calls
- Schema context management
- Document lifecycle control

## 4. Performance Comparison

### 4.1 Memory Usage

- **xml-schema.lua**: Lower memory footprint for simple validation
- **xml-lint.lua**: Higher memory usage due to additional features and error handling

### 4.2 Execution Speed

- **xml-schema.lua**: Faster for direct XSD/DTD validation
- **xml-lint.lua**: Slightly slower due to option processing and delegation

## 5. Code Quality Analysis

### 5.1 Strengths of xml-schema.lua

- Focused responsibility (schema validation only)
- Clear error handling with xmlUtil
- Good separation of concerns
- Minimal dependencies

### 5.2 Strengths of xml-lint.lua

- Comprehensive feature set
- Flexible options system
- Excellent error reporting
- Multiple parser support (xmlua + libxml2)

### 5.3 Areas for Improvement

- **Code Duplication**: DTD/XSD validation logic is duplicated
- **Dependency Chain**: xml-lint → xml-schema creates unnecessary dependency
- **Error Handling**: Two different error collection systems

## 6. Detailed Code Analysis

### 6.1 Duplicated Code Sections

#### XSD Validation in xml-lint.lua (lines 78-88)

```lua
if options.schema then
    -- XSD validation
    local isValid, errorArray = xmlSchema.validateXSD(xmlFile, options.schema)
    -- Handle output if no_out is false and validation passed
    if not options.no_out and isValid and options.output_file then
        local content = fs.readFile(xmlFile)
        if content then
            fs.writeFile(options.output_file, content)
        end
    end
    return cleanup(isValid, errorArray)
```

This is unnecessary delegation that adds complexity without value.

#### DTD Validation in xml-lint.lua (lines 89-99)

```lua
elseif options.dtd then
    -- External DTD validation
    local isValid, errorArray = xmlSchema.validateDTD(xmlFile, options.dtd, {})
    -- Handle output if no_out is false and validation passed
    if not options.no_out and isValid and options.output_file then
        local content = fs.readFile(xmlFile)
        if content then
            fs.writeFile(options.output_file, content)
        end
    end
    return cleanup(isValid, errorArray)
```

Again, unnecessary delegation with duplicated output handling.

### 6.2 Error Handling Inconsistencies

#### xml-schema.lua Error Handling

```lua
-- Simple error collection
local errorArray = xmlUtil.collectErrorArray({}) -- we need errorArray as array, not nil
if ffi.isNull(ctxt) then
    if #errorArray == 0 then
        errorArray[#errorArray + 1] = {type = 0, message = string.format("could not create schema parser context, file '%s'", XMLFileName)}
    end
    xmlUtil.shutdown()
    return false, errorArray
end
```

#### xml-lint.lua Error Handling

```lua
-- Enhanced error collection with stack traces
local function xmlError(message) -- luacheck: no global
    if errorCallCollection == nil then
        errorCallCollection = {}
    end
    if isTestMode then
        errorCallCollection[#errorCallCollection + 1] = {type = ERROR_TYPE.GENERIC, message = util.printColor("bright magenta", "test mode error: '%s'", tostring(message))}
    else
        errorCallCollection[#errorCallCollection + 1] = {type = ERROR_TYPE.GENERIC, message = util.printColor("bright red", "error: '%s'", tostring(message) .. "'\n  [" .. debug.traceback() .. "]")}
    end
end
```

The inconsistency creates confusion and makes unified error handling difficult.

## 7. Recommendations

### 7.1 Immediate Optimizations

1. **Remove Duplicate Wrappers in xml-lint.lua**
   - Delete `validateXSD()` (lines 335-337)
   - Delete `validateDTD()` (lines 340-342)
   - Update documentation to direct users to xml-schema.lua
   - Remove the require for xml-schema in lint() function

2. **Fix Error Handling Inconsistency**
   - Standardize on ERROR_TYPE constants from xml-lint.lua
   - Add these constants to xml-schema.lua
   - Use consistent error structure across both modules

3. **Optimize Dependencies**
   - Remove xml-schema import from xml-lint.lua
   - Create shared constants module if needed
   - Clarify module responsibilities in documentation

### 7.2 Code Changes Required

#### To Remove from xml-lint.lua

```lua
-- Remove these functions (lines 334-342)
function xmlLint.validateXSD(xmlFile, xsdFile)
    return lint(xmlFile, {schema = xsdFile, no_out = true, quiet = true, no_net = true})
end

function xmlLint.validateDTD(xmlFile, dtdFile)
    return lint(xmlFile, {dtd = dtdFile, no_out = true, quiet = true, no_net = true})
end

-- Remove xml-schema require (line 12)
local xmlSchema = require "xml/xml-schema" -- For validation functions
```

#### To Add to xml-lint.lua

```lua
-- Add at top of file
-- Export ERROR_TYPE constants for use by other modules
xmlLint.ERROR_TYPE = ERROR_TYPE
```

#### To Add to xml-schema.lua

```lua
-- Add at top of file
-- Error types (matching xml-lint.lua)
local ERROR_TYPE = {GENERIC = 0, WARNING = 1, VALIDATION = 2}

-- Export for other modules
xmlSchema.ERROR_TYPE = ERROR_TYPE
```

### 7.3 Long-term Architecture

1. **Create xml-core.lua** (Future)
   - Move shared validation logic
   - Consolidate error handling
   - Standardize memory management

2. **Module Reorganization** (Future)

   ```
   xml/
   ├── xml-core.lua      -- Core validation logic
   ├── xml-lint.lua      -- Advanced processing features
   ├── xml-schema.lua    -- Simple wrapper for backward compatibility
   └── xml-util.lua      -- Shared utilities
   ```

## 8. Testing Impact

### 8.1 Tests to Update

- `xml-lint-test.lua`: Remove tests for wrapper functions
- `xml-schema-test.lua`: Ensure ERROR_TYPE compatibility
- Integration tests: Verify direct xml-schema usage

### 8.2 New Tests Needed

- Test ERROR_TYPE constants consistency
- Test memory cleanup without delegation
- Test error structure compatibility

## 9. Migration Guide

### For Existing Code

```lua
-- Old pattern (will need updating)
local xmlLint = require "xml/xml-lint"
local isValid, errors = xmlLint.validateXSD("file.xml", "schema.xsd")

-- New pattern (recommended)
local xmlSchema = require "xml/xml-schema"
local isValid, errors = xmlSchema.validateXSD("file.xml", "schema.xsd")

-- Or use full lint functionality
local xmlLint = require "xml/xml-lint"
local result = xmlLint.lint("file.xml", {
    schema = "schema.xsd",
    no_out = true
})
```

## 10. Implementation Checklist

### Phase 1: Code Cleanup (1 day)

- [ ] Remove validateXSD wrapper from xml-lint.lua
- [ ] Remove validateDTD wrapper from xml-lint.lua
- [ ] Remove xml-schema require from xml-lint.lua
- [ ] Add ERROR_TYPE exports to both modules
- [ ] Update internal error handling to use ERROR_TYPE constants

### Phase 2: Documentation (1 day)

- [ ] Update xml-technical-manual.md
- [ ] Create migration guide in repository
- [ ] Update inline documentation
- [ ] Add deprecation notices to removed functions

### Phase 3: Testing (1 day)

- [ ] Run all existing tests
- [ ] Fix any test failures
- [ ] Add new tests for ERROR_TYPE compatibility
- [ ] Verify memory cleanup works correctly

## 11. Conclusion

The analysis reveals significant code duplication with xml-lint.lua providing unnecessary wrapper functions that delegate to xml-schema.lua. The optimal solution is to remove these wrappers, standardize error handling, and maintain clear separation of concerns:

- **xml-schema.lua**: Focused schema validation (XSD/DTD)
- **xml-lint.lua**: Advanced XML processing (linting, formatting, XPath, etc.)

This approach eliminates duplication, improves maintainability, and preserves the specialized features of each module while maintaining backward compatibility through clear documentation.
