# LPeg Minimal Guide: Finding Partial Strings

LPeg is NOT regex. Use these patterns instead:

- <https://www.inf.puc-rio.br/~roberto/lpeg/>
- <https://www.inf.puc-rio.br/~roberto/lpeg/slides-lpeg-workshop2008.pdf>
- <http://lua-users.org/wiki/LpegTutorial>
- <http://leafo.net/guides/parsing-expression-grammars.html>
- <https://medium.com/@brynne8/lpeg-and-peg-practices-b3d0fc00457e>

## MOST IMPORTANT: USE peg.patternNotPattern()

### WHY patternNotPattern is REQUIRED

**THE FUNDAMENTAL PROBLEM:**

Files are human-written. Humans write inconsistently. Entries appear with space delimiters OR colon delimiters OR no delimiters. Times appear as "9:05-16:20", "9:05 - 16:20", or "9:05:30-16:20:45" with seconds. Format variation is infinite.

**WHY SIMPLE PATTERNS FAIL:**

A pattern like `dash * dayNameCapture * dateCapture` expects day name to be immediately followed by valid date. When input is "- sat: 2025-03-15", the parser sees "sat:" - the colon is NOT a valid date character, so dateCapture fails. The parse fails completely.

You would need separate patterns for every delimiter variation: space, colon, multiple spaces, tab, no space. This is impossible - infinite combinations.

**THE NOTPATTERN SOLUTION:**

notPattern means "match anything EXCEPT this thing, keep going until you find it." The pattern `notDatePattern * dateCapture` tells parser: skip over characters that are NOT valid date, then capture date when found.

This single pattern handles ALL delimiter variations. For "- sat:2025-03-15", notDatePattern skips colon and spaces, then dateCapture captures "2025-03-15". Same pattern works for "- sat 2025-03-15". One pattern, all variations handled.

**FOR TIME EXTRACTION:**

To extract start and end times from "15:50-19:01", you need notTimePattern to skip between the two times. The pattern `notTimePattern * timeCapture * notTimePattern * timeCapture` means: capture first time, skip everything NOT time (the dash), capture second time. Works for "15:50-19:01", "15:50 - 19:01", or "15:50 -19:01" because notTimePattern skips all non-time characters between.

**THIS IS WHY IT IS CRITICAL:**

Without notPattern, pattern parsing FAILS on all but perfectly formatted input. With notPattern, one pattern handles all format variations. notPattern is ESSENTIAL for ANY pattern that needs to skip delimiters to find the target pattern to capture.

### What patternNotPattern returns

`peg.patternNotPattern` returns TWO patterns:

1. **pattern** - matches the specific text exactly
2. **notPattern** - matches anything EXCEPT the specific text, stopping BEFORE it

#### Function Signature

```lua
peg.patternNotPattern(pattern, countLeast, countMost, countAntiLeast)
```

**Parameters:**

- `pattern`: The pattern to match (string or pattern)
- `countLeast`: Minimum repetitions of pattern (default 1)
- `countMost`: Maximum repetitions of pattern (default 1)
- `countAntiLeast`: Minimum repetitions of anti-pattern (default 1)

**Returns:**

- pattern: matches 'pattern' with optional repetition counts
- notPattern: matches anything EXCEPT 'pattern'

#### Parameter Examples

1. **Basic usage (default)**: `peg.patternNotPattern(":")`
   - pattern: matches ":" exactly once
   - notPattern: matches anything except ":"

2. **Optional patterns (countLeast = 0)**: `peg.patternNotPattern(" ", 0)`
   - pattern: matches " " zero or more times
   - notPattern: matches anything except " "

3. **Range of repetitions (countLeast, countMost)**: `peg.patternNotPattern(peg.range("09"), 2, 4)`
   - pattern: matches 2-4 digits
   - notPattern: matches anything except digits

4. **Minimum anti-pattern length (countAntiLeast)**: `peg.patternNotPattern(peg.define.separator, 0, nil, 2)`
   - pattern: matches zero or more separators
   - notPattern: matches anything except separators (needs at least 1 char)

### Pattern Building Rules

1. **ONLY peg.patternNotPattern for character patterns**
2. **Use notPattern to match up to but NOT including text**
3. **Use peg.least() only for repetitions of defined patterns**
4. **ALL patterns at file top** - no inline pattern building
5. **NO string concatenation** - use PEG operators (*, +) only
6. **NEVER use peg.define.anyChar with peg.least()**

```lua
-- Match content between quotes using anti-pattern
local quotePattern, notQuotePattern = peg.patternNotPattern('"')
local quotedContent = quotePattern * notQuotePattern * quotePattern
-- Remove DOCTYPE declarations: "<!DOCTYPE" + (anything except ">") + ">"
local gtPattern, notGtPattern = peg.patternNotPattern(">")
local doctypePattern = peg.pattern("<!DOCTYPE") * notGtPattern * gtPattern
```

### Pattern Caching

For performance, cache patterns outside functions:

```lua
local quote, notQuote, entity, notEntity, entityPattern

function extractVersion(dtdContent)
    if entityPattern == nil then
        quote, notQuote = peg.patternNotPattern('"')
        entity, notEntity = peg.patternNotPattern("<!ENTITY cxml.version ")
        -- notEntity allows anything before "<!ENTITY"
        -- entity matches "<!ENTITY cxml.version " exactly
        entityPattern = notEntity * entity * quote * peg.capture(notQuote) * quote
    end
    return peg.match(dtdContent, entityPattern)
end
```

### WHY patternNotPattern is Essential (XML Example)

**Problem**: When searching for a pattern like `version=` in XML documents, you don't want to match the XML declaration's `version="1.0"` - you want the cXML element's `version="1.2.066"`.

**Solution**: `peg.patternNotPattern("version=")` returns two patterns:

- `version` - matches "version=" exactly
- `notVersion` - matches anything EXCEPT "version="

**Usage**: `notVersion * version` ensures:

1. The pattern can appear ANYWHERE in the document (not at start)
2. It won't match false positives like XML declaration version
3. It correctly handles multiple version attributes

**CRITICAL**: This pattern is almost ALWAYS needed when matching text that does NOT start at the beginning. Without `notVersion * version`, the pattern would only match if it appears at the very start of the text.

**Example for method attribute**:

```lua
-- Find method="..." anywhere in XSLT
method, notMethod = peg.patternNotPattern('method=')
quote, notQuote = peg.patternNotPattern('"')
methodPattern = notMethod * method * quote * peg.capture(notQuote) * quote
-- This matches: anything, then method=", then capture value until next "
```

### Critical: Why peg.least(peg.define.anyChar, n) FAILS

**Common mistake**: Using `peg.least(peg.define.anyChar, 1)` to match characters before a pattern.

**Problem**: `peg.least()` is GREEDY and will consume ANY characters, including the text you're looking for!

**Example**:

```xml
<cXML payloadID="..." timestamp="..." version="1.2.066">
```

**Wrong approach**:

```lua
-- This FAILS because peg.least consumes "version="!
local badPattern = notCxml * cxml * peg.least(peg.define.anyChar, 1) * peg.pattern("version=") * quote * peg.capture(notQuote) * quote
```

What happens:

1. `peg.least(peg.define.anyChar, 1)` consumes: `payloadID="..." timestamp="..." version="1.2.066">`
2. Then `peg.pattern("version=")` looks for ANOTHER "version="
3. Pattern fails because "version=" was already eaten!

**Correct approach**:

```lua
local version, notVersion = peg.patternNotPattern("version=")
local goodPattern = notVersion * version * quote * peg.capture(notQuote) * quote
```

What happens:

1. `notVersion` matches: `payloadID="..." timestamp="..."`
2. **Stops BEFORE "version="** (doesn't consume it)
3. `version` matches "version="
4. Success!

**Key Rule**:

- Use `peg.least()` for minimum repetitions (like regex `*` or `+`)
- Use `peg.patternNotPattern()` to match up to BUT NOT INCLUDING specific text

---

## Basic String Finding

```lua
local peg = require "peg"

-- Simple string finding
peg.found(text, "error")        -- finds "error" anywhere
peg.found(text, "warning")      -- finds "warning" anywhere

-- Pattern matching with peg module (for complex patterns)
local errorPattern = peg.pattern("error") + peg.pattern("warning") -- error OR warning
-- error FOLLOWED BY warning (concatenation)
local errorThenWarning = peg.pattern("error") * peg.pattern("warning")
peg.match(text, errorPattern)
```

## Pattern Matching Numbers

```lua
-- Ranges: peg.range("09") = [0-9], peg.range("az") = [a-z], peg.range("AZ") = [A-Z]
local digit = peg.range("09")  -- single digit 0-9
local letter = peg.range("az") + peg.range("AZ")  -- Any letter
-- Match "1 error", "2 errors", "10 warnings", etc.
local nonZeroDigit = peg.range("19")  -- [1-9]
local singularError = peg.pattern("1") * peg.pattern(" error")      -- "1 error"
local pluralErrors = nonZeroDigit * digit * peg.pattern(" errors") -- "2-99 errors"
```

## Common Patterns

```lua
-- Match "not found"
peg.found(text, "not found")
-- Match "command not found"
peg.found(text, "command not found")
-- Match file not found patterns
peg.found(text, "No such file or directory")
-- Match I/O error
peg.found(text, "I/O error")
-- Match luacheck specific patterns
peg.found(text, "unfinished string")
peg.found(text, "syntax error")
```

## Combining Patterns

```lua
-- Multiple alternatives with OR (use peg for complex patterns)
local errorPatterns = peg.pattern("error") + peg.pattern("warning") + peg.pattern("I/O error")
peg.match(text, errorPatterns)

-- NOT found pattern (negation) - use peg.other()
local goodPattern = peg.pattern("0 errors") + peg.pattern("0 warnings")
local hasErrors = peg.other(goodPattern) * peg.pattern("error")  -- error that's not "0 errors"

-- Search pattern anywhere in string (not just from start)
-- This is a more advanced pattern using recursion - see peg source for details
local function anywherePattern(pattern)
  -- Implementation requires peg grammar table syntax
  return pattern -- Simplified for documentation - use peg.found() instead
end
```

## Key Differences from Regex

- No `[a-z]` character classes - use `peg.range("az")`
- No `+` quantifier - use `peg.least(pattern, 1)` for one or more
- No `*` quantifier - use `peg.least(pattern, 0)` for zero or more
- No `|` OR - use `peg.pattern(p1) + peg.pattern(p2)` for OR
- No `^` start - use `peg.pattern("^")` for literal ^
- No `$` end - use `-1` for end of string

## Important Notes

- `peg.toPattern(pattern)` takes **only one parameter**, not multiple
- Use `peg.found(text, "string-to-find")` for simple string finding
- Use `peg.pattern()`, `peg.range()`, `peg.match()` for complex patterns
- `*` = concatenation (A followed by B) - works with peg patterns
- `+` = OR (A or B) - works with peg patterns
- `peg.least(pattern, 0)` = zero or more repetitions
- `peg.least(pattern, 1)` = one or more repetitions
- `peg.most(pattern, n)` = at most n repetitions

## For fix.lua

Instead of regex `[1-9]%d* errors?` use:

```lua
-- Create proper range patterns using peg
local nonZeroDigit = peg.range("19")  -- digits 1-9 (non-zero)
local digit = peg.range("09")         -- digits 0-9
local singularError = peg.pattern("1") * peg.pattern(" error")      -- "1 error"
local pluralErrors = nonZeroDigit * digit * peg.pattern(" errors") -- "2 errors" to "99 errors"
local luacheckErrorPattern = singularError + pluralErrors + peg.pattern("1 warning") + peg.pattern("I/O error")

-- Usage with peg.found
if peg.found(output, luacheckErrorPattern) then
  -- Found errors like "2 errors", "10 errors", "1 error", etc.
end

```

## Essential PEG Functions

### Pattern Creation

```lua
peg.pattern("string")     -- Literal string pattern
peg.set("abc")           -- Character set: any of a, b, or c
peg.range("az")          -- Character range: a-z
peg.range("09")          -- Character range: 0-9
peg.toPattern("string")  -- Convert string to pattern (single parameter)
peg.patternNotPattern(pattern, countLeast, countMost, countAntiLeast)
                        -- Returns two patterns: pattern, notPattern
                        -- pattern: matches 'pattern' with optional repetition counts
                        -- notPattern: matches anything EXCEPT 'pattern'

-- Parameters:
-- pattern: The pattern to match (string or pattern)
-- countLeast: Minimum repetitions of pattern (default 1)
-- countMost: Maximum repetitions of pattern (default 1)
-- countAntiLeast: Minimum repetitions of anti-pattern (default 1)

-- Examples:
peg.patternNotPattern('">')           -- Match '">" exactly once
peg.patternNotPattern('">"', 0)        -- Match '">' zero or more times
peg.patternNotPattern('">"', 1, 3)     -- Match '">' 1 to 3 times
peg.patternNotPattern('">"', 0, nil, 2) -- Anti-pattern needs at least 2 chars
```

### Predefined Patterns (peg.define)

The `peg.define` table contains useful predefined patterns:

```lua
peg.define.locale           -- Lua locale patterns (alpha, alnum, etc.)
peg.define.digit           -- Digits 0-9 (same as peg.range("09"))
peg.define.space           -- Space character " "
peg.define.whiteSpace      -- Space or tab " \t"
peg.define.anyWhiteSpace   -- Space, tab, return, newline, formfeed " \t\r\n\f"
peg.define.notSpace        -- Any character except space
peg.define.endOfLine       -- Any end-of-line pattern: "\n\r", "\r\n", "\n", "\r"
peg.define.separator       -- Non-alphanumeric, non-underscore characters (used for word boundaries)
peg.define.emptyLineEnd    -- End of string (-1)
peg.define.anyChar         -- Any character except null
peg.define.anyText         -- Zero or more any characters

-- String patterns
peg.define.singlequoted    -- Single-quoted string: 'content'
peg.define.doublequoted    -- Double-quoted string: "content"
peg.define.quoted          -- Either single or double quoted string

-- Long string patterns (Lua style)
peg.define.longStringStart    -- Start of long string: [[ or [=[...=[
peg.define.longCommentStart   -- Start of long comment: --[[
peg.define.longStringEnd      -- End of long string: ]] or ]=...=]
```

### Pattern Operations

```lua
peg.least(pattern, 0)    -- Zero or more repetitions (like regex *)
peg.least(pattern, 1)    -- One or more repetitions (like regex +)
peg.leastMost(pattern, min, max) -- Between min and max repetitions
peg.most(pattern, n)     -- At most n repetitions
peg.pattern1 + peg.pattern2  -- OR (pattern1 OR pattern2)
peg.pattern1 * peg.pattern2  -- Concatenation (pattern1 FOLLOWED BY pattern2)
peg.other(pattern)       -- Match anything EXCEPT pattern (negation)
```

### LPEG Capture Operators

```lua
peg.capture(pattern)     -- C(patt) - capture match
peg.Cc(const)            -- Cc(value) - constant capture (always returns value)
peg.Cc(val1, val2, ...)  -- Cc(...) - multiple constant captures
peg.Cf(func)             -- Cf(patt) - fold capture (pass results to function)
peg.Cg(name, patt)       -- Cg(patt) - named group capture (for tables)
peg.Cp()                 -- Cp() - position capture (returns current position)
peg.Cs(patt)             -- Cs(patt) - substitution capture (returns match as string)
peg.Ct(patt)             -- Ct(patt) - table capture (returns array of captures)
peg.V(name)              -- V(name) - grammar variable reference
```

### String Finding & Matching

```lua
peg.found(text, pattern)      -- Find pattern anywhere in text (returns boolean)
peg.match(text, pattern)      -- Match pattern from start (returns position)
peg.find(text, pattern)       -- Find pattern position (returns number)
peg.startsWith(text, pattern) -- Check if text starts with pattern
peg.endsWith(text, pattern)   -- Check if text ends with pattern
```

### String Manipulation

```lua
peg.replace(text, old, new)           -- Replace all occurrences
peg.replaceOnce(text, old, new)        -- Replace first occurrence
peg.removePattern(text, pattern)       -- Remove all occurrences of pattern
peg.parseBefore(text, divider)         -- Get text before first divider
peg.parseBeforeLast(text, divider)     -- Get text before last divider
peg.parseBeforeWithDelimiter(text, div)  -- Get text before first divider, including divider
peg.parseAfter(text, divider)          -- Get text after first divider
peg.parseAfterLast(text, divider)      -- Get text after last divider
peg.parseAfterStart(text, divider)     -- Get text after divider if at start
peg.parseAfterWithDelimiter(text, div)   -- Get text after first divider, including divider
peg.parseLast(text, divider)           -- Get last part after divider
peg.parseLastWithDelimiter(text, divider)-- Get last part including divider
peg.parse(text, divider, num)          -- Get Nth element (1-indexed), negative for from end
peg.splitToArray(text, separator)      -- Split text to array
peg.split(text, divider)               -- Split at first divider: returns before, after
peg.splitWithDelimiter(text, divider)    -- Split including divider: returns before, from_divider
peg.splitLast(text, divider)           -- Split at last divider: returns before_last, after
peg.trimFunction(separator)            -- Returns trim function, faster to create pattern once
peg.trim(text)                         -- Trim whitespace from both ends of string
peg.removeFromStart(text, remove)     -- Remove pattern from start
peg.removeFromEnd(text, remove)       -- Remove pattern from end
peg.removeFromStartEnd(text, remove)  -- Remove from both start and end
peg.addToEnd(text, suffix)             -- Add suffix if not already present
peg.cleanBeforeAfterDoubleSpace(text) -- Clean double spaces, trim ends
```

### Iteration Functions

```lua
peg.iterateLines(text, func)              -- Call func for each line
peg.iterateSeparators(text, sep, func)     -- Call func for each part separated by sep
peg.iterateSeparatorsCapture(text, sep, func) -- Call func(part, sep) for each
peg.iterateCapture(text, pattern, func)    -- Same as iterateSeparatorsCapture
peg.countSeparators(text, sep)             -- Count separator occurrences (+1 = parts)
```

### Counting Functions

```lua
peg.countOccurrence(text, char)            -- Count occurrences of character/string
peg.countOccurrenceFromTable(tbl, key, char) -- Count in table[key]
```

### Parsing Between Patterns

```lua
peg.parseBetweenWithDelimiter(source, start, end) -- Returns text, startPos, endPos (with delimiters)
peg.parseBetweenWithoutDelimiter(source, start, end) -- Returns text between (without delimiters)
peg.replaceBetweenWithDelimiter(source, start, end, newText) -- Replace text between
```

### Case Conversion

```lua
peg.upper(text)                 -- Convert to uppercase (UTF-8 aware)
peg.lower(text)                 -- Convert to lowercase (UTF-8 aware)
peg.toFirstUpperCase(text)      -- Capitalize first letter of each word
peg.toPascalCase(text)           -- Convert to PascalCase (UserAccount)
peg.toCamelCase(text)            -- Convert to camelCase (userAccount)
peg.toKebabCase(text)            -- Convert to kebab-case (user-account)
```

### Number Functions

```lua
peg.toNumber(text)               -- Convert string to number (handles comma decimal)
peg.parseFirstNumber(text)       -- Extract and parse first number from text
peg.foundNumber(text)            -- Check if string contains digits
```

### Concatenation

```lua
peg.concat(...)                  -- Concatenate all arguments
peg.concatWith(separator, ...)   -- Concatenate with separator between non-empty parts
peg.concatString(...)            -- Concatenate only non-empty string arguments
```

### Advanced Functions

```lua
peg.substitute(open, repl, close)      -- Create substitution pattern for use in grammar
peg.matchString(text, pattern)         -- Match pattern against text
peg.replaceFromStart(text, from, to)   -- Replace prefix if present
peg.splitterFunction(option)           -- Create text splitter function for chunking
peg.patternNotPattern(patt, least, most) -- Returns (pattern, notPattern) pair
peg.other(patt)                        -- Match anything except pattern
peg.stringToTable(text)                -- Convert string to character array
peg.utfNormalize(text)                 -- Normalize UTF-8 to NFC form
```

### Examples

```lua
-- Find digits in text
local digitPattern = peg.range("09")
if peg.found("file123", digitPattern) then print("Found digits") end

-- Replace words
peg.replace("hello world", "world", "universe") -- "hello universe"

-- Parse file extension
peg.parseAfter("script.lua", ".") -- "lua"

-- Split CSV
local arr = peg.splitToArray("a,b,c", ",") -- {"a", "b", "c"}

-- Trim whitespace
local trimmed = peg.trim("  hello world  ") -- "hello world"
```
