# Calc Server

Production planning engine that loads data from PostgreSQL, builds BOM trees, calculates product need, schedules work phases against capacity, and serves results to the browser.

## Architecture

```mermaid
flowchart TD
    Browser -->|"POST /rest/nc/calc/product-need"| API
    Browser -->|"POST /rest/nc/calc/work-center"| API
    API --> defaultCalc
    defaultCalc --> loadData
    loadData -->|"SQL queries"| PostgreSQL
    loadData --> fixData
    loadData --> sortAndIndex
    sortAndIndex --> calculateData
    calculateData --> capacity
    calculateData --> load
    defaultCalc --> productNeed
    productNeed --> addSourceArray
    productNeed --> buildBOMTree
    productNeed --> balanceLoop
    productNeed --> checkNeedDates
    productNeed --> resultCache
```

## Data Flow

### 1. Data Loading

Tables are loaded from PostgreSQL in dependency order. Each table has a query definition in `query/` and column definition in `column/`.

**Core tables loaded on every request:**

| Table | Purpose |
|-------|---------|
| `product` | Products and their BOM structures |
| `product_work-estimated` | Planned work phases per product |
| `product_work-schedule` | Scheduled work phases linked to production orders |
| `product_work-actual` | Actual production reports from shop floor |
| `product_material-estimated` | Planned material needs per product |
| `product_material-actual` | Actual material usage |
| `production_order` | Production orders (open or all) |
| `order-sales` / `order_row-sales` | Sales orders driving demand |
| `order-purchase` / `order_row-purchase` | Purchase orders |
| `capacity` | Shift schedules per resource |
| `resource-load_group` | Resource to load group mapping |
| `resource_efficiency` | Resource efficiency over time |

**Loading modes:**

- **Full load** — all tables from database, used on first load or when `reload_data` is true
- **Incremental load** — only changed rows since last `modify_id`, used when `load_only_changed` is true
- **Cached** — data is kept in memory per organization, subsequent requests reuse it unless flagged for reload

### 2. Data Fixing

After loading, `fixData` normalizes the raw database records:

- Converts product IDs to uppercase for 4D database compatibility
- Validates dates against minimum/maximum date range
- Ensures required fields exist on records
- Creates the `unproductive` virtual product, resource, and production order used by work center

### 3. Sorting and Indexing

All major tables are sorted and indexed for fast lookup during calculation:

| Index | Key | Purpose |
|-------|-----|---------|
| `product` | `product_id` | Product lookup |
| `production_order` | `production_order_id` | Production order lookup |
| `product_work-schedule-resource-arr` | `resource_id` | Find schedules by resource |
| `product_work-schedule-production_order_id-arr` | `production_order_id` | Find schedules by order |
| `product_work-estimated-product-arr` | `product_id` | Find estimated phases by product |
| `product_work-actual` → `production_order-actual` | `production_order_id` | Actual production amounts per order |
| `order-sales` | `order_id` | Sales order lookup |
| `resource-load_group` | `resource_id` | Resource to load group |

**Sort order matters** — schedules are sorted by `start_dt > end_dt > production_order_id > work_phase_number`, estimated phases by `product_id > work_phase_number > resource_id`.

### 4. Production Order Actual Index

The `production_order-actual` index aggregates shop floor actuals from `product_work-actual` into a per-order summary:

**Structure:**

```text
production_order-actual = {
  ["order-123"] = {
    produced_amount = 150,
    failed_amount = 3,
    actual_record_id = {"rec-1", "rec-2"},
    phase = {
      [1] = { work_phase_state = "AL", produced_amount = 80 },
      [2] = { work_phase_state = "KU", produced_amount = 70 }
    }
  }
}
```

**Per-phase state resolution:**

- Uses `actual.work_phase_state` directly when it has a value
- When actual state is empty `""`, falls back to the corresponding schedule row's `work_phase_state`
- Skips the `unproductive` virtual product

**Open phase filtering:**

- When `actual_only_open` is enabled (default), only actuals with a state in `phase_open_state` are included
- Open states: `""`, `EK`, `KU`, `AL`, `TA`, `PY`, `confirmed`
- Closed states like `done`, `done2`, `cancelled` are excluded from the index

**Consumers:**

- **Product need (gantt chart)** — uses per-phase data to show actual production state and amounts on each work phase row
- **Work center** — uses order-level totals for `doneAmount`, calculates `produced_fraction` and `remaining_amount`

### 5. Capacity Calculation

Capacity is the available working time per resource, generated from shift schedules:

- **Time window** is determined by the earliest schedule start and latest schedule end, expanded by `days_before_first_work` and `days_after_last_work`
- **Shift generation** creates capacity slots per resource from shift schedule definitions in preferences
- **Default capacity** is used when a resource has no specific capacity configuration
- **Efficiency** from `resource_efficiency` adjusts available hours

Capacity is never empty — if a resource has no configuration, default capacity from preferences is applied.

### 6. Load Calculation

Load assigns scheduled work phases to capacity slots:

- Each work phase consumes `work_hours` from the resource's capacity
- Setup and extra hours are consumed as physical time outside the capacity multiplier
- Load is calculated per resource per time slot
- Phase parts track how a work phase is split across capacity slots

## Product Need Calculation

The main calculation that determines what needs to be produced, when, and how much.

### Source of Need

Need comes from multiple sources, each contributing rows to the need calculation:

| Source | Flag | Description |
|--------|------|-------------|
| **Sales orders** | always | Open sales order rows create direct need |
| **Production orders** | always | Open production orders create need for their products |
| **Lower-level BOM** | always | Products needed as components of parent products |
| **Estimate need** | `estimate_need` | Forecast based on historical sales |
| **Minimum stock** | `minimum_stock` | When stock falls below minimum |
| **Action limit** | `action_limit` | When stock falls below action limit |
| **Negative end balance** | `negative_end_balance` | When projected balance goes negative |
| **Negative start balance** | `negative_start_balance` | When starting balance is already negative |
| **Critical date** | `critical_date` | Need based on critical delivery dates |

### BOM Tree

The bill-of-materials tree is built top-down from each product:

1. Start with top-level products that have need
2. For each product, find its estimated materials (`product_material-estimated`)
3. Each material references a lower-level product
4. Recurse down the tree, calculating required amounts at each level
5. Amount conversion uses `amountInStockUnit1` for unit differences between BOM levels
6. Maximum material delivery time across all purchase materials becomes the material phase `extra_hours`

### Balance Loop

The core iteration that determines how much to produce:

1. **Add sources** — collect all need rows from sales, production, estimate, etc.
2. **Build balance** — for each product, stack all need rows chronologically
3. **Calculate balance** — running balance = stock + receipts - demand
4. **Detect gaps** — when balance goes below zero, a production need is created
5. **Add lower-level need** — the new production need triggers BOM tree expansion
6. **Repeat** — lower-level needs may create further lower-level needs, up to `max_need_down_loop` iterations
7. **Check dates** — validate that need dates are achievable against capacity

### Virtual Production Orders

When `virtual_production_order` is enabled:

- Products that need to be produced but have no existing production order get a virtual one
- Virtual orders are prefixed with `V-` and stored alongside real orders
- They carry the calculated need date as their schedule

### Schedule Creation

When `create_schedule` is enabled:

- After need calculation, production orders that have no scheduled work phases get them
- Phases are created from the product's estimated work phases
- Phase timing is set based on the calculated need dates

### Need Date Calculation

Need date passed to lower-level products always comes from the first **real** (non-material) phase start date:

- Material phase extra time only determines when to purchase — it never affects child scheduling
- Parent phase lookup skips material phases, linking to the first real work phase

### Result Cache

Results are cached per organization in memory:

- Subsequent requests with same parameters return cached result
- `recalculate: false` skips calculation and returns cache directly
- Incremental updates track `last_modify_id` to send only changed rows to the browser

## Work Center

The work center view shows what is happening on the shop floor right now.

### Data Sources

- Uses the same loaded data as product need (shared cache)
- Additional per-phase calculations for person assignments, started amounts, and can-start logic

### Schedule Sorting

Work center sorts schedules differently from product need — by critical end date, then by production order, then by phase number. This puts the most urgent work first.

### Production Order Progress

For each production order in the work center:

- `doneAmount` = external DB `produced_amount` + index `produced_amount` from `production_order-actual`
- `remaining_amount` = `amount_to_produce` - `doneAmount`
- `produced_fraction` = `doneAmount / amount_to_produce`
- When `amount_to_produce` is zero and order is not closed, fraction is set to `1.1` (error indicator shown in red)

### Per-Phase Calculation

Each work phase in the schedule has:

- **Started amount** — how much has been started on this phase
- **Can start amount** — how much can be started based on previous phase completion
- **Person assignments** — who is working on this phase
- **Days left** — days until critical end date

### Actual Data Integration

The work center uses the `production_order-actual` index to combine shop floor actuals with planned data. This gives real-time visibility into what has actually been produced versus what was planned.

## Gantt Chart

The production gantt shows work phases visually with color coding.

### Color Mapping

Phase colors come from `load_phase_state_color` in `constant.json`:

| State | Color | Meaning |
|-------|-------|---------|
| `confirmed` | black | Planned, confirmed |
| `AL` | blue | Started (working) |
| `KU` | black | Waiting |
| `EK` | grey | Queued |
| `PY` | red | Stopped |
| `LT` | dark grey | Finished |
| `delivery` | olive | Total/delivery row |
| `running` | blue | Currently producing |
| `done` | dark grey | Completed |
| `cancelled` | red | Cancelled |

### Total Row

The TOTAL row in the gantt always has `work_phase_state = 'delivery'` — this is intentional and never overridden by actuals.

### Per-Phase Rows

When actual data exists for a phase:

- `work_phase_state` comes from the `production_order-actual` index
- `produced_amount` shows actual produced amount
- `remaining_amount` shows what is left to produce

When no actual data exists:

- `work_phase_state` comes from the sales order `origin` field
- If origin is empty, state is `'unknown'`

## Configuration

### Constants (`constant.json`)

System-wide settings that define states, colors, and behavior defaults:

- **Order states** — open, confirmed, delivered, cancelled
- **Phase states** — open states, finished states, started states
- **Production order states** — draft, confirmed, progress, done, cancelled, halted
- **Colors** — CSS color mapping for each phase state
- **Thresholds** — max loop counts, date ranges, rounding decimals

### Preferences (`rec.json`)

User-adjustable calculation parameters:

- Toggle features: `minimum_stock`, `action_limit`, `negative_end_balance`, `critical_date`, `estimate_need`
- Date ranges: `minimum_date`, `maximum_date`, `min_days_in_past`, `min_days_in_future`
- Timing: `production_order_start_hour`, `production_need_hour`
- Performance: `max_need_down_loop_count`, `load_all_products`
- Actuals: `actual_only_open` — filter actuals to open phases only

## Performance

### Memory Management

- Data is kept in memory per organization between requests
- Garbage collection runs between full loads
- Memory usage is logged on start and end of each calculation

### Caching Layers

1. **Data cache** — `use_calc_server_cache` saves loaded grid to disk for faster startup
2. **Result cache** — calculation results cached in memory per organization
3. **Incremental updates** — only changed rows sent to browser via `last_modify_id` tracking

### Timing

Each calculation phase is timed and logged:

- Query time, data combine time, data fix time
- Sort time (count and rows)
- Index creation time
- Capacity creation time
- Load calculation time
- Total calculation time

## General Rules

### Capacity

- `days_before_first_work` and `days_after_last_work` are handled inside `createCapacityArray` — time window is expanded before shift generation
- `max_days_before` and `max_days_after` limits are enforced in `capacity.calculate` before calling `createCapacityArray`
- Capacity array is never empty — if resource has no capacity config, default capacity from preferences is used

### Need Calculation (Backward Scheduling)

- Scheduling is backward from delivery date through all work phases
- Each phase consumes capacity time in reverse order (last phase first)
- Extra time (setup, resource extra hours) is consumed as physical time outside capacity multiplier
- Phase start must be before or at the next phase start — overlap is an error
- Consecutive phase check reports `work hours`, `extra hours` and overlap duration

### Material Phase

- Material phase represents purchase ordering lead time, not manufacturing
- Material phase is only created when there are **purchase materials** (non-work items) with non-zero delivery time
- If all lower parts are work items (manufactured) or longest delivery time is 0, no material phase is created
- Material phase `extra_hours` = longest `raw_material_delivery_time` across all purchase materials of that work, in hours
- The longest delivery time is calculated during BOM tree construction and stored on the tree record as `max_material_delivery_weeks`

### Need Date for Lower Levels

- Need date passed to lower-level products always comes from the first **real** (non-material) phase start date
- Material phase extra time only determines when to purchase — it never affects child scheduling
- Parent phase lookup skips material phases, linking to the first real work phase

### Extend Capacity

- When required hours is 0 or negative, it is clamped to 1 hour before extending
- If capacity already covers the range (neither backward nor forward), extend returns early without logging
- After extend, the `phase_part_cache` value is cleared so the retry uses fresh capacity
