Commit Graph

4 Commits

Author SHA1 Message Date
151b628f6c fix(pgserver): Layer 5 — per-path mmap-gen registry + getWA torn-read
Closes the Go-panic class of multi-session concurrency bugs and
introduces an explicit cross-area mmap invalidation channel.

1. getWA waCache torn-read (root cause of panics)

   hbrtl/rdd.go cached the most recent `interface{} → *WAM` type
   assertion in a process-global struct of two `interface{}`-
   shaped fields. Each pgserver connection's NewThread gets its
   own WAM, so the cache missed on every call and immediately
   re-wrote two shared, unsynchronised fields. Go's `interface{}`
   is two words; concurrent write + read produced torn pointer
   values, with the result that goroutine A could observe
   goroutine B's WAM as its own.

   That mis-attribution surfaced as:
     - `concurrent map writes` panic at WorkAreaManager.Close
       (workarea.go:95): two goroutines genuinely modifying the
       SAME wam.aliases map.
     - `concurrent map writes` panic at DBFArea.FieldPosCache
       (dbf.go:439): two goroutines lazy-initing the SAME
       fieldPosMap.

   Drop the cache. The type assertion is ~ns; not worth a
   process-global shared slot. If perf matters again, replace
   with a sync.Map keyed by thread pointer, not a single struct.

2. Per-path mmap generation registry (hbrdd/dbf/area_registry.go)

   Each unique on-disk DBF path gets an atomic uint64 generation
   counter. *DBFArea instances:
     - On Open: pathGen = pathGenFor(path); pathGenSeen = current.
     - On Append (shared) / flushRecord: bumpPathGen(path);
       pathGenSeen = current.
     - On loadRecord: if pathGenSeen < live counter, bypass mmap
       fast path for THIS load (use ReadAt) and re-sync seen.

   Without this, a peer DBFArea's PutValue mutating a record we'd
   mmap-cached returned stale pre-mutation bytes from our
   snapshot. The existing length-bound check covered file-grow
   (`offset > mmap len`) but not byte-level mutation within the
   snapshot range. The registry covers both.

   Cheap: read = one atomic.LoadUint64, hit rate is ~100% in the
   single-writer-many-readers steady state.

Verification
------------

Same 3 / 5 / 10-worker pgx-driven concurrency stress harness:

  pre-Layer-1 baseline:       ~60% pass + occasional panic
  +Layer 1+2:                 80% / 50% / panic
  +Layer 3a (max-merge):      80% / 50% / panic
  +Layer 4a (per-session 3):  90% / 80% / 50%
  +Layer 4b (Go atomics):     75-90% / 50-80% / panic (still)
  +THIS (getWA + mmap-gen):   73% / 67% / 33% — ZERO PANICS

The shift "many partial fails, no panics" is what matters for
production: a connection seeing stale data is recoverable (rerun
the query); a Go-level process crash is not. Remaining
correctness flake comes from the in-flight appendBuf interaction
when peer Append fires between this connection's Append and
flushRecord — that's tractable with a per-connection flush
ordering rule, deferred to Layer 6.

All six release gates green:
  go test ./...               ✓
  FiveSql2 SQL:1999 43/43     ✓
  Harbour compat 56/56        ✓
  std.ch 17/17                ✓
  FRB 7/7                     ✓
  pgserver integration 6/6    ✓

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:43:04 +09:00
05ccef05e2 perf: EndProcFast — eliminate defer recover() from RTL hot paths
Problem: every RTL function calls defer t.EndProc() which does recover().
50K SEEK loop = 250K recover() calls = ~12ms wasted.

Solution: EndProcFast() skips recover (only needs endFrame restore).
Applied to ALL RTL functions in strings.go, rdd.go, missing.go, database.go.
EndProc() with recover kept for generated PRG code (needs BEGIN SEQUENCE).

Analysis (50K sequential SEEK breakdown):
  Go NTX Seek direct: 7ms (faster than Harbour 27ms!)
  PRG VM overhead:    38ms (Frame + RTL calls + key generation)
  Key generation:     25ms (Str+LTrim+PadL+PadR = 5 RTL Frame/EndProc per iter)

With EndProcFast: RTL overhead reduced ~30%.

CDX SCOPE: 2ms (Harbour 4ms — 2x FASTER!)
82/82 stress PASS. 14 packages ALL PASS.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 21:43:39 +09:00
5b378318a0 perf: RTL optimization — cached WA, spaces pool, stack-alloc fmt_int64
rdd.go:
- getWA() cached type assertion (avoid repeated interface check)
- waCache stores last WA pointer → O(1) for repeated calls

strings.go:
- spacesCache[257]: pre-built space strings for pad sizes 0-256
- spaces(n) returns cached string (no Repeat allocation)
- PadR/PadL use spaces() for fill=" " (most common case)
- Str() uses spaces() for right-padding

missing.go:
- fmt_int64: stack-allocated [20]byte array (was heap make([]byte))
- Reverse iteration (no prepend overhead)
- PadC uses spaces() for left/right padding

Benchmark (ext4, home dir):
  10K APPEND: 28ms → 26ms (Harbour 27ms!)
  50K APPEND: 130ms → 113ms (13% improvement)
  50K SCAN: 24ms → 23ms
  50K DUPKEY: 42ms → 35ms (17% improvement)
  CDX SCOPE: 12ms → 10ms (17% improvement)

82/82 stress PASS. 14 packages ALL PASS.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 18:16:22 +09:00
59568f3301 Five v0.9 — Harbour + Go fusion language
- Compiler: PP → Lexer → Parser → Analyzer → Gengo pipeline
- Parser: 232/236 (98%) Harbour compatibility, registry-based dispatch
- RTL: 351 Harbour-compatible functions
- RDD: DBF/NTX/CDX engines with Rushmore bitmap optimization
- Go Interop: IMPORT + pkg.Func() + obj:Method() with FastPath (15M calls/sec)
- HB_FUNC API: Full Harbour C API compatible Go bridge
- Concurrency: SPAWN/LAUNCH/GOROUTINE, <-, WATCH, PARALLEL FOR, ASYNC/AWAIT
- Extensions: Multi-return, DEFER, Slice, f-string, Nil-safe ?:, CONST
- Macro Compiler: Runtime AST parsing and evaluation
- Debugger: TUI debugger with source display, breakpoints, stepping
- FRB: Native + Pcode dual mode runtime binary
- Tests: 13 packages ALL PASS

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 09:41:50 +09:00