fix(dbf): Layer 6 — EOF marker max-merge + disable append batching in shared

Closes two more multi-session correctness bugs surfaced by the
post-Layer-5 stress harness. Combined with Layer 5's panic-free
result, three-worker concurrency now sits around 80% pass with
zero Go-level crashes; higher worker counts trade reliability
for throughput against the inherent single-file-multi-writer
limit of the DBF format.

1. EOF marker write at Close (max-merge with disk)

   `Close()` writes the EOF marker `0x1A` at
   `header.HeaderLen + a.recCount * RecordLen`, computed from
   our LOCAL recCount. A peer Append between our last refresh
   (under the append-intent lock at Append-time) and Close-time
   may have bumped the disk recCount above ours. Writing EOF
   at our stale offset overwrites byte 0 of the peer's record
   — flipping the delete-flag from ' ' (RecordActive) to 0x1A.
   The field bytes survive, but downstream code that depends on
   byte 0's exact value misclassifies the record.

   Fix mirrors updateHeader's max-merge (Layer 3a): in shared
   mode, re-read the disk header right before computing
   EOFOffset and use max(disk.RecCount, local). Cheap (~1 stat-
   sized read per Close) and the eventual close-fd is already
   the serial bottleneck of any meaningful churn.

2. Append-batching disabled in shared mode

   The appendBuf optimisation accumulates several consecutive
   APPENDs into a single WriteAt at flushRecord time. In single-
   process EXCLUSIVE mode that's a clean throughput win. In
   shared mode, though, a peer SELECT can open the file while
   our slots N..N+M are buffered but still on-disk only as
   reserved-but-zero bytes. The peer iterates 1..recCount and
   ReadAts zeros at offsets [N..N+M), treating the records as
   garbage / empty markers.

   Skip the batch path when `a.shared`: each Append writes its
   record straight through via flushRecord on the next state
   change. EXCLUSIVE single-process flows are unaffected.

Observed stress numbers (3 trials × 30 runs each, average):

  pre-Layer-1 baseline:   ~60% / panics
  +Layer 1+2:             80% / 50% / panic
  +Layer 4a/4b:           75-90% / 50-80% / panic
  +Layer 5 (mmap-gen):    ~73% / ~67% / ~33% / NO PANICS
  +THIS (EOF + no-batch): ~83% / ~50% / ~22% / NO PANICS

The remaining flake at 5+ concurrent writers reflects the
fundamental constraint of FiveSql2's DBF model: no table-level
write lock, no MVCC. PostgreSQL solves this with snapshot
isolation; the equivalent for FiveSql2 would need a
write-ahead log or per-table writer mutex. Tracked as a
post-1.0 R&D direction.

For typical pgserver use — many read clients, few write
clients — the current correctness is production-acceptable.
The pgserver Phase 7 integration suite (3/3 in the basic
psql harness + 3/3 in the auth/TLS harness) remains 6/6 green
because each suite uses one connection at a time.

All six release gates green:
  go test ./...               ✓
  FiveSql2 SQL:1999 43/43     ✓
  Harbour compat 56/56        ✓
  std.ch 17/17                ✓
  FRB 7/7                     ✓
  pgserver integration 6/6    ✓

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-21 22:03:56 +09:00
parent 151b628f6c
commit 12fcb8d249

View File

@@ -393,6 +393,25 @@ func (a *DBFArea) Close() error {
if a.dirty {
a.flushRecord()
}
// Shared-mode EOF write — same max-merge as updateHeader. A
// peer Append between our last refresh and this write may have
// grown the file past our local recCount. Writing the EOF
// marker at our stale offset (HeaderLen + localCount*RecLen)
// would overwrite byte 0 of the peer's just-appended record
// (its delete flag) — the record body bytes survive but byte 0
// flips from ' ' (RecordActive) to 0x1A (EOFMarker). Re-read
// the disk header to pick the larger of {local, disk} before
// computing EOFOffset.
if a.shared {
if _, err := a.dataFile.Seek(0, 0); err == nil {
if hdr, err := ReadHeader(a.dataFile); err == nil {
if hdr.RecCount > a.recCount {
a.recCount = hdr.RecCount
a.header.RecCount = hdr.RecCount
}
}
}
}
a.dataFile.WriteAt([]byte{EOFMarker}, a.header.EOFOffset())
a.updateHeader()
// Release any held byte-range locks before closing the fd — POSIX
@@ -784,8 +803,15 @@ func (a *DBFArea) Append() error {
return fmt.Errorf("table is read-only")
}
// Batch consecutive APPENDs: save current dirty record to appendBuf instead of writing to disk.
if a.dirty && a.ghost {
// Batch consecutive APPENDs: save current dirty record to
// appendBuf instead of writing to disk. SHARED mode opts out
// of batching — a peer SELECT that opens the file while one
// of our slots is still buffered would iterate to that slot,
// ReadAt zero bytes (slot is reserved on disk but unwritten),
// and treat the record as garbage. Writing each APPEND straight
// through trades some throughput for "self+peer-coherent on
// every cursor move" semantics. EXCLUSIVE mode still batches.
if a.dirty && a.ghost && !a.shared {
// Previous was also an APPEND — accumulate in batch buffer (no syscall)
recLen := int(a.header.RecordLen)
if a.appendBuf == nil {