CharlesKWON 12fcb8d249 fix(dbf): Layer 6 — EOF marker max-merge + disable append batching in shared
Closes two more multi-session correctness bugs surfaced by the
post-Layer-5 stress harness. Combined with Layer 5's panic-free
result, three-worker concurrency now sits around 80% pass with
zero Go-level crashes; higher worker counts trade reliability
for throughput against the inherent single-file-multi-writer
limit of the DBF format.

1. EOF marker write at Close (max-merge with disk)

   `Close()` writes the EOF marker `0x1A` at
   `header.HeaderLen + a.recCount * RecordLen`, computed from
   our LOCAL recCount. A peer Append between our last refresh
   (under the append-intent lock at Append-time) and Close-time
   may have bumped the disk recCount above ours. Writing EOF
   at our stale offset overwrites byte 0 of the peer's record
   — flipping the delete-flag from ' ' (RecordActive) to 0x1A.
   The field bytes survive, but downstream code that depends on
   byte 0's exact value misclassifies the record.

   Fix mirrors updateHeader's max-merge (Layer 3a): in shared
   mode, re-read the disk header right before computing
   EOFOffset and use max(disk.RecCount, local). Cheap (~1 stat-
   sized read per Close) and the eventual close-fd is already
   the serial bottleneck of any meaningful churn.

2. Append-batching disabled in shared mode

   The appendBuf optimisation accumulates several consecutive
   APPENDs into a single WriteAt at flushRecord time. In single-
   process EXCLUSIVE mode that's a clean throughput win. In
   shared mode, though, a peer SELECT can open the file while
   our slots N..N+M are buffered but still on-disk only as
   reserved-but-zero bytes. The peer iterates 1..recCount and
   ReadAts zeros at offsets [N..N+M), treating the records as
   garbage / empty markers.

   Skip the batch path when `a.shared`: each Append writes its
   record straight through via flushRecord on the next state
   change. EXCLUSIVE single-process flows are unaffected.

Observed stress numbers (3 trials × 30 runs each, average):

  pre-Layer-1 baseline:   ~60% / panics
  +Layer 1+2:             80% / 50% / panic
  +Layer 4a/4b:           75-90% / 50-80% / panic
  +Layer 5 (mmap-gen):    ~73% / ~67% / ~33% / NO PANICS
  +THIS (EOF + no-batch): ~83% / ~50% / ~22% / NO PANICS

The remaining flake at 5+ concurrent writers reflects the
fundamental constraint of FiveSql2's DBF model: no table-level
write lock, no MVCC. PostgreSQL solves this with snapshot
isolation; the equivalent for FiveSql2 would need a
write-ahead log or per-table writer mutex. Tracked as a
post-1.0 R&D direction.

For typical pgserver use — many read clients, few write
clients — the current correctness is production-acceptable.
The pgserver Phase 7 integration suite (3/3 in the basic
psql harness + 3/3 in the auth/TLS harness) remains 6/6 green
because each suite uses one connection at a time.

All six release gates green:
  go test ./...               ✓
  FiveSql2 SQL:1999 43/43     ✓
  Harbour compat 56/56        ✓
  std.ch 17/17                ✓
  FRB 7/7                     ✓
  pgserver integration 6/6    ✓

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 22:03:56 +09:00

Five — Harbour + Go Fusion Language

Harbour PRG 코드를 네이티브 Go 바이너리로 컴파일하는 퓨전 언어.

30년간 축적된 Harbour/xBase 비즈니스 로직을 Go의 성능, 동시성, 크로스 플랫폼 위에서 실행합니다.

employees.prg  →  five build  →  employees (단일 실행파일, 18MB)

주요 특징

  • Harbour 문법 100% 지원 (CLASS, CODE BLOCK, BEGIN SEQUENCE, ...)
  • Go 네이티브 바이너리 출력 (CGo 없음, 순수 Go)
  • DBF/NTX/CDX 데이터베이스 엔진 내장
  • 479개 RTL 내장 함수
  • FiveSql2: DBF 위에서 SQL:1999 쿼리 (43/43 테스트 통과)
  • Goroutine/Channel 확장 (GO BLOCK, CHANNEL)
  • @byref 참조 전달, mutable closure
  • 대화형 디버거 (TUI/CLI)

빌드 방법

1. Go 설치

Five는 Go 1.21 이상이 필요합니다.

Linux/WSL:

# 이미 설치되어 있는지 확인
go version

# 없으면 설치
wget https://go.dev/dl/go1.22.5.linux-amd64.tar.gz
sudo rm -rf /usr/local/go
sudo tar -C /usr/local -xzf go1.22.5.linux-amd64.tar.gz
echo 'export PATH=$PATH:/usr/local/go/bin:$HOME/go/bin' >> ~/.bashrc
source ~/.bashrc
go version

macOS (Apple Silicon — M1/M2/M3/M4):

brew install go
# 또는 직접 다운로드:
wget https://go.dev/dl/go1.22.5.darwin-arm64.tar.gz
sudo tar -C /usr/local -xzf go1.22.5.darwin-arm64.tar.gz
echo 'export PATH=$PATH:/usr/local/go/bin:$HOME/go/bin' >> ~/.zshrc
source ~/.zshrc

macOS (Intel):

brew install go
# 또는 직접 다운로드:
wget https://go.dev/dl/go1.22.5.darwin-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.22.5.darwin-amd64.tar.gz
echo 'export PATH=$PATH:/usr/local/go/bin:$HOME/go/bin' >> ~/.zshrc
source ~/.zshrc

Windows:

https://go.dev/dl/ 에서 .msi 설치

2. Five 컴파일러 빌드

git clone https://gitea.fivego.org/fivedev/five.git
cd five
go build -o five ./cmd/five

빌드 확인:

./five version

3. PRG 프로그램 컴파일 및 실행

단일 파일:

# 컴파일
./five build examples/hello.prg -o hello

# 실행
./hello

다중 파일 (FiveSql2 등):

./five build _FiveSql2/test/test_sql1999.prg _FiveSql2/src/*.prg -o test_sql
./test_sql

4. 테스트 실행

# Go 유닛 테스트
go test ./...

# FiveSql2 SQL 테스트 (43/43)
./five build _FiveSql2/test/test_sql1999.prg _FiveSql2/src/*.prg -o /tmp/test_sql
cd /tmp && ./test_sql

# Harbour 호환 테스트 (51/51)
./five build tests/compat_harbour.prg -o /tmp/test_compat
/tmp/test_compat

Five 명령어

five run <file.prg>              컴파일 후 즉시 실행
five build <file.prg> [-o out]   네이티브 바이너리 생성
five gen <file.prg>              생성된 Go 코드 출력 (디버깅용)
five debug <file.prg>            대화형 디버거 실행
five version                     버전 정보

Hello World

// hello.prg
PROCEDURE Main()
   ? "Hello, Five!"
   ? "Date:", Date()
   ? "Time:", Time()
RETURN
./five build hello.prg -o hello && ./hello

SQL 예제

// sql_demo.prg
#include "FiveSqlDef.ch"

PROCEDURE Main()
   // 테이블 생성
   five_SQL("CREATE TABLE employees (id INTEGER, name CHAR(30), salary NUMERIC(12,2))")

   // 데이터 삽입
   five_SQL("INSERT INTO employees (id, name, salary) VALUES (1, 'Alice', 8000)")
   five_SQL("INSERT INTO employees (id, name, salary) VALUES (2, 'Bob', 7000)")

   // SQL 쿼리
   LOCAL aR := five_SQL("SELECT name, salary FROM employees WHERE salary > 6000 ORDER BY salary DESC")

   ? "Results:", Len(aR[2]), "rows"
   LOCAL i
   FOR i := 1 TO Len(aR[2])
      ? "  ", aR[2][i][1], aR[2][i][2]
   NEXT
RETURN
./five build sql_demo.prg _FiveSql2/src/*.prg -o sql_demo && ./sql_demo

프로젝트 구조

five/
├── cmd/five/          Five CLI (main entry point)
├── compiler/          PRG → Go 컴파일러
│   ├── lexer/         토크나이저
│   ├── parser/        구문 분석기
│   ├── analyzer/      의미 분석기
│   ├── gengo/         Go 코드 생성기
│   └── pp/            전처리기 (#include, #define)
├── hbrt/              런타임 (VM, Stack, Value, Class)
├── hbrtl/             RTL 표준 라이브러리 (479개 함수)
├── hbrdd/             RDD 데이터베이스 엔진
│   ├── dbf/           DBF 파일 드라이버
│   ├── ntx/           NTX 인덱스 드라이버
│   ├── cdx/           CDX 인덱스 드라이버
│   └── mem/           메모리 RDD
├── _FiveSql2/         SQL:1999 엔진 (PRG)
│   ├── src/           SQL 엔진 소스 (14 파일, 10,458 LOC)
│   └── test/          SQL 테스트 스위트
├── tests/             호환성 테스트
├── examples/          예제 프로그램
└── docs/              기술 문서

현재 상태

항목 수치
Go 프로덕션 코드 ~36,000 LOC
RTL 내장 함수 479개
RDD 드라이버 4종 (DBF, NTX, CDX, Memory)
FiveSql2 테스트 43/43 (100%)
호환 테스트 51/51 (100%)
Go 테스트 ALL PASS

라이선스

Copyright (c) 2025-2026 Charles KWON OhJun (charleskwonohjun@gmail.com) All rights reserved.

Description
Five — Harbour+Go Fusion Language (PRG→Go native binary)
Readme 64 MiB
Languages
Go 57.9%
xBase 22%
C 19.5%
Shell 0.5%
Makefile 0.1%