main
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| d7a81af7db |
feat(pgserver): binary-format param decoding (Phase 4.1)
pgx defaults to binary wire format for INT2/INT4/INT8/FLOAT4/FLOAT8/ BOOL/NUMERIC/DATE/TIMESTAMP/TIMESTAMPTZ — Go's most-used PG driver ships nearly every typed parameter as binary unless explicitly told to use text mode. The Phase 3 implementation only decoded INT4/INT8/ BOOL, so any pgx call with a decimal price, a timestamp, or a date was silently mis-quoted into the SQL stream. Decoders now cover the seven additional OIDs. The interesting one is NUMERIC: PG's wire format is base-10000 digit groups plus a separate displayed-scale, so the decoder rebuilds the decimal string from weight+sign+ndigits+digits[] without going through float (which would lose precision for NUMERIC(38,*) values). Pinned by vectors covering zero / positive / negative / fractional-only / NaN / multi-group integer + fraction cases. DATE / TIMESTAMP decoders assume integer_datetimes=on (which the server advertises in ParameterStatus); the 8-byte microsecond delta from the PG epoch (2000-01-01 UTC) is converted via Go's time.Time machinery and re-emitted as a quoted SQL literal. Text-format path also broadened: FLOAT4/FLOAT8/INT2 now transit unquoted alongside INT4/INT8/BOOL/NUMERIC; the regression would have been clients sending text-format floats getting them rewritten as '1.5' (string literal) instead of 1.5 (numeric). Verified: all 6 mandatory gates green (go test, SQL 43/43, compat 56/56, std.ch 17/17, FRB 7/7, pgserver 11/11). Five new decoder tests pin each wire format against handcrafted PG payloads. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| e83787750a |
feat(pgserver): SCRAM-SHA-256 authentication (Phase 5.1)
PG14+ clients (libpq, pgx, JDBC) prefer SCRAM over MD5 when offered;
this lands the five-message exchange (SASL / SASLInitialResponse /
SASLContinue / SASLResponse / SASLFinal) so they get their preferred
path. MD5 stays as the universal fallback.
Storage stays plaintext in the in-memory role registry — per-auth we
generate a fresh salt + iter, derive SaltedPassword on the fly. Same
net security as the existing MD5 path, while matching wire output to
RFC 5802 byte for byte.
Critical detail: pgproto3's Backend multiplexes PasswordMessage,
SASLInitialResponse, and SASLResponse onto the same 'p' byte tag.
Without SetAuthType() the decoder picks PasswordMessage and the
handshake fails immediately. Switch state to AuthTypeSASL before
the client-first receive and AuthTypeSASLContinue before the
client-final receive.
Verified:
* SCRAM math (PBKDF2 / HMAC / proof verify / server signature)
via pinned unit test
* Live psql round-trip — correct password accepted, wrong password
rejected with proper SQLSTATE 28P01
* All 6 mandatory gates green (go test, SQL 43/43, compat 56/56,
std.ch 17/17, FRB 7/7, pgserver 11/11)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| ed1aeeb212 |
feat(pgserver): pg_catalog stub for BI-tool connection compatibility
PostgreSQL clients (psql, pgx, DBeaver, Tableau, DataGrip,
pgAdmin) fire a barrage of catalog probes at connection time —
SELECT version(), SHOW server_version, SELECT FROM pg_namespace
/ pg_class / pg_type / pg_database / pg_settings. FiveSql2 can't
parse most of them. Without interception the BI tool either
errors out on connect or proceeds with a half-broken view of
the database (zero tables, no type info, no schema list). This
commit lands the minimum-viable catalog shim so the common
connect-and-list-tables flow succeeds.
Strategy
--------
Pattern-match catalog probes BEFORE handing the SQL to five_SQL.
Recognised shapes get synthesised result envelopes — same
`{ aFieldNames, aRows }` hbrt.Value shape the engine returns,
so the existing dispatchSimpleQuery / executePortal pipelines
stream them identically to a normal query.
Covered (v1.0)
--------------
* SET / RESET / DISCARD <name> → success, no-op
* SHOW <name> → single-row response
(server_version, server_encoding,
client_encoding, DateStyle,
transaction_isolation, etc.)
* SELECT version() / current_database() / current_schema() /
current_user / session_user / pg_backend_pid() → single-row
* SELECT … FROM pg_namespace → 2 rows (pg_catalog + public)
* SELECT … FROM pg_class → list of open workareas
(relkind='r', relnamespace=public)
* SELECT … FROM pg_attribute → empty (stub; column-shape
introspection deferred to v1.1)
* SELECT … FROM pg_type → 7 OIDs FiveSql2 actually emits
(bool, int4, int8, text, numeric,
date, timestamp)
* SELECT … FROM pg_database → 1 row, the connect-time db name
* SELECT … FROM pg_settings → name/setting pairs matching SHOW
* Anything else mentioning pg_catalog. / pg_<name> / information_schema.
→ empty result with generic field names (BI tool sees "0 rows" rather
than a parse error)
Deliberate non-goals
--------------------
* WHERE / JOIN evaluation — psql, pgx, DBeaver all filter
client-side on the rows we return. We send the whole
catalog and let them apply their predicates.
* pg_attribute introspection — would need to re-derive
column types from the open workarea + map back to PG OIDs.
Tracked as v1.1 work.
* Recursive CTE catalog queries (pgAdmin's tree builder uses
them) — too brittle to pattern-match. Falls through to
five_SQL where it errors loudly. pgAdmin's table-tree pane
will then show "0 tables" but the connection itself stays
alive.
Files
-----
hbrtl/pgserver/catalog.go (new, ~280 LOC)
catalogIntercept(sql) → (handled, value)
synthPgNamespace / synthPgClass / synthPgAttribute /
synthPgType / synthPgDatabase / synthPgSettings
simpleSelectFunction (version/current_*/pg_backend_pid)
showResponse (SHOW <name>)
hbrtl/pgserver/dispatch.go
dispatchSimpleQuery: catalogIntercept ahead of runSQL.
hbrtl/pgserver/extended.go
executePortal: same intercept, ahead of runSQL.
Verification
------------
psql against a running pgserver, with sslmode=require + MD5:
$ psql -c 'SELECT version()' -At
PostgreSQL 14.0 (FiveSql2) (FiveSql2 wire-compat shim)
$ psql -c 'SELECT * FROM pg_namespace' -At
11|pg_catalog|10
2200|public|10
$ psql -c 'SELECT * FROM pg_type' -At
16|bool|1
23|int4|4
20|int8|8
25|text|-1
1700|numeric|-1
1082|date|4
1114|timestamp|8
$ psql -l # \\l now works
데이터베이스 목록
oid | datname | datdba | 인코딩
-----+---------+--------+--------
1 | alice | 10 | 6
Integration script gates grew from 6/6 → 9/9:
PASS Catalog probe: SELECT version()
PASS Catalog probe: pg_namespace lists public + pg_catalog
PASS Catalog probe: SHOW server_version_num
All six release gates green:
go test ./... ✓
FiveSql2 SQL:1999 43/43 ✓
Harbour compat 56/56 ✓
std.ch 17/17 ✓
FRB 7/7 ✓
pgserver integration 9/9 ✓ (+3 from catalog stubs)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 3b2dd365ad |
feat(pgserver): Phase 6 — TLS + source-IP allowlist
Closes the v1.0 hardening surface: encrypted transport + a
coarse pg_hba.conf-equivalent CIDR allowlist. Together with the
Phase 5 auth flows, this is the security-baseline an internet-
exposed PostgreSQL-wire server needs.
TLS subsystem
-------------
`hbrtl/pgserver/tls.go`:
* `LoadTLSFromFiles(certPath, keyPath)` — cert/key PEM pair load
with tls.VersionTLS12 floor. Installed as the *pending* config
that the next PG_SERVER_START consumes (matches PG's
"must-set-before-pg_ctl-start" semantics).
* `GenerateSelfSignedCert(certPath, keyPath, hostname)` — ECDSA
P-256 + 365-day validity + DNSNames+IPAddresses SANs covering
the hostname plus 127.0.0.1 / ::1. Dev/CI helper; production
ships a CA-signed cert via the loader.
* `upgradeToTLS()` wraps `tls.Server(conn, cfg).Handshake()` so
pgproto3 reads plaintext on top of the encrypted stream.
Source-IP allowlist
-------------------
* `AllowIP(cidr)` parses a CIDR and appends it to a per-server
list snapshotted at PG_SERVER_START time.
* `peerAllowed(remote, list)` runs at accept() — empty list →
accept any, otherwise drop connections whose RemoteAddr falls
outside every registered range.
* `ClearAllowList()` resets to allow-all.
Coarse but compatible with the "host alice 10.0.0.0/8 md5"-style
entries every pg_hba.conf author already knows; a fuller per-
role/per-database matcher is Phase 6.1+.
PRG bindings (register.go)
--------------------------
New HB_FUNCs, all idempotent and composable in any order before
PG_SERVER_START:
pg_tls_load( certPath, keyPath ) → .T. | cErr
pg_tls_self_signed( cert, key, hostname ) → .T. | cErr
pg_allow_ip( cidr ) → .T. | cErr
pg_clear_allowlist() → NIL
Bootstrap idiom:
PROCEDURE Main()
PG_TLS_SELF_SIGNED( "/tmp/cert.pem", "/tmp/key.pem", "localhost" )
PG_ADD_ROLE( "alice", "swordfish" )
PG_ALLOW_IP( "127.0.0.1/32" )
PG_ALLOW_IP( "10.0.0.0/8" )
PG_SERVER_START( ":5432", "md5" )
The startup banner now reports TLS + allowlist state so the PRG
operator sees the security posture at a glance:
pgserver: listening on :5432 (auth=md5 tls=on allowlist=2)
Verification
------------
End-to-end via real psql against a self-signed server:
$ PGPASSWORD=swordfish psql \
"postgres://alice@127.0.0.1:15432/alice?sslmode=require" \
-c "SELECT 'tls-works' AS x" -At
tls-works
$ # off-allowlist source (192.168.x.x mock) → connection refused
$ # (verified manually; psql can't easily spoof src IP for CI)
Integration script gates expanded to 6/6:
PASS Simple Query
PASS Multi-statement Simple Query
PASS Transaction control
PASS MD5 auth: wrong password rejected
PASS MD5 auth: correct password accepted
PASS TLS handshake + MD5 auth via sslmode=require
All six release gates green:
go test ./... ✓
FiveSql2 SQL:1999 43/43 ✓
Harbour compat 56/56 ✓
std.ch 17/17 ✓
FRB 7/7 ✓
pgserver integration 6/6 ✓
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 90eafcfc06 |
feat(pgserver): Phase 5 — password + MD5 authentication
Trust mode (v1.0 default) accepts anyone; that's fine for embedded
demo but unshipping a multi-client database without credentials
would be irresponsible. This commit adds two of libpq's three
standard auth flows. SCRAM-SHA-256 is Phase 5.1 — pgx/psql both
fall back to MD5 cleanly when the server advertises only md5, so
v1.0's functional coverage is complete with the pair landed here.
Auth subsystem
--------------
`hbrtl/pgserver/auth.go` adds:
* An in-memory role registry: `roleMap map[string]*role` guarded by
sync.RWMutex. Reads (lookupRole) are hot-path during connection
startup so the RWMutex lets multiple sessions auth in parallel
without serialising through a plain Mutex.
* `AddRole(name, password)` / `RemoveRole(name)` Go API consumed
by the new HB_FUNCs `PG_ADD_ROLE` / `PG_REMOVE_ROLE` (see
register.go). Bootstrap PRG idiom:
PG_ADD_ROLE("alice", "swordfish")
PG_ADD_ROLE("bob", "hunter2")
PG_SERVER_START(":5432", "md5")
* `authPassword()` — cleartext PasswordMessage exchange. The wire
payload is plain so intended for TLS-protected links only;
Phase 6 ties the warning to actual TLS detection on the session.
* `authMD5()` — libpq's md5 challenge:
server → AuthenticationMD5Password{salt: 4 random bytes}
client → "md5" || md5_hex( md5_hex(password || user) || salt )
We recompute the canonical hash from the stored plaintext and
compare. md5Challenge() is exported for pinning by a Go unit
test (vector cross-checked against libpq's fe-auth-md5.c).
Salt is sourced from crypto/rand on every challenge so replay
attacks against a captured wire trace can't reuse a prior hash.
Dispatch matrix (Config.AuthMode → flow):
"" / "trust" → AuthenticationOk immediately, no lookup
"password" → authPassword()
"md5" → authMD5()
anything else→ 28000 + connection close
Tests
-----
Unit (hbrtl/pgserver/pgserver_test.go):
PASS TestMD5Challenge (vector + determinism + diff)
PASS TestRoleRegistry (add/replace/remove/lookup)
Integration (tests/pgserver/run.sh):
PASS Simple Query: SELECT 1, 'hello'
PASS Multi-statement Simple Query
PASS Transaction control: BEGIN/COMMIT round-trip
PASS MD5 auth: wrong password rejected
PASS MD5 auth: correct password accepted
End-to-end matrix with real psql:
wrong password → "ERROR: md5 authentication failed for user 'alice'"
correct password → SELECT returns row
unknown user → "ERROR: md5 authentication failed for user 'eve'"
password mode → cleartext exchange works equivalently
All six release gates green:
go test ./... ✓
FiveSql2 SQL:1999 43/43 ✓
Harbour compat 56/56 ✓
std.ch 17/17 ✓
FRB 7/7 ✓
pgserver integration 5/5 ✓ (up from 3/3 in Phase 4)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 8472928102 |
feat(pgserver): Phase 4 — Extended Protocol (Parse/Bind/Execute)
pgx and most drivers default to PostgreSQL's Extended Protocol
(named prepared statements). Phase 2 only handled Simple Query,
so every pgx caller had to force `QueryExecModeSimpleProtocol` —
unworkable for a production deployment. This commit lands the
full Parse → Bind → Describe → Execute → Sync state machine,
enough that pgx (and any other libpq-protocol-v3 client) works
without any client-side knobs.
Implementation lives in `hbrtl/pgserver/extended.go`:
* Per-session caches `stmts map[string]*preparedStmt` and
`portals map[string]*portal`, lazily allocated on first use.
Stored as fields on `session` so they don't leak across
connections.
* Parameters are inlined at Bind time via `substituteParams` —
the resolved SQL is a normal Simple-Query-shaped string the
engine sees through the existing `five_SQL(cSQL, …, oSession)`
pipeline. Avoids teaching FiveSql2 a second param-shape; the
trade-off is that binary timestamps/numerics round-trip through
text (Phase 4.1 will plumb `?`-params through aParams for the
binary fast path).
* `paramToLiteral` decodes the binary-format encodings pgx uses
by default for INT4/INT8/BOOL (big-endian fixed-width). Other
binary OIDs fall back to a hex-escaped quoted literal which
errors loudly rather than silently misparsing.
* `countPgPlaceholders` scans the SQL outside string literals for
the highest `$N` so the server can answer Describe-statement
with a correctly-sized ParameterDescription even when the
client didn't pre-declare param OIDs. Without this, pgx errored
with "expected 0 arguments, got 2" on the very first prepared
query.
* RowDescription emission: Describe-statement still returns NoData
(we can't infer row shape without execution). When Execute fires
on a portal the client never Described, we emit RowDescription
inline from the cached result before DataRow streams. pgx and
psql both tolerate this ordering.
* Execute → CommandComplete tag derives from the SQL verb via the
existing `commandTagFor` helper. Row counts in the tag remain
"VERB 0" for v1.0; threading real counters through the engine
is Phase 5.
Wire dispatch in `session.go:queryLoop` now handles Parse, Bind,
Describe, Execute, Close, Sync, Flush — the full v3 message set.
Verification
------------
End-to-end pgx (default mode, no SimpleProtocol flag) successfully
runs:
SELECT $1 AS n, $2 AS s with 42 + "hi" → [42 hi]
Same statement re-executed with different bound values → reuses
the cached prepared statement
SELECT $1 AS b, $2 AS s with true + "binary-bool" → [t binary-bool]
`tests/pgserver/run.sh` expanded from 1 → 3 integration assertions:
PASS Simple Query: SELECT 1, 'hello'
PASS Multi-statement Simple Query
PASS Transaction control: BEGIN/COMMIT round-trip
(Extended Protocol can't be driven from psql's -c CLI directly
because psql's PREPARE/EXECUTE is a separate SQL-level feature
that FiveSql2 doesn't parse; the pgx-driven path verifies it
manually, and a self-contained Go integration that drives pgx
from inside a process bootstrap is Phase 7 work.)
All six release gates green:
go test ./... ✓
FiveSql2 SQL:1999 43/43 ✓
Harbour compat 56/56 ✓
std.ch 17/17 ✓
FRB 7/7 ✓
pgserver integration 3/3 ✓
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 708329785a |
test(pgserver): wire-protocol roundtrip via net.Pipe
Adds an in-process startup-handshake test using net.Pipe so we
can pin the protocol envelope (StartupMessage → AuthenticationOk
→ ParameterStatus×N → BackendKeyData → ReadyForQuery) without
binding a real TCP port. Runs in <1ms; safe for CI.
The PRG-dispatch path (runSQL → FIVE_SQL → row encoding) is
already covered manually by spinning a `five run` of
`pg_server_start(":15432")` and connecting with pgx — that flow
verified post-MVP that a real PostgreSQL client receives
`{ONE (INT4), GREET (TEXT)}` + row `[1 hello]` for
`SELECT 1 AS one, 'hello' AS greet` over the wire. An automated
shell harness will land in Phase 7 with the psql integration
tests.
Also rolls go.mod / go.sum forward with the pgx v5 toolchain pulled
in by Phase 2's pgproto3 dependency. Module bump 1.21.13 → 1.25.0
matches what `go get github.com/jackc/pgx/v5/pgproto3` selected;
cross-builds for windows/linux/darwin all still succeed (verified
locally).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| d98f5e1767 |
feat(pgserver): PostgreSQL-wire MVP — psql can SELECT from FiveSql2
First end-to-end working version of the PostgreSQL-wire-compatible
TCP server frontend. A standard `psql` client now connects, runs
`SELECT * FROM employees`, and gets back a properly typed result
set rendered by psql with the right column alignment:
ID | NAME | SALARY
----+----------------------+----------
1 | Alice | 50000.00
2 | Bob | 42000.50
3 | Cho | 77500.00
This is the Phase 2 deliverable from the approved plan at
/Users/charleskwon/.claude/plans/compiled-launching-shore.md.
Builds on the session-state refactor in
|
