Article

How far can a Lua-integrated psql go? Inside the VACUUM progress wrapper

An experimental patch embeds Lua in psql so users can define client-side behavior. Using its VACUUM wrapper as the concrete example, this article explains the two-connection monitoring design, the meaning of PostgreSQL progress fields, and the compatibility checks required before testing it.

Share

Koharu's reading tip

The useful idea is not Lua syntax itself, but the design that turns psql into a small operational console. We will separate the experiment from shipped functionality while extracting a monitoring pattern that is useful today.

Koharu's reading tip

An experimental patch published on August 29, 2026 embeds Lua in PostgreSQL's psql client and lets users bring functions into the terminal. Its concrete example wraps VACUUM, processes relations one at a time, and presents progress in a more readable form.

The interesting question is not merely whether Lua can run. It is how the same terminal can keep observing the database while a long SQL command occupies a connection. That turns connection management, usually hidden behind a script, into the central design problem.

Following the VACUUM wrapper shows what the patch is trying to change, why it needs two connections, and how far it can reasonably be treated as an operational tool today.

Lua turns psql behavior into user-defined functions

Today's psql is more than an interactive SQL prompt. It provides backslash meta-commands, variables, conditionals, and other scripting features; the PostgreSQL 18 psql manual describes it as a terminal front end for both administration and automation.

The experimental repository adds commands such as \luacode, \luafile, \lua, and \luaset. Lua code can run SQL, iterate over results, use psql's display functions, and even register new backslash commands.

An initial demonstration from August 6, 2026 defined and called a small function with \luacode. The VACUUM example moves further by loading a function from a file and invoking it with an option.

text
\luafile ~/vacuum.lua
\lua vacuum {verbose=true}

The useful shift is therefore not just moving between SQL and Lua. It is the ability to combine several queries, formatting, and wait logic behind one terminal operation.

VACUUM execution and progress monitoring use separate connections

The wrapper's core design is to split database work across two connections. After collecting the target relations, the Lua function clones the current connection settings, sends VACUUM through the clone, and records that backend's PID.

text
psql and Lua
├─ cloned connection: run VACUUM asynchronously
└─ current connection: poll progress for the target PID

A PostgreSQL connection cannot process a monitoring query while its backend is busy executing another command. Separating the maintenance connection from the observation connection is what allows execution and monitoring to happen together.

The waiting loop combines operations corresponding to sendquery, consumeinput, isbusy, and getresult. PostgreSQL's libpq asynchronous command documentation defines the state transition: send a query, consume available input, inspect whether the command remains busy, and retrieve all results when they become available. Lua makes that state machine expressible inside psql.

pg_stat_progress_vacuum does not report one overall percentage

While a regular VACUUM is running, pg_stat_progress_vacuum exposes one row per active backend. The official progress-reporting reference defines fields such as phase, heap_blks_total, heap_blks_scanned, indexes_total, and indexes_processed.

If heap_blks_scanned is 8 million and heap_blks_total is 10 million, the heap-scanning phase is 80% complete. That does not mean the complete VACUUM job is 80% done: index vacuuming, heap vacuuming, truncation, and final cleanup may still follow.

The index counters become meaningful only during the relevant index phases. VACUUM FULL is another boundary because its progress appears in pg_stat_progress_cluster, not pg_stat_progress_vacuum. A readable monitor must therefore interpret values by phase rather than place every counter on one percentage scale.

The progress view itself was introduced in PostgreSQL 9.6. The indexes_total and indexes_processed fields printed by this example were added in PostgreSQL 17. Testing against an older server requires checking the monitoring query's column compatibility as well as building a Lua-enabled client.

A Lua wrapper does not change VACUUM permissions or load

Wrapping VACUUM in Lua does not alter the command's server-side rules. Under the VACUUM reference, the role ordinarily needs the MAINTAIN privilege on each target, VACUUM cannot run inside a transaction block, and its I/O can affect other active sessions.

At least three parts of the proof of concept need review before it approaches operational use.

  • Relation selection: the sample filters pg_class.relkind for r, s, and n, while the current pg_class catalog reference defines r for ordinary tables, m for materialized views, p for partitioned tables, and uppercase S for sequences. The target policy should be made explicit.
  • Argument handling: the sample evaluates options with load("return " .. args). It should not become an input path for untrusted strings; accepted keys and values need to be constrained.
  • Asynchronous result handling: libpq does not allow another PQsendQuery on the same connection until PQgetResult has returned null. The consecutive sendquery calls in the sample's verbose branch need testing, including their return values and errors.

Testing requires a Lua-enabled PostgreSQL master build

The repository describes the work as an experimental patch for the PostgreSQL master branch. Its build path compiles PostgreSQL itself with Lua enabled; it is not an extension installed into an existing packaged psql.

sh
./configure --with-lua
make all
sudo make install

This is not yet a shipped feature or a stable plugin API. The available material does not establish an upstream review state, a target release, or an API compatibility promise, so an isolated build and a test cluster are more appropriate than replacing an existing administration client.

The prototype nevertheless suggests a useful boundary. Existing psql variables and fixed meta-commands can remain the default, while interactive workflows that need loops, formatting, and multiple connections could be candidates for Lua.

The design points toward a small operational console

The answer to the opening question is not that Lua instantly turns psql into a universal automation platform. The patch demonstrates that exposing psql's connection, query, and display facilities to a scripting language could let users package recurring administration tasks as their own commands.

The immediately reusable lesson from the VACUUM example is to separate execution from observation and interpret progress by phase. Even if Lua integration is not yet something to adopt, those two ideas transfer directly to external scripts and existing operational tools.

Source

Share

Related Articles

These articles share nearby categories or tags, so you can keep reading along the same thread.