Skip to content

Commit 6c28097

Browse files
committed
Add pg_lake extension for Iceberg and data lake access
- Add pg_lake extension from Snowflake Labs to the Docker image - Build pg_lake with all core extensions (pg_map, pg_extension_base, pg_extension_updater, pg_lake_engine, pg_lake_copy, pg_lake_iceberg, pg_lake_table, pg_lake) - Build and include Apache Avro library required by pg_lake - Add runtime dependencies for pg_lake (snappy, jansson, lz4, xz, zstd, libpq) - Update README to document pg_lake extension - Update Makefile test to verify pg_lake extension loads correctly Note: DuckDB/pgduck_server integration is not included due to Alpine Linux compatibility constraints. The core pg_lake extensions provide Iceberg table support and data lake file access capabilities.
1 parent c0019b3 commit 6c28097

3 files changed

Lines changed: 48 additions & 4 deletions

File tree

Dockerfile

Lines changed: 45 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,12 @@
1-
# Lean PostgreSQL image with pgvector, PostGIS, pg_textsearch, and pgsodium
1+
# Lean PostgreSQL image with pgvector, PostGIS, pg_textsearch, pgsodium, and pg_lake
22
# Multi-stage build - all toolchains discarded, only artifacts kept
33

44
ARG PG_VERSION=17
55
ARG PGVECTOR_VERSION=0.8.0
66
ARG POSTGIS_VERSION=3.5.1
77
ARG PG_TEXTSEARCH_VERSION=0.2.0
88
ARG PGSODIUM_VERSION=3.1.9
9+
ARG PG_LAKE_VERSION=main
910

1011
#############################################
1112
# Stage 1: Build extensions
@@ -16,6 +17,7 @@ ARG PGVECTOR_VERSION
1617
ARG POSTGIS_VERSION
1718
ARG PG_TEXTSEARCH_VERSION
1819
ARG PGSODIUM_VERSION
20+
ARG PG_LAKE_VERSION
1921

2022
RUN apk add --no-cache \
2123
git \
@@ -36,7 +38,18 @@ RUN apk add --no-cache \
3638
# PostGIS build tools
3739
perl \
3840
flex \
39-
bison
41+
bison \
42+
# pg_lake dependencies
43+
cmake \
44+
ninja \
45+
openssl-dev \
46+
snappy-dev \
47+
jansson-dev \
48+
lz4-dev \
49+
xz-dev \
50+
zstd-dev \
51+
libpq-dev \
52+
linux-headers
4053

4154
WORKDIR /build
4255

@@ -70,6 +83,25 @@ RUN git clone --branch v${PGSODIUM_VERSION} --depth 1 https://github.com/michelp
7083
make -j$(nproc) && \
7184
make install
7285

86+
# pg_lake - Postgres with Iceberg and data lake access
87+
RUN git clone --branch ${PG_LAKE_VERSION} --depth 1 --recurse-submodules https://github.com/Snowflake-Labs/pg_lake.git && \
88+
cd pg_lake && \
89+
# Build and install avro library
90+
cd avro && git checkout -f . && git apply --ignore-whitespace ../avro.patch && \
91+
mkdir -p lang/c/build && cd lang/c/build && \
92+
cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=RelWithDebInfo && \
93+
make -j$(nproc) && make install && \
94+
cd /build/pg_lake && \
95+
# Build pg_lake extensions (without DuckDB/pgduck_server for Alpine compatibility)
96+
make -C pg_map && make -C pg_map install && \
97+
make -C pg_extension_base && make -C pg_extension_base install && \
98+
make -C pg_extension_updater && make -C pg_extension_updater install && \
99+
make -C pg_lake_engine && make -C pg_lake_engine install && \
100+
make -C pg_lake_copy && make -C pg_lake_copy install && \
101+
make -C pg_lake_iceberg && make -C pg_lake_iceberg install && \
102+
make -C pg_lake_table && make -C pg_lake_table install && \
103+
make -C pg_lake && make -C pg_lake install
104+
73105
#############################################
74106
# Stage 2: Final lean runtime image
75107
#############################################
@@ -84,11 +116,20 @@ RUN apk add --no-cache \
84116
protobuf-c \
85117
libxml2 \
86118
pcre2 \
87-
libsodium
119+
libsodium \
120+
# pg_lake runtime dependencies
121+
snappy \
122+
jansson \
123+
lz4-libs \
124+
xz-libs \
125+
zstd-libs \
126+
libpq
88127

89128
# Copy compiled extensions from builder
90129
COPY --from=builder /usr/local/lib/postgresql/ /usr/local/lib/postgresql/
91130
COPY --from=builder /usr/local/share/postgresql/ /usr/local/share/postgresql/
131+
# Copy avro library for pg_lake
132+
COPY --from=builder /usr/local/lib/libavro* /usr/local/lib/
92133

93134
LABEL org.opencontainers.image.source="https://github.com/constructive-io/docker"
94-
LABEL org.opencontainers.image.description="PostgreSQL 17 with pgvector, PostGIS, pg_textsearch, and pgsodium"
135+
LABEL org.opencontainers.image.description="PostgreSQL 17 with pgvector, PostGIS, pg_textsearch, pgsodium, and pg_lake"

Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ test: build
4040
CREATE EXTENSION postgis; \
4141
CREATE EXTENSION pg_textsearch; \
4242
CREATE EXTENSION pgsodium; \
43+
CREATE EXTENSION pg_lake; \
4344
SELECT 'all extensions OK';"
4445
@docker stop $(CONTAINER_NAME)-test > /dev/null
4546
@docker rm $(CONTAINER_NAME)-test > /dev/null

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ Lean PostgreSQL 17 image with essential extensions for modern applications.
1414
| [PostGIS](https://postgis.net/) | Spatial and geographic data |
1515
| [pg_textsearch](https://www.tigerdata.com/docs/use-timescale/latest/extensions/pg-textsearch) | BM25 full-text search |
1616
| [pgsodium](https://github.com/michelp/pgsodium) | Encryption using libsodium |
17+
| [pg_lake](https://github.com/Snowflake-Labs/pg_lake) | Iceberg and data lake access |
1718

1819
## Usage
1920

@@ -36,6 +37,7 @@ CREATE EXTENSION vector;
3637
CREATE EXTENSION postgis;
3738
CREATE EXTENSION pg_textsearch;
3839
CREATE EXTENSION pgsodium;
40+
CREATE EXTENSION pg_lake;
3941
```
4042

4143
## Build

0 commit comments

Comments
 (0)