pgspider influxdb 扩展使用以及docker 镜像构建说明
influxdb 是官方提供的fdw，对于influxdb 的访问是通过cgo 直接到处golang 的调用为c 调用（实际上也有一个直接使用golang 编写pg 扩展的项目plgo）
一些说明
对于pgspider 的构建是通过一个base 镜像构建的，但是以为依赖golang 所以提供了一个额外的golang base 镜像，同时在
测试的时候发现base 镜像的基础镜像对于golang 的cgo 支持有问题（debian:stretch-slim）所以使用了debian:buster，golang
版本使用13.3
- Dockerfile-base-go
FROM debian:busterRUN apt-get update && apt-get install -y build-essential git wget libreadline-dev zlib1g-devWORKDIR /appRUN set -eux; \ if [ -f /etc/dpkg/dpkg.cfg.d/docker ]; then \ # if this file exists, we're likely in "debian:xxx-slim", and locales are thus being excluded so we need to remove that exclusion (since we need locales) grep -q '/usr/share/locale' /etc/dpkg/dpkg.cfg.d/docker; \ sed -ri '/\/usr\/share\/locale/d' /etc/dpkg/dpkg.cfg.d/docker; \ ! grep -q '/usr/share/locale' /etc/dpkg/dpkg.cfg.d/docker; \ fi; \ apt-get update; apt-get install -y locales; rm -rf /var/lib/apt/lists/*; \ localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8ENV LANG en_US.utf8RUN set -eux; \ apt-get update; \ apt-get install -y --no-install-recommends libnss-wrapper; \ rm -rf /var/lib/apt/lists/*RUN wget https://ftp.postgresql.org/pub/source/v11.6/postgresql-11.6.tar.gzRUN wget https://raw.githubusercontent.com/pgspider/pgspider/master/pgspider.patchRUN tar xvf postgresql-11.6.tar.gzRUN patch -p1 -d postgresql-11.6 < /app/pgspider.patchRUN cd postgresql-11.6 \ && ./configure \ && make && make install \ && cd /app/postgresql-11.6/contrib/pgspider_core_fdw \ && make && make install \ && cd /app/postgresql-11.6/contrib/pgspider_fdw \ && make && make install \ && cd /app/postgresql-11.6/contrib \ && make && make installENV PATH $PATH:/usr/local/pgspider/bininflux pg 扩展构建
- dockerfile
FROM dalongrong/pgspider:base-go as buildWORKDIR /appRUN wget https://dl.google.com/go/go1.13.3.linux-amd64.tar.gz && tar zxvf go1.13.3.linux-amd64.tar.gzENV PATH $PATH:/app/go/binENV GOPATH /opt/goRUN go get github.com/influxdata/influxdb1-client/v2RUN git clone https://github.com/pgspider/influxdb_fdw.git /app/postgresql-11.6/contrib/influxdb_fdwRUN cd /app/postgresql-11.6/contrib/influxdb_fdw && make USE_PGXS=1 with_llvm=no && make install USE_PGXS=1 with_llvm=noFROM debian:stretch-slimENV GOSU_VERSION 1.11RUN apt-get update && apt-get install -y wget# explicitly set user/group IDsRUN set -eux; \ groupadd -r postgres --gid=999; \# https://salsa.debian.org/postgresql/postgresql-common/blob/997d842ee744687d99a2b2d95c1083a2615c79e8/debian/postgresql-common.postinst#L32-35 useradd -r -g postgres --uid=999 --home-dir=/var/lib/postgresql --shell=/bin/bash postgres; \# also create the postgres user's home directory with appropriate permissions# see https://github.com/docker-library/postgres/issues/274 mkdir -p /var/lib/postgresql; \ chown -R postgres:postgres /var/lib/postgresqlRUN wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture)" \ && chmod +x /usr/local/bin/gosu \ && gosu nobody true# make the "en_US.UTF-8" locale so postgres will be utf-8 enabled by defaultRUN set -eux; \ if [ -f /etc/dpkg/dpkg.cfg.d/docker ]; then \ # if this file exists, we're likely in "debian:xxx-slim", and locales are thus being excluded so we need to remove that exclusion (since we need locales) grep -q '/usr/share/locale' /etc/dpkg/dpkg.cfg.d/docker; \ sed -ri '/\/usr\/share\/locale/d' /etc/dpkg/dpkg.cfg.d/docker; \ ! grep -q '/usr/share/locale' /etc/dpkg/dpkg.cfg.d/docker; \ fi; \ apt-get update; apt-get install -y locales; rm -rf /var/lib/apt/lists/*; \ localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8ENV LANG en_US.utf8# install "nss_wrapper" in case we need to fake "/etc/passwd" and "/etc/group" (especially for OpenShift)# https://github.com/docker-library/postgres/issues/359# https://cwrap.org/nss_wrapper.htmlRUN set -eux; \ apt-get update; \ apt-get install -y --no-install-recommends libnss-wrapper; \ rm -rf /var/lib/apt/lists/*RUN mkdir /docker-entrypoint-initdb.dCOPY --from=build /usr/local/pgspider /usr/local/pgspiderRUN sed -ri "s!^#?(listen_addresses)\s*=\s*\S+.*!\1 = '*'!" /usr/local/pgspider/share/postgresql/postgresql.conf.sample; \ grep -F "listen_addresses = '*'" /usr/local/pgspider/share/postgresql/postgresql.conf.sampleRUN mkdir -p /var/run/postgresql && chown -R postgres:postgres /var/run/postgresql && chmod 2777 /var/run/postgresqlENV PATH $PATH:/usr/local/pgspider/binENV PGDATA /var/lib/postgresql/dataRUN mkdir -p "$PGDATA" && chown -R postgres:postgres "$PGDATA" && chmod 777 "$PGDATA"VOLUME /var/lib/postgresql/dataCOPY docker-entrypoint.sh /usr/local/bin/RUN ln -s usr/local/bin/docker-entrypoint.sh / # backwards compatENTRYPOINT ["docker-entrypoint.sh"]EXPOSE 5432CMD ["postgres"]使用
依赖的influxdb 使用docker-compose 运行
- docker-compose 文件
version: "3"services: pgspider-influxdb: image: dalongrong/pgspider:influxdb ports: - "5432:5432" influxdb: image: influxdb environment: - "INFLUXDB_ADMIN_USER=dalong" - "INFLUXDB_ADMIN_PASSWORD=dalong" - "INFLUXDB_USER=demo" - "INFLUXDB_USER_PASSWORD=demo" - "INFLUXDB_DB=mydb" ports: - "8086:8086" chronograf: image: chronograf ports: - "8888:8888" command: --influxdb-url=http://influxdb:8086- 启动
docker-compose up -d- 使用chronograf添加测试数据
- 使用扩展
// 启动扩展CREATE EXTENSION influxdb_fdw;// 创建serverCREATE SERVER influxdb_server FOREIGN DATA WRAPPER influxdb_fdw OPTIONS(dbname 'mydb', host 'http://influxdb', port '8086') ;// 创建用户映射CREATE USER MAPPING FOR CURRENT_USER SERVER influxdb_server OPTIONS(user 'dalong', password 'dalong');// 创建外部表 方法一 直接定义CREATE FOREIGN TABLE t1(time timestamp with time zone , age int,name text, email text,user_id int) SERVER influxdb_server OPTIONS (table 'demouser');// 查询数据SELECT * FROM t1;// 创建外部表 方法二 使用schem 导入IMPORT FOREIGN SCHEMA public FROM SERVER influxdb_server INTO public;// 查询数据select * from demouser;- 效果
参考资料
https://github.com/pgspider/influxdb_fdw
https://github.com/pgspider/pgspider
https://github.com/rongfengliang/pgspider-docker
https://hub.docker.com/repository/docker/dalongrong/pgspider
https://github.com/influxdata/influxdb
https://github.com/influxdata/chronograf
