centos7.9上面搭建superset开发环境

公司需要在superset上面接入sso单点登录,目前superset还不支持,需要在superset上面进行二次开发。曾经尝试过在windows10上面搭建开发环境,无奈python虚拟环境运行的各种失败后,放弃。
也尝试了在Ubuntu20.04上面搭建开发环境,最终npm打包功能不断失败,最终也放弃,在Ubuntu16上面尝试安装能够成功,但是桌面UI做的太烂了,遂放弃。
总结了下,Ubuntu20.04的桌面做的很炫,比windows做的漂亮,但是内核版本太高5.11,对于目前很多软件的兼容性不是太好。windows就别提了,superset官网说的好像是放弃了windows环境上面的开发。所以,最后尝试在centos7.9的桌面版本上面部署开发环境得以成功,记录本文如下。

在win10上面centos7.9桌面版本虚拟机

安装VMware

在VMware上面创建centos虚拟机

参考文档就不写了,网上搜索一大堆;
开通防火墙

firewall-cmd --zone=public --add-port=9000/tcp --permanent
 firewall-cmd --zone=public --add-port=8088/tcp --permanent


挂载磁盘过程,参考:

装anaconda环境

  1. 下载anaconda
    进入到anaconda官网下载linux版本:https://www.anaconda.com/download/#linux
  2. 安装anaconda
#按照漫长的提示,完成安装anaconda
[cyxinda@centos7 superset]$ bash Anaconda3-2021.05-Linux-x86_64.sh
# 加载环境变量
[cyxinda@centos7 superset]$ source ~/.bashrc	
[cyxinda@centos7 superset]$ conda --version
conda 4.10.1
  1. 创建虚拟环境
#创建虚拟环境
(base) [cyxinda@centos7 superset]$ conda create -n python3.8.11 python=3.8.11

(python3.8.11) [cyxinda@centos7 superset]$ conda env list
# conda environments:
#
base                     /home/cyxinda/anaconda3
python3.8.11          *  /home/cyxinda/anaconda3/envs/python3.8.11

#切换虚拟环境
(base) [cyxinda@centos7 superset]$ source activate python3.8.11
(python3.8.11) [cyxinda@centos7 superset]$
  1. 修改pip源
(python3.8.11) [cyxinda@centos7 ~]$ mkdir ~/.pip
(python3.8.11) [cyxinda@centos7 ~]$ vim ~/.pip/pip.conf
[global]
trusted-host =  mirrors.aliyun.com
index-url = http://mirrors.aliyun.com/pypi/simple/

下载superset源码

加速github

如下两种方法任意即可

下载superset源码

superset的github托管地址为:https://github.com/apache/superset 找到superset的1.3.1release下载地址:https://github.com/apache/superset/releases/tag/1.3.1 下载源码:

(python3.8.11) [cyxinda@centos7 superset]$ wget https://github.com/apache/superset/archive/refs/tags/1.3.1.zip
(python3.8.11) [cyxinda@centos7 superset]$ unzip 1.3.1.zip
(python3.8.11) [cyxinda@centos7 superset]$ ls
1.3.1.zip  Anaconda3-2021.05-Linux-x86_64.sh  install.sh  superset-1.3.1

目录结构如下:

code
└── superset
    ├── 1.3.1.zip
    ├── Anaconda3-2021.05-Linux-x86_64.sh
    ├── install.sh
    └── superset-1.3.1
        ├── CHANGELOG.md
        ├── CODE_OF_CONDUCT.md
        ├── CONTRIBUTING.md
        ├── docker
        ├── docker-compose-non-dev.yml
        ├── docker-compose.yml
        ├── Dockerfile
        ├── docs
        ├── helm
        ├── INSTALL.md
        ├── LICENSE.txt
        ├── lintconf.yaml
        ├── Makefile
        ├── MANIFEST.in
        ├── NOTICE
        ├── pytest.ini
        ├── README.md
        ├── RELEASING
        ├── requirements
        ├── RESOURCES
        ├── scripts
        ├── setup.cfg
        ├── setup.py
        ├── superset
        ├── superset-frontend
        ├── superset-websocket
        ├── tests
        ├── tox.ini
        └── UPDATING.md

构建superset

  1. 根据开发文档,构建superset
    文档地址:https://github.com.cnpmjs.org/apache/superset/blob/master/CONTRIBUTING.md
  2. 安装依赖
(python3.8.11) [cyxinda@centos7 superset-frontend]$ pwd
/code/superset/superset-1.3.1/
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ pip install -r requirements/testing.txt
Looking in indexes: http://mirrors.aliyun.com/pypi/simple/
Obtaining file:///. (from -r requirements/base.txt (line 8))
Collecting aiohttp==3.7.2
  Downloading http://mirrors.aliyun.com/pypi/packages/63/c8/7c58c9ef4b7ac8f12c83b21e20e6566c48660ae49e403afa025d01df2550/aiohttp-3.7.2-cp38-cp38-manylinux2014_x86_64.whl (1.5 MB)
     |████████████████████████████████| 1.5 MB 1.8 MB/s
# 经过漫长的等待,以及调错即可完成
  1. 安装superset
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ pwd
/code/superset/superset-1.3.1
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ pip install -e .
  1. 创建管理员用户
# Create an admin user in your metadata database (use `admin` as username to be able to load the examples)
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ superset fab create-admin
Username [admin]: admin
User first name [admin]: cao
User last name [user]: yong
Email [admin@fab.org]: caoyong1@lenovo.com
Password:
Repeat for confirmation:
Recognized Database Authentications.
Admin User admin created.
# Initialize the database
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ superset db upgrade
# Create default roles and permissions
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ superset init
  • 异常情况1:
    ERROR: No matching distribution found for mysqlclient==1.4.2.post1
    参考:
    需要安装
sudo yum install python3-devel mysql-devel
  • 异常情况2:
    Running setup.py install for sasl … error
    参考:
    需要安装依赖包:
(python3.8.11) [cyxinda@centos7 superset-1.3.1]$ sudo yum -y install cyrus-sasl cyrus-sasl-devel cyrus-sasl-lib
  1. 构建前端资源
  • 下载nvm安装文件
(python3.8.11) [cyxinda@centos7 superset-frontend]$ get https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.0/install.sh
#执行nvm安装
(python3.8.11) [cyxinda@centos7 superset]$ bash install.sh
=> Downloading nvm from git to '/home/cyxinda/.nvm'
=> Cloning into '/home/cyxinda/.nvm'...
remote: Enumerating objects: 317, done.
remote: Counting objects: 100% (317/317), done.
remote: Compressing objects: 100% (269/269), done.
remote: Total 317 (delta 36), reused 136 (delta 23), pack-reused 0
Receiving objects: 100% (317/317), 170.27 KiB | 0 bytes/s, done.
Resolving deltas: 100% (36/36), done.
=> Compressing and cleaning up git repository

=> Appending nvm source string to /home/cyxinda/.bashrc
=> Appending bash_completion source string to /home/cyxinda/.bashrc
=> Close and reopen your terminal to start using nvm or run the following to use                                                                   it now:

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads                                                                   nvm bash_completion
(python3.8.11) [cyxinda@centos7 superset]$ nvm
bash: nvm: command not found...
(python3.8.11) [cyxinda@centos7 superset]$ source ~/.bashrc
(base) [cyxinda@centos7 superset]$ nvm --version
0.37.0
(base) [cyxinda@centos7 superset-frontend]$ nvm use --lts
Now using node v14.18.1 (npm v6.14.15)
  • 安装npm和nodejs
(base) [cyxinda@centos7 superset-frontend]$ pwd
/code/superset/superset-1.3.1/superset-frontend
(base) [cyxinda@centos7 superset-frontend]$ nvm install --lts
Installing latest LTS version.
Downloading and installing node v14.18.1...
Downloading https://nodejs.org/dist/v14.18.1/node-v14.18.1-linux-x64.tar.xz...
########################################################################################################################################### 100.0%
Computing checksum with sha256sum
Checksums matched!
Now using node v14.18.1 (npm v6.14.15)
Creating default alias: default -> lts/* (-> v14.18.1)
(base) [cyxinda@centos7 superset-frontend]$ npm -version
6.14.15
  • 安装node依赖包
(base) [cyxinda@centos7 superset-frontend]$ pwd
/code/superset/superset-1.3.1/superset-frontend
(base) [cyxinda@centos7 superset-frontend]$ npm ci
#漫长的等待过程,可能重复执行几次才能成功。。。。
  • 安装前端资源
(base) [cyxinda@centos7 superset-frontend]$ npm run build
#经过漫长的等待

搭建superset开发环境

  1. 下载pycharm IDE
    首先在下面的网址下载安装包:
    https://www.jetbrains.com/pycharm/download/#section=linux
(python3.8.11) [cyxinda@centos7 pycharm]$ pwd
/code/ide/pycharm
(python3.8.11) [cyxinda@centos7 pycharm]$ tar -xvf pycharm-community-2021.2.2.tar.gz
(python3.8.11) [cyxinda@centos7 pycharm]$ cd pycharm-community-2021.2.2/
(python3.8.11) [cyxinda@centos7 pycharm-community-2021.2.2]$ pwd
/code/ide/pycharm/pycharm-community-2021.2.2
(python3.8.11) [cyxinda@centos7 pycharm-community-2021.2.2]$ cd bin
  1. 在centos桌面上面启动pycharm
    可以直接使用脚本启动:
(python3.8.11) [cyxinda@centos7 bin]$ ./pycharm.sh

并且,为了方便起见,可以创建桌面的快捷方式

[root@centos7 ~]# vim /usr/share/applications/Pycharm.desktop
#在Pycharm.desktop中编辑如下内容,强制保存:
[Desktop Entry]
Type=Application
Name=Pycharm
GenericName=Pycharm3
Comment=Pycharm3:The Python IDE
Exec=sh /code/ide/pycharm/pycharm-community-2021.2.2/bin/pycharm.sh
Icon=/code/ide/pycharm/pycharm-community-2021.2.2/bin/pycharm.png
Terminal=pycharm
Categories=Pycharm;

在桌面上面出现快捷方式如下:

centOS7支持4090显卡吗 centos7支持九代cpu吗_centOS7支持4090显卡吗


单击快捷方式,打开pycharm:

centOS7支持4090显卡吗 centos7支持九代cpu吗_linux_02

  1. 启动superset
    打开位于/code/superset/superset-1.3.1的项目:点击Open按钮图标,选择位置
  2. 设置Python Interpreter
    在Settings中,设置上文中创建好的python Interpreter,如下图:
  3. 创建启动程序
    在目录/code/superset/superset-1.3.1/中创建superset_run.py

    内容如下:
from superset.app import create_app
app = create_app()
if __name__ == '__main__':
    app.debug = True # 设置调试模式,生产模式的时候要关掉debug
    app.run(host='0.0.0.0', port=8888, debug=True)

debug模式启动superset_run.py

/home/cyxinda/anaconda3/envs/python3.8.11/bin/python /code/ide/pycharm/pycharm-community-2021.2.2/plugins/python-ce/helpers/pydev/pydevd.py --multiproc --qt-support=auto --client 127.0.0.1 --port 46778 --file /code/superset/superset-1.3.1/superset_run.py
Connected to pydev debugger (build 212.5284.44)
logging was configured successfully
2021-10-16 03:24:59,197:INFO:superset.utils.logging_configurator:logging was configured successfully
2021-10-16 03:24:59,220:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/home/cyxinda/anaconda3/envs/python3.8.11/lib/python3.8/site-packages/flask_caching/__init__.py:201: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
  warnings.warn(
 * Serving Flask app "superset.app" (lazy loading)
 * Environment: development
 * Debug mode: on
2021-10-16 03:25:08,026:INFO:werkzeug: * Running on http://0.0.0.0:8888/ (Press CTRL+C to quit)
2021-10-16 03:25:08,029:INFO:werkzeug: * Restarting with stat
2021-10-16 03:25:13,091:INFO:superset.utils.logging_configurator:logging was configured successfully
2021-10-16 03:25:13,113:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/home/cyxinda/anaconda3/envs/python3.8.11/lib/python3.8/site-packages/flask_caching/__init__.py:201: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
  warnings.warn(
2021-10-16 03:25:21,591:WARNING:werkzeug: * Debugger is active!
2021-10-16 03:25:21,650:INFO:werkzeug: * Debugger PIN: 320-957-409
  1. 前端访问验证

    在/code/superset/superset-1.3.1/superset/initialization/init.py的723行,添加断点
    输入用户名密码admin/****后:
  • 后端:
    F9继续执行
  • 前端

至此,superset在centos7.9上面的开发环境,搭建完毕。

如上,是通过安装过程启动superset,但是实际开发,应该是前后端分离的。所以,应该先启动后端项目;再启动前端项目,分开启动,分开调试。

  1. 后端在pycharm中直接启动,不用执行npm run build,启动了默认端口是8088
  2. 启动vscode,加载前端项目/code/superset/superset-1.3.1/superset-frontend,如下图:

如果后端项目启动的不是默认的8088端口,而是8081,则在vscode中,需要执行的命令变为:npm run dev-server – --supersetPort=8081

接下来就可以使用vscode进行代码调试了。