安装Python3

./configure --prefix=/usr/local/python360make && make install#备份python 和pipmv /usr/bin/python /usr/bin/python_bakmv /usr/bin/pip/usr/bin/pip_bak#添加Python3软连接ln -s /usr/local/python360/bin/python3.6 /usr/bin/pythonln -s /usr/local/python360/bin/pip3 /usr/bin/pip

官网的很简单：

export AIRFLOW_HOME=~/airflow# install from pypi using pippip install apache-airflow# initialize the databaseairflow initdb# start the web server, default port is 8080airflow webserver -p 8080

airflow默认的数据库是sqlite3,当你初始化数据库的时候会报错：

ModuleNotFoundError: No module named '_sqlite3'

一般解决方法：

yum install sqlite-devel重新编译Python：./configure --prefix=/usr/local/python360make && make install

如果还是报错：可以重新安装sqlite3

wget https://www.sqlite.org/2017/sqlite-autoconf-3170000.tar.gz --no-check-certificatetar zxvf sqlite-autoconf-3170000.tar.gzcd sqlite-autoconf-3170000./configure --prefix=/usr/local/sqlite3 --disable-static --enable-fts5 --enable-json1 CFLAGS="-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1"make && make install

然后重新编译Python：

LD_RUN_PATH=/usr/local/sqlite3/lib ./configure --prefix=/usr/local/python360 LDFLAGS="-L/usr/local/sqlite3/lib" CPPFLAGS="-I /usr/local/sqlite3/include"LD_RUN_PATH=/usr/local/sqlite3/lib makeLD_RUN_PATH=/usr/local/sqlite3/lib make install

目前使用的是mysql数据库：

运行 mysql -uroot -p 报错：

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)

解决方法：重启mysql

/etc/rc.d/init.d/mysqld restart

发现没有这个文件，解决方法是未安装mysql-server：

yum install mysql-server

建用户：

#设置root密码mysql -uroot -pSET PASSWORD=PASSWORD("123");FLUSH PRIVILEGES;mysql -uroot -p123create user 'airflow'@'%' identified by '123';flush privileges;create database airflowdb;grant all privileges on `airflowdb`.* to 'airflow'@'%' identified by '123';flush privileges;

运行命令 airflow  生成在根目录文件 ~/airflow,修改airflow.cfg配置文件

vi ~/airflow/airflow.cfgsql_alchemy_conn = mysql://airflow:123@xx.xx.xx.xx/airflowdbpip install cryptography mysqlclientyum install MySQL-python#初始化airflow数据库airflow initdb

运行airflow webserver 外网访问

vi ~/airflow/airflow.cfg[webserver]endpoint_url = http://xx.xx.xx.xx:8081base_url = http://xx.xx.xx.xx:8081web_server_port = 8081airflow webserver

浏览器输入：  http://xx.xx.xx.xx:8081

 

界面加上用户在~/airflow/airflow.cfg [webserver]下添加（其他模块会有同名，不要加错了）：

authenticate = Trueauth_backend = airflow.contrib.auth.backends.password_auth

并且在~/airflow/目录下运行以下代码：

cd ~/airflow./add_user.py 
     
      
      
     
    

add_user.py 代码：

#!/usr/bin/python# This little script creates users in an airflow instance so it can be open to the public.# It gets the password in plain text so be careful where you run it.# You can properly invoke the script as follows:# ./add_user.py 
     
      
      
       import airflow, sysfrom airflow import models, settingsfrom airflow.contrib.auth.backends.password_auth import PasswordUseruser = PasswordUser(models.User())user.username = sys.argv[1]user.email = sys.argv[2]user.password = sys.argv[3]session = settings.Session()session.add(user)session.commit()session.close()exit()
      
     
    

但是运行的时候，通常会报错（“AttributeError: can't set attribute”）

原因是 sqlalchemy这个Python包版本大于1.2

解决方法：

pip uninstall sqlalchemypip install 'sqlalchemy<1.2'

Airflow添加CeleryExecutor：

添加依赖：

yum -y install make gcc gcc-c++ kernel-devel m4 ncurses-devel openssl-devel

安装Erlang：

wget http://erlang.org/download/otp_src_18.3.tar.gztar xvf otp_src_18.3.tar.gzcd otp_src_18.3./configure --prefix=/usr/local/erlang --with-ssl -enable-threads -enable-smmp-support -enable-kernel-poll --enable-hipe --without-javacmake && make install#配置环境vim /etc/profileexport ERLANG_HOME=/usr/local/erlangexport PATH=$ERLANG_HOME/bin:$PATHsource /etc/profile#输入命令检验是否安装成功erl

 

按照Rabbitmq：

wget http://www.rabbitmq.com/releases/rabbitmq-server/v3.6.1/rabbitmq-server-generic-unix-3.6.1.tar.xzxz -d rabbitmq-server-generic-unix-3.6.1.tar.xztar -xvf rabbitmq-server-generic-unix-3.6.1.tarcp -rf ./rabbitmq_server-3.6.1 /usr/local/cd /usr/local/mv rabbitmq_server-3.6.1 rabbitmq-3.6.1#开启管理页面插件cd ./rabbitmq-3.6.1/sbin/./rabbitmq-plugins enable rabbitmq_management#启动命令，该命令ctrl+c后会关闭服务./rabbitmq-server#在后台启动Rabbit./rabbitmq-server -detached#关闭服务./rabbitmqctl stop#添加用户#rabbitmqctl add_user Username Password./rabbitmqctl add_user rabbitadmin 123456#分配用户标签#rabbitmqctl set_user_tags User Tag#[administrator]:管理员标签./rabbitmqctl set_user_tags rabbitadmin administrator

登录管理界面

浏览器输入地址：http://服务器IP地址:15672/ 

 

~/airflow/airflow.cfg 配置celery

创建Rabbitmq用户：

cd /usr/local/rabbitmq361/sbin./rabbitmqctl add_user ct 123./rabbitmqctl add_vhost ct_airflow./rabbitmqctl set_user_tags ct airflow./rabbitmqctl set_permissions -p ct_airflow ct ".*" ".*" ".*"

修改airflow.cfg文件: 

[core] executor = CeleryExecutor [celery] broker_url = amqp://ct:152108@localhost:5672/ct_airflow celery_result_backend = amqp://ct:152108@localhost:5672/ct_airflow

 

pip install celery

进入/usr/lib/python2.7/site-packages/airflow/executors，修改celery_executor.py源码文件

from celery import Celery, platformsplatforms.C_FORCE_ROOT = True