通过梳理 celery_app 与flask_app关系理解 “working outside of application context” 报错

背景

flask项目中使用celery，有时遇到“working outside of application context”报错。究其原因是因为celery的实例app运行在与flask app 独立的进程空间，当在celery 任务中与flask app交互，会因为不存在flask的上下文抛出此异常

解决思路

在执行flask app交互的逻辑前，使用 “with app.app_context():” 手动调用flask上下文，如在使用flask_mail模块发送邮件中（以下代码仅作演示,无法运行）

from flask_mail import Mail, Message


app = Flask(__name__)
app.config['MAIL_SERVER']='smtp.qq.com'
# ...... 省略部分配置信息
app.config['CELERY_BROKER_URL']='redis://localhost:6379/0'
app.config['CELERY_RESULT_BACKEND']='redis://localhost:6379/0'

mail = Mail(app)

celery = Celery(app.name, broker=app.config['CELERY_BROKER_URL'])
celery.conf.update(app.config)

@celery.task
def send_async_email(email_data):
    """Background task to send an email with Flask-Mail."""
    msg = Message(email_data['subject'],
                  sender=app.config['MAIL_DEFAULT_SENDER'],
                  recipients=[email_data['to']])
    print(msg)
    msg.body = email_data['body']
    with app.app_context():  # 如果缺少此行代码，抛出outside of application context异常
        mail.send(msg)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

优化方向

如果在celery task逻辑中每涉及到flask app的交互，都要with app.app_context()：,显然不是很方便，根据官方文档的推荐，将celery集成到flask中

from celery import Celery

def make_celery(app):  # 将flask app当作参数传递给celery 的make函数
    celery_app = Celery(app.import_name, broker=app.config['CELERY_BROKER_URL'])
    celery_app.conf.update(app.config)
    TaskBase = celery_app.Task
    
    class ContextTask(TaskBase):
        abstract = True
        def __call__(self, *args, **kwargs):
            with app.app_context():
                return TaskBase.__call__(self, *args, **kwargs)
    celery_app.Task = ContextTask
    return celery_app
1
2
3
4
5
6
7
8
9
10
11
12
13
14

示例

from flask import Flask

app = Flask(__name__)
app.config.update(
    CELERY_BROKER_URL='redis://localhost:6379',
    CELERY_RESULT_BACKEND='redis://localhost:6379'
)
celery = make_celery(app)


@celery.task()
def add_together(a, b):
    return a + b
1
2
3
4
5
6
7
8
9
10
11
12
13

Tips

1. celery 可以单独配置独立于flask使用，但是讲flask的上下文与celery集成并没有坏处。如果在不涉及与flask 交互的场景（如仅调用一个外部请求，或更新数据库等），仍然可以将celery的生成与配置完全独立开来；
2. 建议实际使用过程中与flask做集成；
3. 参考链接： http://docs.jinkan.org/docs/flask/patterns/celery.html