赞
踩
由上一博客Linux(Centos7.8)中Anaconda3使用虚拟环境搭建Django3应用Django应用的基础环境已搭建完备。
第三方外部应用能够通过调用api接口的方式,获取由LSTM(长短期记忆)递归神经网络预测的未来30天的活期存款余额趋势数据。
进入到由上篇博客搭建的虚拟环境目录/opt/anaconda3/envs/ml_api并激活环境
cd /opt/anaconda3/envs/ml_api
# 激活环境
conda activate ml_api
使用conda安装lstm神经网络依赖包,考虑到版本兼容性问题指定版本下载
conda install numpy
conda install matplotlib
conda install pandas
conda install pymysql
python -m pip install django-cors-headers #跨域
python -m pip install --upgrade --force-reinstall tensorflow==2.2.0 --ignore-installed
python -m pip install --upgrade --force-reinstall keras==2.3.0 --ignore-installed
python -m pip install --upgrade --force-reinstall sklearn --ignore-installed
conda list 查看该虚假环境已安装的包
(ml_api) [root@localhost api]# conda list # packages in environment at /opt/anaconda3/envs/ml_api: # # Name Version Build Channel _libgcc_mutex 0.1 main _tflow_select 2.3.0 mkl absl-py 0.11.0 py38h06a4308_0 aiohttp 3.6.3 py38h7b6447c_0 asgiref 3.2.10 pypi_0 pypi astunparse 1.6.3 py_0 async-timeout 3.0.1 py38_0 attrs 20.2.0 py_0 blas 1.0 mkl blinker 1.4 py38_0 brotlipy 0.7.0 py38h7b6447c_1000 c-ares 1.16.1 h7b6447c_0 ca-certificates 2020.10.14 0 cached-property 1.5.2 pypi_0 pypi cachetools 4.1.1 py_0 certifi 2020.6.20 pyhd3eb1b0_3 cffi 1.14.3 py38he30daa8_0 chardet 3.0.4 py38_1003 click 7.1.2 py_0 cryptography 3.1.1 py38h1ba5d50_0 cycler 0.10.0 py38_0 dbus 1.13.18 hb2f20db_0 django 3.1.2 py_0 django-cors-headers 3.5.0 pypi_0 pypi expat 2.2.10 he6710b0_2 fontconfig 2.13.0 h9420a91_0 freetype 2.10.4 h5ab3b9f_0 gast 0.3.3 py_0 glib 2.66.1 h92f7085_0 google-auth 1.23.0 pyhd3eb1b0_0 google-auth-oauthlib 0.4.2 pyhd3eb1b0_2 google-pasta 0.2.0 py_0 grpcio 1.31.0 py38hf8bcb03_0 gst-plugins-base 1.14.0 hbbd80ab_1 gstreamer 1.14.0 hb31296c_0 h5py 3.0.0 pypi_0 pypi hdf5 1.10.6 hb1b8bf9_0 icu 58.2 he6710b0_3 idna 2.10 py_0 importlib-metadata 2.0.0 py_1 intel-openmp 2020.2 254 joblib 0.17.0 pypi_0 pypi jpeg 9b h024ee3a_2 keras 2.3.0 pypi_0 pypi keras-applications 1.0.8 pypi_0 pypi keras-preprocessing 1.1.2 pypi_0 pypi kiwisolver 1.3.0 py38h2531618_0 krb5 1.17.1 h173b8e3_0 lcms2 2.11 h396b838_0 ld_impl_linux-64 2.33.1 h53a641e_7 libedit 3.1.20191231 h14c3975_1 libffi 3.3 he6710b0_2 libgcc-ng 9.1.0 hdf63c60_0 libgfortran-ng 7.3.0 hdf63c60_0 libpng 1.6.37 hbc83047_0 libpq 12.2 h20c2e04_0 libprotobuf 3.13.0.1 hd408876_0 libstdcxx-ng 9.1.0 hdf63c60_0 libtiff 4.1.0 h2733197_1 libuuid 1.0.3 h1bed415_2 libxcb 1.14 h7b6447c_0 libxml2 2.9.10 hb55368b_3 lz4-c 1.9.2 heb0550a_3 markdown 3.3.2 py38_0 matplotlib 3.3.2 0 matplotlib-base 3.3.2 py38h817c723_0 mkl 2020.2 256 mkl-service 2.3.0 py38he904b0f_0 mkl_fft 1.2.0 py38h23d657b_0 mkl_random 1.1.1 py38h0573a6f_0 multidict 4.7.6 py38h7b6447c_1 ncurses 6.2 he6710b0_1 numpy 1.19.4 pypi_0 pypi numpy-base 1.19.2 py38hfa32c7d_0 oauthlib 3.1.0 py_0 olefile 0.46 py_0 openssl 1.1.1h h7b6447c_0 opt_einsum 3.1.0 py_0 pandas 1.1.3 py38he6710b0_0 pcre 8.44 he6710b0_0 pillow 8.0.1 py38he98fc37_0 pip 20.2.4 py38_0 protobuf 3.13.0.1 py38he6710b0_1 psycopg2 2.8.5 py38h1ba5d50_0 pyasn1 0.4.8 py_0 pyasn1-modules 0.2.8 py_0 pycparser 2.20 py_2 pyjwt 1.7.1 py38_0 pymysql 0.10.1 py38_0 pyopenssl 19.1.0 py_1 pyparsing 2.4.7 py_0 pyqt 5.9.2 py38h05f1152_4 pysocks 1.7.1 py38_0 python 3.8.3 hcff3b4d_2 python-dateutil 2.8.1 py_0 pytz 2020.1 py_0 pyyaml 5.3.1 pypi_0 pypi qt 5.9.7 h5867ecd_1 readline 8.0 h7b6447c_0 requests 2.24.0 py_0 requests-oauthlib 1.3.0 py_0 rsa 4.6 py_0 scikit-learn 0.23.2 pypi_0 pypi scipy 1.5.3 pypi_0 pypi setuptools 50.3.0 py38hb0f4dca_1 sip 4.19.13 py38he6710b0_0 six 1.15.0 py_0 sklearn 0.0 pypi_0 pypi sqlite 3.33.0 h62c20be_0 sqlparse 0.4.1 py_0 tensorboard 2.3.0 pyh4dce500_0 tensorboard-plugin-wit 1.6.0 py_0 tensorflow 2.2.0 mkl_py38h6d3daf0_0 tensorflow-base 2.2.0 mkl_py38h5059a2d_0 tensorflow-estimator 2.2.0 pyh208ff02_0 termcolor 1.1.0 py38_1 threadpoolctl 2.1.0 pypi_0 pypi tk 8.6.10 hbc83047_0 tornado 6.0.4 py38h7b6447c_1 urllib3 1.25.11 py_0 werkzeug 1.0.1 py_0 wheel 0.35.1 py_0 wrapt 1.12.1 py38h7b6447c_1 xz 5.2.5 h7b6447c_0 yarl 1.6.2 py38h7b6447c_0 zipp 3.4.0 pyhd3eb1b0_0 zlib 1.2.11 h7b6447c_3 zstd 1.4.5 h9ceee32_0
settings.py
(ml_api) [root@localhost webapi]# vim settings.py ALLOWED_HOSTS = ['*'] # Application definition INSTALLED_APPS = [ 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'corsheaders', #跨域配置 ] MIDDLEWARE = [ 'django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'corsheaders.middleware.CorsMiddleware',#跨域配置 'django.middleware.common.CommonMiddleware',#跨域配置 ] CORS_ORIGIN_ALLOW_ALL = True #跨域配置 ROOT_URLCONF = 'webapi.urls' TEMPLATES = [ { 'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [os.path.join(BASE_DIR, "templates")], 'APP_DIRS': True, 'OPTIONS': { 'context_processors': [ 'django.template.context_processors.debug', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', ], }, }, ] WSGI_APPLICATION = 'webapi.wsgi.application' # Database # https://docs.djangoproject.com/en/3.1/ref/settings/#databases DATABASES = { 'default': { 'ENGINE': 'django.db.backends.sqlite3', 'NAME': BASE_DIR / 'db.sqlite3', } } # Password validation # https://docs.djangoproject.com/en/3.1/ref/settings/#auth-password-validators AUTH_PASSWORD_VALIDATORS = [ { 'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator', }, { 'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator', }, { 'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator', }, { 'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator', }, ] # Internationalization # https://docs.djangoproject.com/en/3.1/topics/i18n/ LANGUAGE_CODE = 'en-us' TIME_ZONE = 'UTC' USE_I18N = True USE_L10N = True USE_TZ = True # Static files (CSS, JavaScript, Images) # https://docs.djangoproject.com/en/3.1/howto/static-files/ STATIC_URL = '/static/'
在项目下新建视图文件 views.py
touch view.py # -*- coding: utf-8 -*- import json from django.http import HttpResponse from django.views.decorators.csrf import csrf_exempt import numpy as np import matplotlib.pyplot as plt import pandas as pd from sklearn.preprocessing import MinMaxScaler from tensorflow.keras.models import Sequential # Sequential 用于初始化神经网络 from tensorflow.keras.layers import Dense # Dense 用于添加全连接的神经网络层 from tensorflow.keras.layers import LSTM # LSTM 用于添加长短期内存层 from tensorflow.keras.layers import Dropout # Dropout 用于添加防止过拟合的dropout层 import os import datetime import matplotlib.dates as mdates import matplotlib.pyplot as plt import tensorflow as tf #os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2' import pymysql.cursors # 连接数据库 connect = pymysql.Connect( host='192.168.6.100', port=3306, user='root', passwd='123456', db='test', charset='utf8' ) def plot_show(axis_X, real_alive_amt, predicted_stock_price): """ :param axis_X: :param real_alive_amt: :param predicted_stock_price: :return: z展示图 """ plt.rcParams['font.sans-serif'] = ['SimHei'] plt.rcParams['axes.unicode_minus'] = False plt.plot(axis_X, real_alive_amt, color='green', label='真实活期余额') # 显示活期余额 plt.plot(axis_X, predicted_stock_price[:, -1], ls='--', color='red', label='预测活期余额') plt.title('活期余额预测') plt.gca().xaxis.set_major_locator(mdates.DayLocator()) plt.xlabel('Time') plt.ylabel('余额(元)') plt.legend() plt.tick_params(axis='both', which='major', labelsize=16) plt.gcf().autofmt_xdate() # 自动旋转日期标记 plt.show() def train_data(dataset_train,step_len): """ 构建训练数据 :param training_set: :return: X_train, y_train, sc """ sc = MinMaxScaler(feature_range=(0, 1)) training_set = dataset_train.values training_set_scaled = sc.fit_transform(training_set) X_train = [] y_train = [] for i in range(step_len, len(training_set_scaled)): X_train.append(training_set_scaled[i - step_len:i, :]) y_train.append(training_set_scaled[i, :]) X_train, y_train = np.array(X_train), np.array(y_train) X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], X_train.shape[2])) return X_train, y_train, sc def test_data(dataset_train, dataset_test, sc,step_len): """ 构建测试数据 :param dataset_train: :param dataset_test: :param sc: :return: X_test """ # 合并训练集和测试集 dataset_total = pd.concat((dataset_train, dataset_test), axis=0) db_all = dataset_total[:].values np.random.shuffle(db_all) # shuffle() 方法将序列的所有元素随机排序。 inputs = db_all[len(dataset_total) - len(dataset_test) - int(step_len):] inputs = inputs.reshape(-1, dataset_test.shape[1]) inputs = sc.transform(inputs) X_test = [] for i in range(int(step_len), int(step_len) + len(dataset_test)): curr_frame = inputs[i - int(step_len):i - 1, :] curr_frame = np.insert(curr_frame, step_len-1,dataset_test.values, axis=0) X_test.append(curr_frame) X_test = np.array(X_test) X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], X_test.shape[2])) return X_test def bulid_model(X_train, y_train): """ 创建股票预测模型 :param X_train: :param y_train: :return: regressor """ regressor = Sequential() # LSTM的输入为 [samples, timesteps, features],这里的timesteps为步数,features为维度 这里我们的数据是11维的 regressor.add(LSTM(units=50, return_sequences=True, input_shape=(X_train.shape[1], 11))) regressor.add(Dropout(0.2)) regressor.add(LSTM(units=50, return_sequences=True)) regressor.add(Dropout(0.2)) regressor.add(LSTM(units=50, return_sequences=True)) regressor.add(Dropout(0.2)) regressor.add(LSTM(units=50)) regressor.add(Dropout(0.2)) # 全连接,输出11个 regressor.add(Dense(units=11)) regressor.compile(optimizer='adam', loss='mean_squared_error') regressor.fit(X_train, y_train, epochs=1, batch_size=60) #print(regressor.summary()) return regressor def predict_test_data_fun(dataset_train,regressor,sc,X_test,real_alive_amt,step_len,predict_custs): """ 最新预测数据 :param dataset_train: :param regressor: :param sc: :param X_test: :param real_alive_amt: :param step_len: :return: """ predicted_alive_amt = regressor.predict(X_test) # 反向转换 trans_predicted_alive_amt = sc.inverse_transform(predicted_alive_amt) predict_custs.insert(step_len,trans_predicted_alive_amt[-1]) # 添加表头 result_dataframe = pd.DataFrame(predicted_alive_amt, columns=['idcard_first_6','idcard_middle_8','idcard_last_4','gender','age','alive_avg_amt','c_alive_amt', 'd_alive_amt', 'is_arrear','arrear_amt','total_alive_amt']) return test_data(dataset_train,result_dataframe,sc,step_len),trans_predicted_alive_amt[:, -1],predict_custs def recursion_call(dataset_train,regressor,sc,X_test,real_alive_amt,step_len,recursion_day,predict_custs): """ 预测指定天数函数 :param dataset_train: :param regressor: :param sc: :param X_test: :param real_alive_amt: :param step_len: :param recursion_day: :return: """ if recursion_day == 0: return 0 else: X_test,real_alive_amt,predict_custs = predict_test_data_fun(dataset_train,regressor,sc,X_test,real_alive_amt,step_len,predict_custs) return recursion_call(dataset_train,regressor,sc,X_test,real_alive_amt,step_len,recursion_day - 1,predict_custs) #@csrf_exempt def alive_api(request,custno): print(custno) custno=int(custno) result = {"result": 0, "msg": "执行成功"} # 读取文件 df = pd.read_csv('/opt/anaconda3/envs/ml_api/api/webapi/alives.csv',parse_dates=['dt']) df['idcard_last_4'] =df['idcard_last_4'].apply(lambda x:str(x).replace('X','10').zfill(4)) # 根据客户号去重 cust_ids = df.drop_duplicates(subset=['customer_no'], keep='first')['customer_no'] # 步长 step_len = 60 # 递归天数 recursion_day = 30 #custno=1000340109 # 训练集 dataset_train = df[(df['dt'] < '2020-10-18') & (df['customer_no'] == custno)].loc[:, ['idcard_first_6','idcard_middle_8','idcard_last_4','gender','age','alive_avg_amt', 'c_alive_amt','d_alive_amt','is_arrear','arrear_amt','total_alive_amt']] # 测试集 dataset_test = df[(df['dt'] == '2020-10-18') & (df['customer_no'] == custno)].loc[:, ['idcard_first_6','idcard_middle_8','idcard_last_4','gender','age','alive_avg_amt', 'c_alive_amt','d_alive_amt','is_arrear','arrear_amt','total_alive_amt']] # 获取存款余额真实值 real_alive_amt = dataset_test['total_alive_amt'].values # 训练数据 X_train, y_train, sc = train_data(dataset_train,step_len) # 构建模型 regressor = bulid_model(X_train, y_train) # 构造测试数据 X_test = test_data(dataset_train, dataset_test, sc,step_len) # 插入数据 predict_custs = list() recursion_call(dataset_train,regressor,sc,X_test,real_alive_amt,step_len,recursion_day,predict_custs) #print(predict_custs) jsonObj ={} for index in range(len(predict_custs)): key = ((datetime.datetime.now() + datetime.timedelta(days=index)).strftime("%Y-%m-%d")) jsonObj[str(key)]=str(predict_custs[index][-1]) #json返回为中文 #print(jsonObj) return HttpResponse(json.dumps(jsonObj,ensure_ascii=False),content_type="application/json,charset=utf-8")
urls.py
from django.contrib import admin
from django.urls import path
from django.conf.urls import url
from . import views
urlpatterns = [
path('admin/', admin.site.urls),
url(r'^api/(?P<custno>\d+)/$',views.alive_api,name='alive_api'),
]
核心代码到此为止。
python manage.py runserver 0.0.0.0:8000
图标展示采用echart.js 核心代码如下:
服务接口:http://192.168.2.9:8000/api/1001324295/,其中1001324295为参数
<div id="main" style="width: 1200px;height:400px;"></div> <script type="text/javascript"> $(function(){ xdata=[] ydata=[] var myChart = echarts.init(document.getElementById('main'),'macarons'); myChart.showLoading({ text: '大数据正在预测...', textStyle: { fontSize : 30 , color: '#444' }, effectOption: {backgroundColor: 'rgba(0, 0, 0, 0)'} }); $.get('http://192.168.2.9:8000/api/1001324295/',function(dic) { console.log(dic) $.each(dic,function(index,item){ console.log(index+"|"+item) xdata.push(index) ydata.push(item) }) //console.log(xdata) //console.log(ydata) // 基于准备好的dom,初始化echarts实例 // 指定图表的配置项和数据 var option = { title: { text: '未来30天活期存款余额预测趋势图', textStyle:{ color:'#0DB9F2', //颜色 fontStyle:'normal', //风格 fontWeight:'normal', //粗细 fontFamily:'Microsoft yahei', //字体 fontSize:24, //大小 align:'center' //水平对齐 }, x:'center' }, tooltip: {}, legend: { data:['未来30天活期存款余额预测趋势图'] }, xAxis: { data: xdata, axisLabel: { interval:0, rotate:50 } }, yAxis: { type: 'value', //默认以千分位显示,不想用的可以在这加一段 axisLabel : { //调整左侧Y轴刻度, 直接按对应数据显示 show:true, showMinLabel:true, showMaxLabel:true, formatter: function (value) { return value; } } }, series: [{ name: '活期存款余额(元)', type: 'line', data: ydata, markPoint: { itemStyle: { normal: { borderColor: '#87cefa', borderWidth: 1, // 标注边线线宽,单位px,默认为1 label: { show: true } } }, data: [ { type: 'max', name: '最大值' }, { type: 'min', name: '最小值' } ] }, markLine: { data: [ {type: 'average', name: '平均值'}, [{ symbol: 'none', x: '90%', yAxis: 'max' }, { symbol: 'circle', label: { normal: { position: 'start', formatter: '最大值' } }, type: 'max', name: '最高点' }] ] } }] }; // 使用刚指定的配置项和数据显示图表。 myChart.setOption(option); myChart.hideLoading(); }); }); </script>
第三方应用调用预测api接口,由预测模型需要递归多次处理时间比较久,加载过程中提示loading为大数据正在预测…
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。