devbox
Gausst松鼠会
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

t3_Predicting the Markets w ML_sklearn_scatter_PairGrid_R-squared_log returns_Lasso_ridge_KNN_SVM_LR

作者：Gausst松鼠会 | 2024-03-16 17:24:57

赞

踩

predicting the markets

     In the last chapter, we learned how to design trading strategies, create trading signals, and implement advanced concepts, such as seasonality in trading instruments. Understanding those concepts in greater detail is a vast field comprising stochastic processes, random walks, martingales, and time series analysis, which we leave to you to explore at your own pace.

     So what's next? Let's look at an even more advanced method of prediction and forecasting: statistical inference and prediction. This is known as machine learning, the fundamentals of which were developed in the 1800s and early 1900s and have been worked on ever since. Recently, there has been a resurgence in interest in machine learning algorithms and applications owing to the availability of extremely cost-effective processing power and the easy availability of large datasets. Understanding machine learning techniques in great detail is a massive field at the intersection of linear algebra, multivariate calculus, probability theory, frequentist and Bayesian statistics, and an in-depth analysis of machine learning is beyond the scope of a single book. Machine learning methods, however, are surprisingly easily accessible in Python and quite intuitive to understand, so we will explain the intuition behind the methods and see how they find applications in algorithmic trading. But first, let's introduce some basic concepts and notation that we will need for the rest of this chapter.

This chapter will cover the following topics:

  • Understanding the terminology and notations
  • Creating predictive models that predict price movement using linear regression methods
  • Creating predictive models that predict buy and sell signals using linear classification methods

Understanding the terminology and notations

     To develop ideas quickly and build an intuition regarding supply and demand, we have a simple and completely hypothetical dataset of height, weight, and race of a few random samples obtained from a survey. Let's have a look at the dataset:

Let's examine the individual fields:

  • Height in inches and weight in lbs are continuous data types because they can take on any values, such as 65, 65.123, and 65.3456667.
  • Race, on the other hand, would be an example of a categorical data type, because there are a finite number of possible values that can go in the field. In this example, we assume that possible race values are Asian, African, and Caucasian.

     Now, given this dataset, say our task is to build a mathematical model that can learn from the data we provide it with. The task or objective we are trying to learn in this example is to find the relationship between the weight of a person as it relates to their height and race. Intuitively, it should be obvious that height will have a major role to play (taller people are much more likely to be heavier), and race should have very little impact. Race may have some impact on the height of an individual, but once the height is known, knowing their race also provides very little additional information in guessing/predicting a person's weight. In this particular problem, note that in the dataset, we are also provided the weight of the samples in addition to their height and race. 

     Since the variable we are trying to learn how to predict is known, this is known as a supervised learning problem. If, on the other hand, we were not provided with the weight variable and were asked to predict whether, based on height and race, someone is more likely to be heavier than someone else, that would be an unsupervised learning problem. For the scope of this chapter, we will focus on supervised learning problems only, since that is the most typical use case of machine learning in algorithmic trading.

     Another thing to address in this example is the fact that, in this case, we are trying to predict weight as a function of height and race. So we are trying to predict a continuous variable. This is known as a regression problem, since the output of such a model is a continuous value. If, on the other hand, say our task was to predict the race of a person as a function of their height and weight, in that case, we would be trying to predict a categorical variable type. This is known as a classification problem, since the output of such a model will be one value from a set of finite discrete values.

     When we start addressing this problem, we will begin with a dataset that is already available to us and will train our model of choice on this dataset. This process (as you've already guessed) is known as training your model. We will use the data provided to us to guess the parameters of the learning model of our choice (we will elaborate more on what this means later). This is known as statistical inference of these parametric learning models. There are also non-parametric learning models, where we try to remember the data we've seen so far to make a guess as regards new data.

     Once we are done training our model, we will use it to predict weight for datasets we haven't seen yet. Obviously, this is the part we are interested in. Based on data in the future that we haven't seen yet, can we predict the weight? This is known as testing your model and the datasets used for that are known as test data. The task of using a model where the parameters were learned by statistical inference to actually make predictions on previously unseen data is known as statistical prediction or forecasting.

     We need to be able to understand the metrics of how to differentiate between a good model and a bad model. There are several well known and well understood performance metrics for different models. For regression prediction problems, we should try to minimize the differences between predicted value and the actual value of the target variable. This error term is known as residual errors; larger errors mean worse models and, in regression, we try to minimize the sum of these residual errors, or the sum of the square of these residual errors (squaring has the effect of penalizing large outliers more strongly, but more on that later). The most common metric for regression problems is R^2, which tracks the ratio of explained variance vis-à-vis unexplained variance, but we save that for more advanced texts.

     In the simple hypothetical prediction problem of guessing weight based on height and race, let's say the model predicts the weight to be 170 and the actual weight is 160. In this case, the error is 160-170 = -10, the absolute error is | -10| = 10, and the squared error is (-10)^2 =100. In classification problems, we want to make sure our predictions are the same discrete value as the actual value. When we predict a label that is different from the actual label, that is a misclassification or error. Obviously, the higher the number of accurate predictions, the better the model, but it gets more complicated than that. There are metrics such as a confusion matrix(https://blog.csdn.net/Linli522362242/article/details/120093948), a Receiver Operating Characteristic(the ROC curve plots the true positive rate (another name for recall) against the false positive rate(TPR vs FPR)), and the area under the curve(https://blog.csdn.net/Linli522362242/article/details/103786116, A perfect classifier will have a ROC AUC equal to 1, whereas a purely random classifier will have a ROC AUC equal to 0.5.), but we save those for more advanced texts. Let's say, in the modified hypothetical problem of guessing race based on height and weight, that we guess the race to be Caucasian while the correct race is African. That is then considered an error, and we can aggregate all such errors to find the aggregate errors across all predictions, but we will talk more on this in the later parts of the book.

     So far, we have been speaking in terms of a hypothetical example, but let's tie the terms we've encountered so far into how it applies to financial datasets. As we mentioned, supervised learning methods are most common here because, in historical financial data, we are able to measure the price movements from the data. If we are simply trying to predict that, if a price moves up or down from the current price, then that is a classification problem with two prediction labels – Price goes up and Price goes down. There can also be three prediction labels since Price goes up, Price goes down, and Price remains the same. If, however, we want to predict the magnitude and direction of price moves, then this is a regression problem where an example of the output could be Price moves +10.2 dollars, meaning the prediction is that the price will move up by $10.2. The training dataset is generated from historical data, and this can be historical data that was not used in training the model and the live market data实时市场数据 during live trading. We measure the accuracy of such models with the metrics we listed above in addition to the PnL盈亏 generated from the trading strategies. With this introduction complete, let's now look into these methods in greater detail, starting with regression methods.

Exploring our financial dataset

     Before we start applying machine learning techniques to build predictive models, we need to perform some exploratory data wrangling[ˈræŋɡlɪŋ]数据整理 on our dataset with the help of the steps listed here. This is often a large and an underestimated prerequisite when it comes to applying advanced methods to financial datasets.

  1. Getting the data: We'll continue to use Google stock data that we've used in our previous chapter: 
    1. import pandas as pd
    2. from pandas_datareader import data
    3.  
    4. def load_financial_data( start_date, end_date, output_file='', stock_symbol='GOOG' ):
    5.     if len(output_file) == 0:
    6.         output_file = stock_symbol+'_data_large.pkl'
    7.         
    8.     try:
    9.         df = pd.read_pickle( output_file )
    10.         print( "File data found. . . reading {} data".format(stock_symbol) )
    11.     except FileNotFoundError:
    12.         print( "File not found. . . downloading the {} data".format(stock_symbol) )
    13.         df = data.DataReader( stock_symbol, 'yahoo', start_date, end_date)
    14.         df.to_pickle( output_file )
    15.             
    16.     return df
          In the code, we revisited how to download the data and implement a method, load_financial_data , which we can use moving forward. It can also be invoked, as shown in the following code, to download 17 years' of daily Google data: 
    1. goog_data = load_financial_data( start_date='2001-01-01',
    2.                                  end_date='2018-01-01',
    3.                                )
    4. goog_data.head()
         The code will download financial data over a period of 17 years from GOOG stock data. Now, let's move on to the next step.
  2. Creating objectives/trading conditions that we want to predict: Now that we know how to download our data, we need to operate on it to extract our target for the predictive models, also known as a response or dependent variable; effectively, what we are trying predict.

         In our hypothetical example of predicting weight, weight was our response variable. For algorithmic trading, the common target is to be able to predict what the future price will be so that we can take positions in the market right now that will yield a profit in the future. If we model the response variable as future price - current price, then we are trying to predict the direction of the future price with regard to the current price (does it go up, does it go down, or does it remain the same), as well as the magnitude of the price change. So, these variables look like +10, +3.4, -4, and so on. This is the response variable methodology that we will use for regression models, but we will look at it in greater detail later. Another variant of the response variable would be to simply predict the direction but ignore the magnitude, in other words, +1 to signify the future price moving up, -1 to signify the future price moving down, and 0 to signify that the future price remains the same as the current price. That is the response variable methodology that we will use for classification models, but we will explore that later. Let's implement the following code to generate these response variables:

    ---The classification response variable is +1 if the close price tomorrow is higher than the close price today, and -1 if the close price tomorrow is lower than the close price today.

    ---For this example, we assume that the close price tomorrow is not the same as the close price today, which we can choose to handle by creating a third categorical value, 0.
      
    1. def create_classification_trading_condition( df ):
    2.     df['Open-Close'] = df.Open - df.Close
    3.     df['High-Low'] = df.High - df.Low
    4.     df = df.dropna( axis=0)
    5.     X = df[ ['Open-Close', 'High-Low'] ]
    6.     #             the close price tomorrow > the close price today 
    7.     Y = np.where( df['Close'].shift(-1) > df['Close'], 
    8.                   1, -1
    9.                 )
    10.     return (X,Y)
    The regression response variable is Close price tomorrow-Close price today for each day.

    ---It is a positive value if the price goes up tomorrow, a negative value if the price goes down tomorrow, and zero if the price does not change.
    ---The sign of the value indicates the direction, and the magnitude of the response variable captures the magnitude of the price move. 
    1. def create_regression_trading_condition( df ):
    2.     df['Open-Close'] = df.Open - df.Close
    3.     df['High-Low'] = df.High - df.Low
    4.     # the difference between the close price tomorrow and the close price today
    5.     df['Target'] = df['Close'].shift(-1) - df['Close']
    6.     
    7.     df = df.dropna( axis=0 ) # the last item after doing shift(-1) will be nan 
    8.     X = df[ ['Open-Close', 'High-Low'] ]
    9.     Y = df[['Target']]
    10.     return (df, X,Y)
  3. Partitioning datasets into training and testing datasets:
         One of the key questions regarding a trading strategy is how it will perform on market conditions or datasets that the trading strategy has not seen. Trading performance on datasets that have not been used in training the predictive model is often referred to as out-sample performance for that trading strategy. These results are considered representative of what to expect when the trading strategy is run in live markets. Generally, we divide all of our available datasets into multiple partitions, and then we evaluate models trained on one dataset over a dataset that wasn't used in training it (and optionally validated on yet another dataset after that). For the purpose of our models, we will be partitioning our dataset into two datasets: training and testing.

    ---We used a default split ratio of 80%, so 80% of the entire dataset is used for training, and the remaining 20% is used for testing.
    ---There are more advanced splitting methods to account for distributions of underlying data (such as we want to avoid ending up with a training/testing dataset that is not truly representative of actual market conditions). 
    1. from sklearn.model_selection import train_test_split
    2.  
    3. def create_train_split_group( X,y, split_ratio=0.8 ):
    4.     # shufflebool, default=True
    5.     # Whether or not to shuffle the data before splitting. 
    6.     # If shuffle=False then stratify must be None.
    7.     # stratify
    8.     # https://blog.csdn.net/Linli522362242/article/details/103387527
    9.     return train_test_split( X, Y, shuffle=False, # since the stock data is a kind of time-series data
    10.                              train_size = split_ratio
    11.                            )


Creating predictive models using linear regression methods

     Now that we know how to get the datasets that we need, how to quantify what we are trying to predict (objectives), and how to split data into training and testing datasets to evaluate our trained models on, let's dive into applying some basic machine learning techniques to our datasets: 

Ordinary Least Squares

     Given observations of the target variables, m x 1 rows of features values, and each row of dimension 1 x n, OLS seeks to find the weights of dimension that minimize the residual sum of squares of differences between the target variable and the predicted variable predicted by linear approximation: 

  • , which is the best fit for the equation , where
    --- X is the matrix of feature values,
    --- W is the n x 1 matrix/vector of weights/coefficients assigned to each of the feature values, and
    --- y is the m x 1 matrix/vector of the target variable observation on our training dataset.

    Here is an example of the matrix operations involved for m = 4 and n = 2 :
  •  Intuitively, it is very easy to understand OLS with a single feature variable and a single target variable by visualizing it as trying to draw a line that has the best fit.
  • OLS is just a generalization of this simple idea in much higher dimensions, where m is tens of thousands of observations, and n is thousands of features values.
  • The typical setup in m is much larger than n (many more observations in comparison to the number of feature values), otherwise the solution is not guaranteed to be unique.
  • There are closed form solutions to this problem where  OR (Equation 4-4. Normal Equationhttps://blog.csdn.net/Linli522362242/article/details/104005906) but, in practice, these are better implemented by iterative solutions, but we'll skip the details of all of that for now.
  • The reason why we prefer to minimize the sum of the squares of the error terms is so that massive outliers are penalized more harshly and don't end up throwing off the entire fit.

     There are many underlying assumptions for OLS in addition to the assumption that

  • the target variable is a linear combination of the feature values, such as
  • the independence of feature values themselves, and
  • normally distributed error terms.

The following diagram is a very simple example showing a relatively close linear relationship between two arbitrary variables. Note that it is not a perfect linear relationship, in other words, not all data points lie perfectly on the line and we have left out省略了 the X and Y labels because these can be any arbitrary variables. The point here is to demonstrate an example of what a linear relationship visualization looks like. Let's have a look at the following diagram:

1. start by loading up Google data in the code, using the same method that we introduced in the previous section: 

  1. goog_data = load_financial_data( start_date='2001-01-01',
  2.                                  end_date='2018-01-01',
  3.                                  output_file='goog_data_large.pkl'
  4.                                )
  5. goog_data.head()

 

2. Now, we create and populate the target variable vector, Y, for regression in the  following code. Remember that what we are trying to predict in regression is magnitude and the direction of the price change from one day to the next: 

  1. # def create_regression_trading_condition( df ):
  2. #     df['Open-Close'] = df.Open - df.Close
  3. #     df['High-Low'] = df.High - df.Low
  4. #     df = df.dropna( axis=0 )
  5. #     X = df[ ['Open-Close', 'High-Low'] ]
  6. #     # the difference between the close price tomorrow and the close price today
  7. #     Y = df['Target'] = df['Close'].shift(-1) - df['Close']
  8. #     return (df, X,Y)
  9.  
  10. goog_data, X, Y = create_regression_trading_condition( goog_data )
goog_data.head()

 

3. With the help of the code, let's quickly create a scatter plot for the two features we have: High-Low price of the day and Open-Close price of the day against the target variable, which is Price-Of-Next-Day - Price-Of-Today (future price):

  1. import matplotlib.pyplot as plt
  2.  
  3. pd.plotting.scatter_matrix( goog_data[['Open-Close', 'High-Low', 'Target']],
  4.                             grid=True,
  5.                             figsize=(10,6),
  6.                             diagonal='kde'# kernel density estimate
  7.                           )# computing an estimate of a continuous probability distribution
  8.                            # that might have generated the observed data
  9. plt.show()

     Using this scatter matrix, we can now quickly eyeball how the data is distributed and whether it contains outliers. For example, we can see in the kde (the lower right subplot in the scatter plot matrix) that the Target variable seems to be normally distributed but contains several outliers. Besides, the relationship between the variables (Open-Close and High-Low) and the target variable (target) is not linear.https://blog.csdn.net/Linli522362242/article/details/111307026
#####################

  1. import seaborn as sns
  2. import matplotlib.pyplot as plt
  3.  
  4.  
  5. # g = sns.pairplot( goog_data[cols],
  6. #                   # If True, don’t add axes to the upper (off-diagonal) triangle of the grid 
  7. #                   corner=False,
  8. #                   height=1.5,
  9. #                   aspect=2, # Aspect * height gives the width (in inches) of each facet.
  10. #                   diag_kind='kde'
  11. #                 )
  12.  
  13. g = sns.PairGrid( goog_data[cols],
  14.                   height=1.5,
  15.                   aspect=2
  16.                   )
  17. # g = g.map( sns.scatterplot )
  18. g.map_diag( sns.kdeplot )
  19. g.map_offdiag( plt.scatter ) # since I want to display all xlabels and ylabels
  20. # g.map_offdiag( sns.scatterplot )
  21.  
  22.  
  23. # sns.despine( left=False, 
  24. #              bottom=False,
  25. #              # right=False,top=False
  26. #            )
  27.  
  28. # remove the upper axes and better than setting corner=False,
  29. # g.fig.get_axes()
  30. # return:
  31. # [<matplotlib.axes._subplots.AxesSubplot at 0x1830b588>,
  32. #  <matplotlib.axes._subplots.AxesSubplot at 0x183b5e08>,
  33. #  <matplotlib.axes._subplots.AxesSubplot at 0x18491388>,
  34. #  <matplotlib.axes._subplots.AxesSubplot at 0x1877e988>,
  35. #  <matplotlib.axes._subplots.AxesSubplot at 0x187b7608>,
  36. #  <matplotlib.axes._subplots.AxesSubplot at 0x187ebfc8>,
  37. #  <matplotlib.axes._subplots.AxesSubplot at 0x18827d88>,
  38. #  <matplotlib.axes._subplots.AxesSubplot at 0x18971788>,
  39. #  <matplotlib.axes._subplots.AxesSubplot at 0x189a3c48>]
  40. for ax in g.fig.get_axes():
  41.     if ax.get_geometry()[2] in [2,3,6]:
  42.         ax.remove()
  43.  
  44. xlabels,ylabels = [],[]        
  45. for ax in g.axes[-1,:]: # g.axes[-1,:] get the last row axes  
  46.     xlabel = ax.xaxis.get_label_text()
  47.     xlabels.append(xlabel)
  48. for ax in g.axes[:,0]: # g.axes[:,0] get the first column axes
  49.     ylabel = ax.yaxis.get_label_text()
  50.     ylabels.append(ylabel)
  51.  
  52. for row in range( len(ylabels) ):
  53.     for col in range( len(xlabels) ):
  54.         if g.axes[row,col] != None :
  55.             g.axes[row,col].xaxis.set_label_text( xlabels[col] )
  56.             g.axes[row,col].yaxis.set_label_text( ylabels[row] )
  57.  
  58. plt.subplots_adjust( top=1.5 )
  59. plt.show()


g.map_offdiag( sns.scatterplot ) instead of g.map_offdiag( plt.scatter )

#####################

4. Finally, as shown in the code, let's split 80% of the available data into the training feature value and target variable set ( X_train , Y_train ), and the remaining 20% of the dataset into the out-sample testing feature value and target variable set ( X_test , Y_test ):

  1. X_train, X_test, Y_train, Y_test = create_train_split_group( X,Y, split_ratio=0.8 )
  2. X_train.shape, X_test.shape, Y_train.shape, Y_test.shape

X_train.head()

Y_train.head()


5. Now, let's fit the OLS model as shown here and observe the model we obtain: 

  1. conda install sklearn
  2. conda install -c anaconda scikit-learn
  1. from sklearn import linear_model
  2.  
  3. # Fit the model
  4. ols = linear_model.LinearRegression()
  5. ols.fit( X_train, Y_train )

 https://blog.csdn.net/Linli522362242/article/details/104005906

6. The coefficients are the optimal weights assigned to the two features by the fit method. We will print the coefficients as shown in the code:

  1. print( 'Intercept: \n', ols.intercept_,
  2.        '\nCoefficients: \n' , ols. coef_
  3.      )

7. The next block of code quantifies two very common metrics that test goodness of fit for the linear model we just built. Goodness of fit means how well a given model fits the data points observed in training and testing data. A good model is able to closely fit most of the data points and errors/deviations between observed and predicted values are very low. Two of the most popular metrics for linear regression models are mean_squared_error, which is what we explored as our objective to minimize when we introduced OLS, and R-squared (), which is another very popular metric that measures how well the fitted model predicts the target variable when compared to a baseline model whose prediction output is always the mean of the target variable based on training data, that is, .
############################https://blog.csdn.net/Linli522362242/article/details/111307026
Let's compute the MSE of our training and test predictions:

  1. from sklearn.metrics import mean_squared_error
  2.  
  3. print('MSE train: %.3f, test: %.3f' % ( mean_squared_error(y_train, y_train_pred),
  4.                                         mean_squared_error(y_test, y_test_pred)
  5.                                       ) )

        You can see that the MSE on the training dataset is 19.96, and the MSE on the test dataset is much larger, with a value of 27.20, which is an indicator that our model is overfitting the training data in this case. However, please be aware that the MSE is unbounded in contrast to the classification accuracy, for example. In other words, the interpretation of the MSE depends on the dataset and feature scaling. For example, if the house prices were presented as multiples of 1,000 (with the K suffix后缀), the same model would yield a lower MSE compared to a model that worked with unscaled features. To further illustrate this point, ($10K − 15K)^2 < ($10,000 − $15,000)^2 .

     Thus, it may sometimes be more useful to report the coefficient of determination决定系数 () , which can be understood as a standardized version of the MSE, for better interpretability of the model's performance. Or, in other words, is the fraction of response variance响应方差的分数 that is captured by the model. The  value is defined as:
  OR 

  • Here, SSE is the sum of squared errors(OR the sum of squared of residuals)
    This yields a list of errors squared, which is then summed and equals the unexplained variance.
  • and SST is the total sum of squares(total variance):
    the average actual value y.

Let's quickly show that  is indeed just a rescaled version of the MSE:
     #rescaled by the variance of y : 
     For the training dataset, the is bounded between 0 and 1, but it can become negative for the test dataset. If  = 1, the model fits the data perfectly with a corresponding MSE = 0(since Var(y)>0).

     Evaluated on the training data, the of our model is 0.765, which doesn't sound too bad. However, the on the test dataset is only 0.673, which we can compute by executing the following code:

  1. from sklearn.metrics import r2_score
  2.  
  3. print('R^2 train: %.3f, test: %.3f' % ( r2_score(y_train, y_train_pred),
  4.                                         r2_score(y_test, y_test_pred)
  5.                                       ) )


     Coefficient of determination, in statistics,  (or r^2), a measure that assesses the ability of a model to predict or explain an outcome in the linear regression setting. More specifically, R2 indicates the proportion of the variance方差 in the dependent variable 因变量(Y) that is predicted or explained by linear regression and the predictor variable (X, also known as the independent variable自变量).

     The coefficient of determination shows only association. As with linear regression, it is impossible to use  to determine whether one variable causes the other. In addition, the coefficient of determination shows only the magnitude of the association, not whether that association is statistically significant.

     In general, a high R2 value indicates that the model is a good fit for the data, although interpretations of fit depend on the context of analysis.

  • An R2 of 0.35, for example,
    indicates that 35 percent of the variation变动,差异 in the outcome has been explained just by predicting the outcome using the covariates(协变量, ) included in the model
    表明仅通过使用模型中包含的协变量, 预测结果即可解释结果差异的35％.
  • That percentage might be a very high portion of variation to predict in a field such as the social sciences;
  • in other fields, such as the physical sciences, one would expect R2 to be much closer to 100 percent. The theoretical minimum R2 is 0. However, since linear regression is based on the best possible fit, R2 will always be greater than zero, even when the predictor(X) and outcome variables(Y) bear no relationship to one another.



     R2 increases when a new predictor variable is added to the model, even if the new predictor is not associated with the outcome. To account for that effect, the adjusted R2 (typically denoted with a bar over the R in R2) incorporates the same information as the usual  but then also penalizes for the number(k) of predictor variables included in the model. As a result, R2 increases as new predictors are added to a multiple linear regression model, but the adjusted R2 increases only if the increase in R2 is greater than one would expect from chance alone仅当新项对模型的改进超出偶然的预期时，the adjusted R2 才会增加. It decreases when a predictor improves the model by less than expected by chance.In such a model, the adjusted R2 is the most realistic estimate of the proportion of the variation that is predicted by the covariates included in the model.
############################

     We will skip the exact formulas for computing but, intuitively, the closer the  value to 1, the better the fit, and the closer the value to 0, the worse the fit. Negative values mean that the model fits worse than the baseline model. Models with negative values usually indicate issues in the training data or process and cannot be used:
VVVVVVVVVVVVVV

  1. from sklearn.metrics import mean_squared_error, r2_score
  2.  
  3. # The mean square error
  4. print( "Mean squared error: %.2f" % mean_squared_error( Y_train,
  5.                                                         ols.predict( X_train )
  6.                                                       )
  7.      )
  8. # Explained variance score: 1 is perfect prediction
  9. print( "Variance score: %.2f" % r2_score( Y_train,
  10.                                           ols.predict(X_train)
  11.                                         )
  12.      )
  13. # The mean square error
  14. print( "Mean squared error: %.2f" % mean_squared_error( Y_test,
  15.                                                         ols.predict(X_test)
  16.                                                       )
  17.      )
  18. # Explained variance score: 1 is perfect prediction
  19. print( "Variance score: %.2f" % r2_score( Y_test,
  20.                                           ols.predict(X_test)
  21.                                         )
  22.      )

 ValueError: Input contains NaN, infinity or a value too large for dtype('float32').

  1. error_list=Y_test-Y_predicted
  2. np.any( np.isnan( error_list ) )

error_list[-3:]

Y_predicted[-3:]

Y_test[-3:]

 I forgot to dropna after df.shift(-1)

if not dropna in the data process: 

  1. from sklearn.metrics import mean_squared_error, r2_score
  2.  
  3. # The mean square error
  4. print( "Mean squared error: %.2f" % mean_squared_error( Y_train,
  5.                                                         ols.predict( X_train )
  6.                                                       )
  7.      )
  8. # Explained variance score: 1 is perfect prediction
  9. print( "Variance score: %.2f" % r2_score( Y_train,
  10.                                           ols.predict(X_train)
  11.                                         )
  12.      )
  13. # The mean square error
  14. print( "Mean squared error: %.2f" % mean_squared_error( Y_test[:-1],
  15.                                                         ols.predict(X_test[:-1])
  16.                                                       )
  17.      )
  18. # Explained variance score: 1 is perfect prediction
  19. print( "Variance score: %.2f" % r2_score( Y_test[:-1],
  20.                                           ols.predict(X_test[:-1])
  21.                                         )
  22.      )

Negative values mean that the model fits worse than the baseline model. Models with negative values usually indicate issues in the training data or process and cannot be used
^^^^^^^^^^^^^^^^^^^^

  1. from sklearn.metrics import mean_squared_error, r2_score
  2.  
  3. # The mean square error
  4. print( "Mean squared error: %.2f" % mean_squared_error( Y_train,
  5.                                                         ols.predict( X_train )
  6.                                                       )
  7.      )
  8. # Explained variance score: 1 is perfect prediction
  9. print( "Variance score: %.2f" % r2_score( Y_train,
  10.                                           ols.predict(X_train)
  11.                                         )
  12.      )
  13. # The mean square error
  14. print( "Mean squared error: %.2f" % mean_squared_error( Y_test,
  15.                                                         ols.predict(X_test)
  16.                                                       )
  17.      )
  18. # Explained variance score: 1 is perfect prediction
  19. print( "Variance score: %.2f" % r2_score( Y_test,
  20.                                           ols.predict(X_test)
  21.                                         )
  22.      )

 

8. Finally, as shown in the code, let's use it to predict prices and calculate strategy returns:

  • Normal log returns
    Log returns between two times 0 < s < t are normally distributed.
  • Log-normal values
    At any time t > 0, the values  are log-normally distributed.

The regression response variable is Close price tomorrow-Close price today for each day.
---It is a positive value if the price goes up tomorrow, a negative value if the price goes down tomorrow, and zero if the price does not change.
---The sign of the value indicates the direction, and the magnitude of the response variable captures the magnitude of the price move. 

  1. # # the difference between the close price tomorrow and the close price today 
  2. # df['Target'] = df['Close'].shift(-1) - df['Close']
  3. goog_data['Predicted_Signal'] = ols.predict(X)
  4. # Normal log returns         log( Close price today ) - log( Close price yesterday )
  5. goog_data['GOOG_Returns'] = np.log( goog_data['Close']/goog_data['Close'].shift(1) )
  6.  
  7. def calculate_return( df, split_value, symbol ):
  8.     cum_goog_return = df[split_value:][ "%s_Returns" % symbol ].cumsum() * 100
  9.     # Calculates the log returns of the trading strategy 
  10.     # given the prediction values and the benchmark log returns.
  11.     #                        log actual return today   * Predicted return today    
  12.     df['Strategy_Returns'] = df["%s_Returns" % symbol] * df['Predicted_Signal'].shift(1)
  13.     # for classification
  14.     # df['Strategy_Returns']=df["%s_Returns" % symbol] * np.sign( df['Predicted_Signal'].shift(1) )
  15.     return cum_goog_return
  16.  
  17. def calculate_strategy_return( df, split_value, symbol ):
  18.     cum_strategy_return = df[split_value:]['Strategy_Returns'].cumsum() * 100
  19.     return cum_strategy_return
  20.  
  21. cum_goog_return = calculate_return( goog_data, 
  22.                                     split_value=len(X_train), symbol='GOOG' )
  23. cum_strategy_return = calculate_strategy_return( goog_data, 
  24.                                                  split_value=len(X_train), symbol='GOOG' )
  25.  
  26. def plot_chart( cum_symbol_return, cum_strategy_return, symbol ):
  27.     plt.figure( figsize=(15,6) )
  28.     plt.plot( cum_symbol_return, label='%s Returns' % symbol )
  29.     plt.plot( cum_strategy_return, label='Strategy Returns' )
  30.     plt.legend()
  31.     plt.show()
  32.     
  33. plot_chart( cum_goog_return, cum_strategy_return, symbol='GOOG' )

The simplified approach taken here does not account for transaction costs  

      Here, we can observe that the simple linear regression model using only the two features,
Open-Close and High-Low, returns positive returns. However, it does not outperform the
Google stock's return because it has been increasing in value since inception[ɪnˈsepʃn]. But since that cannot be known ahead of time, the linear regression model, which does not assume/expect increasing stock prices, is a good investment strategy. 

  1. # Sharpe ratio: The risk-adjusted return. This ratio is important 
  2. # because it compares the return of the strategy with a risk-free strategy
  3. def sharpe_ratio( symbol_returns, strategy_returns ):
  4.     strategy_std = strategy_returns.std()
  5.     sharpe = (strategy_returns-symbol_returns) / strategy_std
  6.     return sharpe.mean()
  7.  
  8. print( sharpe_ratio(cum_strategy_return, cum_goog_return) )

 

goog_data.shift(1).tail(10)


VVVVVVVVVVVVVV

  1. def calculate_return( df, split_value, symbol ):
  2.     cum_goog_return = df[split_value:][ "%s_Returns" % symbol ].cumsum() * 100
  3.     # Calculates the log returns of the trading strategy 
  4.     # given the prediction values and the benchmark log returns.
  5.     #                        log actual return today   * Predicted return today    
  6.     # df['Strategy_Returns'] = df["%s_Returns" % symbol] * df['Predicted_Signal'].shift(1)
  7.     # for classification
  8.     df['Strategy_Returns']=df["%s_Returns" % symbol] * np.sign( df['Predicted_Signal'].shift(1) )
  9.     return cum_goog_return
  10.  
  11. cum_goog_return = calculate_return( goog_data, 
  12.                                     split_value=len(X_train), symbol='GOOG' )
  13. cum_strategy_return = calculate_strategy_return( goog_data, 
  14.                                                  split_value=len(X_train), symbol='GOOG' )
  15. plot_chart( cum_goog_return, cum_strategy_return, symbol='GOOG' )
  16.  
  17. print( sharpe_ratio(cum_strategy_return, cum_goog_return) )

 np.sign( goog_data['Predicted_Signal'].shift(1) )[-10:]


^^^^^^^^^^^^^^^^^^^^

Regularization and shrinkage – LASSO and Ridge regression 

     Now that we have covered OLS, we will try to improve on that by using regularization and coefficient shrinkage using LASSO and Ridge regression. One of the problems with OLS is that occasionally, for some datasets,

  • the coefficients assigned to the predictor variables can grow to be very large. Also,
  • OLS can end up assigning non-zero weights to all predictors and the total number of predictors in the final predictive model can be a very large number.

     Regularization tries to address both problems, that is, the problem of too many predictors and the problem of predictors with very large coefficients. Too many predictors in the final model is disadvantageous because it leads to overfitting, in addition to requiring more computations to predict. Predictors with large coefficients are disadvantageous because a few predictors with large coefficients can overpower the entire model's prediction, and small changes in predictor values can cause large swings in predicted output. We address this by introducing the concepts of regularization and shrinkage. 

     Regularization is the technique of introducing a penalty term on the coefficient weights and making that a part of the mean squared error, which regression tries to minimize. Intuitively, what this does is that it will let coefficient values grow, but only if there is a comparable decrease in MSE values. Conversely, if reducing the coefficient weights doesn't increase the MSE values too much, then it will shrink those coefficients. The extra penalty term is known as the regularization term, and since it results in a reduction of the magnitudes of coefficients, it is known as shrinkage.

     Depending on the type of penalty term involving magnitudes of coefficients, it is either L1 regularization or L2 regularization. When the penalty term is the sum of the absolute values of all coefficients, this is known as L1 regularization (LASSO), and, when the penalty term is the sum of the squared values of the coefficients, this is known as L2 regularization (Ridge)OR . It is also possible to combine both L1 and L2 regularization, and that is known as elastic net regression. To control how much penalty is added because of these regularization terms, we control it by tuning the regularization hyperparameter. In the case of elastic net regression, there are two regularization hyperparameters, one for the L1 penalty and the other one for the L2 penalty.
##############
https://blog.csdn.net/Linli522362242/article/details/104070847

  • If α = 0 then Ridge RegressionOR is just Linear Regression.
  • If α(or 声明：本文内容由网友自发贡献，转载请注明出处：【wpsshop】
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。