5 – M1L6 09 Statistical Analysis V10

Now we are ready to perform our analysis. The resulting returns time series is the theoretical monthly performance of our long-short portfolio. Our goal is to see if the mean monthly return is greater than zero. Let’s calculate that. So, our mean is indeed greater than zero at 0.53 percent, but does this mean that we got this value because on average this trading strategy yields positive returns? That is to say that the true mean is greater than zero? It could be that on average this trading strategy will not yield positive returns. Maybe the true mean is less than or equal to zero, and this value is just a random fluctuation. Well, one approach to assess this is to perform a statistical test on our hypothesis. A t-test is a way of testing the probability of getting as bigger mean as we did, assuming all the assumptions we made to build our model of strategy returns were correct. In our case, we can compute the t-statistic by dividing the mean return x bar by the standard error of the mean SE x bar. Using this t-statistic, we can measure the probability of getting a mean monthly return of 0.53 percent or larger if the true mean monthly return is zero given that the assumptions we made to build our model are correct. This probability is called the p-value. If the probability is very small, we might infer that it’s unlikely that the true mean is zero. Now, before running the test, we should decide how small the p-value needs to be for us to conclude that the true mean is not zero. To denote this threshold, we usually use the Greek symbol alpha. A commonly used value is 0.1. Setting alpha sets the false positive rate for the test. So, if we set alpha equals 0.1, we’re effectively saying that if we use this threshold, we are incorrectly rejecting the null hypothesis when the true mean is zero in 10 percent of many hypothetical users of this test. By performing a one-tail t-test on the alternative hypothesis that the mean of these returns is statistically greater than zero, we calculate the p-value with the stats package using a degrees of freedom of n minus one equals 47, and we find that the t-statistic is about 1.618, with the p-value of 0.0566. Meaning that the result is not significant at P less than 0.05 level, but insignificant at P less than 0.1. This means that we are fairly unlikely to get a mean of 0.53 percent if the true mean is zero. This result shows some initial promise that there could be some alpha contained in the strategy, and thus serves as a quick sanity check to justify investigating further and fine-tuning the strategy leading to a full backtesting exercise in the end. If we measure the monthly performance of the SPDR, S&P 500 ETF, SPY over the same period, we would find that the mean monthly return is 0.83 percent, which is actually greater than that of the strategy, at least at this preliminary stage. To be clear, comparing our momentum strategy to the strategy of investing directly in S&P 500 ETF is not really a fair comparison. These are two very different strategies. For one thing, ours involves taking long and short positions, while investing in the S&P only involves taking long positions. However, this gives us a vague idea of what values portfolio returns might take. Also, while our strategy’s obviously simplistic to begin with, this shows that it is actually not an easy task to find a strategy that can outperform the market. Note that if you get a very large p-value at this early stage, you wouldn’t want to go back and make meaningless changes to your strategy like changing a parameter a little until you got a very low p-value. Doing this is called data snooping or p-hacking, and would just fine tune your strategy to exploit the particular random fluctuations in the dataset you have. A strategy device that way would be unlikely to perform well on future data. What you should do instead is make meaningful changes to your strategy. In other words, you should come up with a different idea for a strategy and test that instead.

%d 블로거가 이것을 좋아합니다: