So far we’ve learned a few common ways to formulate and solve a portfolio optimization problem. But there are a number of practical limitations to these methods. First of all, mean returns are notoriously difficult to estimate accurately. Past returns are frequently poor estimates of future returns. To mitigate this problem, people frequently avoid estimating returns directly altogether. We’ve already introduced one possible solution which is to simply minimize the difference between your portfolio weights and another set of target weights. Another technique is to use an alternative quantity thought to be predictive of future returns as a substitute for actual returns value estimates. We’re going to talk more about these methods in a future lesson. There are also problems inherent to the estimation of portfolio variance. The first is that variants may not be a good enough measure of risk. This is the theme that might come up whenever you rely upon variants to estimate risk. To understand what I mean, think about the distribution of log returns. If the distribution has a different shape on the left side and on the right, this means there’s a different probability of getting very high positive returns than getting very high negative returns, but mean and standard deviation won’t tell you about this asymmetry. For example, these two distributions have the same mean and the same standard deviation, but obviously very different shapes. As a quantitative trader, you really care about the shape of that negative tail because that part of the distribution tells you how likely you are to get large negative returns. As a workaround for this, you can formulate your optimization problem using alternative measures of risk, some of which we’ll discuss in future lessons. Furthermore, the need to estimate an n by n covariance matrix, where n is the number of assets in your portfolio, means the problem size grows very quickly as the number of assets increases and the matrix becomes very large. Estimating large numbers of parameters introduces the possibility of aggregating many estimation errors. When the number of stocks n is of the same order of magnitude as the number of historical return data points per stock t, the total number of parameters to estimate in the covariance matrix is of the same order as the total size of the dataset, which is problematic. The need for a covariance matrix also means that we need to have a long enough historical stock price dataset to produce reliable estimates of covariances. For example, estimating a covariance matrix of 50 assets requires at least five years of daily data. If there are not enough daily return data, estimates of variances and covariances will not be accurate. But in fact any estimate is noisy. How do you handle noise? Well, instead of using raw values of quantities like means, you can utilize their ranks so that you only keep track of their relative magnitude. You can also simply include a term in the objective function to penalize turnover. This means that by default you are less likely to trade, therefore your results will be less reactive to noise. Finally, there’s a method called robust optimization which takes into account confidence intervals of estimates. Another issue which we discussed when we mentioned rebalancing is that portfolio optimization is typically done using a chunk of data from some time period, but then needs to be performed again as time passes and things change. In other words, the optimized weights are really only valid as long as the estimates of the parameters fed into the problem are accurate. There’s no built-in way to account for change over time. But the changing nature of the stock market may lead to conflicting predictions over different time horizons. For example, what if your analysis tells you that a return over one day horizon will be positive, but over a one month horizon it will be negative? In a single period optimization framework, it’s not clear how to account for the different timescales while taking into account the conflicting predictions. One approach to account for the changing nature of the stock market over time as part of optimization is called multi-period optimization. In essence, this involves modeling a plan for trading over the next several periods of time using the best information available at the current time, and optimizing over this series of future trades but only ever executing the trades for the first period. This basically amounts to a planning exercise, the idea is simply to try to ensure that we don’t place any traits now that will put us in a bad position in the future. In the example I just mentioned, where we have a prediction that returns over one day will be positive, returns over one month will be negative, we can see intuitively that it makes sense to go long and short to take advantage of these predictions, if transaction costs are low but to refrain from trading if transaction costs are high. If transaction costs are high, we may waste more money making trades than we could gain by following the predictions of positive and negative returns. A multi-period optimization analysis that includes transaction costs would provide an optimal solution to this problem. Another difficulty of portfolio optimization is the ambiguity about how to deal with transaction costs. Every time we make trades to rebalance the portfolio, we incur trading costs. But in general, they’re difficult to precisely model. In order to account for them, we model them as being proportional to turn over, the sum total of changes and portfolio weights. But you can also include measures of turnover in the optimization problem formulation, either as a constraint, or as a term to minimize in the objective function. To address some of these weaknesses, factor-based models of risk are often seen as more practical but if this discussion seem rather abstract to you, just hang on. Many of these topics will be elaborated upon in a future lesson.