The average return from experiment A is indeed higher than from B. Why is that? It’s due to a phenomenon known as survivor bias or survivorship bias. If you only picked from the stocks that have survived until today, that already filtered out all the other stocks that failed during that period. Another way to think about experiment A, is that you essentially traveled forward in time from 2005 to the present day. You know then which companies have survived, went back and bought from among those stocks. Of course you are going to do well, but what determines you? Experiment B is more honest. You buy stocks based on information available in 2005. No time travel mumbo jumbo and as expected some of these stocks will fail giving you a lower return on average. This is very critical when you’re testing the efficiency of a trading signal or strategy using historical prices. If you allow survivor bias to creep into your analysis, you will get results that are overly optimistic and probably won’t work in the real world.