The higher energies involved. If you want to cram more information in a given space, you have to use higher frequencies, and thus more power/space. If you attempt to deliver a message in this 'format' while using the same energy you needed to deliver it over the 4G network, this message would only travel a fraction of what it would otherwise on the older and slower network. Thus, you need to build more antennas along the message's path. That involves a higher cost, and potentially, use of land. As a side effect, you might end up placing antennas in forests and reserves where they previously were none. In addition, IPCs might focus on densely urban areas where the number of subscribers divided by antennas will be the highest, thus maybe contributing to a technological gap with the people living on the land. And on top of that we still don't know the long-term effects of being exposed to these waves of different frequencies. For all we know, it could even fuck with other equipments, let alone biomass.
So just because of the power requirements of 5G, we end up with an economical, environmental, societal and technological problem. Some of these issues can be fixed quicker than others, yes. But we will need to learn sooner than later what effect the introduction of all those different techs have on our daily life, in order to better forecast the future.