A free market does optimize but what it optimizes may not be necessarily what you want. For example, when it comes to building homes, making toys or preparing food, we want to make sure that there is an incentive for the producer to make safe products. Milton Friedman would argue that if a company made shoddy or unsafe products, the market would eventually punish that company. This might be true but who wants to be the parent of the first child that gets lead poisoning from a toy or be the first occupant of a house that burns down because of an electrical fault. Sure, the company may go out of business after it is known that they make dangerous products but some people have to be sacrificed to obtain this knowledge. In some cases, like cigarettes or asbestos, the health consequences of a product may not be known for decades so millions of people could be affected before the market corrects. That is why we have regulations for toys, buildings and food.
In terms of health insurance, it would seem like the most sensible business model is to do your best to not pay claims. What you are taught in business school is that to increase profits you must decrease cost and paying out claims is the main cost. Hence, it is not surprising that you hear all sorts of horror stories about health insurers denying coverage. Paul Krugman gives some examples in today’s New York Times. This seems to be perfectly logical and a perfect example of a misaligned incentive. The goal of the customer (which is to get health care) and that of the provider (which is to not give health care) are diametrically opposed. This is not true of all industries. For example, in violin making, the goals of the customer and the producer, which is to get/make the best quality violin for the lowest cost, are aligned. Hence, it is necessary to regulate insurers. However, in many ways this is an unstable situation because the goal of the insurer is always be to find loop holes around the regulations.
The misalignment in incentives is further compounded because the health care providers are reimbursed by the number of procedures they perform so their incentive is to provide as much health care as possible. Thus, on the one hand the providers are doing their best to maximize the cost of a visit and on the other hand the insurers are doing their best to not honor the claims. It’s not hard to see why the result is sometimes an astronomical bill for an unsuspecting patient who just got their insurance retroactively terminated. There is also a conflict between the insurers and the providers because it is in the interest of the insurer to not reimburse the provider. Hence, we have this three way battle between insurers, providers and consumers. The ideal situation would be to align the incentives of the health care providers and the insurers with the consumers. After all, the receiver of health care doesn’t really want more health care, she just wants to be healthy. I don’t really know what the optimal system would be but I do know that our current system is pretty far away from that solution.