Ykhjuy

There is no one answer because some agencies are more efficient than others.

For example, Medicare is much more efficient in processing and paying medical claims than private insurance companies are at the same task. Medicare devotes just 1.4% of expenditures for administrative costs (a more skeptical analysis that used a broader definition of administrative costs came up with 5.2%), with the rest of its funds going towards payment of claims, while private health insurers devote 11%-20% of their expenditures to administrative costs, leaving a smaller share of its resources to pay claims. See also here ("Medicare outperforms private sector plans in terms of patients' satisfaction with quality of care, access to care, and overall insurance ratings.") and here (health care spending goes down while service utilization is unchanged when an average person transitions from private health insurance to Medicare). You can argue that Medicare is able to do that, in part, because by having a policy of paying the prevailing insurance company negotiated price of any provider that it is free riding on costs incurred by private insurance companies to negotiate lower prices with providers, but certainly, Medicare dispenses with significant expenditures associated with advertising and executive compensation that private insurance companies must pay that it need not. Less obviously, Medicare claims processing is more efficient because every medical provider processes lots of Medicare clams and learns how to do it correctly, while a medical provider may process claims from a smaller health insurer only sporadically.

Similarly, Social Security is more efficient in its administration of benefits dispersal than than almost any private sector retirement plan (which average 1.5% to 1.8% administrative fees) in existence. Annuities, the closest private sector analogy to Social Security, have particular high administrative fees (on average 2.3%).

Social Security may not maximize investment return (not so much because it couldn't, lots of states have public employees retirement systems that do so at a quite low administrative cost, as because it is mandated by law to invest in Treasury securities and nothing else, the safest investment in existence often operationally treated by economists as the risk free rate of return), but this is, in part, to provide maximum security when markets fail (Social Security was devised after the collapse of financial markets that led to the Great Depression left many people who had saved diligently all their lives penniless.) Social security isn't just an investment, it is insurance against other investments turning out poorly when bear markets destroy investment wealth.

The U.S. Department of Education took over direct servicing of student loans because that turned out to be more efficient than what the private sector was doing. As Wikipedia explains, the system was put in place, while "[g]uaranteed loans—loans originated and funded by private lenders but guaranteed by the government—were eliminated because of a perception that they benefited private student loan companies at the expense of taxpayers, but did not help reduce costs for students."

In general, government agencies can be very efficient at handling large volumes of administrative paperwork involving large numbers of small individual cases as the examples above illustrate, in part, due to economies of scale.

Also, in some cases, it is hard to compare because there is no close private sector counterpart. One could say that the U.S. Department of Defense is very inefficient (and no one seriously disputes that there are many ways it could be made significantly more efficient even though the magnitude of potential savings varies greatly), but there is no private sector counterpart running the largest military force in the world by many measures. One could try to compare, for example, the efficiency of military defense procurement projects done by private companies under contract with the government to private sector manufacturing enterprises of other kinds, like automobile manufacturers. But, private sector manufacturing is arguably under pressure to achieve very different objectives - the best possible outcome at an affordable price, rather than the best possible outcome at any price because failing means the end of civilization as we know it.

There are also areas where there is considerable private sector competition with public sector institutions, providing a means of comparison, but the public sector institutions aren't notably better or worse in efficiency than the private sector counterparts and both are equally inefficient. There is a good case to be made that this is true of schools, colleges, universities, and hospitals ("5 studies reveal that public and not-for-profit hospitals are more efficient than those in private ownership. One study concludes the opposite, and 2 could not demonstrate any significant differences between the different hospital ownerships."). (Although per student or per degree costs of community college programs in the public sector are much lower for better outcomes than their private sector counterparts despite low rates of government subsidies per student than any other form of public higher education).

There is likewise no compelling evidence that private prisons are run more efficiently, controlling for the makeup of the prisoners supervised, than public prisons.

Similarly, there are private sector businesses that are very inefficient and dysfunctional, often because they don't face much competition. The cable company is a private company, but has a monopoly franchise in any given place for an extended time period (because it is a "natural monopoly", or at least used to be one), and unsurprisingly it has poor customer service and lots of inefficiencies and mismanagement that is allowed to persist. In addition to utilities, companies with patents (e.g. drug companies) can sometimes be inefficient because they have government monopolies. Indeed, one of the justifications for patents is that they are necessary to allow something inefficient (research and development that will often not pan out) to be conducted.

Certainly, this isn't to say that there are no areas where the private sector does not get the job done better and cheaper than the public sector. But, even in those cases, this isn't universal. Some countries, like France and Sweden, do a particularly good job of public administration (anecdotal evidence here). Indeed, the most prestigious university in all of France is a school for public administrators. Other countries, like the U.S., are worse at public administration, such as Italy.

But, as these examples illustrate, there is no one compelling reason.

Probably the most powerful reason that the private sector is sometimes better than the public sector comes from countries with many state-owned enterprises. Comparisons in those cases indicate that the single greatest factor is that money-losing private businesses promptly go out of business or are reorganized in bankruptcy, while deficits in state-owned enterprises are often subsidized and keeping them in business for a long time. (Also, some state owned enterprises are intentionally inefficient as a means of providing political patronage).

The fact that government institutions don't automatically shut down when they cease to be profitable as currently organized is sometimes a feature, rather than a flaw, however. Half of all private businesses fail within their first five years. Government institutions are frequently established to provide services that cannot for practical reasons, or should not, for moral reasons, be discontinued.

For example, while in Japan and the Netherlands (both places where space is at a premium), public sensibilities are O.K. with tossing human remains in a dumpster if the surviving family fails to pay an annual cemetery bill, in the U.S., cemetery administration is often vested in a government because the time horizon over which we expect that it will continue to need to be maintained is infinite.

Similarly, you don't want to nuclear waste disposal facility to just shut down and have its employees all laid off because fees from new waste deposits no longer cover its operating costs, even if the problem arose because an incompetent nuclear waste facility administrator a decade ago underestimated how long it would cost to keep the facility open in what was supposed to be a self-supporting enterprise.

We want water and sewer systems, the military, law enforcement agencies, historical archives, the courts, real estate record keeping, a school for the deaf, and the like to continue indefinitely, even if this eliminates one of the easier ways to control inefficiency, by shutting down inefficient institutions entirely. Government ownership is often an alternative to utility regulation for natural monopolies that must also endure in the long run.

When shutting down inefficient institutions entirely isn't an option, you look for alternative solutions like developing strong audit and oversight institutions with the power to bring about change when an institution operates inefficiently.

Amtrak exists, for better or worse, because somebody in government decided in 1971 that the United States needed at least one passenger rail company when all of them were going bankrupt and all of the infrastructure involved in once private passenger rail system was at risk of going to waste. There are legitimate reasons to say that maybe the United States doesn't actually need a slow speed passenger rail system using 1960s technology. But, once somebody has decided that the service must be provided even though it doesn't make economic sense to do so, it is impossible to run that service in an efficient manner.

President Obama gave a speech at Carnegie Mellon in 2016 where he responded a bit to the idea that government should be run more like a Silicon Valley business, which often comes up when government efficiency is mentioned:

The final thing I’ll say is that government will never run the way Silicon Valley runs because, by definition, democracy is messy. This is a big, diverse country with a lot of interests and a lot of disparate points of view. And part of government’s job, by the way, is dealing with problems that nobody else wants to deal with.

So sometimes I talk to CEOs, they come in and they start telling me about leadership, and here’s how we do things. And I say, well, if all I was doing was making a widget or producing an app, and I didn’t have to worry about whether poor people could afford the widget, or I didn’t have to worry about whether the app had some unintended consequences -- setting aside my Syria and Yemen portfolio -- then I think those suggestions are terrific. (Laughter and applause.) That's not, by the way, to say that there aren't huge efficiencies and improvements that have to be made.

But the reason I say this is sometimes we get, I think, in the scientific community, the tech community, the entrepreneurial community, the sense of we just have to blow up the system, or create this parallel society and culture because government is inherently wrecked. No, it's not inherently wrecked; it's just government has to care for, for example, veterans who come home. That's not on your balance sheet, that's on our collective balance sheet, because we have a sacred duty to take care of those veterans. And that's hard and it's messy, and we're building up legacy systems that we can't just blow up.

Government has to be for everyone, in a way that private businesses generally don't deal with. If you're entitled to social security, you get social security, even if you live in a remote community, don't speak English, and are incapable of managing your own affairs. A private investment plan would likely determine that you're not an economically worthwhile customer to serve. Social security sets up a small resident station, arranges access to interpreters, and appoints a representative payee.

As another example, the federal voter registration form says that, if you don't have a home address or are homeless, you can literally draw a map of your residence on the voter registration form, indicating nearby landmarks. That's not something that fits neatly into any database. Some poor clerk has to decipher your map and figure out which voting districts you live in. Plenty of private businesses will just refuse to serve you unless you have an address, a phone number, email, etc... That's not an option for the government.

Efficiency is inherently different in government. An app developer might decide to withdraw support for an old version of an operating system, because losing 2% of legacy users is worth the cost of not having to support the old version. The government can't just deny services to 2% of citizens even if it's costly to serve them; it has to provide some path that works.

Just like anything else, most things humans do arise out of incentives.

• A private company employee has incentive to be efficient because if they are not, their manager has incentives to fire them, and replace them with a more efficient employee.




• A private company manager has incentive to do that because efficiency has direct impact on productivity and/or customer satisfaction; both of which have direct impact on revenues; due to competition (a low-productivity producer must charge more leading to customers picking competing product; unsatisfied customer can also chose competing product).




• A private company manager has incentive to care about revenues because on the upside, if revenues are higher, they get better pay; and promotions; whereas on the downside, if revenues are sufficiently lower, they get fired in layoffs; or in worst case, company goes under and everyone loses their job.

Now, compare that to public institution. Those institutions are characterized by 3 features:

• 
  

  Monopoly and lack of competition.

  
  
  

  If you as a customer do not like the product or a service offered by a public institution, you are typically S.O.L. There is usually no competing product or service being offered; due to a government rule, law or mandate. You cannot choose to get a driving ID from a private company if your motor vehicle department works poorly or demands bribes. You cannot have your house improvements certified by a private inspection company, you have to wait for a never-in-a-hurry government bureaucrat to do that.

  
  
  

  Or, the service or product has barriers to entry too high, due to scarce resources. For example, water delivery requires physical pipes, so a private company would be unlikely to be able to do this at a competing price. Ironically, this isn't unique to public institutions - same problems caused the rise of landline phone monopolies earlier and wired Internet service provider monopolies nowadays.

  
  
  

  Or, in case of some services, the competing private ones cost too much for the poor; and government one is the only available option within near-free price range (as the service is subsidized by government budget).

  
  


• 
  

  No need to ensure revenue.

  
  
  

  Revenues for public institutions are generated by forcing taxpayers to pay taxes and allocating government revenue by the government out of that. As such, there is no direct impact on revenues from quality of service or product; or productivity^[1].

  
  
  

  _{[1] - one could argue there is an indirect impact in that poor service/product may lead to lower overall economic growth which leads to lower tax revenues down the line. But that is way way too indirect to matter in this case.}

  
  


• 
  

  Employee incomes are not correlated with efficiency.

  
  
  

  If you work better at a private company, you have a chance of being paid more (performance bonuses, pay raises) - see the incentive structure above. At a public institution, they pay is typically very static, with no performance impact on compensation.

  
  


• 
  

  In some cases, it's impossible or difficult to fire poor performers, even if the institution and its managers would want that.

  
  
  

  First, there are unions. At least in USA, public institutions are far far far more heavily unionized than private ones. Second, there are rules about firing people in public institutions, as the employer is "government".

  
  

Others have noted the issues with incentives and competition, but there is actually another reason, which is dis-economies of scale.

Back in 1937 economist Ronald Coase asked the deceptively simple question: "why do companies exist?". He reasoned that if the open market was the most efficient way of getting things done then firms (i.e. companies with lots of employees) shouldn't exist; the economy would consist entirely of individual contractors. However firms do exist, and different markets seem to favour different sizes of firms. This led him to set out his "theory of the firm".

As a company grows it can exploit economies of scale; if you make 100 widgets a week it costs $2/widget, but if you can make 1,000 widgets a week you can reduce the cost to $1.90/widget by increasing automation, using your staff in more specialised roles that require less training, getting bulk discounts on your inputs, and so on. Hence in the widget business companies grow or go out of business; the company making only 100 widgets/week is not going to be able to compete with the one making 1,000/week.

So why isn't the economy dominated by a single giant monopoly in everything? The answer is that increasing scale brings dis-economies:

• Coordination becomes more difficult. The guy on the factory floor may realise that screw No. 7 is difficult to get in, but that information doesn't get back to the design department where it could be fixed.




• New products take longer to bring out because of the work required to retool and reskill the production line.




• Employees no longer need to work hard to "succeed". Instead they are given targets and can relax once these are achieved. Their managers have no incentive to make these targets a real challenge because then they would be seen as failing too. This disincentive goes all the way from the bottom to the top, which is why so many CEOs pocket big bonuses for lack-lustre performance.




• Some middle managers also play empire-building games, seek to undermine each other, and generally mess up the efficiency of the company.

So in any industry there is an optimum point where the economies of scale due to additional growth are matched by the dis-economies. Companies in this industry will tend towards this size.

Now consider the government. Its size is fixed by the size of the country and the list of jobs it does (which in turn is set by law). It is probably the biggest organization in the country, doing a vast range of different jobs under very variable circumstances.

So on one hand there are likely to be huge dis-economies of scale where it does jobs, such as running lots of small offices, or schools, which would be more efficient if done by a small organisation.

On the other hand it is also likely to behave in ways that prevent economies of scale, for instance running lots of small local schools instead of one big one with a fleet of buses.

I've worked at both private and public institutions. Neither seem significantly more or less efficient than the other - there's tons of waste in private institutions but people don't seem to care about that as much.

Public institutions are inefficient because they are removed from the natural market place.
They don't have any competition. Private businesses must be efficient to succeed in a free enterprise market economy. When you have a monopoly on any good or service there's zero motivating factors to increase efficiency.
Also add the bureaucratic attitude that all government agencies inherently have, which is a need and desire to spend all allocated funds or risk a reduction in those funds in the following year; combined with a tenor based, versus a productivity based, pay scale and you have a tasty recipe for low productivity.

You're asking a political question, but the answer is simple economics. Put simply, what you are seeing is a feature of Monopolies, not Governments.

Your stereotypical company with competitors has a limit to how incompetent it can be. Past a certain point, enough customers will use their competitors and they will go out of business.

Even in a world where its pretty much random how efficient or how incompetent each different organization is, you'd expect to see survivorship bias. Simply, competitive companies can die, so the worst will do so. You're only seeing the survivors, and to you that looks like competence and efficiency by comparison.

When an organization doesn't have any competitors, that's called a "monopoly". Since their customers have no choice (other than not getting the service at all), this organization plays by different rules. The inefficiency that many have come to associate with Government agencies typically is a feature of the fact that they are inherently monopolies.

Anyone familiar with large private utility monopolies can tell you they tend to behave the same way as government agencies. There is no danger to the organization if you aren't happy with your service, and so there's nothing really stopping anyone from treating you badly, other than personal initiative. Some monopolies are better, some are worse, but of course its the worse ones you really remember.

Private companies and institutions fill a need where there is profit available.

So what happens when there is an overwhelming, moral or otherwise important need that does not stand to enrich private investors? What happens when profiteering from the need and misery of others is considered immoral?

The government or other public institutions step in because government, ultimately, is supposed to serve the needs of the citizens as a priority.

Do we know that government can't make a bigger profit or be more efficient in areas that are more easily profitable? No, but since those needs are often adequately being served by private enterprises, there is no need for the government to step in or meddle in those areas, and they don't wade in to those areas in non-socialist economies.

So, for those who are short-sighted enough to only look at profit/loss in assessing the value of a process or entity, they are going to see a lot more profitable services being filled by private entities, and more guaranteed money-pits being run by the government.

A better measure, perhaps, is whether competitive profit is the best determining factor for filling a need. I want a better, faster, fancier car, cell phone, computer? Sure, I can shop and choose and the market will sort that out.

Difficult to make a "market choice" if you are deathly ill, have a rare or poorly understood disorder, or need life-saving treatment or medication. When the "choice" is "pay whatever we choose to demand or die" the consumer really doesn't have much in the way of options or ability to vote via shopping, if you want to take health care as an example.

If our nation is at war, are we going to let the "market" determine which arms our soldiers use via equipment failures on the battlefield? No, so government pays a steep premium for up-front rigorous specification and quality controls and chooses a single winning contractor as the standard, and will wind up throwing money far beyond the original price tag to fill a need.

Government services, generally, are driven by service, first, profit last. In the for-profit world, companies will balance service vs profit, and will give the least service they can before it becomes a losing proposition in terms of consumer choice.

Anyone who tells you that government would be able to do everything better is probably more than a little naive. The same is also true for those who believe that greed and profit is automatically the best choice for all situations.