Tim Harford summarises the problem rather nicely in his book Adapt:
There is a limit to how much honest feedback most leaders really want to hear; and, because we know this, most of us sugar-coat our opinions whenever we speak to a powerful person. In a deep hierarchy, that process is repeated many times, until the truth is utterly concealed inside a thick layer of sweet-talk. There is some evidence that the more ambitious a person is, the more he will choose to be a yes-man - and with good reason because yes-men tend to be rewarded.
Even when leaders and managers genuinely want honest feedback, they may not receive it. At every stage in a plan, junior managers or petty bureaucrats must tell their superiors what resources they need and what they propose to do with them. There are a number of plausible lies they might choose to tell, including over-promising in the hope of winning influence as go-getters, or stressing the impossibility of a task and the vast resources needed to deliver success, in the hope of providing a pleasant surprise. Actually telling the unvarnished truth is unlikely to be the best strategy in a bureaucratic hierarchy. Even if someone does tell the truth, how is the senior decision-maker to separate the honest opinion from some cynical protestation?
The resulting filtering can have devastating effects in a steep hierarchy: What starts out as bad news becomes happier and happier as it travels up the ranks -- because after each boss hears the news from his or her subordinates, he or she makes it sound a bit less bad before passing it up the chain.
Building on this analysis, we are all likely, of course, to be upset by, or resent, inconvenient or unwelcome advice, especially if it carries the implication that we are ill-informed, or lack good judgement. And our advisors won't want to deal with the negative emotions that unwelcome advice provokes. We all therefore take more care when delivering such advice, and often ‘aim off’ to some extent, and/or sugar coat our advice with positive feedback before delivering the less welcome stuff.
There is also a "shoot the messenger" problem. Bearers of bad news, even when they aren't responsible for it in any sense, tend to be blamed and to have negative feelings directed toward them. We are accordingly likely to soften the message – or maybe not deliver it at all – if we fear the reaction of the recipient, and we are much more likely to fear their reaction if they are powerful and so in a position to harm us. Politicians – for reasons which I explore below – are particularly likely to resent advice which they regard as obstructive or negative. Some will even try to surround themselves with courtiers and other yes-men (and women) to the great detriment of good government – which is why civil servants are entreated to resist this and insist on ‘speaking truth to power’.
It is sometimes the case that there isn't time, or there isn't a convenient opportuity, to frame the message in an effective way. And the wider the power gap, the more difficult it can be to communicate even urgent concerns. My Understanding Regulation website describes the distress of junior doctor Rachel Clarke when she failed to challenge the appalling behaviour of one senior consultant doctor.
It follows from this analysis that the private and public sectors face similar issues. IT project failures are just as common in the private sector as in government, though much less publicised, for obvious reasons. And private sector managers face just as intense pressures - from their Boards - to deliver quickly and to a tight budget. Margaret Hefernan (in her book Wilful Blindness) notes that a high proportion of executives had, at least some of the time, felt unable to raise an issue or concern with their boss, and that recruits into senior positions are pre-selected so as to assimilate into an organisation’s culture and not likely to ‘rock the boat’. This is no doubt especially true in civil service and other establishment roles.
One difference, I think, is that private sector senior managers are usually more realistic than Ministers, and much less subject to the political career and media pressures summarised later in this paper.
A second difference is that a private sector manager can usually get out from under an unreasonable boss and find a similar job in another company. But civil servants do not have a choice of an alternative career, unless they are, say, IT professionals. So key policy advisers can get locked into acting more like courtiers than professional advisers, and give up on 'speaking truth to power' - with foreseeable and depressing results.
Third. it is also often the case that Permanent Secretaries nowadays feel under great pressure to deliver their own department's policies and programmes, and not those of the government as a whole. They are often, for instance, reluctant to replace or move staff whom they, or their Minister, think able to deliver policies effectively. There are similar tensions in the private sector but corporate Boards and the CEO do determine key corporate strategies and are responsible for seeing their implementation company-wide. In contrast, in government, there is a very limited role for individual Ministers to determine key strategies once the manifesto has been published, and no responsibility for delivering policies and programmes other than those of their own department.
A fourth difference is that newly appointed Ministers will readily suspect officials of being out-of-touch with social and political priorities, over-sympathetic to the views of political opponents, naturally opposed to change, and so on. Cabinet and Permanent Secretaries famously got off to a bad start with Prime Ministers Margaret Thatcher and Tony Blair, and failed to establish an effective working relationship with Gordon Brown in either the Treasury or No.10. All three seem to have preferred to work with those who did not ‘push back’ too hard – ‘courtiers’ even. Follow this link to read a discussion of how 'speaking truth' may have become more difficult in recent years. (Opens in Understanding the Civil Service website.)
This may have led to another problem. Some argue that recent new administrations have been particularly vulnerable to error as a result of suspicious ministers meeting senior officials who are over anxious to please. Indeed, it is hard to read Anthony King’s The Blunders of our Governments or Richard Bacon and Christopher Hope’s Conundrum – Why Every Government Gets Things Wrong - let alone the Chilcot Report - without wondering whether very senior officials could not have done more to persuade their political masters and mistresses to take more sensible decisions. If not, then what were we employing them for?