|If someone is being "overpaid" here, it surely isn't workers|
First of all, the title of the post, "Are federal government workers overpaid?" is misleading. We usually say that someone is overpaid when they are contributing less than they are being compensated. But the CBO report is examining compensation differentials, which is a different issue. The problem is complicated by the fact that public sector workers are engaged in activities (such as education or security) which have clear positive externalities on society, so it's even more difficult to talk about being over or underpaid. (Are the participants in the securitization of bad loans over or underpaid?) It's clear what Mankiw is actually trying to accomplish in his post, and we at Anti-Mankiw can assure you that it has nothing to do with accurately conveying economic research.
Second of all, the report is only for workers in the federal government. In this excellent report by John Schmitt at the Center for Economic Policy Research it was found that state and local government employees are actually paid less than their private sector counterparts, after controlling for observable characteristics (similar to the methodology in the CBO report).
Finally, if a dynamic analysis were performed (i.e., examining the trend of wages in each sector over the last 30 to 40 years), readers would quickly realize that there are more pressing issues at play here. The most important of which may be the fact that wages for both public and private employees have been stagnating since the 1970s. Sure, public employees may have been paid more (according to the study from which the above graph is pulled, however, even that is a contested issue), but the story is more like: private sector wages are falling behind as public sector wages struggle to keep above water. Comparing with the elite 1% over the same 30, 40 year period, you see the drastic explosion of inequality that is the main subject of public debate today. In short, Mankiw is really doing a disservice to the more pressing macroeconomic issues by choosing to focus on whether one sector is more compensated than the other.
Here is another, similar report on this issue ("Public and Private Sector Workers Are in This Together") which focuses on college graduates: http://vox-nova.com/2011/03/17/public-and-private-sector-workers-are-in-this-together/.
Hardly a rosy picture -- for either public or private sector workers. Someone definitely seems to be overpaid here, but it's not workers!