Are Published Compensation Surveys Accurate?

Experiences with two recent clients have raised some red flags about using typical published surveys to evaluate executive compensation.

Both of these clients had previously hired established, well-known consulting firms to evaluate the compensation levels of their senior executives.  Both consulting firms had carefully matched job descriptions to reputable survey data bases and both had used multiple survey sources to support their findings. 

But actual data from IRS 990 forms showed that in both cases, the consultants’  findings were wrong.  Both consultants significantly underestimated competitive market values, especially for top executives. 

Why?  It turns out that the organizations participating in the broad surveys the consultants used were not representative of the organizations with which my clients are in direct competition.  In one case a survey peer group contained 16 organizations with the same type of services as my client and in the same broad geographic region.  But all but one were in lower cost of living areas with salary levels substantially less than a dozen comparable organizations in my client’s immediate market area.

A consultant’s recommendation for another of my clients, an independent nonprofit hospital, was based largely on a compensation survey in which the vast majority of participants are subsidiaries of organizations like Kaiser Permanente and Catholic Healthcare.  Data on the CEO and CFO and other key positions was representative of these positions operating within a larger system, but significantly under the typical compensation levels for leaders of fully independent organizations.

The standard surveys published by firms such as Hay, Hewitt, Towers Perrin, and my former firm – Watson Wyatt – clearly have value.  But if you are going to use them, you should take the following steps:

1. For top positions, check the survey data against the IRS 990 form data for directly comparable organizations.  The 990 data is a bit out of date – but you can update it by 4 or so per year % (maybe 2% for 2009)  and usually get a dependable reading.
2. Obtain a listing of all the organizations in the published survey peer groups against which your data is being compared, to ensure that the group is representative of your organization in terms of mission, structure, size, and geography. 
3. Develop customized peer groups from the participant listings in the published surveys – most of the largest and best known surveys now allow you to do this.  
4. Make sure you understand any assumptions your consultant is using to adjust data from the published surveys.  Common adjustments include aging data to make it current, increasing or decreasing the market values reported for positions that are presumed to be stronger or weaker than positions in your organization, adjusting for regional salary differentials, and weighting data to give more emphasis to surveys the consultant believes are a best match.  These can all be reasonable and useful adjustments, but you should understand them and the effect they have on the conclusions reached.