June 2005 | EPI Book
Grading Places
What Do the Business Climate Rankings Really Tell Us?
by Peter Fisher
(about the author)
Jump to: Table of Contents | Executive Summary | Introduction
Table of contents
Executive Summary
Introduction
Chapter 1: The Small Business Survival Index
Are the measures that make up the SBSI appropriate?
What drives the SBSI?
How well does the SBSI predict outcomes?
Chapter 2: The State Business Tax Climate Index
What measures make up the SBTCI?
How well does the SBTCI measure business tax burdens?
Chapter 3: Beacon Hill’s Competitiveness Reports
What measures make up the Competitiveness Index?
Are the variables appropriate?
Does BHI’s index work?
Chapter 4: The Cato Institute’s Fiscal Policy Report Card
What measures make up the Report Card?
What drives the Fiscal Report Card?
How does the Fiscal Report Card perform?
Chapter 5: The Economic Freedom Index
What makes up the Economic Freedom Index?
How well does the Economic Freedom Index predict?
Chapter 6: Other competitiveness rankings
Economy.com’s cost of doing business indexes
Forbes‘ best places
Expansion Management’s Quotients
What can we learn from these indexes?
Chapter 7: Conclusion: Do business climate rankings serve a useful purpose?
Appendix A: The overall state rankings
Appendix B: The 50 largest metropolitan areas ranked
Appendix C: Performance rankings of cities
Appendix D: Creating an index
Endnotes
References
About EPI
Executive summary
In the United States today, at least eight groups produce rankings of states and cities that are widely cited by public officials and businesses to make the case for changing public policy to enhance a state’s prospects for growth. These rankings are based on each organization’s version of an “index” that purports to show which states or cities have the best “business climate,” or the most “competitive” tax and regulatory environment, or the conditions most conducive to small business growth and entrepreneurialism.
In this report we critique five major rankings that claim to measure the capacity or potential for economic growth. Are they based on science? Do they have biases? Do they in fact work as predictors of economic activity? The indices analyzed here vary widely in the factors that underlie them, but they have one thing in common: they claim that places with lower taxes and fewer government regulations are better. Many of the reports based on these rankings then draw explicit policy recommendations: cut taxes, shrink government, and reduce regulations, and your state or city will experience more business investment, more job creation, or more small business development. These policy recommendations are valid only if the index is a valid measure of the state or city’s growth climate. The indexes of competitiveness described and critiqued in detail in this report are:
- The “Small Business Survival Index,” produced annually by the Small Business and Entrepreneurship Council.
- The “State Business Tax Climate Index,” produced annually by the Tax Foundation.
- The “Metro Area and State Competitiveness Report,” produced annually by the Beacon Hill Institute.
- The “Fiscal Policy Report Card on America’s Governors,” prepared biennially by the Cato Institute.
- The “Economic Freedom Index,” first published by the Pacific Research Institute in 2004.
These five indexes produce widely different rankings of the states, despite the fact that all of the organizations creating these indexes assert that they are measuring something of critical importance to a state’s economic future and its potential for growth. Thirty-four of the 50 states can claim that they are in the top 10 in terms of business climate or competitiveness; they just have to pick which of the five indexes they want to point to. Business interests in just about any state can find at least one ranking to support an argument for cutting business taxes to make the state more competitive. In all but eight states, one can find at least one index that puts the state in the bottom half of all states.
The underlying problem with the five indexes is twofold: none of them actually do a very good job of measuring what it is they claim to measure, and they do not, for the most part, set out to measure the right things to begin with. The Small Business Survival Index is almost entirely about tax burdens on upper-income residents rather than about state programs or policies to assist entrepreneurship or small business growth. The State Business Tax Climate Index is a large and complex undertaking but ends up generating a number that has little relation to the actual taxes falling on new business investment in a state. The Beacon Hill Competitiveness Index is a mishmash of causal and performance variables that render it useless as an overall predictor of anything. The Cato Institute’s Fiscal Policy Report Card is little more than a rating of governors on their aggressiveness in promoting an agenda of limited government. And the Economic Freedom Index is a sometimes bizarre collection of policies and laws libertarians love, or love to hate, but few have any plausible connection to a state’s economic potential.
Do the businesses making investment and location decisions pay any attention to these state rankings? A look at the publications aimed at corporate location executives and site location consultants suggest that they do not. Current rankings conducted by business magazines tend to be much broader in scope. The two that are aimed at creating an index of growth potential or competitiveness look at the whole range of factors that are important to business and/or to employees, including labor costs, cultural and recreational amenities, climate, energy costs, transportation, educational attainment, school quality, and health care. Tax levels are part of the equation, but only a small part.
It is precisely because the competitiveness indexes produced by the ideological think tanks are aimed at promoting particular kinds of legislation that they do a poor job of predicting state economic growth: the measures used must pass an ideology screen, so the validity and relevance criteria go by the wayside. T
his is also why these rankings are ignored by the business people actually making the decisions. They should be ignored by policy makers for the same reason.
Introduction
Newspapers love rankings. Their readers have an apparently insatiable interest in how a particular state or city compares to others. Recognizing this, a number of advocacy think tanks have accommodated the press in recent years by creating and publicizing rankings that purport to show which states or cities have the best “business climate,” the most “competitive” tax and regulatory environment, or the conditions most conducive to small business growth and entrepreneurialism.
These rankings are based on each organization’s version of an “index” that is supposed to summarize the various factors that make a place more competitive or conducive to economic growth. The purpose of this report is to dissect these various indices to see what really drives them. Are they based on science? Do they have biases? Do they in fact work as predictors of economic activity?
Competitiveness rankings date back to the 1970s, when a so-called “economic war between the states” was heating up. In 1979, the accounting firm Grant Thornton began to produce an annual ranking of the states based on their business climates. In the ensuing decade, the “Grant Thornton Index” gained considerable publicity and was widely cited by public officials and businesses to make the case for changing public policy to enhance a state’s prospects for growth. But in 1986 it was harshly criticized in a report by the Corporation for Enterprise Development for its methodology and its bias in favor of certain kinds of policies (CFED 1986). In the face of this and other criticism, the index was abandoned a few years later.
It didn’t take long for the void to be filled: at least eight groups now produce rankings of states or cities on a regular basis. In this report we critique in detail five rankings that claim to measure the capacity or potential for economic growth. The indices analyzed here vary widely in the factors that underlie them, but they have one thing in common: they claim that places with lower taxes and fewer government regulations are better. The reports based on these rankings then draw explicit policy recommendations: cut taxes, shrink government, and reduce regulations, and your state or city will experience more business investment, more job creation, or more small business development.
These policy recommendations are valid only if the index is a valid measure of the state or city’s growth climate. That is the issue investigated in this report. For each index we ask a series of questions aimed at assessing the validity of its components and the way in which the components are combined.
The first question is: does the index include all of the relevant variables, and only relevant variables? For example, an index may purport to measure the capacity for growth, but are the major factors that research has shown to contribute to growth included in the index? Does the index include factors that are not plausibly related to growth? An index could be called “The Best State Economic Policy Index,” but if the ranking is determined by the number of letters in the state’s name, or other implausible factors, it will not be informative about which states have the best economic policies.
Just as bad, an index that purports to measure the climate for growth may include indicators of the state’s actual performance, such as new business starts or growth in per capita income. Creating a multidimensional measure of state economic performance may well be a useful thing, but including performance measures in a supposedly causal index, and then showing that the index predicts performance, is circular reasoning.
The second question that should be asked of an index is: do the causal variables in fact measure what they claim to measure? For example, a sub-index might be labeled “business tax burden.” This may be a legitimate thing to include in a causal index, but only if the business tax burden is measured appropriately.
The third question is: how does the index deal with the problem of combining disparate measures into a single index number? For example, if one believed the only important factors in economic growth were the state corporate income tax rate and state per-capita personal health care expenditures, how would one construct an index? To start with, corporate income tax rates are expressed as a percentage, with all rates under 10%, and health care expenditures range from approximately $3,000 to $7,000. If these were just added together for each state the index would really only measure the health care expenditures because the numbers and range are so much greater than for the corporate income tax. The index components should be converted to a similar scale before they are combined.
Combining disparate measures also entails explicit or implicit weighting. Even if the corporate income tax and health expenditures are scaled so that one does not dominate the other in the index, the question remains as to whether one is more important than the other as a cause of economic growth. An index might weight components according to their assumed importance, and one sure sign of an index that isn’t serious is that it weights all its components the same. We know that every factor is not of equal importance in causing economic growth, and a failure to appropriately weight factors is a sign of a failed index. (Appendix D contains a more complete discussion of the issues involved in combining factors to create an index.)
Finally, does the index do a good job of predicting why some states or cities grew more rapidly than others over some time period? In other words, an index can be put to the test, and this study does just that. It uses simple statistical models to evaluate whether there is a connection between competitiveness and business climate index rankings and actual economic performance.
These questions raise a broader one: is there a “right way” to measure what these indexes purport to measure? Are these indexes legitimate tools? Is there a science of evaluating competitiveness and business climate? There is indeed a science—it is called the statistical analysis of factors contributing to state or metro area growth. A large body of scholarly research has focused on this question, and the methodology used is generally some form of multiple regression analysis. The explanatory variables in these models are like the individual measures that go into the making of an index.
The key difference between an index and a statistical model is that in a statistical analysis the variables are not weighted arbitrarily, as they are in an index. Instead, the weights are produced by the statistical tools used in the analysis. Each weight (or regression coefficient) tells us how that variable contributes to explaining the differences among states in terms of economic growth. For many variables, the contribution turns out to be small or nonexistent (“statistically insignificant”).
It still might be the case that a given index, while not scientifically constructed, in fact does a reasonable job of including and measuring appropriate variables, excluding inappropriate ones, and weighting them in a sensible fashion. To a significant degree, the legitimacy of an index depends on how well it mimics a more sophisticated statistical approach.
In addition to their lack of statistical underpinnings, there is another reason to question the types of indexes examined here. It is not clear that the conc
ept of business climate or competitiveness for an entire state or metro area makes sense to begin with. Charles Skoro has argued that “the usefulness of the business climate concept depends on the existence of a set of indicators that are measurable, that have substantial effects on business outcomes, and that are truly generic—they influence business activity in a more or less uniform manner regardless of industry, region, or time period” (Skoro 1988). Others have made similar arguments: that the factors important to location and expansion decisions are industry specific, and that the conditions conducive to growth can vary tremendously within a state (and that metropolitan regions, not states, are the focus of business decisions).1 For manufacturing in particular, the crucial factors may in fact be project specific, since access to suppliers and markets will often be key.
So why even bother with an index? Why not just rely on scholarly research to address the policy issues? For example, a recent study by Robert Lynch (2004) reviewed the large body of research on the effects of taxes on growth, and concluded that the effects are small or nonexistent. Most research in this area has found other factors to be more important determinants of business location and investment decisions, namely quality of public services in general and education in particular, utility costs, access to markets, transportation infrastructure, the education level of the labor force, and wage rates. The reason for creating an index, perhaps, is that index numbers, and rankings based on them, are simpler, require little in the way of analytical expertise, and are easier to write about in the popular press.
Five indexes of competitiveness are described and critiqued in some detail in this report. We also review briefly three other rankings or indexes of competitiveness for which there is less information available. Other rankings exist that differ from those reviewed here in that they are explicitly intended to be measures of economic performance or outcomes rather than business climate or competitiveness. Some of these are widely publicized, and we describe these briefly in an appendix to make it clear how they differ from the allegedly causal indexes that are the focus of this report.
In a concluding chapter, we summarize the common themes and methodological problems that characterize the indexes reviewed and present some thoughts on the use and misuse of business climate indexes as a guide to public policy.
About the Author
Peter Fisher is a professor in the Graduate Program in Urban and Regional Planning at the University of Iowa, where he has taught since 1977. His research and teaching interests are centered on state and local government finance, economic development policy, and poverty and income inequality. He has served as a consultant on tax issues, welfare reform, and economic development policy for state government agencies, labor unions, and nonprofit organizations. More recently, he has become part-time research director for the Iowa Policy Project, a state-level non-profit public policy research organization focusing on the Iowa economy and state budget and tax policy. He is the co-author, with Alan Peters, of Industrial Incentives: Competition Among American States and Cities (1998), and State Enterprise Zone Programs: Have they Worked? (2002), both published by the W.E. Upjohn Institute for Employment Research. He has also published a number of journal articles on tax incentives and economic development policy. He received a B.A. from Haverford College in 1968 and a Ph.D. in economics from the University of Wisconsin – Madison in 1978.
