ARTICLE IN PRESS
Socio-Economic Planning Sciences 38 (2004) 233–253
DEA evaluations of long- and short-run efficiencies of digital vs. physical product ‘‘dot com’’ companies Anitesh Baruaa, P.L. Brocketta, W.W. Coopera,*, Honghui Dengc, Barnett R. Parkerb, T.W. Rueflia, A. Whinstona a
The Red McCombs School of Business, University of Texas at Austin, Austin, TX 78712-1174, USA b Pfeiffer University, Misenheimer, NC 28109, USA c University of Nevada at Las Vegas, Las Vegas, Nevada 39154-6034, USA
Abstract This paper applies Data Envelopment Analysis to determine relative efficiencies between internet dot com companies that produce only physical products and those that produce only digital products. To allow for the fact that the latter are relatively inexperienced, a distinction is made between long- and short-run efficiencies and inefficiencies, with a finding of no statistically significant difference in the short run but digital product companies are significantly more efficient in the long run. A new way of distinguishing between long- and short-run performances is utilized that avoids the need for identifying the time periods associated with long-run vs. short-run efficiencies and inefficiencies. In place of ‘‘time,’’ this paper utilizes differences in the ‘‘properties’’ that economic theory associates with long- and short-run performances. r 2003 Elsevier Ltd. All rights reserved. Keywords: Internet; Long-run efficiency; Short-run efficiency; Data Envelopment Analysis
1. Introduction Barua et al. [1] initiated a new direction of research into the assessment of Information Technology (IT) productivity by investigating the relative performances of ‘‘digital’’ vs. ‘‘physical’’ ‘‘dot com’’ companies. The distinction between these two types of business is detailed
*Corresponding author. Graduate School of Business, The Foster Parker Professor of Finance and Management, The University of Texas at Austin, Austin, TX 78712-1175, USA. Tel.: +1-512-471-3322; fax: +1-512-471-0587. E-mail addresses:
[email protected] (A. Barua),
[email protected] (P.L. Brockett), cooperw@mail. utexas.edu (W.W. Cooper),
[email protected] (H. Deng),
[email protected] (B.R. Parker) tim.ruefli @bus.utexas.edu (T.W. Ruefli),
[email protected] (A. Whinston). 0038-0121/$ - see front matter r 2003 Elsevier Ltd. All rights reserved. doi:10.1016/j.seps.2003.10.002
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
234
in the following quote from [1]: Digital dot coms are Internet based companies such as Yahoo, Ebay and America Online, whose products and services are digital in nature, and which are delivered directly over the Internet. The physical dot coms are also based entirely on the Internet, but sell physical products (e.g., books, CDs, jewelry, toys) that are shipped to consumers. They are often referred to as electronic retailers (e-tailers) by the business press. Does IT have the same impact on these two categories of dot coms? We hypothesize that IT investments contribute more to various output measures (e.g., revenue, revenue per employee, gross margin and gross margin per employee) for digital dot coms than for physical dot coms. The rationale is that the level of digitization of business processes is fundamentally higher in digital products companies than in Internet based firms selling physical goods. While the Internet and electronic commerce applications are equally accessible to both types of companies, electronic retailers of physical products often build warehouses, handle inventory, and are subject to many of the physical constraints of bricks-andmortar companies. By contrast, due to the very nature of their business, most of the processes and delivery mechanisms of digital dot coms are implemented online. Further, the ability of a digital dot com to differentiate itself from its competitors directly depends on being able to translate innovative business strategies into online capabilities. An underlying theme in [1] is that failure to distinguish these two types of dot com companies is a source of some well known difficulties and paradoxes1 in research seeking to identify increases in productivity with increases in the use of IT. Employing a distinction between digital and physical product users of IT, Barua et al. [1] generate four different regressions for each of these two classes of companies. The regressions are of the form m Y xai i ; ð1Þ q ¼ A elt i¼1
where q is the level of the output—specified differently in each of the regressions—while xi is the level of input i ¼ 1; y; m; which may also differ in the regressions. A is a constant to be estimated, and ai is the output elasticity of input i: The exponential elt ; with t representing time in years of existence of each company, is examined in more detail later in this paper. Here, we only note that it is intended to allow for the fact that many digital product companies are relatively new and, hence, may not yet have achieved their full capabilities. Among the regression inputs, [1] distinguishes between investments in information technology, which they symbolize as IT (information technology investments), and other investments, denoted by NIT (non-information technology investments). The analysis in [1] yielded the following results: (1) IT capital investments were found to have a significantly positive effect on output (¼ q) for ‘‘digital,’’ but not for ‘‘physical,’’ dot coms. (2) Physical dot coms were found to do better by investing in the labor component of NIT, whereas digital dot coms did better by investing in IT. We can gain additional insight into the issues raised in [1] by employing Data Envelopment Analysis (DEA) as an alternative approach to evaluating the relative efficiencies of these two types 1
See the often-cited remarks of the Nobel Laureate in economics, Solow [2]: ‘‘You can see the computer age everywhere but in the productivity statistics.’’ See also [3–5] where the notion of the ‘‘productivity paradox’’ is developed. However, it is not our purpose to explore these paradoxes, which involves a very large and growing literature. See, e.g., the articles and accompanying exchanges appearing in [6].
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
235
of firms. In the process, we can also perform a cross check by determining whether our results are consistent (or in conflict) with the results reported in [1]. Such cross checking by using different methodologies, such as statistical regressions and DEA, can help to reduce the possibility of ‘‘methodological bias’’ that entered so prominently into the exchange between Charnes et al. [7] and Evans and Heckman [8]. We also go further and introduce a new way of empirically utilizing concepts from comparative statics in economics to distinguish between short- and long-run behavior. By turning to properties associated with these two states, we also avoid the need for using statistical time series analyses covering ‘‘sufficiently long’’ periods. Our short-run results are relatively inconclusive in a manner consistent with results obtained from the relatively short periods covered by data for the time series used in some of the studies described in the next section. In the long run, however, when technical inefficiencies are removed, our results favor dot com over physical-product companies in a manner that is highly significant statistically. 2. Long- and short-run effects The time dependent relation between output and inputs in (1) is not a time series analysis in the usual sense. It may be regarded rather as a ‘‘way station’’ en route to the more fundamental distinctions between the long- and short-run efficiencies that are of concern to us. The use of time in (1) is, however, consistent with the time needed to treat the ‘‘productivity paradox’’—also referred to as the ‘‘Solow paradox,’’ as in footnote 1—by arguing that this paradox arises from inadequate allowance of the time needed for the effects of IT investments to be manifested. To quote from Wang et al. [9, p. 203] ‘‘ythe results (from IT investments) may be misleading (as reported in contemporary research) because of the assumption that there are no time lags between IT investments and performancey.2’’. In order to bypass specifying the time period needed to distinguish between long- and short-run periods, Wang et al. [9] follow a suggestion of Kauffman and Weill [10], and treat IT output as an ‘‘intermediate variable’’ in the sense of Koopmans [11]. In the process, they add to previously available ways of treating intermediate products—such as those to be found in the earlier work of Charnes et al. [12], that dealt with US Army advertising for recruitment in a two-phase manner as follows: Phase one treated advertising expenditures as an input with ‘‘advertising awareness’’ (e.g., awareness of military career possibilities) as an output. Phase two then treated these ‘‘ad awares’’ as an input, which, together with other inputs, was used to obtain outputs such as various classes of recruits. Inter alia, this made it possible to identify and evaluate various types of inefficiencies in the Phase one production of ‘‘ad awares’’ as well as in the uses of these outputs in Phase two. However, no attempt was made to distinguish long-run from short-run effects in either phase. As already noted, Wang et al. [9] did not explicitly treat time dimensions in their models. This is done, however, by Shafer and Byrd [13] who, for this purpose, ‘‘yuse the average level of the inputs (for each DMU)3 over a 3-year period andy (then use) the annual compound growth rate 2 3
We are indebted to a referee for this, and the other references, we discuss in this section. DMU=Decision Making Unit defined as the entity responsible for converting inputs into outputs.
ARTICLE IN PRESS 236
A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
for the output over a 5-year period.’’ However, they did not explicitly address the topic of longrun vs. short-run effects from these IT investments in ways that are of interest to us. Although Kao [14] does not address the productivity paradox, he is concerned with treatments of long- and short-run performances using DEA. In his opening remarks, Kao points out that prior DEA treatments fail to say ‘‘yhow long is required for the ideal performance to be accomplished. In economic terminology, this means that long- and short-run effects need to be addressed’’. For his purposes, Kao utilizes a multi-plant formulation for each DMU. He then uses these models to evaluate the various districts (each with several sub-districts) that are responsible for managing the national forests in Taiwan (The Republic of China). The short-run efficiency frontier is then identified with the efficient boundary generated from the sub-districts. The longrun efficiency frontier is obtained by applying DEA to the districts in which the sub-districts are located. Using this approach, Kao is able to evaluate the long- and short-run efficiencies of both the sub-districts and districts. However, he does not discuss the properties of the thus identified short- and long-run frontiers with their accompanying economic rationales other than to note [14, p. 383] that the resulting ‘‘long run frontier is super-imposed upon all of the short-run production frontiers’’.
3. Our approach Our approach differs from those reviewed above in that we focus on properties associated with long- and short-run efficiencies that are of interest in economics and provide accompanying economic rationales. The two properties on which we focus are: (i) All ‘‘technical inefficiencies’’ (=waste) that may exist in short-run performances are eliminated in the longrun, and (ii) the long-run frontier is at least as efficient, technically, as the short-run frontier at every point used for an evaluation. Both properties are needed to satisfy a long-run equilibrium that is consistent with assumed economic behavior such as profit maximization or cost minimization. Our application involves two different types of DMUs (here, business firms) according to whether they are digital- or physical-product dot coms. The DMUs in each category are identified as operating in the short run with two different technologies, one for digital-product and the other for physical-product companies. These two sets of DMUs are each evaluated by reference to their respective frontiers. Using formulas identified below, the technical inefficiencies of the DMUs are eliminated by projecting the observed inputs and outputs for each DMU onto the two different frontiers. See the next section of this paper for the mathematical meaning of such a ‘‘projection.’’ The thus modified values for each DMU then satisfy condition (i). To satisfy condition (ii), described in the first paragraph of this section, we generate a new frontier which is always at least as efficient as the two underlying frontiers. The generated frontier can then be regarded as a long-run frontier that can be used to evaluate the two underlying shortrun frontiers. This is accomplished by applying DEA to the already modified DMUs in order to obtain an efficiency measure for each of the short-run frontiers. The results of these DEA evaluations are then ranked and the Mann–Whitney rank order statistic used to examine which of the two sets of DMUs outranks the other.
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
237
We can thus determine which set of DMUs is likely to displace the other in the long run, e.g., in a manner analogous to the way manufacturing replaced agriculture in the first half of the 20th century, the way service replaced manufacturing in the second half, and the way the knowledge industry is now beginning to replace the others (See [15]).
4. Proposed DEA model Drawing from the DEA literature, we employ a ‘‘BCC model,’’ [16], to which we accord the ‘‘output oriented’’ objective in (2): P Pm Max fo þ e sr¼1 sþ r þ i¼1 si Pn s:t: fo yro ¼ j¼1 yrj lj sþ r ¼ 1; y; s; r ; Pn xio ¼ j¼1 xij lj þ si ; i ¼ 1; y; m; ð2Þ Pn 1 ¼ j¼1 lj ; lj ; sþ r ; si X0
8i; j; r:
Here, the yrj and xij refer to observed values of r ¼ 1; y; s outputs and i ¼ 1; y; m inputs, respectively, for each of j ¼ 1; y; n DMUs. These DMUs take the form of business firms but, more generally, the term may refer to any entity that is regarded as responsible for converting inputs into outputs. The objective in (2) seeks to maximize the outputs; hence, the name ‘‘output oriented.’’ (See [17] for a discussion of the different types of DEA models.) As can be seen in (2) the optimization of fo is effected in a manner that does not change the proportions of the outputs in the constraints associated with this variable (a scalar). This focuses attention on what is referred to as ‘‘technical efficiency’’ in economics.4 We choose this focus, in part, because technical efficiency is fundamental to achieving any other type of efficiency. Note that the objective in (2) multiplies the slacks by e > 0; which is a ‘‘non-Archimedean element’’ defined to be smaller than any positive real number. (See [18] for a fuller development of this concept as taken from ‘‘non-standard mathematics.’’) There need be no concern about assigning a value to this element since most of the available DEA computer codes treat e > 0 in a two-stage manner as follows: Stage 1 identifies max fo ¼ fo without reference to any slack values. Stage 2 sets fo ¼ fo in the constraints of (2) and then maximizes the sum of the slacks. This arranges matters so that no reduction, however small, can occur in the value of f ; in exchange for an increase in the slacks, however large. The result is an ordering in which the maximal value of fo is given preemptive status. 4.1. Definitions Most uses of DEA focus on the amounts of inefficiency in each input and each output for every DMU. This is accomplished by successively positioning the terms for each 4
This is but one type of efficiency. We do not extend our analysis to other types, such as returns-to-scale or returnsto-scope efficiency, or even to the simpler ‘‘allocative efficiency’’ (where unit prices and costs are needed).
ARTICLE IN PRESS 238
A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
DMUj on the left in the constraints of (2)—when setting DMUj=DMUO, j ¼ 1; y; n—while also leaving the terms for this same DMUj on the right-hand side of (2). In other words, DMUO is one of the j ¼ 1; y; n DMUs singled out for evaluation. Hence, a solution can P always be achieved by setting lj ¼ lo ¼ fo ¼ 1 with all other lj ¼ 0: Moreover, the condition nj¼1 lj ¼ 1 limits the solutions to f Z1 values that are finite. Hence, we can be sure that the result described in this and the following sections are not vacuous. Proceeding in this manner for the DMUs to be evaluated, one can identify the sources and amounts of inefficiency for each DMUj from a solution to (2). We write this solution as follows: n X ¼ yrj lj Xyro ; r ¼ 1; y; s; y# ro ¼ fo yro þ sþ r j¼1
¼ x# io ¼ xio s i
n X
xij lj pxio ;
i ¼ 1; y; m;
ð3Þ
j¼1
where ‘‘’’ refers to an optimum value. From this solution, we can identify the amount of inefficiency in each output and input via Dyro ¼ y# ro yro X0; r ¼ 1; y; s; Dxio ¼ xio x# io X0; i ¼ 1; y; m ð4Þ with Dxio > 0 representing an excess in input i ¼ 1; y; m and Dyro > 0 representing a shortfall in output r ¼ 1; y; s for each DMUj=DMUO. We also use (3) to effect the projections to be described later in this paper. Here we note that an application of (2) to observed data provides access to (3). This allows us to identify technical inefficiencies in each input and each output for every DMUj, j ¼ 1; y; n; in a manner that is consistent with the following (purely verbal) definition of efficiency. Definition 1. Full efficiency is attained by DMUj=DMUO, the DMU to be evaluated in (2), if and only if it is not possible to improve any input or output without worsening some other input or output for this DMUO. Referred to as the Pareto–Koopmans definition of efficiency this definition is an adaptation of a concept in welfare economics first formulated in the 19th century by Pareto [19] to avoid the need for interpersonal comparisons of subjective utilities. Adapted for use in production economics by Koopmans [11], it was further adapted by Farrell [20] who also applied it in empirical studies. Initially intended to correct uses and abuses of productivity indexes, the work in [20] extended these ideas first to the concept of ‘‘productive efficiency’’ and then to the even more general concept of ‘‘efficiency’’—extensions which have guided subsequent work that now goes under the name ‘‘Data Envelopment Analysis’’. We can give Definition 1 mathematical form as follows: Definition 2. Full (DEA) efficiency is attained by DMUO if and only if Dyro ¼ Dxio ¼ 0 8r; i in (4). Correspondence with (4) is obtained from Definition 2 by referring to (3) and noting that any Dxio > 0 or any Dyro > 0 provides an opportunity for improvement in the associated input or
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
239
output, that can be realized without worsening any other input or output. In economic terms, the elimination of such inefficiencies amounts to the elimination of waste, since such improvements can be effected without any resource cost (in the form of input increases) or sacrifice of benefits (in the form of output reductions). We now reformulate Definition 2 to make it amenable for use with (2) as follows: Definition 3. DMUO is fully (DEA) efficient if and only if the following conditions are both fulfilled (i) fo ¼ 1; and (ii) All slacks are zero. As already noted, the positioning of DMUO on both the left- and right-hand sides of (2) means that we can always get a solution with fo ¼ 1: Hence, we must have fo Z1: If fo > 1; it is possible or sþ to improve all of DMUO’s outputs without worsening any input. Moreover, nonzero s i r show where additional improvement is possible since any such slack can be removed by reducing the input or increasing the output of DMUO in the constraint with which it is associated without affecting any other constraints, see (3). In this sense, our evaluations avoid any need for recourse to subjective values or relative weights since each such improvement can be effected without worsening any other input or output. In the DEA literature, fo is sometimes referred to as a measure of ‘‘Farrell efficiency.’’ It is also referred to as a measure of ‘‘technical efficiency,’’ in conformance with economic usage. However, this usage is accompanied by an assumption referred to as ‘‘strong (or free) disposal,’’ meaning that non-zero slacks may be disposed of at no cost. This is equivalent to replacing e > 0 with e ¼ 0 in the objective of (2). Finally, fo is referred to as a measure of ‘‘weak efficiency’’ in the OR/MS literature because it does not include the additional inefficiencies associated with the non-zero slacks specified in Definition 3. Free disposal and weak efficiency both assume that the inputs or outputs associated with non-zero slacks have no value. This is not true for the formulation we use in (2). The two-stage solution procedure described for (2) in the preceding section maximizes the sum of the slacks after the value of fo has been determined. We can therefore be certain that possibilities for identifying non-zero slacks are included as required in Definition 3 along with the maximal value of fo : Further, all of our projections will be onto the efficiency frontier. Hence, the conditions for full (DEA) efficiency as given in Definitions 2 and 3 are satisfied by our projections. It is important to note that we use fo as the measure of efficiency, in part, because it more closely corresponds with the use of fixed weights in indexes of productivity. This is in contrast to the weights obtained from the dual to (2), which can vary from one DMUO to another as each of the DMUj ¼ DMUO ; j ¼ 1; y; n is positioned for evaluation in (2).
5. Choice of variables and data As noted above, our interest is in fo as a measure of efficiency and, in the appendix, we check our results by reference to a more comprehensive measure that includes all non-zero slacks. We therefore do not pursue this topic further in the text of this article. We turn, instead, to the data for the inputs and outputs employed herein.
ARTICLE IN PRESS 240
A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
Table 1 Variables employeda Outputs: 1. Sales=Gross sales, in 1000’s of dollars, net of returns and allowances. 2. Gross margins=Dollar value of difference between sales and cost of goods sold. Inputs: 1. IT capital=Dollar value of investments in computer hardware, software and network equipment. 2. NIT capital=Dollar value of non-IT capital. 3. Labor=Number of full-time equivalent employees. 4. Number of years in business. (This is the t in expression (1).) a
Source: [1].
These data are the same as in [1] and were obtained from Hoover’s Online, Inc. (http:// www.hoovers.com). Since only data for 1998 and 1999 were used in [1], we also restrict our analysis to these years. Further, only publicly traded companies that generate all their sales online were selected from the set in [1]. This yielded a list of approximately 300 companies for which data from Securities & Exchange Commission (SEC) filings were collected. Reductions in the number of these companies were then effected in [1] for the following reasons: (1) The financial reports of some DMUs suggested that they did not generate all their revenues online, or (2) critical data, such as IT capital were not available for the omitted DMUs. As reported in [1], these efforts left a sample of 199 companies of which 154 are digital, and 45 are physical-product. Moreover, the operations of these two sets of companies are such that no company deals with both digital and physical products at the same time. The variables employed in our study consist of the two outputs and four inputs recorded in Table 1. Our DEA evaluations are all effected in comparative fashion within each year. Hence, it is not necessary to make adjustments for price level changes. Finally, we do not follow [1] into examining combinations of the above inputs and outputs such as sales per employee. Instead, we confine our attention to the variables listed in Table 1.
6. Mann–Whitney rank order statistic Our tests are directed to the following set of hypotheses: Ho. The two types of companies are equally efficient. HA. One type of company is less efficient than the other. The approach we use to test these hypotheses will utilize relative rankings for the individual companies. We then use the Mann–Whitney rank order statistic to test for statistically significant differences. We use Mann–Whitney since, as noted in [21], it is non-parametric and, hence, is
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
241
suited to the non-parametric form of our DEA analysis. Further, it does not require equal sample sizes. For detailed discussions of this statistic and its properties see [22, p. 314 ff]. Following is a brief review of the manner in which the Mann–Whitney statistic is determined in our study: 1. Rank the DMUs by relative efficiency using their f values as determined from (2). To preserve o
correspondence with the number of firms to be ranked, all ties are assigned the mid-rank value. See [22] for a discussion of this use of mid-rank values. 2. Compute the sum of all rankings for the digital product company types. This value is labeled R: 3. Use R to compute the Mann–Whitney rank statistic: n1 ðn1 þ 1Þ R; U ¼ n1 n2 þ 2 where n1 and n2 are the number of observations in digital and physical product companies, respectively, and n ¼ n1 þ n2 : 4. Compute U ðn1 n2 =2Þ Ztest ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi; n1 n2 ðn1 þ n2 þ 1Þ=12 which is approximately normally distributed for sufficiently large values of n1 and n2 : As Ztest is approximately normal for n1 ; n2 X10; we accept Ho at the a significance level if Za=2 pZtest XZa=2 ; where Za is the ath centile of the standard normal probability distribution. Alternatively, we reject Ho, in favor of HA, if Ztest o Za=2 or if Ztest > Za=2 : We apply the two-sided symmetric test Za=2 pZtest pZa=2 and use the number of digital product companies to compute the R value. For reasons that will subsequently become apparent, we conduct our tests with t (=number of years in business) present and absent. The Mann–Whitney results are exhibited in Table 2. The Z scores in the far right column show that the null hypotheses cannot be rejected at any of the customary significance levels. Indeed, the negative values favor the hypothesis that digital product companies are less efficient. Only when t
Table 2 Mann–Whitney values with unadjusted data Year
n1
n2
R
U
Z
1998 1998a 1999 1999a
115 115 102 102
35 35 28 28
8861 8962 6762 6502
1834 1733 1347 1607
0.790139138 1.241941156 0.45873396 1.013745419
Note: The number of DMUs was reduced from the 154 digital and 45 physical product companies, as described in Section 3, because the computer code we used could not treat the negative gross margins in some of these companies. However, a part of the appendix deals with this problem by using a different DEA model. This approach achieves results that are consistent with results reported in the text of the article. n1 —number of digital product companies. n2 —number of physical product companies. a t (=years in business), which is input 4 in Table 1, is omitted.
ARTICLE IN PRESS 242
A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
is omitted as an input in 1999 is the Z value positive; but, this is offset by the more negative value of Z in row 2. In neither case, however, is a significance of even 10% achieved (which requires only that Za=2 ¼ 1:645). These results motivated us to extend the analysis, as outlined in the next section, to obtain distinctions between long- and short-run efficiencies with properties typically assumed in economics.
7. Efficiency-adjusted data The exponential term, elt ; in expression (1) is described in [1] as allowing for maturity, especially in newly formed firms. There is trouble, however, in using such estimates. All the estimated l values are positive, thus generating output estimates that are augmented in exponentially increasing amounts with increasing t: These augmentations may be excessive when applied to fully efficient entities, and may be deficient (in reaching the efficiency frontier) when applied to very poor performers. We therefore omit t as a variable from explicit consideration and turn to an alternative approach along the lines described in Section 2 of this paper. To begin, we note that the analysis in Section 5 effected its evaluations of all DMUs (digital and physical products) relative to a common ‘‘efficiency frontier.’’ We now alter the analysis in a two step manner as follows: In Step 1 we erect two efficiency frontiers—one for digital, and one for physical product companies. Each such efficiency frontier can be regarded as providing a boundary for what the associated technologies make possible. The inefficiencies for each of these two types of entities can be estimated by applying (4) separately to the two types of companies. Using these same formulas— or, equivalently, using (3)—we then eliminate the inefficiencies identified in (3) in a manner that geometrically projects the DMUs associated with the observations onto their respective frontiers. In step 2 we erect a new (common) frontier against which the thus adjusted observations can again be evaluated by (4). In this manner, we evaluate only points that show each DMU to be operating as efficiently as its own boundary will allow. This enables us to evaluate the underlying capabilities of these two types of companies relative to each other when they are operated as efficiently as their two technologies allow. 7.1. Common frontier Fig. 1 helps to illustrate the first of the two-step procedure described above. We have simplified matters by confining attention to the case of a single output produced from a single input used by each of DMUs A,y,G. The model uses DMUs A, B, C, and D to generate the efficiency frontier from which DMU E, say, can be evaluated. If an output-oriented model, such as (2), is employed, the output shortfall designated by Dy > 0 in (4) represents the inefficiency that could be eliminated without requiring any increase in input. See the first expression in (3). Conversely, if an input-oriented version of this BCC model is employed, then Dx > 0 would represent an inefficiency in the form of an input amount that could be eliminated without requiring any reduction in output. In either case, the projection is onto a point on the frontier, such as the ones emanating from E that are located at the heads of the arrows in Fig. 1.
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
243
y 8
DMU D DMU C
7
OUTPUT
6
DMU G
∆y
DMU B
5 4 3
DMU E
∆x
2 DMU A
1
DMU F
0
x 0
2
4
6
8
10
INPUT
Fig. 1. Data Envelopment Analysis (one input and one output).
To see that points A, B, C, D and the lines connecting them are on the efficiency frontier, we need only observe that no further output improvement can be effected while staying within the production possibility set without increasing (hence, worsening) the input required; and, vice versa, the same condition is confronted for the input at such a point since any attempt at input reduction is necessarily accompanied by an output (worsening) reduction. See Definition 1 in Section 2. Thus, Dy > 0 and Dx > 0 show the presence of technical inefficiencies in either case, i.e., irrespective of whether an output- or input-oriented version of (2) is used to effect these evaluations. 7.2. Separate frontiers Fig. 2 helps to clarify some of the problems to be dealt with in developing the distinction between long- and short-run frontiers. The solid lines depict portions of the efficiency frontiers for two different types of DMUs (e.g., business firms), which we characterize as ‘‘Type 1’’ and ‘‘Type 2,’’ respectively. The Type 1 frontier is used to evaluate the points ‘‘m’’ while Type 2 is used to evaluate points ‘‘’’. This use of two frontiers brings new possibilities and problems into view. For instance, C evidently outperforms D. However, the latter is closer to its own efficiency frontier. Thus, D has a lower value of fo and therefore receives a higher efficiency rating when each of C and D are evaluated relative to their own frontiers. Evidently, our use of two frontiers, one for each type of DMU, reveals a potential for improvement that might not be apparent if both types of firms were evaluated relative to a common frontier—e.g., in the manner used to arrive at the results exhibited in Table 2. Conversely, use of a common frontier can reveal improvement possibilities that are not available from the pertinent individual frontiers. 7.3. Long- and short-run frontiers The common frontier we used in Section 6 was generated from the original observations for all DMUs. We now generate the common frontier in a different manner. First, we evaluate the points
ARTICLE IN PRESS 244
A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253 Output B
Type 2
Type 1 A
C D
Type 1
Type 2
Legend = Type 1 DMUs • = Type 2 DMUs
input
Fig. 2. Long- and short-run efficiency frontiers.
for the physical and the digital product companies relative to their different efficiency frontiers. Next, we project all DMUs onto their respective efficiency frontiers in the manner described for the one efficiency frontier in Fig. 1. Finally, we use the thus adjusted points to generate a new common frontier that we refer to as the ‘‘long-run’’ efficiency frontier because it has the properties noted in Section 2—viz., (i) all short-run technical inefficiencies have been eliminated, and (ii) the new frontier dominates the short-run frontier from which it was generated. In the process of making projections, C and D in Fig. 2 are projected onto their respective efficiency frontiers to eliminate their ‘‘short-run’’ technical inefficiencies. The thus adjusted points are then used to generate a new ‘‘long-run’’ efficiency frontier that is constructed from the points on these two efficiency frontiers. As can be seen in Fig. 2, the frontier for Type 1 firms forms a segment of this common frontier that dominates until A is reached. A new segment represented by the broken line is then generated that strictly dominates both of the short-run efficiency frontiers. Hence, this common (long-run) frontier always dominates the short-run efficiency frontiers from which it was generated. This approach makes it possible to decompose the technical inefficiencies into two components. One represents the short-run inefficiencies that are evaluated from each of the short-run frontiers. The long-run inefficiency component is then obtained by using the common frontier to evaluate the points located on these short-run frontiers. This puts us in a position to evaluate long-run as well as short-run potentials for improvement. 7.4. Mathematical development and results To implement these procedures, we utilize (3) to replace the original yro and xio with new coordinates y# ro and x# io so that Dy ¼ Dx ¼ 0 as described in (4). We then apply the model in (2) to all y# ro ; x# io values in order to obtain the desired long-run efficiencies. This use of (2) also produces the values of fo which we use for the rankings. Finally, we apply the Mann–Whitney rank order statistic to obtain the results in Table 3 where, as can be seen, we are able to reject Ho in favor of HA at very high levels of statistical significance.
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
245
Table 3 Mann–Whitney values for efficiency adjusted dataa Year
n1
n2
R
U
Z
1998 1999
115 102
35 28
7015 5761
3680 2348
7.409434268 5.21031165
a
Data for the input variable t have been omitted.
Table 4 Results of Mann–Whitney rank order statisticsa Year
n1
n2
R
U
Z
1998 1999
35 28
115 102
4310 2754
345 508
7.409434268 5.21031165
a Ranks of physical product companies are used to compute the R-values. Hence the definitions of n1 and n2 are reversed from Table 3. See instruction 3 for the Mann–Whitney statistic in Section 6.
In this application, we use the sum of the ranks of the digital product companies to compute the value of R for the Mann–Whitney statistic. See instruction 2 in Section 6. We conclude that, in the long-run (when the short- and long-run technical inefficiencies are both eliminated) the digital product companies will prove to be more efficient. This means they will attract more resources and thus tend to displace the physical product companies. These results also allow us to infer that Table 2 fails to reflect such possibilities because inefficient performances are present in the original observations. That is, the presence of these technical inefficiencies obscures potentials for performance that greatly favor digital product companies. The conclusions described above were realized by computing R as the sum of all rankings for the digital product companies. We now reverse this calculation and compute R as the sum of the ranks for physical product companies. This helps ensure that the choice of digital products used to compute R did not produce results that favor them. Using physical products to compute R; we thus obtain the Mann–Whitney values shown in Table 4. The negative Z values lead us to strongly reject the assumption that the physical product companies have potentials for efficient performances (which, as reported in Table 2, are not statistically distinguishable from digital product companies). Finally, we calculated the average ranks of digital and physical product companies, as reported in Table 5. Smaller numerical values being associated with higher ranks, the averages for the digital product companies exhibit superior ranks, on average, in every case, in this (simple) way of looking at such matters.5
5
See also Tables 9 and 13 in the appendix. They show that the averages remain stable even after being determined with a different model, and after an increase in sample size that results from including the DMUs with negative profits.
ARTICLE IN PRESS 246
A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
Table 5 Average ranks for digital and physical product companies Year
Digital product companies
Physical product companies
1998 1999
61.0 56.5
123.1 98.4
8. Conclusions and suggestions for further research Results reported herein clarify issues beyond those addressed in [1]. Table 2, for example, reports results that suggest physical-product companies might be more efficient than their digital product counterparts. However, these results are clouded by technical inefficiencies present in the data. Clearing out these inefficiencies provided evidence that reversed this finding and indicated, instead, that digital product companies are superior, and significantly so. Our method for distinguishing between short- and long-run efficiencies was thus able to bring into view fundamental features obscured by technical inefficiencies present in the data. In looking for ways to extend the results, we note the possibility of investigating other types of efficiency such as ‘‘scale’’ and ‘‘allocative’’ efficiencies. Another promising area is provided by developments reported in [23] where Bardhan et al. suggest ways to combine regressions with DEA so as to take account of efficient and inefficient performances. This is done in a two-stage fashion as follows: Stage one applies DEA to identify efficient and inefficient performers. Stage two utilizes the Stage one results by incorporating them as dummy variables into regression expressions such as (1). In this manner, it becomes possible to extend and refine the results reported both in this paper and in [1]. This two-stage process allows one to obtain regressions that simultaneously distinguish between efficient and inefficient performers in both digital and physical product firms. This makes it possible to examine whether results such as those reported in [1] hold for both efficient and inefficient performers. Such an analysis helps to clarify the types of investments that each class (inefficient and efficient) might make as ‘‘longer run’’ tendencies toward efficient behavior are realized.
Acknowledgements Acknowledgement is gratefully made to the RGK Foundation in Austin, Texas, and the IC2 Institute of the University of Texas at Austin for support of the research of W.W. Cooper. Comments by Chester Wilson helped to improve the presentation. Appendix A Reference to (2) in the text of this article shows that f does not reflect any input inefficiencies. It also does not reflect any of the non-zero slack that may be present in the objective of (2). The radial measure, f ; is therefore ‘‘incomplete’’.
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
247
One also encounters another problem in the use of f for ranking purposes. The f value for one DMU may not be directly comparable to the f value for another DMU because they are evaluated on different segments of the frontier. This means that such points are evaluated by different DMUs. This does not create problems in identifying the sources and amounts of inefficiencies for each of these DMUs—since these amounts are determined relative to the efficient DMUs that are used to evaluate the performances of each of these two different DMUs. It does, however, create problems of ranking. For instance, DMU A may be 90% as efficient as the combination of DMUs used to evaluate its performance. DMU B may be 80% as efficient as the (different) DMUs used to evaluate its performance. This does not, however, mean that DMU B is less efficient than DMU A and hence these two DMUs should be ranked by reference to their f values of 80% and 90%, respectively. Given these considerations, we turn to a measure that does not suffer from such deficiencies in order to cross-check the results reported in the text. The measure we use is RAM, the range adjusted measure of efficiency, described in Cooper et al. [24]. See also the exchange recorded in [25,26]. Inter alia, this use of different models can provide a safeguard against ‘‘choice-of-model bias’’ analogous to the bias discussed in the exchanges recorded in [7,8]. See also the more complete record of these exchanges in Charnes et al. [27]. A.1. RAM Because RAM is fairly new, it may not be familiar to many readers. We therefore begin by describing the RAM model and its uses in a manner that emphasize features pertinent to the topics of interest here. As described in [24], the RAM model can be represented in the following manner: Pm Ps þ þ i¼1 si =Ri þ r¼1 sr =Rr max ðm þ sÞ Pn s:t: xio ¼ j¼1 lj xij þ s i ¼ 1; 2; y; m; i ; Pn ðA:1Þ r ¼ 1; 2; y; s; yro ¼ j¼1 lj yrj sþ r ; P 1 ¼ nj¼1 lj ; þ 0plj ; s i ; sr ;
8i; j; r;
þ where R i is the range of observed values for input i and Rr is the range of observed values for output r: That is,
R % i xi ; i ¼x %
Rþ % r yr ; r ¼y %
where x% i ¼ max fxij g; j¼1;y;n
y% r ¼ max ðyrj Þ; j¼1;y;n
xi ¼ min fxij g j¼1;y;n % yr ¼ min fyrj g j¼1;y;n %
for i=1,y,m; r=1,y,s.
ðA:2Þ
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
248
In the exchange between [24] and [25] it is shown that Pm Ps þ þ i¼1 si =Ri þ r¼1 sr =Rr 0p p1; ðA:3Þ mþs Pn þ when the convexity condition j¼1 lj ¼ 1 forms part of the model. Notice that Ri and Ri þ þ provide the range of inefficiencies allowed by the data for every DMU with s i rRi and sr pRr : The ratios in the numerator of (A.3) thus represent the proportion of the possible inefficiencies utilized by these slacks. The ratios are evidently units invariant. Hence, we can interpret the value of (A.3) as an average of these inefficient proportions. The complementary (efficiency) measure is Pm Ps þ þ i¼1 si =Ri þ r¼1 sr =Rr 0pG 1 p1; ðA:4Þ mþs where (A.4) is derived from (A.1) and referred to as the RAM model. To provide contact with the measure used in the objective of (2), we modify (A.1) as follows: Pm Ps þ þ i¼1 si =Ri þ r¼1 sr =Rr min y mþs P s:t: yxio ¼ nj¼1 lj xij þ s i ¼ 1; 2; y; m; i Pn yro ¼ j¼1 lj yrj r ¼ 1; 2; :::; s; ðA:5Þ Pn 1 ¼ j¼1 lj ; 1 ¼ y; þ 0plj ; s i ; sr
8i; j; r:
Here, we have added the constraint y ¼ 1 to achieve the usual RAM model. Hence, there is no need to utilize the non-Archimedean element e > 0; as in (2), because no other variables can influence the value of y: We therefore have a measure in the form of G that incorporates all inefficiencies that the model can identify. The interpretation we provided in our discussion of þ (A.3) and (A.4) continues to apply—i.e., R i and Rr represent the ranges of inefficiency, as shown by the data, that are available in each input and output for every DMU. We can also regard the þ R i and Rr as weights, in reciprocal form. These weights are the same for all DMUs and hence provide a valid basis for the rankings we use. See the discussion in [24]. See also [25]. A.2. Cross checking We now utilize the same data in the text to see whether the different models and measures represented by RAM produce results that are consistent with the results secured from the model and measure in (2). Tables 6–9, below, are numbered to match the table numberings in the text. The values obtained from (A.5) and (2) are thus very similar. All algebraic signs point in the same directions and the numerical magnitudes are generally close. One exception occurs in Table 6 where, for 1998, the Z value achieves statistical significance at the 1% level whereas these values do not achieve significance in Table 2. We thus conclude that the digital product companies exhibit significantly less efficient performances when all the inefficiencies are reflected in the measure used—as in G but not in f : This suggests that these additional (short-run) inefficiencies
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
249
Table 6 Mann–Whitney values with unadjusted data Year
n1
n2
R
U
Z
1998 1998a 1999 1999a
115 115 102 102
35 35 28 28
9259 9017 6745 6482
1436 1678 1364 1627
2.561642492 1.486330293 0.362456463 1.127013063
n1 —number of digital product companies. n2 —number of physical product companies. a the variable t (=years in business) is omitted.
Table 7 Mann–Whitney values for efficiency adjusted dataa Year
n1
n2
R
U
Z
1998 1999
115 102
35 28
7126 5880
3569 2229
6.916212557 4.536369165
a
Data for the input variable t have been omitted.
Table 8 Results of Mann–Whitney rank order statistics with physical product companies used for R Year
n1
n2
R
U
Z
1998 1999
35 28
115 102
4199 2635
456 627
6.916212557 4.536369165
Table 9 Average ranks for digital and physical product companies Year
Digital product companies
Physical product companies
1998 1999
61.965 57.647
119.971 94.107
tend to occur in the digital rather than the physical product, companies. This, too, is consistent with the relative newness and lack of experience in digital product companies. See the discussion in Section 6. As reflected in Tables 7 and 3, the algebraic signs are reversed when ‘‘short-run’’ operating inefficiencies are eliminated by means of projection formulas for (A.1). Hence, (3) is replaced by i ¼ 1; y; m; x# io ¼ xio s i ; ðA:6Þ r ¼ 1; y; m y# ro ¼ yro þ sþ r ;
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
250
for use in Table 7 in contrast to formulas (4) for Table 3. In both 1998 and 1999, statistical significance is again achieved, favoring the digital product companies in both tables. Hence, as in the text, we conclude that the digital product companies are inherently more efficient. Further, we conclude that this will become clearer as operating inefficiencies are eliminated in the long run for both types of companies. Comparisons between Tables 8 and 4 show very similar results when the Mann–Whitney R value is computed for physical rather than digital product companies. Finally, comparison shows a slight increase in the average rank for digital product companies and a slight decrease in the average rank for physical product companies when going from Tables 5 and 9. Although consistent with the fact that G reflects inefficiencies not present in fo ; the slight increase in average rank for digital product companies fails to be significant. We therefore conclude that our use of (A.5) in place of (1) reinforces the finding in the text: The long-run potential for relative efficiency favors digital vs. physical product companies. A.3. Negative data We note that our use of two different measures and two different models provides a cross-check on possible ‘‘choice-of-models bias.’’ Consistency in the results strengthens our confidence in the conclusions we have drawn. Similarly, we have a cross-check on ‘‘methodological bias’’ as described in the exchange reported in [7,8]. That is, our confidence is increased since results using the methodologies of DEA are consistent with the results obtained with the regression methodologies in [1]. We can also take advantage of the fact that the RAM model (and measure) is both ‘‘translation invariant’’ and ‘‘units invariant.’’ That is, adding an arbitrary constant, k; to any input or output does not change the solution set or the subset of optimal solutions. See the exchange reported in [25,26]. In the preceding treatments—in the text and in this appendix—we simply dropped those DMUs for which a negative gross margin was reported for either a physical or digital product dot com because the computer code we used could not handle negative values. However, the translation invariance property of the RAM model allows us to proceed as follows: The most negative of the gross margin values was $29,214 103. Hence, adding $30,000 103 to each of the observed gross margins converts them all to positive values without affecting the solutions. To see that this is so, we replace yro ¼
n X
yrj lj sþ r
ðA:7Þ
j¼1
in (A.1) with yro þ kr ¼
n X ðyrj þ kr Þlj sþ r j¼1
¼
n X j¼1
yrj lj þ kr
n X j¼1
lj sþ r ;
r ¼ 1; y; s;
ðA:8Þ
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
251
where kr isP the arbitrary constant that we add to all yrj in output constraint r ¼ 1; y; s: We then n note that j¼1 lj ¼ 1; as prescribed in (A.1). The constant kr thus cancels on both sides of Eq. (A.8) and the solution for (A.8) also satisfies (A.7). Conversely, any solution for (A.7) satisfies (A.8). A similar proof holds for the inputs. Hence, the solution set, including the slack values, is not altered. Finally, we note that addition of the kr values does not alter the value of the ranges as defined in (A.2). The subset of optimal solutions is also not altered and our replacement of negative values by adding a sufficiently large positive constant is justified. After adding k=$30,000 103 to all observed gross margins, we utilized (A.5) and (A.6) to obtain the results reported in Tables 10–13. These tables are presented in the same numerical order as Tables 6–9 so that comparisons can be readily made. The results are almost the same as before. Indeed, the previously noted conclusions are reinforced. For instance, the positive value of Z for 1999 in the last row of Table 6 is replaced by a negative value in Table 10. The increase in sample size, as seen in the changed values for n1 and n2 in A2 and B2, etc., conforms to the results previously achieved with different models, measures, and sample sizes.
Table 10 Mann–Whitney values with unadjusted data Year
n1
n2
R
U
Z
1998 1998a 1999 1999a
133 133 122 122
41 41 31 31
12303 11767 9500 9414
2061 2597 1785 1871
2.359945153 0.459222986 0.481144323 0.090781948
n1 —number of digital product companies. n2 —number of physical product companies. a the variable t (=years in business) is omitted.
Table 11 Mann–Whitney values for efficiency adjusted dataa Year
n1
n2
R
U
Z
1998 1999
133 133
41 31
9642.5 8430.5
4721.5 2854.5
7.074516272 4.373420329
a
Data for input variable t have been omitted.
Table 12 Results of Mann–Whitney rank order statistics with physical product companies used for R Year
n1
n2
R
U
Z
1998 1999
41 31
133 122
5582.5 3350.5
731.5 927.5
7.074516272 4.373420329
ARTICLE IN PRESS 252
A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
Table 13 Average ranks for digital and physical product companies Year
Digital product companies
Physical product companies
1998 1999
72.5 69.10246
136.1585 108.0806
The observed (short-run) behavior shows the physical product dot coms to be more efficient, or at least not less efficient, than the digital dot coms. However, this conclusion is reversed when efficiency-adjusted data are used to reveal the improvements that might be expected with increasing experience.
References [1] Barua A, Whinston AB, Yin F. Not all Dot Coms are created equal: an exploratory investigation of the productivity of internet based companies. Management Science 2002; submitted for publication. [2] Solow RM. We’d better watch out. New York Times, July 12, 1987. p. 36. [3] Roach SS. Stop rolling the dice on technology spending. Interview with G. Harrar, Editor, Computer World Extra 1988, June 20. [4] Roach SS.The case of the missing technology payback, Paper presented at the tenth International Conference on Information Systems, Boston, MA, December 1989. [5] Roach SS. Service under siege: the restructuring imperative. Harvard Business Review 1991;69:82–91. [6] Oliner SD, Sichel DE. The resurgence of growth in the late 1990s: is information technology the story? Journal of Economic Perspectives 2000;14:3–23; Brynjolfson E, Hitt LM. Beyond computation: information technology, organizational transformation and business performance. Journal of Economic Perspectives 2000;14:23–48; Gordon RJ. Does the new economy measure up to the great inventions of the past? Journal of Economic Perspectives 2000;14:49–74. [7] Charnes A, Cooper WW, Sueyoshi T. A goal programming/constrained regression review of the Bell system breakup. Management Science 1988;34:1–26. [8] Evans DS, Heckman JJ. Natural monopoly and the Bell system: response to Charnes, Cooper and Sueyoshi. Management Science 1988;34:27–38. [9] Wang CH, Gopal RD, Zionts S. Uses of Data Envelopment Analysis in assessing information technology impact on firm performance. Annals of Operations Research 1997;73:191–213. [10] Kauffman RJ, Weill P. An evaluative framework for research on the performance effects of information technology investment. In: Proceedings of the 10th International Conference on Information Systems, Boston, MA, December 1989. p. 187–96. [11] Koopmans TC. Analysis of production as an efficient combination of activities. In: Koopmans TC, editor. Activity analysis of production and allocation, Cowles Commission Monograph No. 13. New York: Wiley; 1951. [12] Charnes A, Cooper WW, Golany B, Thomas D, Halek R, Schmitz E. A two-phase procedure for efficiency analysis and evaluation of advertising in US Army recruiting activities. CCS Report 521, Center for Cybernetic Studies, University of Texas at Austin, Austin, TX, 1985. [13] Shafer SM, Byrd TA. A framework for measuring the efficiency of organizational investment in information technology using Data Envelopment Analysis. Omega 2000;28:125–41. [14] Kao C. Short-run and long-run efficiency measures for multiplant firms. Annals of Operations Research 2000;97:379–88.
ARTICLE IN PRESS A. Barua et al. / Socio-Economic Planning Sciences 38 (2004) 233–253
253
[15] Kozmetsky G, Yue P. The Economic Transformation of the United States, 1950–2000. Austin, TX: The IC2 Institute of the University of Texas at Austin (Lafayette, IN: The Purdue University Press; 2003, to appear). [16] Banker RD, Charnes A, Cooper WW. Some models for estimating technical and scale inefficiencies in Data Envelopment Analysis. Management Science 1984;30:1078–92. [17] Cooper WW, Seiford LS, Tone K. Data Envelopment Analysis: a comprehensive text with models, applications, references and DEA-Solver software. Boston: Kluwer Academic Publishers; 2000. [18] Arnold VL, Bardhan I, Cooper WW, Gallegos A. Primal and dual optimality in computer codes using two-stage solution procedures in DEA. In: Aranson J, Zionts S, editors. Operations research: methods, models and applications. Westport, CT: Quorum Books; 1998. [19] Pareto V. Manuel d’Economie Politique,, 2nd ed. Paris: Girard et Cie; 1927. [20] Farrell MJ. The measurement of productive efficiency. Journal of the Royal Statistical Society, Series A 1957;III:251–78. [21] Brockett PL, Golany B. Using rank statistics for determining programmatic efficiency differences in Data Envelopment Analysis. Management Science 1996;42:466–71. [22] Brockett PL, Levine A. Statistics and probability and their applications. Fort Worth, TX: CBS College Publishing Co.; 1984. [23] Bardhan I, Cooper WW, Kumbhakar S. A simulation study of joint uses of Data Envelopment Analysis and statistical regressions for production function estimation and efficiency evaluation. Journal of Productivity Analysis 1998;9:249–78. [24] Cooper WW, Park KS, Pastor JT. RAM: a range adjusted measure of inefficiency for use with additive models, and relations to other models and measure in DEA. Journal of Productivity Analysis 1999;11:5–42. [25] Cooper WW, Park KS, Pastor JT. The range adjusted measure (RAM) in DEA: a response to the comment by Lukas Steinman and Peter Zweifel. Journal of Productivity Analysis 2001;15:145–52. [26] Steinmann L, Zweifel P. The range adjusted measure (RAM) in DEA: comment. Journal of Productivity Analysis 2001;15:140–5. [27] Charnes A, Cooper WW, Sueyoshi T. More on breaking up Bell. Center for Cybernetic Studies Research Report 590, Red McCombs School of Business, University of Texas, Austin, TX, 1988.
