Measuring Outsourcing Efficacy: An Examination of Performance in the Aerospace Sector

O n‐site outsourcing can be an effective strategy for organizations but operational performance may be temporarily hindered and recovery from this can take many months. This study explores an instance of outsourcing in an aerospace maintenance and repair operation (MRO) in South Wales, where the inbound materials handling and warehousing (IMHW) function has been wholly outsourced to a third‐party organization. In this respect the study is novel and sheds new light on the operational performance benefits of outsourcing. The host organization, located in South Wales, employs over 1000 staff and its annual turnover is in excess of £1 billion. The outsourcing organization is a provider of workforce solutions to organizations in aviation, defense, government, and technology sectors. After outsourcing the IMHW function, the host organization’s staff were retained and re‐employed in other areas of the facility. Specifically, this study provides a quantitative analysis of the IMHW function, comparing the throughput and accuracy of the function before and after outsourcing. The analysis is made based on data extracted from the host organization’s management information system (MIS), spanning the 13 months prior to, and 16 months post, the function being outsourced. Using trend analysis, correlation, and linear regression analysis it finds that on‐site outsourcing can deliver comparable operational performance, in terms of IMHW functional throughput and accuracy, to the host organization, and therefore be an effective strategy for organizations wishing to focus upon core competencies. It also highlights that operational performance can be severely hindered following functional outsourcing and that recovery from this can take many months. It postulates that preparation for functional handover, especially in the form of training and tacit knowledge exchange between host and outsourcing employees, requires careful planning to minimize disruptions and maximize operational benefits. The


Trends in outsourcing practice and research
Over the last decade, outsourcing has become one of the major issues in many sectors of commerce (Cigolini et al., 2011).In 1998, outsourcing was forecast to increase on average by 10% each year (Elmuti et al., 1998).By 2004, 40% of Fortune 500 companies were outsourcing (Mehta et al., 2006 -cited in Nadkarni andHerrmann, 2010) and in 2005 outsourcing was said to have become a 'mega-trend' in supply chain management (Feeney et al., 2005cited in Wee et al., 2010).According to Pouder et al. (2011), 82% of large and medium-sized firms now out-source.The literature also indicates, however, that out-sourcing is 'not all rosy' (Dekkers, 2011: 936).
Over time, the reasons for outsourcing appear to have shifted, from being predominantly cost related to enabling the organization to focus upon its core competencies and, lately, to access complementary specialist expertise (Table 1).In this particular case of on-site outsourcing, the host company retained its staff and redeployed them in other areas of the organization.This clearly indicates that the reason for outsourcing the function was not one of cost reduction.
As well as the growth of outsourcing as a topic, there is also an emerging and growing trend in the scope of activities that are being outsourced (Jain and Natarajan, 2011).Pouder et al. (2011) have commented how it is now common for companies to manage a portfolio of outsourcing contracts.It is, however, typically applied to non-core business activities (Bailey et al., 2008;Lysons and Farrington, 2006;Reuvid and Hinks, 2001).Elmuti (2003), Elmuti et al. (1998), andPouder et al. (2011) recognize the preponderance of information technology (IT)-related outsourcing activities, and this appears to have led to the outsourcing literature being dominated by investigations in this area (Busi and McIvor, 2008; see also Parker and Russell, 2005;Udo, 1999).McIvor (2010) notes though that outsourcing activity has been particu-larly pronounced in manufacturing, and Klingenberg and Boksma (2010) highlight that materials handling out-sourcing has received little academic attention.
Drawing upon material from the field of business and management, using the Business Source Premier and Emerald knowledge repositories, the literature review con-ducted for this study supports Elmuti's (2003), Elmuti et al.'s (1998), andPouder et al.'s (2011) observations that the majority of outsourcing literature has focused on IT-related outsourcing activities.It also concurs with Klin-genbergand Boksma's(2010) observationthatconsiderably less attention has been paid to the outsourcing of manu-facturing, logistics, and materials handling functions (Table 2).

Author
Reasons for outsourcing

Outsourcing and offshoring
Offshoring has become increasingly prevalent (Cantwell and Santangelo -cited in Mudambi and Venzin, 2010), and has become the focus of much of the outsourcing literature (Jain and Swarup, 2011).The majority of the literature focuses on offshore outsourcing, where business functions or activities are taken on by other organizations and often relocated across the globe: such as Jain and Natarajan's (2011) examination of outsourcing decisions in the Indian banking sector.There have, however, been some recent suggestions that offshoring is in decline (Herath and Kishore, 2009;Weerakkody and Irani, 2010).On-site outsourcing, where the function remains physically located within the host organization but the management and operation of that function is taken over by a third party, has not received much attention: the literature review identified a single study that explores an instance of on-site outsourcing, where a minimal number of third-party staff remain located in the host organization (Weerakkody and Irani, 2010).It found that the venture, which focused on the provision of IT staff to the host organization, was ultimately successful.However, this example is limited to the provision of human resources working under the direction of the host's management.It does not constitute the provider taking over responsibility for an entire business function, as is the focus of this study.Pouder et al. (2011) recognize that there are a growing number of companies using outsourcing as a way to improve performance.After implementing outsourcing it is, however, important to monitor the performance of the outsourced company or function (Elmuti, 2003;Elmuti et al., 1998;Weimer and Seuring, 2009), with Lynch (cited in Trebilcock, 2004) declaring that measuring per-formance is one of the 'ten rules' of outsourcing.

Measuring outsourcing performance
Even though the need to measure outsourcing performance would appear self-evident, several authors have indicated that this requires further investigation (Busi and McIvor, 2008;Elmuti, 2003;Thurm, 2007 -cited in Bardhan et al., 2007).According to Giga Information Group (Vernon, 2002), only 10% of companies actually measure outsourcing performance.Again, it must be stressed that this view is based on research that focuses predominantly on the IT sector.
Much of the outsourcing literature is also subjective and based on business experiences rather than an objective view using historical data (Nieroda, 2003).Jiang and Qureshi (2006) showed that only 19.8% of research literature provides a results-based view of outsourcing.They also observed that much of the literature uses 'soft data,' reviewing people's opinions, and suggest that there is a need for more objective evidence.
One of the problems with measuring performance is finding what exactly should be measured, and even when that is accomplished the data is usually inaccurate (Busi and McIvor, 2008).Kotabe et al. (1998-cited in Elmuti, 2003) state three important measures of performance: strategic, financial, and quality related.Meanwhile, Bardhan et al. (2007) state that there are two methods: gross margin (financial) and on-time delivery (operational).Among the debates within the literature about how to measure outsourcing performance, many have disagreed with measuring cost (Bardhan et al., 2007;Jiang and Qureshi, 2006;Vernon, 2002).This is rather surprising, considering that cost has been one of the most common reasons for undertaking outsourcing in the first place, and this may be partially due to the difficulty in calculating cost savings in a changing environment (Bardhan et al., 2007;Jiang and Qureshi, 2006;Nieroda, 2003).Vernon (2002) states that as long as costs are reduced, many companies do not consider measuring the performance.
One of the benefits of measuring outsourcing performance in terms of long-term benefits, such as acquiring skills and knowledge, instead of short-term benefits such as costs, is that the arrangement is said to last longer (Pouder et al., 2011;Vernon, 2002).Lacey and Blumberg (2005) found that the most important factor when choosing an outsourcing partner was their expertise and capability, arguing that the longer a company uses outsourcing the stronger the performance.This is supported by a study carried out by Accenture (Lacey and Blumberg, 2005), which found that 63% of participants saw business performance improvements within two to seven years of out-sourcing, further increasing to 71% of companies seeing more positive performance after seven years.Gonzalez et al. (2010) warn though that if the outsourcing company subsequently believes they possess some expertise, they can try to increase the price.

Aims of the study
This study aims to address the gaps identified in the literature relating to: lack of research around outsourcing materials handling functions in manufacturing environments; the absence of research that explores the on-site outsourcing of a business function; and the need to conduct an objective investigation of outsourcing performance.Specifically, this study provides a quantitative analysis of the IMHW function, comparing the through-put and accuracy of the function before and after outsourcing.
Recognizing the importance that is placed on measuring operational performance within the literature, this study measures two variables that are key to the effective-ness of the outsourced IMHW activities of the host organization (see Figure 1): (1) Measure of Performance -the time taken to receive goods where the delivery information was incomplete (termed 'transact case file' by the host organization); (2) Measure of Accuracy -the difference in time between goods being physically used in production and the time that the information system is updated to reflect the movement of materials (termed 'book-in' by the host organization).

Research methodology
The nature of the research aim, to measure outsourcing performance, necessitates a positivist approach (Bryman and Bell, 2007;Saunders and Thornhill, 2009).Deductive research is defined as 'explaining phenomena by collecting numerical data that are analysed using mathematically based methods ' (Aliaga and Gunderson, 2000: 2).
Although it is argued that this method can fail to provide a context for findings (Gramatikov et al., 2010), it is recognized that the visible, standardized nature of the findings this approach generates provides a basis for measurement and will help the researcher to achieve the research aim (Amaratunga et al., 2004;Karami, 2007).This study was undertaken in a longitudinal fashion, which is where the 'data is collected on one or more variables for two or more periods of time' (Menard, 2008;Sandra et al., 2009: 3).It incorporates participant observation, where the researchers were immersed in the host organization (Bositis, 1988), to add context and insight to the quantitative analyses.
Data was selectively sampled from the host organization's MIS and was representative of the incoming deliver-ies of all product lines (Agirre and Edmonds, 2007;Collis and Hussey, 2009;Franses and Paap, 2004).The analysis is made based on 29 months' worth of data, spanning the 13 months prior to, and 16 months post, the IMHW function being outsourced.The data sample comprises 5000 individual goods inward receipts.Data is presented visually, in the form of line graphs, and analyzed with trend analysis (t-test), correlation analysis, and regression analysis, using SPSS v18.The data sample was tested and considered sufficiently normal for the use of parametric tests (kurtosis = 0.57, skew = 1.05).
During data collection it was found that the host organization's MIS did not possess a function to allow data to be exported into a format that could be manipulated in a spreadsheet prior to migration into SPSS: the data could not be sorted according to the date of delivery.This was an unexpected issue and required the manual re-entry of all dates for each data item.
Research ethics 'ensures the interests and concerns of those taking part in, or possibly affected by, the research are safeguarded ' (Robson, 2002: 18).The researcher received verbal consent to carry out this study using data that is readily available to persons within the company (Bryman and Bell, 2007).Outside the company, the data is protected by restricting access to the data files using pass-words and anonymizing the company names: throughout the remainder of this paper, the host organization is referred to as HOST and the outsourcing organization is referred to as OUT.

Findings Trend analysis
In this study, a t-test was used to provide an initial confirmation that there was a step change in both measures of performance when the IMHW function was outsourced in October 2010.According to Chapman (1997), there are two main types of trend: monotonic and step (Gilbert, 1987-cited in Chapman, 1997).A step trend exists when the data collected before a specific time is from a distinct population to the data collected after that time (Chapman, 1997;Helsel and Hirsch, 1992).
The procedure used to detect if there is a step trend in the data is the two-sample t-test (Helsel and Hirsch, 1992;Lettenmaier, 1976-cited in Harmancioglu et al., 1999;Rong, 2011).According to Helsel and Hirsch (1992: 348), the 'magnitude of change is measured by the difference in sample means between the two periods.'The results of a t-test will display the probability of a step trend existing and should be considered a significant trend when the probabil-ity is below 5% (Harmancioglu et al., 1999;Rong, 2011).The results of the t-test confirmed that there was a probability of 0.0000461, almost 100% certain, that there was a significant step change in the Measure of Perfor-mance after the IMHW function was outsourced (Rong, 2011).The t-test also confirmed that there was a probabil-ity of 0.03477, over 96% certain, that there was a significant step change in the Measure of Accuracy after the IMHW function was outsourced (Rong, 2011).

Regression analysis
One of the problems associated with the quantitative step change trend analysis is that it is said to project the future using straight lines projected from the past (Folsom and Boulware, 2004;Lloyd-Reason and Sear, 2007).It assumes there are no other changes to the environment other than the outsourcing of the IMHW that resulted in the step change of performance (Folsom and Boulware, 2004).
Intimate knowledge of the organization, gained through participant observation of the outsourcing process, revealed that the number of staff and facilities were constant over the period during which the IMHW function was outsourced; the only other variable that would appear to have a potential effect on the performance of the IMHW function was the volume of goods received.A cursory examination of the data revealed that there was a slight increase in the volume of goods received toward the end of 2010 and into 2011, the time during which the IMHW function was outsourced.
Correlation analysis showed a positive relationship between the volume of goods delivered and the IMHW performance (0.65, significant at p = 0.01).A linear regression analysis was performed to determine the influence of volume fluctuations on the performance of the outsourced IMHW function (Figure 2) (Chatterjee and Hadi, 2006).
The results show, unsurprisingly, that there is a relation-ship between the volume of goods received and the Measure of Performance.However, the R-square value (0.406) demonstrates that the increase in volume only accounts for 40.6% of the variation in time taken (Buglear, 2012).Therefore, the change in volume is responsible for less than half the change in performance, the remaining 60% of the step change being due to the function being outsourced.Over the entire period of analysis the IMHW function had an overall average Measure of Performance of 12 days.Prior to outsourcing the IMHW function the average Measure of Performance was 6 days, and after outsourcing the IMHW function the average Measure of Performance was 17 days.The analysis shows that there is a deterioration in the performance following the outsourcing of the IMHW function, confirmed by the t-test.

Measure of accuracy
Figure 4 presents the Measure of Accuracy before and after the IMHW function was outsourced.The monthly average Measure of Accuracy is displayed above the line in the graph for each month.Over the entire period of analysis the IMHW function had an average Measure of Accuracy of 52 days.Prior to outsourcing the IMHW function the average Measure of Accuracy was 38 days, and after outsourcing the IMHW function the average Measure of Accuracy was worse, at 50 days.

Discussion
As is evident from the results, the performance of the function after being outsourced was significantly worse than when the function was operated in-house.Both a visual inspection of the graphs and the t-test demonstrate a step change in performance and accuracy, the regression analysis confirming that the majority of the change in performance was due to the function being outsourced.
Figure 3 shows that unlike OUT's performance, the HOST's performance is consistent until shortly before the IMHW function was outsourced, whereupon performance drops.This could be a result of the resentment that was experienced by employees in the IMHW function, since they were expected to train the people who were essentially replacing them in their roles.This observation is also supported in the literature, whereby Elmuti (2003) suggests that when outsourcing, existing employees lose morale and performance decreases.
When first taking over the function, OUT's Measure of Performance was significantly worse, taking them twice as long as the previous year when run by HOST. Figure 3 shows that OUT's performance is in fact more than three times longer in some months: taking 21 days more in February (eight times longer than HOST), 27 days more in March (ten times longer than HOST), and 19 days more in April (over three times longer than HOST).Despite the poor outsourcing performance of the IMHW function 15 months after it was outsourced, OUT's Measure of Performance has improved continually, and toward the end of 2011 was comparable with that of HOST (in December OUT took only 7 days and in January improved to only 3 days).Despite little in the literature being known about the effects of outsourcing on performance, these results support the claims made by Lacey and Blumberg (2005), whereby it is suggested that the longer outsourcing is used the stronger the performance.From Figure 1 it can be speculated that as OUT enters its second year of outsourcing it could further improve its performance.Further analysis is required to see if HOST experiences continual improvements in OUT's performance beyond that which they were capable of themselves.
Figure 4 shows that OUT's Measure of Accuracy is more than double that of HOST: taking 41 days more in January (three times longer than HOST), 37 days more in March (over twice as long as HOST), and 40 days more in August (twice as long as HOST).The graph also shows that HOST's performance against this measure fluctuated each month, with December being particularly high (68 days).However, OUT continually takes longer than HOST, with August taking the longest to rectify the issues: OUT took on average 80 days to correct the issue, double that of HOST.
One of the issues to bear in mind is that within the IMHW function, and in similar functions in other organizations, there are many challenges to address on a daily basis, and new problems arise continually.For example: issues around efficiency and effectiveness (Kelly, 1983;Won and Olafson, 2005); technology adoption (White et al., 2007); and the introduction of new legislation such as that pertaining to packaging regulations (White et al., 2011).It can be seen that it would be extremely difficult, indeed maybe impossible, to transfer the tacit knowledge skills that are necessary to operate this type function under all circumstances and eventualities (Alonderience et al., 2006;Ambrosini and Bowman, 2001;Bengtsson and Dabhilkar, 2009;Cigolini et al., 2011;Evans et al., 2004;Nonaka, 1994;Nonaka and Konno, 1998;Oyeleran-Oyeyinka, 2004;Sennet, 2008).
The overall deterioration in performance, in both the Measure of Performance and the Measure of Accuracy adopted in this study, raises questions about the capability and training of OUT.It could be argued that OUT's employees or management have simply not received adequate training, echoing Vora's (2010) observation that over-investment in initial training is never bad.It is significant that the persons conducting the training were HOST's employees, whose roles were being replaced by OUT's employees.Even though Vora (2010) notes that when functions are outsourced there are likely to be teething issues, in this instance one should question the quality and completeness of the training that was delivered, along with the degree to which the outsourcing organization was integrated with the host.Parker and Russell's (2005) exploration of IT outsourcing, for instance, highlights the rich and complex working relations and behavioral issues -including psychological contracts, power, and trust -that underpin successful outsourcing projects.Our observations revealed that OUT's employees frequently sought assistance from the HOST's former IMHW staff in order to overcome operational problems.
These findings suggest that once the functional duties have been transferred from the host organization to the outsourcing organization the handover should not be considered complete.There is a need to first establish and regularly, if not continually, monitor meaningful measures of performance.Furthermore, an extended period of tacit support should be maintained, or at least made available, to the outsourced organization to assist in overcoming the infrequent or unusual disruptions that occur during normal operating conditions but which formal training may not encompass.

Conclusion
The literature shows that organizations' motives to out-source have changed over time, and that while they have primarily been for reasons of cost, outsourcing has also been undertaken to enable the organization to focus on its core competencies and to gain competitive advantage.It also reveals that despite outsourcing being an increasingly popular business strategy, only a relatively small number of organizations actually measure their outsourcing partner's performance.
This study provides a quantitative analysis of the IMHW function, comparing the throughput and accuracy of the function before and after outsourcing.The analysis is made based on data extracted from the host organization's MIS, spanning a period of two and a half years.It finds that on-site outsourcing can deliver comparable operational performance, in terms of IMHW functional throughput and accuracy, to the host organization, and therefore be an effective strategy for organizations wishing to focus on core competencies.
It also highlights that operational performance can be severely hindered after the function has been outsourced, and that recovery from this can take many months: some of the literature suggests that it can take several years.It observes that preparation for functional handover, especially in the form of training and tacit knowledge exchange between host and outsourcing employees, requires careful planning to minimize disruptions and maximize operational benefits.It also indicates that host organizations have a vested interest in continuing to support the out-sourced function and organization after operational handover, to maximize performance.
Further investigation is required to confirm if on-site, outsourced functions can deliver improved operational performance, over and above that which the host organization could achieve.This is proposed as a potentially productive avenue for future research.
An unexpected benefit of this research was the discovery of a lack of functionality of the host organization's MIS.This subsequently led to the organization undertaking a minor system modification to address the issue, and has resulted in an improvement in their ability to query and analyze goods inward performance data.
The limitations of this study center on its investigation of a single organization, and thereby the generalizability of its findings are somewhat constrained.However, in offering an approach to quantitatively assessing the operational performance of an outsourced function, in terms of throughput and accuracy, it endeavors to provide an approach to the further study of outsourcing that may corroborate its findings.

Figure 2 .
Figure 2. Linear regression analysis resultsMeasure of performance Figure3presents the Measure of Performance before and after the IMHW function was outsourced, expressed in number of days.In-house and outsourced performance are plotted parallel to each other to allow a clearer visual comparison.In-house performance is plotted between September 2009 and September 2010, outsourced performance is plotted between October 2010 and January 2011.The monthly average Measure of Performance is displayed above the line in the graph for each month.

Table 1 .
Summary of literature reasons for outsourcing

Table 2 .
Focus of outsourcing literature