Chapter Listing | Return to 1997 Topical Index


DATA AND MEASURES

Model I and II are tested on data from the medical instruments and supplies industry (SIC 384). The sample population was obtained from the Company Profiles Database. 1011 business–units were identified was listing SIC 384 as their primary business. Each business–unit was contacted via telephone and asked to provide two people who had a good knowledge of research and product development. The goal of surveying two people per organization was to enable inter–rater agreements assessment and to provide a more accurate picture of the knowledge acquisition and dissemination activities. 644 business–units agreed to provide names (386 provided two names and 258 would only provide one name). This resulted in a sample population of 1030, with 644 unique business–units.

The administration of the survey took place in several stages, following Dillman’s (1978) Total Design Method (TDM). A week after the initial questionnaires were sent out, reminder post cards were mailed. A follow up questionnaire was sent to non respondents two weeks after the post card. Reviews of the methods used to increase the response rate suggest that the use of pre–notification, follow–up mailing, monetary incentive, return postage, and personalization of the cover letter increases the response rate (e.g. Coz et. al., 1974; Dillman, 1978). To maximize the response rate in this study a number of inducements were used including, a personalized cover letter, commemorative stamps on the initial survey packet, a colored questionnaire, a stamp for return postage, a dollar bill for the participant to purchase a beverage to enjoy while filling out the survey, and guarantee of participant anonymity. 614 responses were received from 444 business units, representing a response rate of 59.6% for individuals and 68.9% for business units. The reader is cautioned that not all participants provided financial data and a reduced data set of completed surveys is used to evaluate the models.

To assess the inter–rate agreement between respondents, the surveys from the 170 business–units with two respondents were compared. Given the wide variety of interrater agreement indices available the analysis of interrater agreement is restricted to two of the more common indices to enable the comparison with existing research. The average pair–wise correlation coefficient was calculated for the sales, employment, and research expenditure figures (Jones, et. Al., 1983). While no cut–off level of average pair–wise correlation has been agreed upon, the result of 0.80 from this data appears be comparable to published values that been deemed generally high (e.g. Borman, 1982).

To evaluate the agreement between the perceptual questions scored on a 7–point LIKERT scale, the Tinsley and Weiss (1975) T index is used. This index evaluates the exact agreement between raters. Exact agreement is difficult to obtain with high–point rating scales (Schmidt and Kilmoski, 1991); therefore, the Tinsley–Weiss index is also computed on the number of items that were within one point of each other. While there is inherent variation in the responses to the perceptual questions due to the individual differences, the results of this analysis show that, after correcting for chance agreement, on average the individuals agree 26% of the time and are within one point of each other 39% of the time. This suggests a good degree of inter–rater agreement (Schmidt and Kilmoski, 1991). In conclusion, the results of the inter–rater agreement analysis are consistent with accepted values in the business literature. Because the unit of analysis is the business unit, questions with two responses per business–unit were pooled to provide a sample of 444 questionnaires used for scale development and validation.

The next series of analyses focuses on refining the scales of external knowledge acquisition and intra–firm knowledge dissemination. Exploratory factor analysis and TETRAD analysis was conducted on a training set of the data. The robustness of the solution is then confirmed via structural equation modeling on a test set of the data. The sample of 444 firms was randomly separated into a training and test data set. The process of splitting data into two random samples is widely recognized in the statistics literature as a procedure for evaluating the robustness of results (e.g. Mosteller and Tukey, 1977). A summary of the stages and results can be found in Table 1.

The first exploratory technique used was factor analysis with no pre–specified structure to the test data set. From the theoretical development, three factors representing external knowledge acquisition and intra–firm knowledge dissemination were anticipated; however, five factors have an eigenvalue in excess of one. Examination of the pattern of the eigenvalues of the factors shown in the scree revealed that a significant break occurs between the fourth and fifth factor. In the four factor solution each item, except for one, has a loading of 0.40 or greater. Inspection of the questions loading on the four factors identified one factor relating to intra–firm knowledge dissemination, and three factors pertaining to external knowledge acquisition. Examination of the external knowledge acquisition factors indicates that each factor pertains to adistinct area of knowledge. One factor relates to knowledge about competitors, one to knowledge about customers, and one to new developments.

The content validity of the factors was assessed by examining each of the items that loaded on particular factors and checking to see that it was consistent with the theoretical development. While the separation of external knowledge acquisition into three distinct areas was not hypothesized, this result is consistent with the other work investigating external knowledge acquisition (e.g. Narver and Slater, 1990).

The results of the exploratory factor analysis were then purified using the Purify module of the TETRAD II program to find unidimensional measures. This procedure relies on the fact that structural equation models with latent variables include certain vanishing tetrad differences independent of the numerical value of the parameters (for further details see Scheines, 1994). The TETRAD purification program eliminated a number of questions and the refined factors were then assessed for internal reliability by computing the Cronbach’s alpha. Nunnally (1978: 245–246) suggests that reliability measures should exceed 0.7 for preliminary research and 0.8 for basic research. Each construct exceeds the 0.7 threshold and two exceed the more stringent 0.8 figure.

TABLE 1

Overview of Exploratory and Confirmatory Procedures

Exploratory Analysis (Training Data Set)

Confirmatory Analysis

(Test Data Set)

Stage 1  

Stage 2

 

Stage 3

Stage 4

Stage 5

Stage 6

Literature Review.

Interviews.

Pilot Testing.

 

Exploratory Factor Analysis

 

Tetrad Analysis

Internal Reliability of Constructs (Chronbachs a )

Internal Reliability of Constructs (Chronbachs a)

Independence of Constructs (c 2 test)

Construct

# Items

Construct

# Items

# Items

     
External Knowledge Acquisition

17

Knowledge Acquisition about Competitors

7

6

0.81

0.74

Confirmed

   

Knowledge Acquisition about Customers

5

4

0.73

0.71

Confirmed

   

Knowledge Acquisition about New Developments

5

4

0.74

0.71

Confirmed

Intra–Firm Knowledge Dissemination

7

Intra–Firm Knowledge Dissemination

7

6

0.80

0.84

Confirmed

To evaluate the robustness of the exploratory factor analysis, the four factor solution was tested on the test data set. The purified four factor model was fitted to the test data set using structural equation modeling with each item loading on its respective factor. To confirm the four factor solution, the chi–square for the three factor models is compared with the chi–square for the four factor model (Richens and Dawson, 1992). In this test items from two factors are constrained to load onto a single factor, and the constrained model (three factors) is compared to the unconstrained model (four factors).

H0: A three factor model fits the data better than a four factor model.

If above hypothesis is rejected for every three factor model (6 cases) then the four factor solution is validated. The results of this analysis confirmed that the four factor solution is superior in fitting the data. Finally, the internal reliability of measures was confirmed by computing the Cronbach’s alpha for each construct using the test data set. Each construct’s alpha exceeds the 0.7 threshold confirming the internal reliability.

In conclusion, using a process of exploratory and confirmatory techniques, the scales were evaluated and refined to generate measures with the desirable psychometric properties of high internal reliability and validity.

Top of page | Chapter Listing | Return to 1997 Topical Index


1997 Babson College All Rights Reserved
Last Updated 03/19/98