(1) The Agreement and Structure of Colleague Ratings
Maria M. Llabre
University of Miami
Harry W. Forgan
University of Miami
Abstract: The interrater reliability and the factor structure of colleague ratings of university faculty were studied. Two approaches to rating colleagues were compared: a global rating and an analytic rating in four areas of responsibility. Reliability indices indicated that means based on four raters using the analytic method were reliable. Results of a factor analysis indicated the presence of two factors: research and publications and, to a lesser extent, teaching.
Citation: Llabre, M. M., & Forgan H. W. (1985). The agreement and structure of colleague ratings. Florida Journal of Educational Research, 27(1), 19-26.
Download: Llabre.271.pdf (1880 downloads )
Stephen F. Olejnik
University of Georgia
James Algina
University of Florida
Abstract: The rank transformation approach to analysis of variance as a solution to the Behrens-Fisher problem is examined. Using simulation methodology four parameters were manipulated for the two group design: (1) ratio of.population variances; (2) distribution form; (3) sample size and (4) population mean difference. The results indicated that while the rank transform approach was less sensitive to variance inequality than the parametric ANOVA F-ratio, unacceptably high Type I error rates were obtained when cell frequencies and group variances were inversely related. With equal cell frequencies and/or when cell frequencies were directly related to group variances, appropriate Type I error rates were obtained. Under these conditions however, the Brown-Forsythe procedure for comparing, group· means provided greater power except when the sampled distribution was leptokurtic.
Citation: Olejnik, S. F., & Algina, J. (1985). Type I error rate and power of rank transform ANOVA when populations are non-normal and have equal variance. Florida Journal of Educational Research, 27(1), 61-82.
Download: Olejnik.271.pdf (1117 downloads )
(3) The Effect of Examinee Pass/Fail Information on the Level of Passing Score Set by Expert Judges
Joseph C. Saunders
South Carolina Department of Education
Dan L. Linton
South Carolina Department of Education
Abstract: Many important applications of criterion-referenced testing require that passing or cutting scores be established. One class of procedures for setting passing scores involves the judgments of subject-matter experts. This study investigated the effect on passing scores of providing judges with pass/fail data for the various possible cut-score levels. Passing scores produced by individual judges using a modified version of the Angoff (1971) procedure are compared with the same judges’ passing scores as revised after considering pass/fail data. For the seven cases studied, no consistent effect was seen on the level of passing score averaged across judges. However, a consistent reduction in the variability of passing scores among judges was observed. Judges with more extreme Angoff passing scores tended to become more moderate in their judgments after considering pass/fail information.
Citation: Saunders, J. C., & Linton, D. L. (1985). The effect of examinee pass/fail information on the level of passing score set by expert judges. Florida Journal of Educational Research, 27(1), 5-18.
Download: Saunders.271.pdf (1473 downloads )
Shlomo Sawilowsky
University of South Florida
Abstract: The use of the parametric ANOVA Test when the underlying population is non-normally distributed is a violation of the test’s assumption of normality. Alternatives include the use of nonparametric procedures. However, to date, useful procedures have not been developed to detect interactions, especially those of the higher order. Based on the rank transform, theRandomNormalScoresTest has been suggested as a powerful alternative to the ANOVATest. The major support rests upon asymptotic theory. This study is an empirical analysis of the Random Normal Scores Test performed under the F and Chi-square distributions. In the balanced 2x2x2 layout, for various population distributions, sample sizes, and nominal alpha levels selected, the Random Normal Scores Tests- were shown to be non-robust and not powerful alternatives to the ANOVA Test.
Citation: Sawilowsky, S. (1985). A comparison of random normal scores test under the f and chi-square distributions to the 2x2x2 ANOVA test. Florida Journal of Educational Research, 27(1), 83-97.
Download: Sawilowsky.271.pdf (1221 downloads )
(5) Evaluation of a Pilot Program for Computer Literacy in Grades K-8
Jacquelyn H. Still
Orange County Public Schools
Abstract: A curriculum guide and suggested instructional resource materials were developed for an Orange County School District Program in computer literacy for grades K-8. The program was piloted with 43 teachers in nine elementary schools and one junior high school in the spring of 1984. Both teachers and students were positive about the program and students’ achievement gains were satisfactory. The evaluation data provided information about revisions that should be considered in the curriculum guide, resource materials, inservice training and instruction.
Citation: Still, J. H. (1985). Evaluation of a pilot program for computer literacy in grades K-8. Florida Journal of Educational Research, 27(1), 43-59.
Download: Still.271.pdf (1210 downloads )
(6) Comparative Performance of Florida Seniors on the GED Test Battery
Howard W. Stoker
University of Tennessee
Abstract: Two separate studies are reported. The first comparedFlorida students’ performance on the GEO to national GEO norms. Results indicated that Florida students compare quite favorably. Mean scale scores were higher for Florida students than the national norm group on all tests and noticeably higher on the science and math tests. The second study compared Florida students’ performance on the GEO with their performance on the Florida SSAT-II. Results indicated that students who cannot pass the SSAT-II are unlikely to succeed on the GEO.
Citation: Stoker, H. W. (1985). Comparative performance of Florida seniors on the GED test battery. Florida Journal of Educational Research, 27(1), 27-41.
Download: Stoker.271.pdf (1341 downloads )