Archives of Current Research International

The main objective of making investigation about the field of psychology, education and behavioral sciences is to achieve a consistent measurement results. When carrying out the evaluation, many researchers use the classical test theory and other researchers use the generalizability theory. Generalizability theory is used less than the classical test theory by researchers. The main reason is generalizability theory analysis cannot perform by popular statistical programs. In this study, generalizability theory analysis was performed with different statistical programs such as GENOVA, EduG, with SPSS and SAS. In the study, GENOVA, EduG, SPSS and SAS programs outcomes were discussed as the results obtained in the study of alternative decisions in terms of variance components, partial and absolute errors and the results obtained generalizability coefficients. After the research results, there was no significant differences between the results obtained with GENOVA, EduG, SPSS and SAS program.

Aims: SPSS and SAS programs are statistical programs and generally used by researchers which help to perform various analyzes. As for GENOVA and EduG program, they are only written for making G theory analysis. It is considered that widely used statistics programs such as SPSS and SAS programs are written for making the analysis of a G theory. Therefore, when making G theory analysis, there is no need for different software like GENOVA and EduG program. However, it should be noted that it is required to take the same results in different programs due to consistency

of results. The purpose of this study, examination of G theory analysis with different programs GENOVA, EduG ve SPSS-SAS and prediction of Eρ² and Ф coefficents.

Study Design:  Data are collected from two different departments of a company with a proficiency exam (tasks). In this study, two different situations were examined.

Methodology: The first situation is crossed pattern [person(p) x task(t) x evaluator(e)] that includes an evaluation of the adequacy for 10 personnel in 4 different tasks and it is made by 2 evaluators. In other words, every staff made all the tasks and the way to do all of the tasks which was made by staff also evaluated by 2 evaluators. The second situation is mixed pattern [person(p) x (task(t): evaluator(e))] that includes an evaluation for each task in different way for 10 personnel in 3 different tasks and it is scored by 4 different evaluators. In G theory analysis to reach Eρ² and Ф coefficients, it must be made analysis which is appropriate for G theory (pattern) by GENOVA, EduG, SPSS and SAS programs.

Results: When comparing results, analysis are made with using two different designs in generalizability theory and also they are made with GENOVA, EduG, SPSS and SAS programs. (a) The estimated variance of the G study results are generated(p): person; (t): task; (e): evaluator; main effect with (pt): person x task; (pe): person x evaluator; (te): task x evaluator; (pte,c): person x task x evaluator common effects and (p):person; (t):task;(t:e): task : evaluator; (pt): person x task; (pt:e): person x task: evaluator, (b) G study is estimated with σ²(δ) relative and σ²(Δ) of the absolute error variance, (c) K study is estimated with Eρ² and Ф coefficients, (d) After alternative D study, number of different evaluators and tasks obtained from the combination of variance and Ф and Eρ² coefficients found to be consistent with each other.