Language Testing Analysis and Research

At SLTI, we conduct a wide range of research and analysis for our clients. Some of the types of studies we have carried out are described below.

Psychometric Analysis

Psychometrics is the field of study concerned with psychological measurement, which includes the measurement of knowledge, skills, and abilities. SLTI staff members have the capability to conduct psychometric analyses of language tests based on field-test or operational-test data. We can conduct both classical item analyses and analyses using Item Response Theory (IRT), to provide item level data (for example, item difficulty and discrimination, and distractor analyses). These item analyses then feed into more complex analyses related to test reliability and validity.

Validity and Reliability Studies

SLTI also conducts work in test validation. SLTI analyses test data and test results for evidence of reliability and validity, including evidence of content, criterion-related, construct, consequential, convergent and discriminant validity. SLTI also provides reports documenting the test development procedures as well as the results of the test validation study.

Standard-Setting Studies

A standard setting study is a process to establish score requirements or cut-off scores in order to rate examinees at various proficiency levels or to determine the minimum score that should be required to pass a test. SLTI conducts standard setting studies for different assessment instruments using a variety of methods including the Angoff and Bookmark methods, among others.

Alignment Studies

SLTI conducts alignment studies of standard-based assessments in order to provide evidence of the alignment between an assessment and a set of standards. An alignment study provides a systematic procedure for quantifying the degree of alignment of academic content standards and assessments.

Teacher Training

SLTI is often called upon to provide training for teachers. This may be related to any aspect of the test development process. It may cover item writing, test administration, or training raters to score examinee performances.

In the areas of foreign language testing, SLTI is frequently called upon to train teachers to score the Simulated Oral Proficiency Interview (SOPI). SLTI also trains teachers in developing SOPIs and in customizing them so that they are maximally appropriate for their student population.

ISO/TC 232 Learning Services Outside Formal Education

SLTI's Executive Director, Dr. Clive Roberts is currently serving as a delegate from the USA on the ISO/TC 232 committee, writing the international standards for 'Learning Services Outside Formal Education.' These important standards regulate, among other areas, language teaching, test development, and assessment outside the formal educational sector, worldwide.

Past Clients

Medical Interpreter Tests

In 2005, SLTI conducted a pilot test and evaluation of four language proficiency tests (English, Spanish, Cantonese, and Hmong) and medical interpreter readiness tests in three language pairs (Spanish-English, Cantonese-English, and Hmong-English) for the California Endowment. SLTI also carried out a diagnostic evaluation of 10 medical interpreter tests that were converted to computer-based tests.

American Council on the Teaching of Russian

From 2004 to 2006, SLTI conducted item analyses of multiple forms of an experimental, computer-delivered AP Russian (Listening and Reading) Language Examination. SLTI also prepared comprehensive reports on these assessments, including reliability, concurrent and criterion-related validity analyses. We also assisted in conducting a standard-setting study using the Bookmark method.

Arabic Reading Comprehension Tests

During 2005 and 2006, under contract with a Washington, DC based translation company, SLTI developed an ILR level 3 reading comprehension test of Modern Standard Arabic (MSA). The level 3 passing score was determined through a formal standard-setting study using the Angoff method with US Government-trained Arabic language specialists. Subsequently, SLTI was contracted to develop another Arabic Reading Comprehension test to identify readers at levels 3+ and 4 on the ILR scale. This was followed-up by another standard setting study using the Angoff method. A report was prepared on the cut-scores and their reliability.

New Mexico Public Education Department

During June 2006, SLTI conducted a three-day alignment study of the New Mexico Spanish Reading Standards-Based Assessments for grades 3-9, using the alignment analysis technique and Web Alignment Tool designed by Norm Webb. To carry out the study, SLTI worked with four groups of New Mexico teachers, with each group evaluating the alignment of two grade level tests. SLTI also sent a staff member to Puerto Rico to assist Dr. Webb in carrying out an alignment study of the Puerto Rico mathematics and Spanish language arts assessments with the island’s content standards.