Below is the first draft version of my Action Research Project. I am currently still in progress of completing this project.
Abstract or Executive Summary
Waiting on results to write this part
Title
Determine the Validity and Reliability of Two Universal Screeners for Fifth Grade Students
Introduction / Background (Section One)
Comal ISD currently uses several different screeners across the district to determine the academic needs of our students in reading and math. The screeners used in Comal include: TPRI, DAR, TMSFA, SRI and SMI. As a district we would like to find one universal screener that is user friendly, valid, and preferably on the commissioners list of reading instruments. At this time the SRI and SMI are not on the commissioners list.
The purpose of this study is to determine the most valid and reliable universal screener that can appropriately identify a student who is struggling on the state assessment. I will be looking at the SRI (Scholastic Reading Inventory) and STAR reading screener at the fifth grade level to determine their accuracy in identifying these students. My objective is to identify a universal screener that most accurately identifies students in need of intervention. My overall research question is: Which of the two screeners is most reliable and valid in determining students that need interventions?
The Response to Intervention model or RTI prevention model is designed to help identify struggling and at risk students and provided evidence-based interventions. Currently universal screeners are being used to help determine student strengths and weaknesses and provide appropriate interventions. The two universal screeners I will be looking at is the STAR reading screener and the SRI, Scholastic Reading Inventory.
Literature Review (Section Two)
Universal screeners are the mechanism for targeting students who struggle to learn when provided a scientific, evidence-based general education (Jenkins, Hudson, & Johnson, 2007). This is the first step in the RTI prevention model process. Needed interventions are initiated immediately following the BOY, beginning of year, testing. They are then followed up with a MOY, middle of year, and EOY, end of year test.
One of the most common concerns with universal screeners and identification is the number of false positives or negatives. However the real concern is the false negatives – those students who are not identified as at-risk but are. These students miss out on early intervention needed to help them achieve. Because of these concerns universal screeners will be measured in four areas: sensitivity, specificity, practicality and consequential validity (Jenkins, 2003). Each of these areas helps determine the quality of a universal screener.
To evaluate each screener I will use the formulas for classification accuracy and sensitivity. Both of these formulas take into account all of the possible outcomes. These include true positives (TP’s – correctly identified as at risk), true negatives (TN’s - correctly identified as not at risk), false positive (FP’s – those incorrectly identified as at risk) and false negative (FN’s – those incorrectly identified as not being at risk). To calculate classification accuracy it is the sum of the true positives and negatives divided by the total students screened. This shows the percentage of students who are correctly identified. To calculate sensitivity it is the number of true positives divided by the total number of students at risk. This shows the proportion of students who are risk that have been correctly identified. (Johnson, Pool, & Carter )
The two universal screeners I will be evaluating are the SRI, Scholastic Reading Inventory and the STAR reading. The SRI universal screener is a research-based, computer adaptive reading assessment program. It is a quick and easy program to use and implement. The SRI is great tool to help support a RTI framework. The main disadvantage of the SRI is the fact it is not on the Commissioners list of approved universal screeners for specific grades. This has caused the district to use multiple screeners across the district and grade levels. The STAR reading universal screener is a research-based computer adaptive program. STAR reading uses Item Response Theory which is an item calibration to determine the difficulty of test items in comparison to thousands of students. STAR is also a user friendly program that is quick and easy to implement. The STAR reading program is on the commissioners list of approved universal screeners for specific grades. The biggest disadvantage to the STAR is the bridge from STAR early literacy to STAR reading. We are experiencing problems with kids who are too high for early literacy but too low for reading.
Based on my research the following are my hypothesizes.
SRI will show equal or greater reliability in identifying students needing intervention.
STAR will show equal or greater reliability in identifying students needing intervention.
There is no significant difference in the two screeners in their ability to identify students that need intervention.
Some delineations of the study are: teacher variability, intervention resources, technical support, and student motivation.
Action Research Design (Section Three)
Subjects
My specific research is going to target one campus using the STAR screener and one campus using the SRI screener, both at the fifth grade level. I am choosing to focus on the fifth grade level because fifth grade is an SSI grade. The ELA coordinator felt she needed more specific information about the screener’s reliability so as to provide the best intervention for the fifth grade students. Since my data is from all of the 5th graders at the two campuses ethnicity, economically disadvantage, special needs, and exceptionality will all be contained in the data itself.
Procedures
This specific research is a spin off from a larger research study being conducted by the ELA and math teams at Comal ISD. The overall objective of the larger study is to identify a screener out of several, that best identifies all students who need intervention. The original research project was initiated with the ELA reading and math coordinators and was advanced to the superintendent of curriculum instruction and superintendent for modifications and approval. This study has also been presented to the DEIC for discussion and comments.
Once the need was brought to my attention I began to gather more information. I discussed the needs with the ELA coordinator and began to do research. I also contacted the ELA coordinator requesting fifth grade data for both the SRI and Star screeners. This data included the BOY, MOY, and spring 2012 STAAR data. I expect this data in about three weeks and the ELA coordinator has told me she will give me the data redacting all student information in compliance with the IRB policy.
I was able to work with the ELA and math team at the initial conception of the research. Because the need was so great the plan met no opposition and was heartedly received. As the district got into the research there were opportunities for rethinking some of the data collections and participants. My part of the research is directly link to this positive embracing and the need confirmed by district data for the study.
Data Collection
I will be gathering data from the 5th grade from two schools. One school was using the STAR universal screener and the other school is using the SRI universal screener. Once I receive the data from the BOY, MOY and 2012 STAAR tests I will analyze the results from the screeners to the student achievement on the state assessments. I will also run the formulas for classification accuracy and sensitivity of the screeners.
Findings (Section Four)
No data yet
Conclusions and Recommendations (Section Five)
none
References (Final Section)
Hughes, Charles and Dexter, Douglas D. Universal Screening Within a Responsive-to-Intervention Model. Retrieved February 2, 2013. http://www.rtinetwork.org/learn/research/universal-screening-within-a-rti-model
Jenkins, J. R., Hudson, R. F., & Johnson, E. S. (2007). Screening for at-risk readers in a response to intervention framework. School Psychology Review, 36, 582-600.
Jenkins, J. R. (2003, December). Candidate measures for screening at-risk students. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO. Retrieved February 2, 2013. http://www.nrcld.org/symposium2003/jenkins/index.html.
Johnson, Evelyn S., Pool, Juli, & Carter, Deborah R. Screening for Reading Problems in an RTI Framework. Retrieved February 2, 2013. from http://www.rtinetwork.org/essential/assessment/screening/screening-for-reading-problems-in-an-rti-framework?tmpl=component&print=1
