Updated Action Research Project - Draft

Below is the first draft version of my Action Research Project. I am currently still in progress of completing this project.

Abstract or Executive Summary

Waiting on results to write this part

Title

Determine the Validity and Reliability of Two Universal Screeners for Fifth Grade Students


Introduction / Background (Section One)

Comal ISD currently uses several different screeners across the district to determine the academic needs of our students in reading and math. The screeners used in Comal include: TPRI, DAR, TMSFA, SRI and SMI. As a district we would like to find one universal screener that is user friendly, valid, and preferably on the commissioners list of reading instruments. At this time the SRI and SMI are not on the commissioners list.

The purpose of this study is to determine the most valid and reliable universal screener that can appropriately identify a student who is struggling on the state assessment. I will be looking at the SRI (Scholastic Reading Inventory) and STAR reading screener at the fifth grade level to determine their accuracy in identifying these students. My objective is to identify a universal screener that most accurately identifies students in need of intervention. My overall research question is: Which of the two screeners is most reliable and valid in determining students that need interventions?

The Response to Intervention model or RTI prevention model is designed to help identify struggling and at risk students and provided evidence-based interventions. Currently universal screeners are being used to help determine student strengths and weaknesses and provide appropriate interventions. The two universal screeners I will be looking at is the STAR reading screener and the SRI, Scholastic Reading Inventory.

Literature Review (Section Two)

Universal screeners are the mechanism for targeting students who struggle to learn when provided a scientific, evidence-based general education (Jenkins, Hudson, & Johnson, 2007). This is the first step in the RTI prevention model process. Needed interventions are initiated immediately following the BOY, beginning of year, testing. They are then followed up with a MOY, middle of year, and EOY, end of year test.

One of the most common concerns with universal screeners and identification is the number of false positives or negatives. However the real concern is the false negatives – those students who are not identified as at-risk but are. These students miss out on early intervention needed to help them achieve. Because of these concerns universal screeners will be measured in four areas: sensitivity, specificity, practicality and consequential validity (Jenkins, 2003). Each of these areas helps determine the quality of a universal screener.

To evaluate each screener I will use the formulas for classification accuracy and sensitivity. Both of these formulas take into account all of the possible outcomes. These include true positives (TP’s – correctly identified as at risk), true negatives (TN’s - correctly identified as not at risk), false positive (FP’s – those incorrectly identified as at risk) and false negative (FN’s – those incorrectly identified as not being at risk). To calculate classification accuracy it is the sum of the true positives and negatives divided by the total students screened. This shows the percentage of students who are correctly identified. To calculate sensitivity it is the number of true positives divided by the total number of students at risk. This shows the proportion of students who are risk that have been correctly identified. (Johnson, Pool, & Carter )

The two universal screeners I will be evaluating are the SRI, Scholastic Reading Inventory and the STAR reading. The SRI universal screener is a research-based, computer adaptive reading assessment program. It is a quick and easy program to use and implement. The SRI is great tool to help support a RTI framework. The main disadvantage of the SRI is the fact it is not on the Commissioners list of approved universal screeners for specific grades. This has caused the district to use multiple screeners across the district and grade levels. The STAR reading universal screener is a research-based computer adaptive program. STAR reading uses Item Response Theory which is an item calibration to determine the difficulty of test items in comparison to thousands of students. STAR is also a user friendly program that is quick and easy to implement. The STAR reading program is on the commissioners list of approved universal screeners for specific grades. The biggest disadvantage to the STAR is the bridge from STAR early literacy to STAR reading. We are experiencing problems with kids who are too high for early literacy but too low for reading.

Based on my research the following are my hypothesizes.

SRI will show equal or greater reliability in identifying students needing intervention.
STAR will show equal or greater reliability in identifying students needing intervention.
There is no significant difference in the two screeners in their ability to identify students that need intervention.

Some delineations of the study are: teacher variability, intervention resources, technical support, and student motivation.

Action Research Design (Section Three)

Subjects

My specific research is going to target one campus using the STAR screener and one campus using the SRI screener, both at the fifth grade level. I am choosing to focus on the fifth grade level because fifth grade is an SSI grade. The ELA coordinator felt she needed more specific information about the screener’s reliability so as to provide the best intervention for the fifth grade students. Since my data is from all of the 5th graders at the two campuses ethnicity, economically disadvantage, special needs, and exceptionality will all be contained in the data itself.

Procedures

This specific research is a spin off from a larger research study being conducted by the ELA and math teams at Comal ISD. The overall objective of the larger study is to identify a screener out of several, that best identifies all students who need intervention. The original research project was initiated with the ELA reading and math coordinators and was advanced to the superintendent of curriculum instruction and superintendent for modifications and approval. This study has also been presented to the DEIC for discussion and comments.

Once the need was brought to my attention I began to gather more information. I discussed the needs with the ELA coordinator and began to do research. I also contacted the ELA coordinator requesting fifth grade data for both the SRI and Star screeners. This data included the BOY, MOY, and spring 2012 STAAR data. I expect this data in about three weeks and the ELA coordinator has told me she will give me the data redacting all student information in compliance with the IRB policy.
I was able to work with the ELA and math team at the initial conception of the research. Because the need was so great the plan met no opposition and was heartedly received. As the district got into the research there were opportunities for rethinking some of the data collections and participants. My part of the research is directly link to this positive embracing and the need confirmed by district data for the study.

Data Collection

I will be gathering data from the 5th grade from two schools. One school was using the STAR universal screener and the other school is using the SRI universal screener. Once I receive the data from the BOY, MOY and 2012 STAAR tests I will analyze the results from the screeners to the student achievement on the state assessments. I will also run the formulas for classification accuracy and sensitivity of the screeners.


Findings (Section Four)

No data yet

Conclusions and Recommendations (Section Five)

none

References (Final Section)

Hughes, Charles and Dexter, Douglas D. Universal Screening Within a Responsive-to-Intervention Model. Retrieved February 2, 2013. http://www.rtinetwork.org/learn/research/universal-screening-within-a-rti-model

Jenkins, J. R., Hudson, R. F., & Johnson, E. S. (2007). Screening for at-risk readers in a response to intervention framework. School Psychology Review, 36, 582-600.

Jenkins, J. R. (2003, December). Candidate measures for screening at-risk students. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO. Retrieved February 2, 2013. http://www.nrcld.org/symposium2003/jenkins/index.html.

Johnson, Evelyn S., Pool, Juli, & Carter, Deborah R. Screening for Reading Problems in an RTI Framework. Retrieved February 2, 2013. from http://www.rtinetwork.org/essential/assessment/screening/screening-for-reading-problems-in-an-rti-framework?tmpl=component&print=1

Web Conference Reflection - School Community Relations Class

I was able to attend one web conference during this course. I enjoyed being able to attend. It allowed me to be able to get questions answered and hear what others were thinking. It was also beneficial to hear other information that was presented regarding requirements of the masters program. Many of which I still need to complete. When I can't attend the web conferences I try to read the conversations on-line. Thanks for having them.

New Action Research Topic

Due to changes with administration and needs of the district my supervisor has decided it would be best to change the action research topic. She has asked me to help her with this action research study on the reliability and validity of universal screeners in math and reading. We currently have a universal screener we have used in the past but have the opportunity to use a couple of other screeners at no or little cost. We are currently looking at using three screeners with a possibility of a fourth. We are going to set up a study at the elementary level to determine which screener is best for our district. We are looking at two key aspects. First, we are going to look at the relationship between a universal screener and the accompanying intervention on student performance as measured by the state assessment. Secondly, we are going to look at the validity and reliability of screeners from various vendors in determining appropriate interventions. We will determine the results by looking at how the students performed on the state test last year in comparison to performance this year. We also realize there are several factors that we cannot control: absolute fidelity to the program, teacher variability, technical support, and student motivation. We have spent time researching and picking the screeners, writing and editing the proposal, and dividing the schools with each screener. We have also had several meetings with key members of the research team. At this time schools have received their assignments and are currently screening or waiting for more information.

Action Research Plan

The Purpose Of My Action Research

The purpose of my action research study is to determine what teaching strategies make the biggest impact on state writing scores.  I am interested in researching this area because the state writing scores were the lowest content scores in our district.  This is a real need to investigate and research.  First I will study the research regarding best practices to teach writing.  I also plan to look at teaching strategies from teachers who scored high as well as the teachers who did not.  I will also take into consideration the student data, student demographics, teacher data, teacher surveys, and any other factors that could impact the data. 

How Educational Leaders Might Use Blogs

There are numerous ways that educational leaders could use blogs.  Leaders could use blogs to monitor and share information regarding any action research studies they are facilitating or participating in.  They could also encourage other committees and/or PLC’s throughout the school to use blogs to record any reflections or ideas they are investigating. This would be a great tool to share minutes or main ideas from staff, curriculum, department, and grade-level meetings.  Since the main purpose of a blog is to reflect on current issues, happenings, and concerns; a blog is a fantastic tool leaders could use to engage with their staff, district and community. 

Action Research and Uses

Administrative action research is the systematic process of looking at a problem, concern or issue in an administrator’s practice or school.  Through action research he/she is able to self evaluate issues and develop a plan to change the outcome.  The process of action research includes addressing the problem, collecting and analyzing data, collaborating with colleagues, exploring patterns and developing a plan.  When a plan is in place implementation can begin.  Throughout implementation it is very important to evaluate and monitor progress with regularly planned checkpoints.  The final step of sustaining the plan will begin once the desired results are met.  Throughout this entire process the administrator will monitor and evaluate progress to make sure the process is still on track to reach the final goal.  Action research is a process that can be utilized anytime depending on the issue at hand.  Becoming a leader who uses action research regularly will discover a better awareness of his/her personal daily activities and school wide concerns.  This process of administrative action research allows professional growth as a leader, school, district and community.  Continuing this practice of action research will allow constant growth and awareness of problems or concerns before they become huge issues.  The overall idea is to continually look into how you are performing and what outcomes you are getting. 

 

Action research can be used on a number of different issues or concerns. My supervisor used action research this past school year to evaluate the efficacy of an instructional resource.  I could see administrators using it to look at programs geared toward raising academic performance, examine interventions for discipline issues, investigate improving student motivation, improve parent involvement by gathering data on strategies implemented, and increase school morale through strategic goals, to name a few.