WASL Quagmire – Read CURE’s response to the Washington State Institute for Public Policy (WSIPP).
April 19, 2010
Response to Consultation Request
for WSIPP K-12 Student Assessment Study
Submitted by
Citizens United for Responsible Education (CURE)
cure@curewashington.org
August 2006
Click to skip to the following sections:
Contents:
- Introduction
- Current state standards
- Current state assessment
- Remedies
- Test suggestions
- Challenges to implementation
- Legislation needed
- Conclusion
- Appendix
Before we analyze the alternative assessment options, we must inspect the foundational premises upon which the WASL rests. Under this current Education Reform law, the state sets and then assesses educational standards which ALL children must meet. In theory, this would be a good plan if the curriculum equipped students with the facts, information, and academic skills needed to handle a variety of life situations and vocations.
However, education reformers take a different approach. They claim that the current standards are “high standards,” and that the curriculum teaches our children “higher order thinking skills” instead of “lower order thinking skills.” This is misleading, because “lower order skills” are foundational concrete facts and skills such as reading, writing, and arithmetic. The teaching of “process”—what is involved in “higher order thinking skills”—is an elusive goal, since we each process our information differently.
WASL scorers attempt to “get inside the student’s head” and divine the thought process. We respectfully submit that while it is the duty of an education system to impart information and assess whether students have learned what has been taught, it is not the purview of the state to pass judgment on a person’s unique thought process. In most cases, the correctness of an answer automatically attests to the accuracy of the process. We recommend that the schools teach information and skills, and allow children to process that information in freedom.
Since the WASL is supposed to measure the state’s academic standards, we should examine the standards themselves before we examine the assessment, to see if they are appropriate and worth being assessed.
The Essential Academic Learning Requirements (EALRs) are not age-appropriate. Dr. Donald Orlich, professor at Washington State University, has done extensive research on the WASL. He has found that the EALRs are not developmentally appropriate. In his report, “A Critical Report to Interested Citizens of the State of Washington,” from March 15, 2005, the summary states:
- The Grade 5 Science WASL exceeds the intellectual level of the vast majority of grade 5 children and appears to be an 8th grade examination.
- While not specifically examined, English language learners will find this assessment to be virtually impossible to pass due to needed vocabulary skills.
- The Grade Level Expectations (GLEs) for Grade 5 science are developmentally inappropriate. The GLEs drive the WASL; thus the test is developmentally inappropriate.
- The 7th Grade Math WASL is in all reality a 9th grade test.
- Test items do not progress from relatively easy to more difficult. They simply appear with no logical sequence. Standardized tests begin with easy items and move to more difficult ones.
- A total of 9 math concepts are tested. Yet, 185 math General Level Expectations are listed for Grade 7.
- Reading and writing are most critical for student success. One could hypothesize a very high correlation between these two skills and success in the Science and Mathematics WASL.
- Reviewing the GLEs for grade 7 and 10 reveals parallel entries. That is, the grade 7 GLEs are almost identical in many cases to those of grade 10.
To obtain the full report, please contact Dr. Orlich at:
SMEEC
PO Box 644237
Pullman, WA 99164-4237
(509) 335-4844 FAX (509) 335-7389
The Washington Essential Academic Learning Requirements received an F-rating from the Fordham foundation, in reading and math.
The Fordham Foundation made these statements about Washington’s reading standards:
“…a large number are processes, strategies, or pretentious statements with no clear academic objective—reductionist in goal and often incomprehensible.”
“Washington should eliminate those standards for the English language arts that seem to serve as expressions of some person’s or group’s particular political and social goals.”
(The State of State English Standards 2005, by the Fordham Foundation, pages 69, 70)
The Fordham Foundation also made these remarks about Washington’s math standards:
“Overall, the Washington standards are poorly written and needlessly voluminous……Other standards have little to do with mathematics….The standards devoted to problem-solving are of especially low quality…..In fact, these sections focus on talking about solving problems, rather than actually solving them.
“…such problems risk miseducating students to believe that mathematics itself is ambiguous, a matter of opinion, and without definite answers.”
(The State of State Math Standards 2005, by the Fordham Foundation, page 113-115)
The Fordham Foundation’s criticisms are supported by an examination of the math EALRs. None of the Math EALR items ever states “finding the correct answer” as an academic requirement. The requirements are to “…use, understand, apply, define, communicate, etc.” Requirement #2 states “solve problems,” but an inspection of the sub-requirements shows that the student is NOT required to find the correct answer, but merely to “investigate”, “formulate”, and “construct solutions.” (See math EALRs included at end of this survey.)
Therefore, many of the math questions on the WASL do not require the student to calculate an answer. Meanwhile, in Singapore, Japan, and other countries which perform well on international tests, math standards do require finding correct answers.
In theory, a criterion-referenced assessment is said to be more valuable than a multiple choice test because it more closely measures what the student is taught. In Washington, however, the reverse is happening. The WASL contains ambiguous, process-oriented questions, and the curriculum is being changed to match the assessment. Thus the WASL is converting the entire curriculum into one of process, with decreasing emphasis on knowledge and information.
Any state educational assessment should be reliable, valid, cost effective, and academically effective. The WASL has none of these attributes.
The WASL is not reliable.
There have been many news articles about scoring mistakes made by Pearson (which scores the WASL) as well as by other testing companies. A Times-Dispatch Article from October 27, 2005 stated that Pearson, “in 1999—then a subcontractor under Riverside Publishing—along with Riverside incorrectly scored 410,000 student essays in Washington State.” The Olympian Online, in a March 27, 2006 article, discussed the costs, stating that the “410,000 WASL essays were scored incorrectly and later regraded at a cost of $600,000….”
No dollar amount can describe the cost of the psychological damage to the students.
Even without actual scoring mistakes, the WASL’s subjective nature makes it difficult to grade, even in math. Two graders could give two different scores to the same paper. About 40% of the math WASL questions are open-ended, leaving room for interpretation and subjective judgments.
The OSPI’s publications containing released WASL items include explanations of the rubrics. These explanations show that correct answers given without explanations and incorrect answers given with explanations may receive the same score. Thus math is being redefined as a “gray” subject rather than a subject of right and wrong answers. It is our position that correct answers should receive higher scores than incorrect answers. Parents do not pay taxes and send their children to school intending that they learn there is no difference between right or wrong answers.
The reading and writing WASLs are even more difficult to score. Concerning essay scoring in general, it was found in a 1961 study by Paul Diederich and his colleagues, as reported by E. D. Hirsch in The Schools We Need and Why We Don’t Have Them :
“When 300 papers were graded by fifty-three graders (a total of 15,900 readings), more than one third of the papers received every possible grade. That is, 101 of the 300 papers received all nine grades: A, A-, B+, B, B-, C+, C, C-, and D. ….94 percent [of the papers] received either seven, eight, or nine different grades; and no essay received less than five different grades from fifty-three readers.“ 100% of the 300 essays received at least 5 different grades!
(Emphasis added.)
WASL results can be controlled by:
- Changing the cut scores for “mastery”
- Changing the grading rubrics
- Choosing easier/harder questions from the item “bank”
The Office of the Superintendent of Public Instruction may do all three without input or oversight from outside parties.
Taking into account the scoring mistakes, the subjectivity of the open-ended responses, and the various ways scoring can be administratively controlled, it is understandable that the WASL is not viewed as a reliable assessment.
The WASL is not valid.
The WASL does not closely match the Essential Academic Learning Requirements. In one instance when a mismatch was brought to the attention of the OSPI, the OSPI changed the Learning Requirements to match the assessment, not the other way around. This further proves our observation that the assessment is driving the curriculum, not assessing it.
When Washington’s education reform law, ESHB1209, was passed, it included this provision:
“After a determination is made by the state board of education that the high school assessment system has been implemented and that it is sufficiently reliable and valid, successful completion of the high school assessment shall lead to a certificate of mastery……” RCW 28A.655.060(3)I.
State Board of Education’s (SBE) May 21, 2003, “Final Report of the Certificate of Mastery Study Committee” stated the WASL was not ready, with respect to validity and reliability, to be used for graduation. There have been no reports since then, describing any changes made to the WASL to render it valid and reliable. Yet passing the WASL was made a graduation requirement.
After the SBE issued its May 2003 report, this provision, RCW 28A.655.060(3)I, was repealed. The SBE’s report has also been removed from the internet, and has not been replaced with any other report which describes any corrections made to the validity and reliability problems.
Past reports of validity have focused on whether the WASL assesses the EALRs. In addition, now that the WASL is being used as a diploma-denying instrument, there should also be studies available on the WASL’s accuracy in measuring graduation-readiness and success after graduation. However, no such studies are available.
The WASL was originally intended as a tool to improve student learning. Instead, it is now being used to measure school compliance with federal guidelines, as well as being a de facto permit for students’ future opportunities. Its validity for its original purpose, and for its new purposes, has yet to be established.
The WASL is not cost effective.
WASL costs could be more easily determined and streamlined if the expenses were reported in one “WASL” category instead of being imbedded under various categories. Currently, it is difficult to discern actual costs, and asking legislators, school administrators, and OSPI officials elicits many different answers. We have pieced together information from various sources.
The WASL costs about $73 per student for four parts, while the Iowa Test of Basic Skills (ITBS) costs less than $3 per student. This is 2003 cost information from a budget report provided by Jennifer Priddy of OSPI. This is just the cost of administration and does not include the late school start times for the non-WASL students at the high school level, the creation of motivational multi-media presentations, the additional time and money spent on WASL rewards for students who passed, and remedial time for students who failed. Furthermore, remedial class time is primarily spent on test-taking strategies, not on improving math or language arts skills. This ITBS cost data came from the actual 2003 contracts with Riverside (which had the ITBS contract) and was divided by the number of students participating.
After the WASL is administered there are still more costs. Tests must be sorted, inventoried, stored, counted before and after every session, and transported. In some cases, a guard is even posted—a further cost.
Beyond these direct costs, there is the value of the education that is lost when time that was formerly spent on learning is instead spent on WASL preparation. Economists call this “opportunity cost,” for the lost opportunity for education. By establishing how many hours are devoted to WASL preparation instead of real learning, we can calculate a dollar value for this lost opportunity, using per student, per hour costs. Informal estimates indicate that the opportunity cost of the WASL dwarfs its other costs.
Here is an example, using 2004-2005 data. There were 75,231 fourth graders, 80,688 seventh graders, and 83,315 tenth graders for a total of 239,234 WASL students. Total state expenditures that year, according to the OSPI website, were $7,724,234,122, spent to educate a total of 980,716 students. This amounts to an average amount of $7876.12 spent per pupil, not including any local contributions to augment state costs. Thus a conservative total cost of educating fourth, seventh, and tenth graders was ($7876.12 per pupil) X (239,234 pupils) = $1,884,235,692. Estimating conservatively, if just a cumulative equivalent of ONE day per week were spent preparing for the WASL, this would be ($1,884,235,692)/5 = $376,847,138. However parents are well aware that the WASL dominates the curriculum. They have complained that the entire curriculum is now nothing but WASL preparation. It is reasonable to estimate the real opportunity cost losses at well over $400,000,000 each year. Opportunity cost losses will further increase now that there must be testing in all the elementary and middle school grades.
Add to this the opportunity costs lost when teacher training is spent on understanding and “de-mystifying” the WASL, instead of on substantive training for subjects such as the teaching of math, reading, writing, and helping special needs students.
The WASL is not academically effective.
Turning the curriculum into all WASL-prep is not a good educational practice. Many other enrichment subjects have been eliminated or given much less time. With much of the budget spent on assessment, districts have had to eliminate other things such as librarian positions, as in the Federal Way school district.
The WASL introduces practices which would be unacceptable in other circumstances. A Seattle Times article revealed in March 17, 2006,
“The state’s education office, to the dismay of some teachers, recently announced that making up facts is acceptable when writing nonfiction, persuasive essays on the WASL — something that on class assignments would mean a failing grade.
‘Statistics in a WASL paper can be made up by you, the writer!’ says a PowerPoint presentation that the Office of Superintendent of Public Instruction (OSPI) created to be used this summer for students who fail the WASL this spring. And, a little later: ‘On the WASL, you can invent an important expert and have that person say something to bolster your opinion.’
…. after hearing about teachers’ concerns, Willhoft said OSPI plans to change the PowerPoint to remove the language about inventing facts and experts. But it will continue to verbally encourage teachers to tell students they can make up facts — but just for the WASL.” [At the time of this article, Joe Willhoft was interim assistant superintendent for assessment and research at OSPI.]
Although the data show WASL scores have risen since the inception of this assessment, what exactly is being taught and measured? An examination of Iowa test scores over the same period shows that student achievement of basic skills has remained flat. (See attached chart of high school results.) The students have consistently scored around the 57th or 58th percentile on the Iowa Test of Basic Skills, a normed, standardized test. The WASL does not meet the scientific criteria for a standardized test. Do the rising WASL scores show that students are getting used to taking the WASL or that WASL results are being controlled? There is no clear indication of improved acquisition of knowledge. Unfortunately, the Legislature has now eliminated the requirement that the Iowa test be given, and comparisons are no longer possible.
Replace the current standards with more concrete standards.
The Fordham Foundation has rated every state’s standards. Since Washington’s standards received an “F” grade from the Fordham Foundation, it would be instructive to study and copy standards from states which received “A” grades.
Replace the WASL correspondingly.
Some educators claim that criterion-referenced assessments such as the WASL are superior to norm-referenced tests such as the Iowa test, Stanford Achievement Test, and others. They complain that norm-referenced tests do not accurately measure what was taught, that they may be culturally biased, there is time pressure, and correct answers can be obtained by guessing. Those conclusions may be drawn if one confuses the purpose of norm-referenced, standardized tests versus criterion-referenced assessments. An assessment with many open-ended questions might be useful for discerning whether a law student knows how to apply the law to a tricky case. However, to test a student’s aptitude in multiplication, a test must contain multiplication questions that have correct answers. To evaluate if a student can punctuate and spell, a test must contain punctuation and spelling questions which have definite correct answers.
The WASL is a poor criterion-referenced assessment that Washington is using in place of a good norm-referenced, standardized test. Such “off-the-shelf” standardized tests from nationally known companies not only more accurately measure basic skills, but also provide timely results, specifying student strengths and weaknesses in specific skills such as punctuation, spelling, long division, etc. This is not the case with WASL results.
The nationally-known, standardized tests are also written to avoid “cultural biases.”
Some companies even provide large print and Braille versions.
The timed nature of the standardized test is unavoidable and ensures that all students take the test under the same conditions. Students have failed parts of the WASL because they missed the original WASL day and had to take the make-up test in a shorter time period than that allotted to the rest of the class. Thus the time variability of the WASL causes unfairness, and further proves its failure to meet a criterion of test standardization.
Perhaps in some cases multiple choice answers can be arrived at by guessing. On the other hand, many students who actually know the right answer to a WASL question may not be able to express it to the satisfaction of a scorer—who may be having a bad day— or a person who does not understand a question may be able to write enough paragraphs to dazzle an unfocused scorer. Therefore, the WASL is not automatically superior on that account.
As stated above, a good assessment should be reliable, valid, cost effective, and academically effective.
Given that no test is perfect, it is vital to choose one that is as reliable, valid, and effective as possible.
1. Any assessment must be reliable.
A norm-referenced, objective test can be machine-graded and will result in the same score no matter how many times it is graded.
- As mentioned before, these normed and standardized tests are already adjusted to eliminate “cultural bias.”
- In mathematics, a test that emphasizes numbers will not be as confusing to the English language learners. Currently, the Math WASL correlates more closely with the Reading WASL than with other math assessments.
- In 2006, Maine started using the SAT (Scholastic Aptitude Test) in the 11th grade. Perhaps Washington can do the same for the high school level test. http://www.state.me.us/education/clips/07-ela/ela17.htm
2. Any assessment must be valid.
If the Essential Academic Learning Requirements are replaced with standards that are clear, practical, and developmentally appropriate, most traditional, content-rich tests will match.
- Subjects covered would more closely match the skills needed for higher education, vocational training, or job situations.
- For 11th graders, using the SAT (Scholastic Aptitude Test) would cause dovetailing of skills needed for college with skills covered in high school.
3. Any assessment must be cost effective.
- Norm-referenced, machine graded tests are much less expensive than open-ended tests that must be scored by a human scorer.
- As mentioned above, in 2003, the ITBS costs less than $3 per person while the WASL costs $73 per person for four parts.
- An off-the-shelf test will already have been analyzed for fairness, cultural appropriateness, etc., by the company, saving our state the expense of this analysis.
4. Any assessment must be academically effective.
- As mentioned above, norm-referenced, standardized tests give timely and detailed feedback. Unlike the WASL, score reports for standardized, norm-referenced tests show students, teachers, and parents how many items were missed in each sub-area such as capitalization, punctuation, word usage, etc. This will allow teachers and parents to work on a student’s specific strengths and weaknesses.
- Within classrooms, the teacher can supplement curriculum with essay assignments in which students explain their answers. Teachers are certificated, and know their students—not the case with $11/hour WASL scorers at Pearson Measurements. Then reasoning and communication skills will still be practiced, but just not placed on the WASL. Teachers will recognize deficiencies immediately, and can give students quick remedial help.
- There will be no need for students to learn to make up facts, information, and experts, as happens now with the WASL.
Nationally known standardized tests.
There are at least four nationally known, norm-referenced, true standardized tests which could replace the WASL. The Iowa Test which was used in Washington until 2006 is just one example. They test the acquisition of basic skills and are machine-scored so that subjective judgments are not involved. Some of these have accommodations for sight-impaired, blind, and deaf students. They are all evaluated to eliminate cultural bias, and they are reasonable in cost. Another advantage is that results can be obtained in a timely manner and are reported in a way that clearly shows students’ strengths and weaknesses. The older versions of the test are better assessments for academic skills, but even the more recent versions are valid, reliable, and effective.
PASS test
The PASS test—the Personalized Achievement Summary System—is for grades 3-8. It is normed but is not a timed test. The low-stress nature of this test makes it good for special education students and English language learners. Students are first given a placement test to determine their level; then they are given a test that corresponds to their level. This works well with students who may be at different levels in different subjects, for example, strong in math, but weak in language arts–often the case for many English language learners. It is developed by Hewitt Research Foundation in Washougal, Washington. More information can be found at: Hewitt Homeschooling Resources.
Maine’s assessment system
Maine originally tested students in the 4th, 8th, and 11th grades, and in 2006, it began using the College Board’s Scholastic Aptitude Test (SAT) for their 11th graders. This may be an option for Washington’s high school students. Maine officials stated the SAT measures their state standards in all but a few areas, and adding a short supplemental test will cover those areas— statistics, probability, and data analysis.
Maine not only pays the SAT fee ($41.50) for each student, but also purchases the SAT online preparation course and makes it available for all high school students. In this way, lower-income students who could not afford expensive SAT preparation classes would have an equal opportunity to take and pass the SAT. The SAT preparation program identifies academic weaknesses and provides remediation before the exam.
Parents have been happy because the SAT takes place on one Saturday, instead of all week long, and schedule disruptions are minimized. That Saturday is considered an official school day, and those students are given a free day off later, to compensate.
Maine’s testing service, (Measured Progress of Dover, New Hampshire) which develops their test for the lower grades, determines the cut scores for the four levels of proficiency and also converts the SAT data into reports for compliance with NCLB. Maine does not use the eleventh grade assessment for high stakes. It is reported that Maine students are motivated and take this exam very seriously because it opens doors to future opportunities. Next year, Maine tenth graders will be taking the PSAT. Good scores on the PSAT can lead to National Merit Scholarships – more positive motivation.
Even though the SAT is not as academically rigorous as it once was, it is still a good option.
VI. Challenges to implementation
There will be several groups who will not be in favor of eliminating the WASL:
- State legislators and school executives who prefer not to look outside of the ESHB1209 educational structure.
- State and local administrators who have invested much time and money into the WASL system don’t want more changes.
- School district personnel who have just spent thousands of dollars, buying new texts to align with the “F-rated” EALRs.
- A State Board of Education which, after its recent reconfiguration, no longer represents the districts or the parents, since only 5 of its 16 members are elected.
- Legislation may be needed to change the EALRs as currently written.
- The process to change EALRs in the future needs to be improved through legislation. If the State Board of Education can be converted to a truly elected body, the SBE would be an appropriate check on any small changes the OSPI wished to make in the EALRs. For large-scale changes to the EALRs, it may be appropriate for the Legislature to give approval. Some oversight must be required.
- Make SBE a fully elected board. A School Board is supposed to represent the citizens, or at least districts, but currently only 5 out of 16 board members are elected.
- Remove WASL as a graduation requirement. It is not required under the federal No Child Left Behind legislation (NCLB). The state has made the WASL a graduation requirement under the mistaken notion that the students would then “take it more seriously.” However, administrators did not consider that the past low scores were earnestly and honestly earned, and that the students truly did not understand the WASL questions. Forcing students to repeatedly take and fail the WASL, an unreliable, invalid assessment, is cruel.
- Investigate the cost of federal dollars. The cost of compliance with NCLB may be more than the federal dollars we receive. Consider rejecting federal money.
The Institute’s task is to “explore options to augment the current system of assessments…” Unfortunately, the current WASL is not reliable, valid, cost effective, or academically effective. Furthermore, augmenting the WASL will not solve educational problems unless the EALRs upon which it is based are first rewritten, and a more rigorous, content-based standard is adopted.
Therefore, we recommend that the EALRs be replaced, and that the WASL and any alternative or other assessment be a normed, standardized test of academic knowledge that is valid, reliable, cost effective, academically effective, and unbiased. A normed but untimed test such as the PASS test may be reasonable for students with special circumstances.
Prompt results with good diagnostic reports should be available to improve students’ academic weaknesses. This would bring it into compliance with the law, ESHB 1209, as originally written, with respect to its use as an educational tool:
Section 202 (3)(b)(ii) “The assessment system shall be designed so that the results under the assessment system are used by educators as tools to evaluate instructional practices, and to initiate appropriate educational support for students who have not mastered the essential academic learning requirements at the appropriate periods in the student’s educational development.”
We further recommend that passing the assessment not be a graduation requirement. Earning the required Carnegie Units should continue to be the graduation requirement in the absence of an exit test that is valid, reliable, cost effective, unbiased, and useful for improving academic learning.
Since the goal of the WSIPP is to find options to augment the WASL, it would be a good idea for the Institute to first thoroughly examine a current WASL for each of the grades. In that way, members of the Institute will know what they are attempting to augment, and how various options might complement it. It would be best to look at an actual WASL, and not just released items. Otherwise the committee will be augmenting something it has never seen. It would also be useful to compare a WASL score report for parents with a score report from a standardized test such as the Iowa Test, to see how much more useful a normed, standardized test is in showing all the student’s areas of strengths and weaknesses. Since cost is one of the Institute’s concerns, it would also be useful to find out the actual cost of the WASL as it currently stands.
The WASL system is currently hurting all students, but does the most harm to economically disadvantaged students and English language learners. A return to the traditional basic skills would be best for all students, their families, and the state.
Math EALRs
1. The student understands and applies the concepts and procedures of mathematics.
To meet this standard, the student will:
1.1 understand and apply concepts and procedures from number sense
number and numeration, computation, and estimation
1.2 understand and apply concepts and procedures from measurement
attributes and dimensions, approximation and precision, and systems and tools
1.3 understand and apply concepts and procedures from geometric sense
shape and dimension, and relationships and transformations
1.4 understand and apply concepts and procedures from probability and statistics
probability, statistics, and prediction and inference
1.5 understand and apply concepts and procedures from algebraic sense
relations and representations, and operations
2. The student uses mathematics to define and solve problems.
To meet this standard, the student will:
2.1 investigate situations
by searching for patterns and exploring a variety of approaches
2.2 formulate questions and define the problem
2.3 construct solutions
by choosing the necessary information and using the appropriate mathematical tools
3. The student uses mathematical reasoning.
To meet this standard, the student will:
3.1 analyze information
from a variety of sources; use models, known facts, patterns and relationships to validate thinking
3.2 predict results and make inferences and make conjectures based on analysis of problem situations
3.3 draw conclusions and verify results
support mathematical arguments, justify results, and check for reasonableness of solutions
4. The student communicates knowledge and understanding in both everyday and mathematical language.
To meet this standard, the student will:
4.1 gather information
read, listen, and observe to access and extract mathematical information
4.2 organize and interpret information
4.3 represent and share information
share, explain, and defend mathematical ideas using terms, language, charts, and graphs that can be clearly understood by a variety of audiences
5. The student understands how mathematical ideas connect within mathematics, to other subject areas, and to real-life situations.
To meet this standard, the student will:
5.1 relate concepts and procedures within mathematics
recognize relationships among mathematical ideas and topics
5.2 relate mathematical concepts and procedures to other disciplines
identify and apply mathematical thinking and notation in other subject areas
5.3 relate mathematical concepts and procedures to real-life situations
understand the connections between mathematics and problem solving skills used every day at work and at home
THE WASL: A CRITICAL REPORT TO INTERESTED CITIZENS OF THE STATE OF WASHINGTON
Dr. Donald C. Orlich
March 15, 2005
Executive Summary
Conclusions. This report is an analysis of the 2004 Grade 5 Science WASL and the Grade 7 Mathematics WASL using criteria from developmental psychology and the scales of the National Assessment of Educational Progress (NAEP). Inferences from this study may be applicable to the entire battery of WASL assessments.
- The Grade 5 Science WASL exceeds the intellectual level of the vast majority of grade 5 children and appears to be an 8th grade examination.
- While not specifically examined, English language learners will find this assessment to be virtually impossible to pass due to needed vocabulary skills.
- The Grade Level Expectations (GLE’s) for Grade 5 science are developmentally inappropriate. The GLE’s drive the WASL; thus the test is developmentally inappropriate.
- The 7th Grade Math WASL is in all reality a 9th grade test.
- Test items do not progress from relatively easy to more difficult. They simply appear with no logical sequence. Standardized tests begin with easy items and move to more difficult ones.
- A total of 9 math concepts are tested. Yet, 185 math General Level Expectations are listed for Grade 7.
- Reading and writing are most critical for student success. One could hypothesize a very high correlation between these two skills and success in the Science and Mathematics WASL.
- Reviewing the GLE’s for grade 7 and 10 reveals parallel entries. That is, the grade 7 GLEs are almost identical in many cases to those of grade 10.
Policy Implications. There are instructional and policy implications associated with the findings and conclusions of this analysis.
First, if the WASL tests are advanced beyond the mental cognition of grade 5 and 7 pupils, then for most children failure will be the ultimate end, regardless of instructional techniques used.
Second, what psychological impact will failing an inappropriate science and math WASL have on students and their ultimate attitudes towards science and math, and schooling in general?
Third, one may predict litigation by concerned parents and child advocacy groups against the State of Washington.
Fourth, scoring errors have been found nationally in virtually all mandatory high-stakes tests. These have led to class action law suits. For example, the state of Minnesota paid out approximately $12 million to students and/or their parents due to scoring errors.
Fifth, the legislature is approaching fiscal irresponsibility or is not practicing fiscal accountability by continuing to fund the exorbitant WASL. With the State of Washington viewing at least a $2.2 Billion budget short fall, the massive $200,000,000 OSPI budget for school reform must be challenged.
Sixth, the legislature should commission an outside research organization to verify or refute this study.
Tags: Fordham, Orlich, WASL, assessments, cost effective, high stakes, reliability, standards, validity
Response to Consultation Request
for WSIPP K-12 Student Assessment Study
Submitted by
Citizens United for Responsible Education (CURE)
cure@curewashington.org
August 2006
Click to skip to the following sections:
Contents:
- Introduction
- Current state standards
- Current state assessment
- Remedies
- Test suggestions
- Challenges to implementation
- Legislation needed
- Conclusion
- Appendix
Before we analyze the alternative assessment options, we must inspect the foundational premises upon which the WASL rests. Under this current Education Reform law, the state sets and then assesses educational standards which ALL children must meet. In theory, this would be a good plan if the curriculum equipped students with the facts, information, and academic skills needed to handle a variety of life situations and vocations.
However, education reformers take a different approach. They claim that the current standards are “high standards,” and that the curriculum teaches our children “higher order thinking skills” instead of “lower order thinking skills.” This is misleading, because “lower order skills” are foundational concrete facts and skills such as reading, writing, and arithmetic. The teaching of “process”—what is involved in “higher order thinking skills”—is an elusive goal, since we each process our information differently.
WASL scorers attempt to “get inside the student’s head” and divine the thought process. We respectfully submit that while it is the duty of an education system to impart information and assess whether students have learned what has been taught, it is not the purview of the state to pass judgment on a person’s unique thought process. In most cases, the correctness of an answer automatically attests to the accuracy of the process. We recommend that the schools teach information and skills, and allow children to process that information in freedom.
Since the WASL is supposed to measure the state’s academic standards, we should examine the standards themselves before we examine the assessment, to see if they are appropriate and worth being assessed.
The Essential Academic Learning Requirements (EALRs) are not age-appropriate. Dr. Donald Orlich, professor at Washington State University, has done extensive research on the WASL. He has found that the EALRs are not developmentally appropriate. In his report, “A Critical Report to Interested Citizens of the State of Washington,” from March 15, 2005, the summary states:
- The Grade 5 Science WASL exceeds the intellectual level of the vast majority of grade 5 children and appears to be an 8th grade examination.
- While not specifically examined, English language learners will find this assessment to be virtually impossible to pass due to needed vocabulary skills.
- The Grade Level Expectations (GLEs) for Grade 5 science are developmentally inappropriate. The GLEs drive the WASL; thus the test is developmentally inappropriate.
- The 7th Grade Math WASL is in all reality a 9th grade test.
- Test items do not progress from relatively easy to more difficult. They simply appear with no logical sequence. Standardized tests begin with easy items and move to more difficult ones.
- A total of 9 math concepts are tested. Yet, 185 math General Level Expectations are listed for Grade 7.
- Reading and writing are most critical for student success. One could hypothesize a very high correlation between these two skills and success in the Science and Mathematics WASL.
- Reviewing the GLEs for grade 7 and 10 reveals parallel entries. That is, the grade 7 GLEs are almost identical in many cases to those of grade 10.
To obtain the full report, please contact Dr. Orlich at:
SMEEC
PO Box 644237
Pullman, WA 99164-4237
(509) 335-4844 FAX (509) 335-7389
The Washington Essential Academic Learning Requirements received an F-rating from the Fordham foundation, in reading and math.
The Fordham Foundation made these statements about Washington’s reading standards:
“…a large number are processes, strategies, or pretentious statements with no clear academic objective—reductionist in goal and often incomprehensible.”
“Washington should eliminate those standards for the English language arts that seem to serve as expressions of some person’s or group’s particular political and social goals.”
(The State of State English Standards 2005, by the Fordham Foundation, pages 69, 70)
The Fordham Foundation also made these remarks about Washington’s math standards:
“Overall, the Washington standards are poorly written and needlessly voluminous……Other standards have little to do with mathematics….The standards devoted to problem-solving are of especially low quality…..In fact, these sections focus on talking about solving problems, rather than actually solving them.
“…such problems risk miseducating students to believe that mathematics itself is ambiguous, a matter of opinion, and without definite answers.”
(The State of State Math Standards 2005, by the Fordham Foundation, page 113-115)
The Fordham Foundation’s criticisms are supported by an examination of the math EALRs. None of the Math EALR items ever states “finding the correct answer” as an academic requirement. The requirements are to “…use, understand, apply, define, communicate, etc.” Requirement #2 states “solve problems,” but an inspection of the sub-requirements shows that the student is NOT required to find the correct answer, but merely to “investigate”, “formulate”, and “construct solutions.” (See math EALRs included at end of this survey.)
Therefore, many of the math questions on the WASL do not require the student to calculate an answer. Meanwhile, in Singapore, Japan, and other countries which perform well on international tests, math standards do require finding correct answers.
In theory, a criterion-referenced assessment is said to be more valuable than a multiple choice test because it more closely measures what the student is taught. In Washington, however, the reverse is happening. The WASL contains ambiguous, process-oriented questions, and the curriculum is being changed to match the assessment. Thus the WASL is converting the entire curriculum into one of process, with decreasing emphasis on knowledge and information.
Any state educational assessment should be reliable, valid, cost effective, and academically effective. The WASL has none of these attributes.
The WASL is not reliable.
There have been many news articles about scoring mistakes made by Pearson (which scores the WASL) as well as by other testing companies. A Times-Dispatch Article from October 27, 2005 stated that Pearson, “in 1999—then a subcontractor under Riverside Publishing—along with Riverside incorrectly scored 410,000 student essays in Washington State.” The Olympian Online, in a March 27, 2006 article, discussed the costs, stating that the “410,000 WASL essays were scored incorrectly and later regraded at a cost of $600,000….”
No dollar amount can describe the cost of the psychological damage to the students.
Even without actual scoring mistakes, the WASL’s subjective nature makes it difficult to grade, even in math. Two graders could give two different scores to the same paper. About 40% of the math WASL questions are open-ended, leaving room for interpretation and subjective judgments.
The OSPI’s publications containing released WASL items include explanations of the rubrics. These explanations show that correct answers given without explanations and incorrect answers given with explanations may receive the same score. Thus math is being redefined as a “gray” subject rather than a subject of right and wrong answers. It is our position that correct answers should receive higher scores than incorrect answers. Parents do not pay taxes and send their children to school intending that they learn there is no difference between right or wrong answers.
The reading and writing WASLs are even more difficult to score. Concerning essay scoring in general, it was found in a 1961 study by Paul Diederich and his colleagues, as reported by E. D. Hirsch in The Schools We Need and Why We Don’t Have Them :
“When 300 papers were graded by fifty-three graders (a total of 15,900 readings), more than one third of the papers received every possible grade. That is, 101 of the 300 papers received all nine grades: A, A-, B+, B, B-, C+, C, C-, and D. ….94 percent [of the papers] received either seven, eight, or nine different grades; and no essay received less than five different grades from fifty-three readers.“ 100% of the 300 essays received at least 5 different grades!
(Emphasis added.)
WASL results can be controlled by:
- Changing the cut scores for “mastery”
- Changing the grading rubrics
- Choosing easier/harder questions from the item “bank”
The Office of the Superintendent of Public Instruction may do all three without input or oversight from outside parties.
Taking into account the scoring mistakes, the subjectivity of the open-ended responses, and the various ways scoring can be administratively controlled, it is understandable that the WASL is not viewed as a reliable assessment.
The WASL is not valid.
The WASL does not closely match the Essential Academic Learning Requirements. In one instance when a mismatch was brought to the attention of the OSPI, the OSPI changed the Learning Requirements to match the assessment, not the other way around. This further proves our observation that the assessment is driving the curriculum, not assessing it.
When Washington’s education reform law, ESHB1209, was passed, it included this provision:
“After a determination is made by the state board of education that the high school assessment system has been implemented and that it is sufficiently reliable and valid, successful completion of the high school assessment shall lead to a certificate of mastery……” RCW 28A.655.060(3)I.
State Board of Education’s (SBE) May 21, 2003, “Final Report of the Certificate of Mastery Study Committee” stated the WASL was not ready, with respect to validity and reliability, to be used for graduation. There have been no reports since then, describing any changes made to the WASL to render it valid and reliable. Yet passing the WASL was made a graduation requirement.
After the SBE issued its May 2003 report, this provision, RCW 28A.655.060(3)I, was repealed. The SBE’s report has also been removed from the internet, and has not been replaced with any other report which describes any corrections made to the validity and reliability problems.
Past reports of validity have focused on whether the WASL assesses the EALRs. In addition, now that the WASL is being used as a diploma-denying instrument, there should also be studies available on the WASL’s accuracy in measuring graduation-readiness and success after graduation. However, no such studies are available.
The WASL was originally intended as a tool to improve student learning. Instead, it is now being used to measure school compliance with federal guidelines, as well as being a de facto permit for students’ future opportunities. Its validity for its original purpose, and for its new purposes, has yet to be established.
The WASL is not cost effective.
WASL costs could be more easily determined and streamlined if the expenses were reported in one “WASL” category instead of being imbedded under various categories. Currently, it is difficult to discern actual costs, and asking legislators, school administrators, and OSPI officials elicits many different answers. We have pieced together information from various sources.
The WASL costs about $73 per student for four parts, while the Iowa Test of Basic Skills (ITBS) costs less than $3 per student. This is 2003 cost information from a budget report provided by Jennifer Priddy of OSPI. This is just the cost of administration and does not include the late school start times for the non-WASL students at the high school level, the creation of motivational multi-media presentations, the additional time and money spent on WASL rewards for students who passed, and remedial time for students who failed. Furthermore, remedial class time is primarily spent on test-taking strategies, not on improving math or language arts skills. This ITBS cost data came from the actual 2003 contracts with Riverside (which had the ITBS contract) and was divided by the number of students participating.
After the WASL is administered there are still more costs. Tests must be sorted, inventoried, stored, counted before and after every session, and transported. In some cases, a guard is even posted—a further cost.
Beyond these direct costs, there is the value of the education that is lost when time that was formerly spent on learning is instead spent on WASL preparation. Economists call this “opportunity cost,” for the lost opportunity for education. By establishing how many hours are devoted to WASL preparation instead of real learning, we can calculate a dollar value for this lost opportunity, using per student, per hour costs. Informal estimates indicate that the opportunity cost of the WASL dwarfs its other costs.
Here is an example, using 2004-2005 data. There were 75,231 fourth graders, 80,688 seventh graders, and 83,315 tenth graders for a total of 239,234 WASL students. Total state expenditures that year, according to the OSPI website, were $7,724,234,122, spent to educate a total of 980,716 students. This amounts to an average amount of $7876.12 spent per pupil, not including any local contributions to augment state costs. Thus a conservative total cost of educating fourth, seventh, and tenth graders was ($7876.12 per pupil) X (239,234 pupils) = $1,884,235,692. Estimating conservatively, if just a cumulative equivalent of ONE day per week were spent preparing for the WASL, this would be ($1,884,235,692)/5 = $376,847,138. However parents are well aware that the WASL dominates the curriculum. They have complained that the entire curriculum is now nothing but WASL preparation. It is reasonable to estimate the real opportunity cost losses at well over $400,000,000 each year. Opportunity cost losses will further increase now that there must be testing in all the elementary and middle school grades.
Add to this the opportunity costs lost when teacher training is spent on understanding and “de-mystifying” the WASL, instead of on substantive training for subjects such as the teaching of math, reading, writing, and helping special needs students.
The WASL is not academically effective.
Turning the curriculum into all WASL-prep is not a good educational practice. Many other enrichment subjects have been eliminated or given much less time. With much of the budget spent on assessment, districts have had to eliminate other things such as librarian positions, as in the Federal Way school district.
The WASL introduces practices which would be unacceptable in other circumstances. A Seattle Times article revealed in March 17, 2006,
“The state’s education office, to the dismay of some teachers, recently announced that making up facts is acceptable when writing nonfiction, persuasive essays on the WASL — something that on class assignments would mean a failing grade.
‘Statistics in a WASL paper can be made up by you, the writer!’ says a PowerPoint presentation that the Office of Superintendent of Public Instruction (OSPI) created to be used this summer for students who fail the WASL this spring. And, a little later: ‘On the WASL, you can invent an important expert and have that person say something to bolster your opinion.’
…. after hearing about teachers’ concerns, Willhoft said OSPI plans to change the PowerPoint to remove the language about inventing facts and experts. But it will continue to verbally encourage teachers to tell students they can make up facts — but just for the WASL.” [At the time of this article, Joe Willhoft was interim assistant superintendent for assessment and research at OSPI.]
Although the data show WASL scores have risen since the inception of this assessment, what exactly is being taught and measured? An examination of Iowa test scores over the same period shows that student achievement of basic skills has remained flat. (See attached chart of high school results.) The students have consistently scored around the 57th or 58th percentile on the Iowa Test of Basic Skills, a normed, standardized test. The WASL does not meet the scientific criteria for a standardized test. Do the rising WASL scores show that students are getting used to taking the WASL or that WASL results are being controlled? There is no clear indication of improved acquisition of knowledge. Unfortunately, the Legislature has now eliminated the requirement that the Iowa test be given, and comparisons are no longer possible.
Replace the current standards with more concrete standards.
The Fordham Foundation has rated every state’s standards. Since Washington’s standards received an “F” grade from the Fordham Foundation, it would be instructive to study and copy standards from states which received “A” grades.
Replace the WASL correspondingly.
Some educators claim that criterion-referenced assessments such as the WASL are superior to norm-referenced tests such as the Iowa test, Stanford Achievement Test, and others. They complain that norm-referenced tests do not accurately measure what was taught, that they may be culturally biased, there is time pressure, and correct answers can be obtained by guessing. Those conclusions may be drawn if one confuses the purpose of norm-referenced, standardized tests versus criterion-referenced assessments. An assessment with many open-ended questions might be useful for discerning whether a law student knows how to apply the law to a tricky case. However, to test a student’s aptitude in multiplication, a test must contain multiplication questions that have correct answers. To evaluate if a student can punctuate and spell, a test must contain punctuation and spelling questions which have definite correct answers.
The WASL is a poor criterion-referenced assessment that Washington is using in place of a good norm-referenced, standardized test. Such “off-the-shelf” standardized tests from nationally known companies not only more accurately measure basic skills, but also provide timely results, specifying student strengths and weaknesses in specific skills such as punctuation, spelling, long division, etc. This is not the case with WASL results.
The nationally-known, standardized tests are also written to avoid “cultural biases.”
Some companies even provide large print and Braille versions.
The timed nature of the standardized test is unavoidable and ensures that all students take the test under the same conditions. Students have failed parts of the WASL because they missed the original WASL day and had to take the make-up test in a shorter time period than that allotted to the rest of the class. Thus the time variability of the WASL causes unfairness, and further proves its failure to meet a criterion of test standardization.
Perhaps in some cases multiple choice answers can be arrived at by guessing. On the other hand, many students who actually know the right answer to a WASL question may not be able to express it to the satisfaction of a scorer—who may be having a bad day— or a person who does not understand a question may be able to write enough paragraphs to dazzle an unfocused scorer. Therefore, the WASL is not automatically superior on that account.
As stated above, a good assessment should be reliable, valid, cost effective, and academically effective.
Given that no test is perfect, it is vital to choose one that is as reliable, valid, and effective as possible.
1. Any assessment must be reliable.
A norm-referenced, objective test can be machine-graded and will result in the same score no matter how many times it is graded.
- As mentioned before, these normed and standardized tests are already adjusted to eliminate “cultural bias.”
- In mathematics, a test that emphasizes numbers will not be as confusing to the English language learners. Currently, the Math WASL correlates more closely with the Reading WASL than with other math assessments.
- In 2006, Maine started using the SAT (Scholastic Aptitude Test) in the 11th grade. Perhaps Washington can do the same for the high school level test. http://www.state.me.us/education/clips/07-ela/ela17.htm
2. Any assessment must be valid.
If the Essential Academic Learning Requirements are replaced with standards that are clear, practical, and developmentally appropriate, most traditional, content-rich tests will match.
- Subjects covered would more closely match the skills needed for higher education, vocational training, or job situations.
- For 11th graders, using the SAT (Scholastic Aptitude Test) would cause dovetailing of skills needed for college with skills covered in high school.
3. Any assessment must be cost effective.
- Norm-referenced, machine graded tests are much less expensive than open-ended tests that must be scored by a human scorer.
- As mentioned above, in 2003, the ITBS costs less than $3 per person while the WASL costs $73 per person for four parts.
- An off-the-shelf test will already have been analyzed for fairness, cultural appropriateness, etc., by the company, saving our state the expense of this analysis.
4. Any assessment must be academically effective.
- As mentioned above, norm-referenced, standardized tests give timely and detailed feedback. Unlike the WASL, score reports for standardized, norm-referenced tests show students, teachers, and parents how many items were missed in each sub-area such as capitalization, punctuation, word usage, etc. This will allow teachers and parents to work on a student’s specific strengths and weaknesses.
- Within classrooms, the teacher can supplement curriculum with essay assignments in which students explain their answers. Teachers are certificated, and know their students—not the case with $11/hour WASL scorers at Pearson Measurements. Then reasoning and communication skills will still be practiced, but just not placed on the WASL. Teachers will recognize deficiencies immediately, and can give students quick remedial help.
- There will be no need for students to learn to make up facts, information, and experts, as happens now with the WASL.
Nationally known standardized tests.
There are at least four nationally known, norm-referenced, true standardized tests which could replace the WASL. The Iowa Test which was used in Washington until 2006 is just one example. They test the acquisition of basic skills and are machine-scored so that subjective judgments are not involved. Some of these have accommodations for sight-impaired, blind, and deaf students. They are all evaluated to eliminate cultural bias, and they are reasonable in cost. Another advantage is that results can be obtained in a timely manner and are reported in a way that clearly shows students’ strengths and weaknesses. The older versions of the test are better assessments for academic skills, but even the more recent versions are valid, reliable, and effective.
PASS test
The PASS test—the Personalized Achievement Summary System—is for grades 3-8. It is normed but is not a timed test. The low-stress nature of this test makes it good for special education students and English language learners. Students are first given a placement test to determine their level; then they are given a test that corresponds to their level. This works well with students who may be at different levels in different subjects, for example, strong in math, but weak in language arts–often the case for many English language learners. It is developed by Hewitt Research Foundation in Washougal, Washington. More information can be found at: Hewitt Homeschooling Resources.
Maine’s assessment system
Maine originally tested students in the 4th, 8th, and 11th grades, and in 2006, it began using the College Board’s Scholastic Aptitude Test (SAT) for their 11th graders. This may be an option for Washington’s high school students. Maine officials stated the SAT measures their state standards in all but a few areas, and adding a short supplemental test will cover those areas— statistics, probability, and data analysis.
Maine not only pays the SAT fee ($41.50) for each student, but also purchases the SAT online preparation course and makes it available for all high school students. In this way, lower-income students who could not afford expensive SAT preparation classes would have an equal opportunity to take and pass the SAT. The SAT preparation program identifies academic weaknesses and provides remediation before the exam.
Parents have been happy because the SAT takes place on one Saturday, instead of all week long, and schedule disruptions are minimized. That Saturday is considered an official school day, and those students are given a free day off later, to compensate.
Maine’s testing service, (Measured Progress of Dover, New Hampshire) which develops their test for the lower grades, determines the cut scores for the four levels of proficiency and also converts the SAT data into reports for compliance with NCLB. Maine does not use the eleventh grade assessment for high stakes. It is reported that Maine students are motivated and take this exam very seriously because it opens doors to future opportunities. Next year, Maine tenth graders will be taking the PSAT. Good scores on the PSAT can lead to National Merit Scholarships – more positive motivation.
Even though the SAT is not as academically rigorous as it once was, it is still a good option.
VI. Challenges to implementation
There will be several groups who will not be in favor of eliminating the WASL:
- State legislators and school executives who prefer not to look outside of the ESHB1209 educational structure.
- State and local administrators who have invested much time and money into the WASL system don’t want more changes.
- School district personnel who have just spent thousands of dollars, buying new texts to align with the “F-rated” EALRs.
- A State Board of Education which, after its recent reconfiguration, no longer represents the districts or the parents, since only 5 of its 16 members are elected.
- Legislation may be needed to change the EALRs as currently written.
- The process to change EALRs in the future needs to be improved through legislation. If the State Board of Education can be converted to a truly elected body, the SBE would be an appropriate check on any small changes the OSPI wished to make in the EALRs. For large-scale changes to the EALRs, it may be appropriate for the Legislature to give approval. Some oversight must be required.
- Make SBE a fully elected board. A School Board is supposed to represent the citizens, or at least districts, but currently only 5 out of 16 board members are elected.
- Remove WASL as a graduation requirement. It is not required under the federal No Child Left Behind legislation (NCLB). The state has made the WASL a graduation requirement under the mistaken notion that the students would then “take it more seriously.” However, administrators did not consider that the past low scores were earnestly and honestly earned, and that the students truly did not understand the WASL questions. Forcing students to repeatedly take and fail the WASL, an unreliable, invalid assessment, is cruel.
- Investigate the cost of federal dollars. The cost of compliance with NCLB may be more than the federal dollars we receive. Consider rejecting federal money.
The Institute’s task is to “explore options to augment the current system of assessments…” Unfortunately, the current WASL is not reliable, valid, cost effective, or academically effective. Furthermore, augmenting the WASL will not solve educational problems unless the EALRs upon which it is based are first rewritten, and a more rigorous, content-based standard is adopted.
Therefore, we recommend that the EALRs be replaced, and that the WASL and any alternative or other assessment be a normed, standardized test of academic knowledge that is valid, reliable, cost effective, academically effective, and unbiased. A normed but untimed test such as the PASS test may be reasonable for students with special circumstances.
Prompt results with good diagnostic reports should be available to improve students’ academic weaknesses. This would bring it into compliance with the law, ESHB 1209, as originally written, with respect to its use as an educational tool:
Section 202 (3)(b)(ii) “The assessment system shall be designed so that the results under the assessment system are used by educators as tools to evaluate instructional practices, and to initiate appropriate educational support for students who have not mastered the essential academic learning requirements at the appropriate periods in the student’s educational development.”
We further recommend that passing the assessment not be a graduation requirement. Earning the required Carnegie Units should continue to be the graduation requirement in the absence of an exit test that is valid, reliable, cost effective, unbiased, and useful for improving academic learning.
Since the goal of the WSIPP is to find options to augment the WASL, it would be a good idea for the Institute to first thoroughly examine a current WASL for each of the grades. In that way, members of the Institute will know what they are attempting to augment, and how various options might complement it. It would be best to look at an actual WASL, and not just released items. Otherwise the committee will be augmenting something it has never seen. It would also be useful to compare a WASL score report for parents with a score report from a standardized test such as the Iowa Test, to see how much more useful a normed, standardized test is in showing all the student’s areas of strengths and weaknesses. Since cost is one of the Institute’s concerns, it would also be useful to find out the actual cost of the WASL as it currently stands.
The WASL system is currently hurting all students, but does the most harm to economically disadvantaged students and English language learners. A return to the traditional basic skills would be best for all students, their families, and the state.
Math EALRs
1. The student understands and applies the concepts and procedures of mathematics.
To meet this standard, the student will:
1.1 understand and apply concepts and procedures from number sense
number and numeration, computation, and estimation
1.2 understand and apply concepts and procedures from measurement
attributes and dimensions, approximation and precision, and systems and tools
1.3 understand and apply concepts and procedures from geometric sense
shape and dimension, and relationships and transformations
1.4 understand and apply concepts and procedures from probability and statistics
probability, statistics, and prediction and inference
1.5 understand and apply concepts and procedures from algebraic sense
relations and representations, and operations
2. The student uses mathematics to define and solve problems.
To meet this standard, the student will:
2.1 investigate situations
by searching for patterns and exploring a variety of approaches
2.2 formulate questions and define the problem
2.3 construct solutions
by choosing the necessary information and using the appropriate mathematical tools
3. The student uses mathematical reasoning.
To meet this standard, the student will:
3.1 analyze information
from a variety of sources; use models, known facts, patterns and relationships to validate thinking
3.2 predict results and make inferences and make conjectures based on analysis of problem situations
3.3 draw conclusions and verify results
support mathematical arguments, justify results, and check for reasonableness of solutions
4. The student communicates knowledge and understanding in both everyday and mathematical language.
To meet this standard, the student will:
4.1 gather information
read, listen, and observe to access and extract mathematical information
4.2 organize and interpret information
4.3 represent and share information
share, explain, and defend mathematical ideas using terms, language, charts, and graphs that can be clearly understood by a variety of audiences
5. The student understands how mathematical ideas connect within mathematics, to other subject areas, and to real-life situations.
To meet this standard, the student will:
5.1 relate concepts and procedures within mathematics
recognize relationships among mathematical ideas and topics
5.2 relate mathematical concepts and procedures to other disciplines
identify and apply mathematical thinking and notation in other subject areas
5.3 relate mathematical concepts and procedures to real-life situations
understand the connections between mathematics and problem solving skills used every day at work and at home
THE WASL: A CRITICAL REPORT TO INTERESTED CITIZENS OF THE STATE OF WASHINGTON
Dr. Donald C. Orlich
March 15, 2005
Executive Summary
Conclusions. This report is an analysis of the 2004 Grade 5 Science WASL and the Grade 7 Mathematics WASL using criteria from developmental psychology and the scales of the National Assessment of Educational Progress (NAEP). Inferences from this study may be applicable to the entire battery of WASL assessments.
- The Grade 5 Science WASL exceeds the intellectual level of the vast majority of grade 5 children and appears to be an 8th grade examination.
- While not specifically examined, English language learners will find this assessment to be virtually impossible to pass due to needed vocabulary skills.
- The Grade Level Expectations (GLE’s) for Grade 5 science are developmentally inappropriate. The GLE’s drive the WASL; thus the test is developmentally inappropriate.
- The 7th Grade Math WASL is in all reality a 9th grade test.
- Test items do not progress from relatively easy to more difficult. They simply appear with no logical sequence. Standardized tests begin with easy items and move to more difficult ones.
- A total of 9 math concepts are tested. Yet, 185 math General Level Expectations are listed for Grade 7.
- Reading and writing are most critical for student success. One could hypothesize a very high correlation between these two skills and success in the Science and Mathematics WASL.
- Reviewing the GLE’s for grade 7 and 10 reveals parallel entries. That is, the grade 7 GLEs are almost identical in many cases to those of grade 10.
Policy Implications. There are instructional and policy implications associated with the findings and conclusions of this analysis.
First, if the WASL tests are advanced beyond the mental cognition of grade 5 and 7 pupils, then for most children failure will be the ultimate end, regardless of instructional techniques used.
Second, what psychological impact will failing an inappropriate science and math WASL have on students and their ultimate attitudes towards science and math, and schooling in general?
Third, one may predict litigation by concerned parents and child advocacy groups against the State of Washington.
Fourth, scoring errors have been found nationally in virtually all mandatory high-stakes tests. These have led to class action law suits. For example, the state of Minnesota paid out approximately $12 million to students and/or their parents due to scoring errors.
Fifth, the legislature is approaching fiscal irresponsibility or is not practicing fiscal accountability by continuing to fund the exorbitant WASL. With the State of Washington viewing at least a $2.2 Billion budget short fall, the massive $200,000,000 OSPI budget for school reform must be challenged.
Sixth, the legislature should commission an outside research organization to verify or refute this study.
Tags: Fordham, Orlich, WASL, assessments, cost effective, high stakes, reliability, standards, validity