After former Iowa Governor Terry Branstad halted the implementation of the Smarter Balanced Assessment, an RFP was released to find a new test vendor. The Iowa Department of Education last week announced the winning bidder for new statewide assessments of student progress in English language arts, mathematics, and science that will be first administered in Spring 2019.
The winning bidder was American Institutes for Research (AIR). Their assessment proposal called Independent College and Career Readiness Solution scored highest overall in a competitive-bidding process that followed criteria set by Iowa lawmakers this year when Branstad signed Senate File 240 into law.
For each bid, in compliance with the new state law, reviewers considered alignment with Iowa’s academic standards and federal law, ability to measure student progress and proficiency, costs, the feasibility of implementation for school districts, time required to administer the assessments, and infrastructure and technology needs. Assessments must be available in both paper-and-pencil and computer-based formats.
A review team consisting of school administrators, educators, and Iowa Department of Education staff considered eight proposals from six vendors. Those who submitted bids were ACT Inc., American Institutes for Research, Data Recognition Corporation, Pearson, Questar, and the University of Kansas.
While Iowa now officially drops out of the Smarter Balanced Assessment Consortium I have some concerns. Five of them in fact:
1. AIR’s Connection to Smarter Balanced.
AIR created Smarter Balanced Assessment’s online delivery system for member states to use.
From AIR’s press release on September 6, 2012:
AIR is partnering with the Smarter Balanced Assessment Consortium, a state-led consortium committed to developing tests that use technology to better measure student knowledge and to make tests accessible to all. The Smarter Balanced tests will be delivered online and include innovative items and performance tasks that take advantage of the potential of technology.
AIR will make available an open-source testing platform to enable all states and others to deliver cutting edge tests.
Smarter Balanced is creating an assessment system aligned to the Common Core State Standards to measure mastery of the content students will need to succeed in colleges and careers. Initial tests will reach millions of students in 2013 and 2014.
The Smarter Balanced tests will be adaptive, meaning the difficulty of the questions will adjust to the knowledge level of the student. The test will be standards-based so each student will be assessed on the full range of the content they should be taught. Those taking the test will have access to a broad array of tools and embedded supports that will enable every student to clearly and fairly show what they know and can do.
AIR is one of the nation’s leading providers of academic assessments, and has been providing accessible, standards-based statewide adaptive testing services since 2007. AIR counts among its partners three of the Consortium’s Governing States—Delaware, Hawaii and Oregon—with whom AIR collaborated to develop some of the innovations that the Consortium is now building upon. The innovations include new ways to enable a test to adapt to student performance, as well as a breakthrough way to provide online Braille tests to improve testing for blind students.
They also ran the initial pilot of Smarter Balanced Assessments in 25 states.
Update: Will Iowa go the way of Indiana?
AIR, a 66-year-old not-for-profit based in Washington, D.C., worked on tests for Smarter Balanced, one of the exam groups affiliated with Common Core. In its business proposal, AIR says it will use its bank of test questions from Smarter Balanced to build Indiana’s English and math tests.
“This approach — using Smarter Balanced as an item bank, rather than a fixed assessment — offers Indiana a hybrid between an off-the-shelf product and a custom test,” the proposal stated.
2. Some blamed AIR for testing disruptions in Nevada.
From Education Week:
A vendor blamed for Nevada’s recent online testing disruptions has pointed the finger at a rival assessment organization, which it said was slow to deliver technology critical to the successful administration of the exam.
The vendor, Measured Progress, a Dover, N.H.-based nonprofit, was taken to task this week by state officials, who said the organization was in breach of contract, along with the Smarter Balanced Assessment Consortium, a group of states that designed Nevada’s common core-aligned tests.
But Measured Progress, without naming names, pointed to “multiple and delayed deliveries of the source code” that the organization said was necessary to ensure that the Nevada assessment could be sufficiently tested so it would perform as expected when large numbers of test takers were on it.
The organization that is under contract to provide the open-source test delivery system is the American Institutes for Research, or AIR, a Washington-based testing vendor, according to Smarter Balanced officials.
This lead to Smarter Balanced ordering a review.
3. AIR Comes With a Hefty Price Tag
The Cedar Rapids Gazette reported that AIR’s selection would come with a hefty price tag.
Students in grades three through eight are to take the statewide tests, which will be available in both paper-and-pencil and computer-based formats. Either high school sophomores or juniors also will take the tests, Iowa Department of Education spokeswoman Staci Hupp said.
The new assessments are expected to cost $31 million over a period of five and a half years, Hupp said. Of the eight assessment proposals considered, she said the American Institutes for Research’s bid had the second-lowest price tag.
“How to pay for the test is going to be a question for the legislature to decide because the bill that directed us to run this procedure didn’t specify that,” Hupp said.
An initial contract is expected to run for 20 months, with an annual renewal option for four years. The initial contract will cost $8 million, Hupp said.
Smarter Balanced Assessments, projected by the Assessment Task Force in their 2014 report, would cost the state $22.50 per student per summative assessment. The annual cost was expected to be $5,632,740. This cost projection did not factor in the cost for online or IT support. Smarter Balanced also did not include science. The AIR assessment does.
The current Iowa Assessments’ cost to the district is $4.25 to $6.25 per student for paper-and-pencil tests or $13.00 per student for online tests for basic scoring and reporting services (plus optional costs for additional reports and other services). The state paid for approximately $2.25 per student ($575,000) for additional data management and reporting. Iowa Testing Programs (ITP) did not charge for item development. ITP submitted a bid in 2014 for the Next Generation Iowa Assessments but did not submit a bid this time around.
The state will spend less for AIR than they would have with Smarter Balanced and a new science assessment. The change still represents a jump in what school districts are paying for assessments.
4. What is AIR’s specialty – social engineering or academic testing?
First, what agenda does AIR promote? This 2014 piece by psychologist and economist Dean Kalahar at The American Thinker is disturbing. Kalahar calls out AIR’s agenda when they were developing Florida’s new Common Core-aligned assessment:
AIR’s “National Clearinghouse on Supportive School Discipline (NCSSD) supports educational practitioners in their efforts to transform the conditions for learning as well as harsh, exclusionary and disproportionate disciplinary practices in our nation’s schools.”
Their goal: “Changing the culture of communities and schools in which certain groups of students — racial and ethnic minorities; lesbian, gay, bisexual, transgender and questioning (LGBTQ) youth; and/or students with emotional, behavioral and cognitive disabilities — face disproportionate school disciplinary action.”
And he continues:
AIR believes that “Institutionalized racism, heterosexism, and biases can interfere with providing culturally and linguistically competent services.” This involves “respecting the individual, their family, culture, traditions, rituals, and preferred language;” and “Integrating cultural knowledge, values, and beliefs into services, policies, and research.” AIR is focused on “Multiculturalism” and “diversity” in “regulations, rules, policies, performance appraisal measures, and budgets/funding.” In short, AIR aims to fight white privilege, traditional American cultural values, and the English language. Sounds like an organization with a distorted worldview and an ax to grind at the expense of children.
AIR promotes “Response to Intervention (RTI)” practices which is education-speak for Common Core’s universal “screening” of children and vast data collection practices.
AIR writes “Race to the Top” guidance papers to foster implementation of President Obama’s nationalized education agenda.
AIR champions “College and career readiness,” which is euphemistic terminology for Common Core’s agenda of centralized control over children’s career paths.
AIR is a driver of “universal pre-school” which is aligned with progressive’s cradle to grave philosophy.
AIR advances the “Value added measures in education,” an empirically discredited methodology of teacher evaluation.
AIR focuses on controversial “social and emotional intelligence” theories that distort achievement differences to blur the line of competency.
AIR works directly with the Center for American Progress, John Podesta’s overarching progressive think tank. Even AIR CEO David Myers, who holds a Ph.D. in Sociology from Washington State University and earns a reported $566,304 yearly salary, believes “in the use of experimental designs in education settings.”
Karen Effrem notes how AIR describes themselves:
What is very important to know but was not reported in either story is that AIR describes itself as “one of the world’s largest behavioral and social science research and evaluation organizations.” This is more evidence that psychological teaching and testing is part of Common Core and the standards, by whatever name they are deceptively being labeled that are taught and tested in Florida, despite the concerns raised in the governor’s executive order.
In addition, if AIR is developing tests for SBAC, then one has to wonder if they signed a similar agreement to PARCC and SBAC to give the US Department of Education individual student test data for “research” and “linking.”
AIR created Utah’s test (SAGE), watch the video below from a 13-year-old in Utah after taking the test.
https://youtu.be/YJEGP1qWzy0
Then a Utah parent shares about her student’s experience.
https://youtu.be/MqMA3_kpoZI
5. AIR’s assessment in Utah has validity concerns.
AIR’s Common Core-aligned assessment in Utah named SAGE saw complaints about its validity. Florida who hired AIR to develop their test paid for a validity report on Utah’s test.
Alpine Testing was the only company that applied to perform the validity study for Florida. Once awarded the contract, they teamed with EdCount, the founder of which had previously worked for AIR.
So, what we have is a questionably independent group stating that this test should not be used for individual students, but it’s ok for the aggregate data to be used for schools and teacher evaluations. If this sounds absurd, it’s because it is. If it’s been shown that this test isn’t good for students, why would we be comfortable using it for the grading or funding of our schools and teachers? The sum of individual bad data can’t give us good data. Nor should we expect it to.
Dr. Gary Thompson, a child psychologist in Utah, wrote an extensive piece about the validity problems with Utah’s SAGE assessment.