How we do our reviews
State Adopted Products
For materials that have gone through the Texas state adoption process, Learning List publishes the state panel’s detailed verification of the material’s alignment.
Non-State Adopted Products
Learning List publishes a detailed, independent alignment report for materials that have not gone through a state adoption process or that have gone through a state adoption process but for which no detailed state alignment report is publicly
A multi-step process is employed for verifying the product’s alignment to state standards. Each publisher-submitted correlation is verified by Learning List subject matter experts under the supervision of the Chief Academic Officer. Our subject matter experts are current educators with at least five years of teaching experience, curriculum alignment experience, and who are certified to teach the grade and subject of each product they review.
Learning List considers an instructional material “aligned” to a standard only if the content addresses the content, cognitive demand, and context specified in the standard. After completing the alignment verification, we calculate the percentage of standards to which the material is aligned.
Publishers are then afforded an opportunity to preview and comment Learning List’s alignment reports for their materials. For any standard Learning List did not initially find the material aligned to, the publisher may submit additional citations to be reviewed. The additional citations will be reviewed and integrated into the alignment report, and the alignment percentage will be recalculated before the review is published. Publishers’ comments that indicate a difference of opinion about the alignment of certain citations are posted in the alignment report so that subscribers get the benefit of both perspectives.
Publishers may refer to the alignments on LearningList.com in their marketing materials but may not reprint them.
Calculating the Percentage of Standards Addressed
Educators increasingly are using a combination of state-adopted, non-state-adopted and open-education resources. To assist educators in determining the combination of resources that achieves full coverage of the state’s standards, Learning List provides an independent verification of the percentage of standards aligned in each featured instructional material.
Percentage of Texas Essential Knowledge and Skills (TEKS)
To calculate the percentage of TEKS addressed in non-state-adopted and open-education resources, Learning List uses the same methodology that the Texas Education Agency (TEA) uses to calculate the percentage of TEKS addressed in materials submitted for state adoption. That methodology is described below. For each featured state-adopted material, Learning List features TEA’s calculation of the percentage of TEKS addressed.
The percentage of TEKS aligned is the percentage of Student Expectations aligned (number of student expectations aligned/total number of student expectations).
Student expectations generally are broken into component pieces (i.e., breakouts). In some instances, these breakouts are further subdivided into elements, and elements may be further subdivided into sub-elements. In order for a student expectation to be considered aligned, each of the constituent breakouts must be aligned. In order for a breakout to be aligned, each element (if any) must be aligned, and in order for an element to be aligned, each sub-element (if any) must be aligned. Therefore, a material is not considered to be aligned to a student expectation unless the material is aligned to each of the constituent breakouts, elements, and sub-elements of that student expectation.
Percentage of Common Core State Standards (CCSS) Addressed
Learning List employs the same methodology for calculating the percentage of CCSS to which a material is aligned. If a standard has related expectations, the material must be aligned to each expectation in order for the material to be considered aligned to the standard.
Editorial Review Methodology
Learning List’s editorial review protocols and methodology draw from nationally available research on effective curriculum design, established rubrics and criteria for evaluating traditional and online instructional materials (e.g., EQuIP, IMET, iNACOL, and the Publisher’s Criteria), as well as states’ adoption processes and a survey of school district RFPs.
Our reviews are formatted to include the information superintendents, curriculum directors, instructional materials coordinators and district technology directors told us they needed most to distinguish instructional materials. We also worked closely with the leaders of state education agencies to ensure that our reviews respect and complement state adoption processes.
To help educators identify the resources best suited to their students’ needs and abilities, our editorial reviews assess:
- the rigor, focus, coherence and relevance of the instructional content;
- accessibility, adaptions for special populations and supports for students at all levels;
- organization, ease of use and professional development; and
- technology requirements, monitoring tools, and assessments.
We aggregate information from the following sources to produce a factual and descriptive editorial review for each of the instructional materials featured on LearningList.com:
- Educators who have used the resources with students;
- Learning List’s subject matter experts;
- The publisher; and,
- Independent product research.