Category: Teaching Social Studies

I first encountered Cornell format note taking in a college education class for teaching reading. I used it with my advanced French classes somewhat, but it became one of the cornerstone activities of my social studies classes beginning around 2006.

Cornell notes is a process that encourages developing reading skills, especially for informational text. It provides a study guide for later, although in truth few of my students used that. In my own experience, this method stimulates long-term memory. I believe this is because to complete the task one returns to the information at different levels of abstraction from text to outline to questions and finally to abstract of the whole. The repetition and organized structure of the information promotes that encoding into memory. In addition, it makes a good class activity: upon completion, students can ask each other their questions in a round-robin or pairs format.

The two informal studies below I conducted in 2013 and 2014 to examine the effects of this method on my students’ progress in social studies. Cornell notes became one of two options students had for processing their reading assignments for each unit. The other was summarizing, an equally effective skill. Consistently, about half my students preferred this method.

Originally posted May 2013 and February, 2014.

Innovation has an app now for students to compose Cornell Notes online! And the AI grading assistant can help you score the notes!

May, 2013

A Study of a Reading-Note Taking Task as  Interim Examination Improvement Strategy

“Interim examination” refers to a regularly occurring examination measuring all course content since the  start of the course. They are given at regular intervals as a progress monitoring method. They should be highly  reliable indicators of achievement in the course (such as being highly predictive of performance on a  standardized test) and teachers ought to be able to use the data to make decisions about instruction. A point  worth emphasizing about the interim examination is that it is a test that spirals: each successive examination  tests the content knowledge of the preceding tests and what had been taught since.

Forty-five students in grade seven through nine social studies at Schroon Lake Central School took the  second interim examination in January 2013. Results for some classes were disappointing. An instructional plan  was devised to improve student performance by the April interim examination. The most important aspect of  this plan was a reading & note taking task. Secondarily, there was some increased exposure to domain-specific  vocabulary. 

The effort appears to have been successful. 17% more students passed the third interim examination  from the second. The mean score went up 6%. The probability that the improvement was not due to random  chance or other variables is 83%.

The Note Taking Task 

The note taking task that was intended to boost student performance had two components: notes from  textbook and notes from lecture. Notes had to be taken in Cornell Note Taking format. Cornell format training  has been regularly included in the courses, including training at the start of quarter 3 on using Bloom’s  Taxonomy to create higher level questions on the notes. The note taking task is graded as a “high order task”  (high order tasks account for 65% of a student’s GPA in the course). Cornell Note Taking is a note taking  technique well supported in research¹. Students have two full class periods to begin the text note taking and  then additional working periods when they may opt to do that. They have twelve days to complete the task as  this is the time a topic usually runs. 

Notes from textbook could come from any of three sources, designated as “below”, “at”, or “above” grade level. Grade level difficulty level was determined using Lexile and gauged by the Common Core State  Standards grade level reading expectations. Students self-select for difficulty level in consultation with me. The  amount of reading ranged from 8-12 pages.

Students doing the standard curriculum normally have 1-2 persuasive composition quizzes and 2  expository composition quizzes in each topic. The lecture included some information and media presentations  intended as background or to reinforce key ideas as well as the direct answer to the composition quizzes. Notes  required from lecture were limited to those aspects of the teacher presentation series that answered specific quiz questions. A modified lecture notes task is optional for students who are not sufficiently able to take notes.  They get a copy of the presentation materials and add notes and create questions as for Cornell notes. The  maximum score on this is 76 owing to the reduced workload.

Student Performance on the Note Taking Tasks 

There were two notes tasks in the third quarter. The average score on the notes task was 70, the median  85. Around a quarter failed the notes task each time. Around half of the people who failed the average of the  notes tasks failed interim three. The average score on the notes task was bore a moderately high correlation to  year-to-date GPA in the course (0.70).

Twenty-seven students responded to a survey in which they were asked how well they like the addition  of reading-note taking to their classroom tasks. 75% responded favorably. Prior to this change, assigned reading  tasks were few. Save for grade nine, who had one short reading task per week as homework, students could get  the information they needed to pass the quizzes elsewhere other than text – including studying the quizzes of  students who took the quiz before them. The amount of regular reading in class had become far too limited. My  focus on performance on content knowledge quizzes and on writing took me too far afield of reading for a while.

February, 2014

TOPICAL READING ASSIGNMENT USING CORNELL NOTE TAKING

For each topic of study over the year from February to February 2013-2014, students in social studies grades seven through nine at a small, rural school (N=~50) were assigned to use Cornell note taking for their assigned textbook
chapter readings. The practice was initiated as a response to weak performance of some groups on the 2013 midterm examination.

Students are assigned ten pages of traditional textbook reading associated with the current topic of study. They  may choose from three levels of text: a fourth-grade text, a grade-level text, and an advanced level text set at two  grade levels higher. Providing reading material close to students’ independent reading levels gives them  meaningful access to the information and support for continued reading growth (Allington, 2009). Students have  two 45-minute class periods to work on the assignment and are expected to complete at least five pages per class  period (this is more than double the time it takes the teacher to do the task). This assignment occurs before  teacher lecture and is intended to support student learning by providing the basic groundwork information of the  topic. 

Students are trained in the Cornell note taking format (Paulk, 2014). Using a form provided by the teacher,  students create an informal or formal outline of the most important top two layers of detail from the source text in  their own words (Marzano, 2001). Next, students create questions to go with the information they recorded.  Students are trained in a basic version of Bloom’s Taxonomy for the development of questions and are encouraged  to devise questions and the analysis and evaluation levels in support of long-term memory of the information.  Finally, students are to construct an abstract of each page of notes at the bottom, summarizing the main idea of  the whole page in one or two sentences. Students are graded on the quality of their notes (Figure 2).The task is  due at the end of the topic, usually around two calendar weeks later. Students have additional “working days”  after the teacher lecture series, some of which they may dedicate to completing whatever was not yet done of the  reading task. 

Students are assigned the Cornell note taking method because of the strong supporting research (Figure 1). Research indicates answering questions on text to be least effective for supporting reading comprehension (Graham, 2010). Cornell note taking supports higher level thinking such as application, synthesis, and analysis (Jacobs, 2008). Note taking is one of the “most powerful skills students can cultivate” by providing “students with  tools for identifying and understanding the most important aspects of what they are learning.” (Marzano, 2001). It supports encoding the information for long term recall more effectively than guided notes and questionnaires (Jacobs, 2008). Note taking is known to be an effective strategy “if it entails attention focusing and processing in a  way compatible with the demands of the criterion task.” (Armbruster, 1984) In effective note taking, research  suggests, happens when “students failed to take notes in a manner that elicited sufficiently deep or thorough  processing.” (Armbruster, 1984)

REASONS TO CONSIDER EXAMINING THIS TASK

Informal feedback from students shows the task is generally disliked. The two periods are not maintained strictly  as silent working periods, though distraction is generally minimal. Weaker students are observed to be often off  task. Examination of work accomplished throughout the period indicates some weaker students complete only a  page during the whole time. The completion rate for this task only averages 80% in each topic September-January  2013-2014 grades seven through nine (N=54). Increasingly, this task is coming in late and poorly done with the  mean score at only 72. The lack of sustained attention to task during the class periods allotted for this task likely  decreases the effectiveness of the task, especially memory of the information (Armbruster, 1984).

WHAT DIFFERENCE DOES PERFORMANCE ON THE READING TASK MAKE?

Five students in the sample who had a passing average for the reading tasks assigned in the 2013-2014 school year to date failed an interim examination¹.

Eighteen of fifty-four students in the sample (33%) have a failing (below 65) average for the reading tasks. This  includes scores of zero assigned for incomplete tasks. Half of the students who have a failing average for the  reading tasks failed an interim exam. Five (9%) failed both interim examinations and four (7%) failed one of two  interim examinations. 

Only nine of eighteen students with a failing average on the reading task were able to pass both interim exams. 

“Interim examinations” are ten-week tests of knowledge of course content going back to the start of the school  year.

“Interim examinations” are ten-week tests of knowledge of course content going back to the start of the school  year.

The reading task score measures how well students extracted the “study-worthy” ideas from the source text and  prepared this content for learning. In this sample it was a weak predictor of performance on both the topic final  test (correlation is 0.419) and the interim examination (correlation is 0.334). This stands to reason, since the  measurements are for different things. Final tests and interim examinations are measures of knowledge of  content. 

For the 16^th topic of study in grade eight, the task was set up as a “test”. Students were given 30 minutes to  complete 5 pages. Students who needed more time received it, though a timer was left obvious and the room  remained silent. Students commented that they felt they got a lot done in the more disciplined atmosphere. I am  now assured that the class has completed the requisite reading assignment to understand the upcoming lessons  and that the task was carried out in the most meaningful way possible. 

Since the pandemic, learning from video has become more and more applied in secondary classrooms. In some quarters there were concerns that the demands of learning from video are different than the consumption of video for entertainment that most students engaged in such that student engagement with video lessons would likely be shallow. This shallowing hypothesis has not been consistently supported in the research, however (P. Delgado · Ø. Anmarkrud · V. Avila · L. Altamura · S. M. Chireac · A. Pérez · L. Salmerón).

It is certainly true that students viewing a video that is more than some six minutes long may find their mind wandering, but there are effective methods to teach students that skill set necessary to learn from video and to maximize the benefit.

Effective use of video as an educational tool is enhanced when instructors consider three elements: how to manage cognitive load of the video; how to maximize student engagement with the video; and how to promote active learning from the video.

Brame

1. The Innovation “Etude” Maximizes Active Learning

Innovation’s Etude is an app that maximizes active learning from video. I used this myself extensively with an 11th grade US History class in the 2021-2022 school year. The Etude is a research-supported application that takes a little time to create, but once done is there for your students year after year. Studies show that students who complete video-based learning tasks with students that include interpolated questions performed significantly better on subsequent tests of the material and reported less mind wandering that those who watched passively (Brame).

Teachers using the Etude to create video lessons can opt to add questions in multiple-choice or short answer format. These serve as guiding questions. Providing guiding questions to students promotes active learning by “share[ing] learning objectives with students, thus increasing the germane load of the learning task and reducing the extraneous load by focusing student attention on important elements. (Brame).” Some of my students liked to run down the questions and answer them before watching the video. They would then correct the wrong responses as they watched the video. This is a highly effective method that I encouraged all students to do.

In an Etude, questions do not appear unexpectedly in a dialogue box to interrupt the video like at some websites. Teachers can opt to set a cue point for their questions so that the question becomes highlighted at the right time when the video has given the answer. the video pauses gently while the student responds.

Students engaged in an Etude lesson can control the video playback and are encouraged to re-watch when they need to. Studies show that “[s]tudents who were able to control movement through the video, selecting important sections to review and moving backward when desired, demonstrated better achievement of learning outcomes and greater satisfaction.” (Brame)

The Etudes are self-scoring for multiple-choice and there is an AI grading assistant to help teachers score short answer questions that is easy to train. The scoring app is designed for maximum efficiency. instead of scoring a whole student page at a time, the scoring app prompts the teacher to view students answers one question at a time. that is, you would score all the question ones, then all the questions twos, etc. This increases accuracy and decreases time on task for grading.

2. Summarize the Content

For the busy teacher who may not have time to develop comprehension questions on video lessons, assigning students to summarize the content or take notes from it is an effective practice. Note taking from video lessons has been found to ” help struggling readers overcome their difficulties when learning from text blogs but not from video blogs.” Further, “Studies with undergraduate students […] demonstrated that students instructed to take notes recalled more information from a video lecture than the control group.” (Hashem Ali Issa Almuslamani, Islam A. Nassar & Omar Rabeea Mahdi)

Innovation can help you with this ion two ways. First, Innovation sets up a handy form next to the video for summarizing. Secondly, and more importantly, the Innovation AI grading assistant is easily trained to help you score the summaries.

3. Proctored Viewing

Another strategy for engaging students in video learning that Innovation can help you accomplish is proctored viewing. This application is very simple: it monitors student screen activity during the playing of a video such that it can increase the chances of active engagement. The proctor notes are then submitted when the video ends.

Proctored viewing is less effective that the Etude or having students summarize the content. Proctored viewing probably works best for videos that are six to nine minutes long. Studies found that “the median engagement time for videos less than 6 minutes long was close to 100%–that is, students tended to watch the whole video. As videos lengthened ,however, student engagement dropped, such that the median engagement time with 9- to 12-minutevideos was ∼50%, and the median engagement time with 12- to 40-minute videos was ∼20%. In fact, the maximum median engagement time for a video of any length was 6 minutes. Making videos longer than 6–9 minutes is therefore likely to be wasted effort” (Brame).

[V]ideos […] have a direct and positive effect on increasing the students’ participation in the classroom.

Hashem Ali Issa Almuslamani, Islam A. Nassar & Omar Rabeea Mahdi

When students have submitted their proctor notes, the teacher can quickly view an easy to read summary of each student’s interaction with the video lesson. From there, teachers can discern the level of engagement to some degree. The proctor notes reveal when the student started or paused the video, for how long, and how much of it ran on the screen.

The proctored video app is useful when students are already likely to be engaged with a short video because of interest or prior knowledge and when there is some followup activity in class using the content. Videos are shown to increase class participation, so proctored video assignments can be useful tools to monitor some aspects of student engagement and support some accountability.

Innovation offers three research-based paths to engagement and accountability in video lessons. The apps were designed by working teachers and tested out by very ambitious teenagers who like to try to find software bugs. Why not give Innovation a try?

Sources

Almuslamani, H., Nassar, I., Mahdi O. (2020, May 12). The Effect of Educational Videos on Increasing Student Classroom Participation: Action Research. College of Administrative Sciences, Applied Science University, Kingdom of Bahrain. Retrieved 24 April 2023, from https://www.sciedu.ca/journal/index.php/ijhe/article/view/17480

Brame, C. J. (October 2017). Effective Educational Videos: Principles and Guidelines for Maximizing Student Learning from Video Content. CBE—Life Sciences Education Vol. 15, No. 4. Retrieved 24 April 2023 from https://www.lifescied.org/doi/10.1187/cbe.16-03-0125.

Delgado, P., Anmarkrud, &O., Avila, V., Altamura, L., Chireac, S. M., Pérez, A., & Salmerón, L. (2021, November 30). Learning from text and video blogs: Comprehension effects on secondary school students – education and Information Technologies. SpringerLink. Retrieved April 24, 2023, from https://link.springer.com/article/10.1007/s10639-021-10819-2

If it’s a conceit of middle age to presume to have some accumulated wisdom or something to share, I apologize and ask your indulgence here. As I transition into retirement, I just can’t resist combing through my long career to try to find something of value.

For this exercise, I would like to imagine myself a guest speaker at a college course for aspiring school administrators. This is the kind of thing I would like to say…

Now, I’ve never been a public school administrator and nor have I ever aspired to such. I appreciate the enormous obstacles to success in this field even if not from personal experience. I have this idea that a person studying to become a school administrator could possibly find something useful in the views of a subordinate.

“Potestas (power) is the ability to do something, while auctoritas (authority) is the capacity to lead, and it comes from the respect one commands by reason of his dignity, his character, his knowledge, and his achievements. True leadership is the combination of both potestas and auctoritas.”

Cicero, De Legibus (On the Laws)

I taught middle and high school social studies. I got a chance in that time to teach about leadership and government in the past; what worked and what did not, maybe with some hypotheses as to why. An explanation from Roman civilization of note differentiated between two leadership powers: potestas and auctoritas. Potestas is the brute force to compel cooperation that societies confer upon their leaders. This is the power to force, physically or though threat of sanction, the cooperation of subordinates. Auctoritas, on the other hand, is the respect subordinates have for the leader that is born of leadership competence and compassion. Inevitably, a leader must get the cooperation of subordinates who may not choose that course of action. Leaders who rely on potestas use threats and sanctions. Leaders who employ auctoritas will get the participation of the unwilling out of respect. Roman philosophers regarded the leader who can lead with auctoritas, who seldom needs to resort to potestas, to be the superior leader. A society under such a leader will prosper and thrive in a more long-term and sustainable way.

Metaphors that work, metaphors that do not.

A metaphor that is apt for the skillful manager, the executive who operates with auctoritas: The Gardener. The gardener does not necessarily know how the tree produces its fruit, and they do not need to, but they do know how much sunlight the tree needs, what kind of solid promotes its growth, what pruning patterns best promote tree health and strong fruit yield. The good admin is a gardener who promotes the growth of their garden by creating the conditions for their success.

“Authority (auctoritas), not coercion (coercitio), creates obedience.”

Livy, Ab Urbe Condita (From the Founding of the City)

Here is a metaphor that is apt for the ineffective administrator, the executive who fails under the modus operandi of auctoritas: The School Bus Driver. The school bus driver is in charge. They are the only adult on the bus, so they are the only real expert on how to safely get to school. Everybody better sit down and behave on this ride! This admin sees themself as the central visionary of the journey; the only legitimate commander. The value of passengers is relative to their obedience, loyalty, and willingness to take the ride wherever the bus driver decides is the best road to take.

“To lead the people, walk behind them.”

Lao Tzu

The reason that the school bus driver is a bad admin is simple: teachers are not analogous to kids on a bus. Kids on a bus are young, innocent, inexperienced, limited by their immaturity. Teachers are highly educated professionals whose knowledge and experience in their subject will surpass that of the admin except for maybe subjects the admin once taught. You can tell when you’re in the school bus driver’s meeting when you realize the supervisor is hearing you but not listening to you.

A metaphor that is apt for the good admin, one who guides by auctoritas and finds success: The Tug-o-war Anchor. The Tug-o-war anchor is the person in the back of the rope. They signal the group to pull together when it is time. They arrange the team members along the rope to pull where they are most suited. The goal is created by the team, not by the leader’s particular vision or prejudices.

The first quality for a commander-in-chief is to be a man of virtue. His authority must be based not only on his power but also on his character.”

Sallust, The Jugurthine War

How can a leader best approach morale problems on their staff?

The proof of the existence of a morale problem is that someone says it; nothing more is needed. If a lot of people say it, then it is a serious problem. If many drop out of committees, it’s not a sign they are lazy, but it’s a sign they no longer have buy-in. Why did that happen?

Potestas leaders eventually have morale problems. There is high turnover is workplaces with weak leaders. Denial or debating whether things are actually all that bad denigrate the listener and trivialize their feelings in ways that have no possible good outcome. That path only earns the contempt of subordinates. An effective leader works to identify the causes of the morale issue and then seeks the counsel of those affected to arrive at reasonable remedies. Leaders accept all morale claims as unquestioningly legitimate and seek remedies as best as possible.

How can a leader best view themself in relation to the educational institution?

Unlike in the private sector where subordinates are often less skilled or knowledgeable than managers, in education the chief executive’s main purpose is to harness the potential of subordinates whose knowledge and expertise is superior to their own in their respective domains. The educational institution that reflects only the vision of the executive officer is culturally impoverished. School leaders understand they do not always know best. They seek regular information from department heads. They accept the direction to which research and regulations point even if it does not jive with their “gut”.

How can a leader enjoy the loyalty and support of subordinates even when they must take decisions they oppose?

Good leaders will have already built a reservoir of goodwill among subordinates. From time to time subordinates will need something extra or unusual: a little time off, an indulgence for a mistake, etc. Such little things are opportunities to build a reservoir of goodwill such that generosity and indulgence from time to time creates an attitude they will fall back on in times when executives must choose an unpopular but principled path. Executives in such cases are also prepared to give sound and convincing reasons for their claims that any reasonable person holding the opposing view would accept as defensible.

A leader’s power is not enough to inspire loyalty and obedience from their subordinates. Instead, a leader must possess personal virtues, such as courage, integrity, and wisdom, that inspire trust and respect in others. This idea reflects the Roman concept of virtus, or the combination of courage, excellence, and morality, as the foundation of leadership

Leaders Must Be Good Communicators

Leaders who are unskilled often communicate poorly. They often fail to convince the listener of an opposing view. They know they have the authority to impose their view and it shows in their manner. Being unresponsive is another communication flaw. Leaving messages unanswered is not a legitimate way to deny a query.

When a leader who relies on potestas attends a meeting to observe the input of subordinates, they arrive with their mind already made up and with a view to performing the show of hearing what subordinates have to say. When a leader who effectively manages with auctoritas arrives at a meeting with subordinates, the plan will be the one the group arrived at by consensus. These effective leaders say things like “once we all agree, we will…”

One should maintain an abiding and unwavering respect for the truth. One should strive to establish a workplace that is stable, predictable, and intellectually prosperous. Subordinates should not come to work wondering what will befall them that day; what aggression they will have to defend against. The ancients knew this well and it is a body of wisdom well worth considering in the present. I am not sure that good leadership can even be taught or studied. It seems very likely to me that some people are simply born with an innate leadership intelligence, a social intelligence, that other people can appreciate and respect.

The purpose for introducing the constructed response question (CRQ) in middle school is to prepare students for this kind of assessment later in their education. Ideally, the task should lay the groundwork for the habits of mind that promote success and should accustom students in a practical way to the assessment itself, its common form and its vocabulary. Experience teaching this to eighth graders shows that one of the first major obstacles is to get students to move away from the reflex ingrained in elementary school: to respond to a text by stating what it says. The second major obstacle to teaching this is that students coming out of elementary school are wholly unfamiliar with the idea that some materials they may be given are quite possibly not reliable. In addition, they lack the vocabulary to manage the concepts of text reliability.

It is difficult for upper elementary students to address primary source material for a variety of reasons. First and foremost is the text complexity. Secondly, a limited ability to comprehend given their severely limited background knowledge (class lessons should remedy this). The middle school CRQ needs to be accessible to most students while still preserving the “primary source” characteristic of the task; the opportunity to see what people of the past had to say in the way they said it. It is often a further revelation to people at this age that the English language has not always existed or that it has existed in variant forms they would find incomprehensible. An appreciation of language change and variety plays an important role in addressing primary sources for this age not only for a deeper understanding but to appreciate reliability concerns of translation and excerpts and secondhand accounts. The documents for analysis in the middle school CRQ will be carefully devised in the following ways:

• An image of the source’s original format and language will be provided for purely observational purposes. This may be merely an incomplete image or fragment.
• A standard translation of the source will be provided despite that it is at a text complexity above the grade level band. This is also purely for observational purposes, though some students may make the attempt to analyze it.
• A translation of the source into a Lexile range of 800-1000 will be provided if necessary. This is the document on which students are to work.
• There will be a citation of the source in the simplified version of the citation format used by genealogists. Students should consider the source in their analysis.

One task will consist of two pairs of documents. Students will give the historical or geographical context of the first document in each pair. Students will assess the reliability of the second document in each pair. In addressing the reliability of the source, students will need more support, naturally, than their compatriots at the high school level. The second document in each 2 pair will ask the student to address reliability in multiple-choice format. This will habituate the student to the typical phrases used in addressing reliability. First, students will be prompted in multiple-choice format to identify the document’s bias, point of view, audience, or purpose. Secondly, students will be asked to identify the best use of the document for a historian or anthropologist. Thirdly, a multiple-choice format question will ask the student to conclude how the reliability factor affects this document’s use as a reliable source of evidence to prove something specific. This latter point is important to help students understand that sources may have different reliability depending on what the historian wishes to do with them.

The last part of the task calls upon the student to synthesize a relationship between documents 1 and 2. This will also be in multiple-choice format. Students will be prompted to use both documents in one of three possible ways: (a) state a similarity or a difference between the documents; (b) explain one change associated with a turning point in history that the documents reflect (the turning point will be identified for the student); (c) explain how some development or idea is the cause of some event, idea, or historical development reflected in both documents.

Gradually toward the end of grade eight, students will move toward short answer format CRQ’s as they will see in high school. Having seen the same wording each unit across grades six through the first part of eighth, the idea of context, reliability, and turning point should be well established. A multiple-choice version for students who are still developing the skill could be offered for a reduced maximum score.

Task Models

Borrowing from the “task model” concept used in developing the New York State Global history and Geography Regents examination part one (stimulus-based multiple-choice), the following are the task models for the multiple-choice version of the middle school CRQ. It will be important to use similar language in constructing the questions for consistency.

• Which of the following [statements | titles] best represents the [historical | geographic] context of the [document | map]? 
• Correct answer will be historical background information not present in the document 
• One incorrect option will be of the type “This document is about…” 

• Correct answer will be geographic background information that explains the origin of the map’s information 
• One incorrect option will be of the type “This map is showing is about…” 
• Which of the following statements best represents the geographic context of the map? 
• Which of the following would be the best use of this document for a historian? 
• Which statement best describes the [point of view, intended audience, purpose, bias] of the document? 

• When point of view is asked, one incorrect option will be “first person’ or “third person”. This is to teach the student to distinguish between how that term is used in English and how it is used in social science. 
• Which of the following factor(s) would [weaken | strengthen] the reliability of this source for the purpose of __. 
• The reliability factors taught are: authorship, format, point of view (objective or biased), time and place, intended audience, purpose. 
• These will often have more than one correct answer. 
• The factors are listed, followed by a colon and a description. Example: 

• Point of view: The author is very biased. 
• These two sources are artifacts from a turning point in history. Which would be the best title for that turning point? 
• Which statement best describes a [similarity | difference] between the two sources? 
• These two sources are artifacts from historical events. Which statement describes a cause-effect relationship of the historical events the sources represent? 

Assessment Task Comparison Across Three Assessments 

The purpose of developing this task is to create a logical early training step for students in middle school working toward the assessment tasks they will see in high school. 

Teachers use video lessons a lot these days, some they create themselves, and other videos that they find on Youtube. Not everyone has time to build a set of cued comprehension questions for each video lesson. Some teachers assign their students to summarize what they saw in the video. This is a fantastic way to keep kids engaged in the video, but it’s a lot of work scoring them.

The Innovation AI grading assistant is the perfect solution. In this post, I would like to show you how to generate a video lesson in Innovation and then quickly train the AI to help you score the summaries very rapidly.

Quick Links to Short Video Tutorials

• Add a video for students to summarize
• Student access to the video and summary space
• Part 1, Train the AI grading Assistant
• Part 2, Train the AI grading assistant
• The AI can coach students to write better summaries.

A Word about the Innovation AI

There are two broad categories of AI: those that work using a complex algorithm and those that work using large language model learning. The Innovation AI is of the former type. The latter, the machine-learning AI, is exemplified by ChatGPT and the Bing AI under development right now. They “learn” by analyzing vast amounts of data across the internet. The Innovation AI is trained on five to seven models that the teacher provides for comparison.

I developed the Innovation AI to help me grade summaries and short answer tests. When I taught social studies, I often assigned summaries of texts instead of comprehension questions. It works by comparing the student text to a number of models and scoring the comparison on a dozen features. These features include measures of similarity like cosine and Jaccard, as well as readability, number of words, level of text complexity, and so forth. The scoring rubric was designed using 500 of my students’ work submissions that I had scored manually so that the AI essentially grades as I would.

The Innovation AI is highly effective for helping you score summaries and short answer responses where the range of possible answers is fairly limited. The AI does effectively recognize different ways to say the same thing using natural language processing algorithms.

Training the AI

When you train the AI, you give it model answers to use in the comparison algorithm. For short answer tasks, the limit is presently five models. For writing samples such as summaries and compositions, you can store up to seven model answers.

The process, in summary, is this: (1) Compose your own summary of the task or let the Innovation AI generate a summary from the source text for you; (2) Manually score the work submissions of your students who usually get full credit. When you find a submission to which you would award full credit, ask the AI to score it. If the AI cannot recognize it as a full credit answer, you “add it to the corpus” of model answers. The next time you ask the AI to score a student submission, it will compare it to each of the models in the corpus and award the highest score earned by the student in those comparisons.

Since many of us reuse our assignments from year to year, you really only have to do this once. I trained the AI on most of my Global 9 and Global 10 assignments in 2018 and just continued to score with those for several years.

Creating an Inbox for a Task with Embedded Video

1. Select the Inbox button from the new course playlist element dashboard.

2. Enter the title and some optional attributes. Paste in the embed code from the video you want students to watch and summarize.

3. Once created, you can click and drag the element to its right position in your class playlist.

Students Save their Summaries

When students access the task from the course playlist, they will see the video you embedded and the space underneath to compose their summary.

AI can coach students to write better!

The AI grading assistant can be engaged to coach students along the way. As they compose their summaries, they can periodically click the “Coach” button to get an estimate of their grade so far. In my experience, this promoted student prolonged engagement for a better work product.

The stimulus-based multiple-choice test item was introduced into the New York State social studies Regents examinations starting in 2019 for Global History and Geography II and for United States history. The task poses challenges for students such that it merits some regular, focused training throughout the year.

In a stimulus-based task, the student is directed to respond to a document, map, or image using their ability to analyze and their knowledge of historical context. In the case of the New York State exams, there are eighteen “task models” used when designing questions. For example, a student may be asked to evaluate and classify (identify) best use of a source or to respond based on knowledge of historical context. Principles of reliability assessment are applied here, such as when students are asked to identify point of view, purpose, context, bias, format of source, location of source in time and/or place, and/or intended audience of sources using background knowledge.

Click here to shop stimulus-based tasks at my store for grades seven through eleven social studies.

The first important habit of thought to train students to engage is to think beyond the document. Habit since their first reading lessons has asked them to find the answer in the text somehow. It takes a lot of practice and reinforcement to get students to activate their schema on the topic; to think of the story of which the document is but a fragment. The question cannot be answered without background knowledge.

• part I of the Global and US History Regents
• 25-30 questions
• primary or secondary source documents
• M-C questions are always paired with stimulus
• primary or secondary source
• maps
• charts
• cartoons
• may have more than one stimulus tied to it
• no more stand-alone questions
• estimate 30-45 minutes for this part of the exam

I used almost exclusively primary source documents for my stimulus-based tasks. This can be challenging for weaker readers, but with practice in skills for addressing difficult texts, this obstacle can be addressed.

My custom was to assign a stimulus-based multiple-choice at the end of every unit starting in October. At first, students find these very difficult. I use a z-score standardization procedure to adjust the scores so as not to bomb out their GPA while they are just learning. Click here to read up on standardized scoring. It is a great way to score students in tasks they are not yet proficient at.

• Practice! Students are generally not good at these at first. 
• Read the question first. be certain you know what it is asking.
• Remind yourself that the answer is rarely found in the document itself.
• Identify the historical time period the documents go with.
• Consciously call to mind the historical context of the document before you read. Try to recite to yourself who, what, when, where, why of the time period.
• Use process of elimination to narrow down the options.

Some of my students always would wonder why ask questions this way. If the test writers want to know whether a student knows something, why not just ask? I don’t have a good answer for this. I strongly support instruction in social studies that calls upon students to think critically and make meaningful connections with knowledge. I also think that students should actually possess knowledge. This assessment method was no doubt inspired by AP history exams. It remains a question in my mind as to whether this level of complexity is necessary for an instrument for secondary school evaluation. If we want to know whether the student knows what caused the French Revolution, for example, maybe we should just ask them that?

The updated New York State Regents examination in United States History and Government, part II, is a short essay task designed to measure students’ ability to work with historic documents. It is a mature version of the “CRQ” found on the tenth grade Global Regents. Students are called upon to understand text, engage it with historical context, and assess a text’s reliability.

In document set 1, students describe the historical context surrounding two documents and identify and explain the relationship between the events and/or ideas found in those documents (Cause/Effect or Similarity/Difference or Turning Point).

Turning point is always the most challenging for students, mainly because it demands a strong knowledge of historical context which only the higher performing students usually possess. In stating similarities and differences, it is important to stress to students that this should be a substantial feature of the two texts, not trivial. For example, some students may respond something like this: “Document one is a cartoon and document two is a newspaper report”. This is trivial and should be discouraged. For cause and effect analyses, remind students that some events may lie outside the documents at hand, so they may need to rely on their historical knowledge.

Click here to short for short essay prompts at my TeachSimple store.

Document set 2 asks students to describe the historical context surrounding two documents and (for one identified document) analyze and explain how audience, or purpose, or bias, or point of view affects the document’s use as a reliable source of evidence.

A good strategy for ensuring students possess the skills to address this task is to be certain to assign one every month or so throughout the year, followed by a debriefing where the class can study their classmates’ work (anonymously) and develop strategies for improvement.

Weaker readers are particularly disadvantaged in this task, although since the test items are field tested before administration it is likely the field testing will mitigate some issues with the difficulty of reading some primary source texts. Students can be taught compensatory reading strategies to help deal with difficult texts.

As always, the challenge is to ensure that students have learned a strong body of historical context. That is, the best marks are reserved for those who actually recall the history and who can analyze it (cause-effect, turning point, etc.) This is best achieved by regularly administering quizzes on historical knowledge. I like to give students time in class to study for these. The apps here at Innovation Assessments are especially suited to that end. A lot of social studies assignments can tend to be just look-it-up and transfer kinds of exercises without real demands on students to remember. This is an easy instruction error to remedy.

Teaching factors affecting the reliability of sources is another matter. This takes a great deal of time and practice and, I would argue, is of upmost importance for a person’s education in this day and age. Students, for the most part, do not intuitively analyze and explain how audience, or purpose, or bias, or point of view affects a document’s use as a reliable source of evidence. They tend to take what we give them on face value. It is important to teach students to think critically and approach all historic documents with a healthy skepticism.

Click here to shop my store at TeachSimple.com

I think teaching reliability should begin young, down in middle school. Engaging students with documents that have very vague reliability weights is a good practice. In a debriefing after the task, it is useful to anonymously display some student reliability evaluations for all to see and to discuss. It is important to do this regularly, starting off right at the start of the school year. There are really no stock phrases that students can learn by rote for this, given the variety of context and source material. I had the benefit of working in a small school where I had the same students grades seven through ten or eleven, so I could implement reliability assessments early in my program. In larger school districts, it would be good to consider a commitment to reliability factor training from an early age. I assigned one longer primary source to analyze each topic (so about once a month). This short essay included an extended analysis of reliability in a conclusion paragraph. The training paid off and when my eleventh graders were preparing for the Regents in US History and Government, they had little difficulty with reliability factors.

A good piece of advice on this is to assign students to do this every month in grade eleven. I suggest assigning it as a test each time. Coach students on the historical context they have to memorize in advance. Lots of teachers assign these for homework, but this entirely misses the point of such training. Student independent work practices are highly efficient in applying the minimal effort to a task, including copying their colleagues’ work or copy-pasting from a source. If they are not doing these without notes, they’re not really practicing.

If you are afraid to assign your students this as a test because they are not likely to do well at first and don’t want to bother their GPA, I recommend using standardized scoring. You can use the z-score calculator here at Innovation Assessments. Use 78 as your standard mean and 14.8 as your standardized standard deviation. Read more about standardized scoring here and where I got those figures. The beauty of this system is you can apply this to their grades every month and as the class improves, as the class average approaches the standardized mean (78 in this case), then the algorithm affects their scores less and less.

Once you have scored their papers, select out some problem responses for class discussion in a debriefing. Keeping the responses anonymous, review how to improve the answers next time.

The short essay is 14% of the Regents score, so for passing the text it’s a good practice. But I would suggest that this kind of work as a regular lesson is extremely valuable as an educational tool. Studying primary sources rightly should take center stage in our social studies lessons. It is not just teaching to the test to do this. It is developing a critical thinking skill set and insisting on recall of historical context that are the values here. Remember, that the highest valued performance is that which is based on a substantial recall of historical context!

Dear fellow social studies teachers,

I would like to play the role of a crusader in this post, if you don’t mind. There’s something I would like you to consider about your teaching practice, or maybe reconsider as the case may be. I’d like you to abandon assigning your students comprehension questions on most kinds of texts you assign in your social studies class. Instead, I’d like you to consider assigning students to summarize instead.

When students do the search and copy instead of reading the text fully, they deny the author the chance to build with the reader an information schema of their own that lends itself to long-term recall.

Okay, so who do I think I am anyway to question a practice that’s at least a century old? Well, I cannot claim to be any super authority. I taught social studies for eighteen of my thirty-two years teaching. I worked in small, rural mountain schools and my roster averaged about one hundred students a year, which is likely smaller than that of my readers. I think these are good starters to convince you to at least hear me, but I have always felt that the argument from authority to be one of the most pernicious fallacies. So in honor of my thirty-two years, grant me the indulgence of reading further. As for accepting my suggestion, let the arguments sway you themselves.

When you assign your students to read a text selection and answer questions on it, you must know what the majority of them do. They skim the text to locate the answer, often they copy it wholesale, and then they’re done. Teachers who accept this kind of work are missing an important opportunity to foster greater reading comprehension skills and greater long-term recall. Writers, even writers of textbooks, build meaning in a schema of content to deliver to the reader, who constructs meaning related to their experience. Ideas are hierarchically arranged and organized to support a set of main ideas. When students do the search and copy instead of reading the text fully, they deny the author the chance to build with the reader an information schema of their own that lends itself to long-term recall. At the risk of putting too fine a point on it, questions on text are largely a useless exercise even when the questions are of high quality.

Once students leave sixth grade in my region, the formal lessons in reading stop. But a great number of sixth graders do not read at a sixth-grade level. They cannot advance their reading without texts at their level. And they have to actually read these. It is possible and desirable in middle and junior high school to promote the development of reading in youngsters by structuring what they do to process text. Teaching students to write competent summaries is one of the best ways to let them develop their reading skills. (This should be paired with offering texts at their independent reading level whenever possible).

What does the research say?

Summarizing and note-taking are two of the most useful academic skills students can have. (Marzano, Pickering, & Pollock, 2005)

Summarizing is in the top two most powerful writing tasks that support the development of student reading. (Graham & Hebert, 2010)

Click here to shop my TpT store for passcodes to give your students access to training videos with embedded auto-corrected questions on composing summaries.

Writing to Read

I was greatly influenced by a paper entitled Writing to Read: Evidence for how Writing can Improve Reading in 2010 published by the Carnegie Corporation of New York. A meta-analysis clearly demonstrates that composing summaries is superior to answering questions on text. The reader may argue that this research was conducted to address methods for using writing to enhance reading. But I would argue that (a) this is rightly a major goal of assigning most secondary students a textbook reading task and (b) the same cognitive processes that go into reading comprehension go into long-term recall of information. Students answering questions on text are less likely to be encoding from working memory into long-term memory than those composing summaries in their own words.

The Five and Three Summary Style

Around 2006 when I was making major investments in curriculum and methodology development, I sat down with a noted professor of literacy who happened to be working in our district, Trudy Walp. I explained my qualms about assigning students questions on text and I asked her, what is the very best thing I can have students do with text from a reading specialist’s perspective. She did not hesitate to say: have them retell what they read.

When I assigned my students summaries on their textbook articles early on, my top students wrote terribly long and detailed compositions that were not brief enough to be a summary. I needed to shorten my students’ summaries and to develop a rubric for evaluating the quality of a summary. The resulting task I used for some fifteen years: the Five and Three.

The assignment for the five and three is to summarize the assigned text (usually four pages in a typical high school textbook) in five sentences exactly, no more and no less. This is the “five”. The “three” is the personal reaction to the text. Students were assigned to connect this text to their own life somehow. What does it remind them of? What do they think about what they read? Why? These had to be exactly three sentences long. This was the second element that the reading specialist recommended to me. Students need to make the text meaningful to them in some personal way that makes new information integrate into their preexisting schema.

The five-and-three became the foundation of my students’ textbook reading work. They also had the option of composing Cornell notes for an assigned reading. I would invite the reader to return to the blog for a post dedicated to extolling the virtues of Cornell note-taking. The five-and-three faded in importance in certain classes where video lessons became an effective information delivery tool when reading instruction was no longer a priority.

The Mechanical Summary

The mechanical summary evolved in 2020 when I had a group of learners who struggled and who just would not produce summaries even given time to do so in class. I offered them the opportunity to write a summary by copying the first sentence of each paragraph word-for-word, then to connect and arrange these into complex sentences such that there were only five sentences. They had to compose the three-sentence personal reaction to the text just like normally done. For this, I offered a maximum score of 76 because, I argued, it had less value not being processed in their own words. I reason that not processing in their own words likely limited the ability of encoding to happen from working memory to long-term memory.

With the Assistance of AI

At this point you are likely wondering how you could possibly grade all these summaries. I will grant you, the workload takes some management, but I would argue that it is only slightly more time consuming than grading answers to a set of questions and the benefit to your students makes it all very worth it.

I’m an amateur programmer and I developed an AI grading assistant that is very good at scoring summaries. This app is available to subscribers to Innovation Assessments. Say, why not sign up for a free 60-day trial? The price is surely right if you decide to subscribe?

The AI grading assistant was accurate enough to save me tons of time scoring summaries. The summarizer app actually let me create models on which to train the AI that were highly accurate when compared to human-generated summaries… eerily so!

This blog post describes the AI grading assistant.

The ability to skillfully summarize is a lost art worth recovering for our students. It enhances the development of their reading ability and it promotes greater recall of the content we teach. It’s my hope my experience developing this system can make this suggestion seem do-able and worthwhile.

SOURCES

Graham, S., & Hebert, M. (2010). Writing to Read: Evidence for How Writing Can Improve Reading. New York: Carnegie Corporation.

Marzano, R. J., Pickering, D. J., & Pollock, J. E. (2005). Classroom Instruction that Works: Research-Based Strategies for Increasing Student Achievement. Upper Saddle River, NJ: Pearson Education, Inc.

When the new Frameworks for social studies in New York State came out about six years ago, the first item that caught my eye was the constructed-response question sets. If you are not familiar with this task, in brief, it calls upon the student to examine historical sources (I use exclusively primary sources) by providing historical or geographic context, identifying the point of view, intended audience, or purpose of a document and then using the two documents in either compare-contrast, cause-effect analysis, or turning point identification.

Originally, the task was supposed to call upon the student to evaluate the reliability of the source. This was scrapped, probably after field testing questions. I regret their decision to scrap this (though it does appear as part of the analogous task, the short essay, in US History grade 11). Students can be taught to evaluate the reliability of sources at an early age (I incorporated it into my sixth- through eighth-grade social studies work). Students can be trained to evaluate based on features like the point of view (bias), intended audience, purpose, time and place, and authorship. Teaching this does require patience and practice. What I think happened is this: few teachers address the reliability of sources and so when they field tested the new exam format, everyone did so poorly that they scrapped the question. I wish I could find out how my kids did. I had been teaching the reliability of sources since they were in middle school.

I advocate a strategy of assigning one CRQ in each unit of study in grades nine and ten without access to notes. I used this as one of the tests at the end of a unit of study. I would tell students about what kinds of documents would appear and what historical context they should be able to recall in advance. It is a challenging task for them.

The first challenge for novices is to understand what it means to provide context. Faced with the question “What is the historical context of this document?”, beginners will retell what the document says. The reason for this mistake is that, since they were little kids, teachers have asked them to relate what a text means to prove they understood it. This task asks students to bring to bear what they know about the history behind the document, which requires a level of recall I don’t think we’re asking students to do enough.

Another challenge for students writing the CRQ is the third question where they must use two documents in analysis. Cause-effect relationships are the easiest for students, it seems, so I teach those first. When students are asked to compare and contrast the documents, some students have to be cautioned to select significant elements to compare. The fact that one document is a map and one is a diary entry, for example, is not a significant fact. The turning point question is the hardest, mostly because it calls on students to recall history and to understand historical trends both before and after the event. I find it useful to break this up into small tasks: first say what the turning point is, then say what is so special about it.

After a difficult task, especially for writing, I like to do a “debriefing”. In the debriefing after the CRQ, I like to share student answers anonymously in a slide show and discuss them. It is a very effective strategy for stamping out common errors early and permanently. Sample debriefing PowerPoints from an eighth-grade and ninth-grade CRQ are posted below.

Certain constant reminders were recurring for all my classes:

— Things are not reliable because of what they say or show.

— Things are not reliable because they are in quotation marks.

— Things are not reliable because they happen to match what you think is already true.

— Unreliable statements can be true.

— When you’re asked for historical context, do not describe what the document says.

And what about middle school? Can the constructed-response task be modified for that level?

Students with an IEP often are very challenged by these tasks because of the reading level of the texts. Since I use primary sources exclusively in my CRQ’s, this is doubly difficult for students with reading limitations. The answer to this is to consider obtaining CRQ’s designed with a lower reading level.

If I may be so bold as to give advice, I would suggest that students do these throughout Global nine and ten, perhaps once a month. I suggest that they be as tests instead of open-book or take-home assignments because students need to prepare for recall situations. The debriefing afterward is vital to class improvement and it makes for a long-lasting correction. I would advocate resisting the temptation to leave this off for the Global ten teacher. It really does take a long time to learn to do this well.