Discussion Analysis

If your course is in Original Course View, see the Original Course View tutorial collection.

 

What is Discussion Analysis?

To better inform grading and provide additional insight into the level of engagement in graded discussions, in the Discussion Analysis area of the Ultra Course View, graders can view sentence complexity, critical thinking, counts of responses and replies, along with statistics for the number of substantive posts, lexical classification (counts of content and functional words), and the percentage of word variation in a student’s posts for a given discussion. These market-leading analytics provide instructors a holistic overview of the level at which students are writing and may better inform differentiation in grading discussions.

See also Grading Discussions.

Procedure

From a discussion's Grades & Participation tab, select a student’s name to view their responses and replies. Discussion Analysis appears next to the student’s discussion activity to give you an overview of their engagement.

 

Why Don't I see Data?

Discussion analysis refreshes nightly for active discussions. The data won't update if there's no new discussion activity on a given day. You'll see data in the Discussion Analysis panel if the student has participated within the last four months. Only instructors and graders see the discussion analysis.

Discussion analysis only appears for graded discussions.
 

What Does the Analysis Mean?

Substantive Posts

The number of responses or replies that contribute to the discussion's development. A substantive post contains sentences that establish or support a student's position or ask thoughtful questions. These posts also show critical thinking or sophisticated composition, based on word choice and variety.

Non-substantive posts may be short or underdeveloped. For example, a simple "Yes" or "No" reply to a discussion prompt isn't substantial. Students need to expand on their answers and explain their positions to make the responses or replies substantial.

 

Sentence Complexity

Measured by the number of sentences, words, and syllables in each response. We look at the complexity of words and how often the words are used. This measurement is a linguistic standard called Flesch-Kincaid. The complexity of each student’s total posts is represented by a grade level from 1st grade to 16th grade. Content with a Flesch-Kincaid grade level of 10 should be easily understood by a person in 10th grade.

 

Lexical Variation

Analyzes the substance of a student's responses or replies based on the words they've used.Comparison of words that support the student’s ideas vs. the words that support proper grammar.

Content words carry meaning in a student's response or reply. These words show a student's feelings or thoughts regarding the prompt. When compared with total word count, content words help show the lexical density of a student's responses and replies. A high count can indicate more sophisticated writing.

Functional words unite the semantic elements of a sentence together and indicate proper grammar. Prepositions, conjunctions, pronouns, and articles are functional words.

Think of functional words as the glue that holds a student's response together. The words may not have substantial meaning themselves.

 

Critical Thinking Level

Indicates words and phrases within a student's total posts that demonstrate critical thinking. Twelve dictionaries are used to identify the words, which then fall into one of the weighted categories of critical thinking:

• Argue a position
• Include supporting data
• Cite literature or experience
• Evaluate
• Summarize
• Reference data
• Offer a hypothesis

 

How Critical Thinking is Measured

The weighted number of the words and phrases in each category are combined and then compared to the class average to create the critical thinking score. The score is the difference between the student’s critical thinking and the class average.

The score falls in a decimal range of -1 to 1. A negative score means the student's critical thinking is below the class average. A positive score means the students critical thinking is above the class average. A score close to 0 means the student's critical thinking is at the class average level. These scores are represented by a range of low to high:

-1 < -0.06 = Low
-0.06 to -0.03 = Below Average
-0.03 to 0.03 = Average
0.03 to 0.06 = Above Average
.06 to 1 = High

Critical thinking is represented visually to show each student's score compared to the class average.

Examples:

• Empirical research shows disagreeing displays a higher level of critical thinking than agreeing. In a discussion, the statement "I agree with John" receives a score of 0.113, while "I disagree with John" receives a score of 0.260.
• If students summarize a passage but add no opinion or argument, they score lower than others who argue a position.
• If students cite literature, they receive a lower score than others who offer a hypothesis.

 

Word Variation

Measures the number of unique words in a student's submission as a percentage. A higher percentage of unique words can show that the student's composition contains multiple ideas and significantly supports a position. A higher percentage can also show that a student engages classmates to think about other perspectives.

You can compare the student's percentage to the class average.

 

Discussion Details

In addition to the calculated readability, the details include average word count, responses, and replies for each student compared to the class average. This information, along with any rubrics or criteria you set, can help you determine a grade.