Monday, February 12, 2024

CRIT: An updated approach to Information Evaluation for Research

 I have always been passionate about information literacy and getting the facts straight. I used to get in trouble as a kid for talking back, which was usually just me trying to correct an adult's factual error. In high school, I wrote a strongly-worded letter to one of our local journalists because he misidentifed turtles as amphibians in a piece about the Teenage Mutant Ninja Turtles. I screamed myself hoarse when people in powerful positions started using phrases like "alternative facts". In an ideal world, I would teach a compulsory course at my high school that is solely devoted to information evaluation. So when, in the fall of 2022, I learned about the shortcomings of that long-touted librarian tool, the CRAAP test, I went straight to work with my colleague Nadia to come up with something that would be more effective, but still suit our students' needs.

Cut the CRAAP: Explore a new acronym to get students thinking CRITically about information by Nadia Komp & Emily Wilt 

We combed through a number of alternative methods for information evaluation, including Mike Caufield's SIFT (the four moves), the book Developing Digital Detectives, the book Fact vs. Fiction, and lessons from The News Literacy Project and the Digital Inquiry Group's (formerly Stanford History Education Group) Civic Online Reasoning (COR) course. These are all excellent resources, but none of them independently satisfied everything we were looking for. We liked the acronym aspect of CRAAP, and felt that SIFT was good for general info evaluation but didn't quite tick all the boxes we wanted for school-related research. So we invented... 
 
CRIT: Credibility, Relevance, Intention, Timeliness

4 letters that pack a punch. I always start CRIT lessons with a terrible joke-- "this will help you think CRIT...ically about the information and media you encounter"-- that will hopefully help my students remember to apply this lens to all the media the come across, whether it's for school or not. Then we dive in and break down each letter. 
 
Full disclosure: I usually only get 90 minutes to do what I'm about to describe (we have block scheduling). Ideally, we would break this lesson down and take at least one class per letter so we would have ample time to practice each skill, but c'est la vie. Maybe next year. 

Credibility. 

What is it? How do you determine whether a source is credible? This is a crucial skill for both academic research and personal knowledge building. Here we have conversations about 
  • web domains ( .orgs have been available for anyone to purchase since 2019, y'all, and top-level domains [TLDs] are an ever changing beast)
  • the open web versus databases (Check out Dr. Kristen Mattson's blog post "Academic Databases are the Netflix for Nerds!" if you struggle to teach this concept to your students) 
  • how human psychology predisposes us to trust information shared by people we know, and how social media blurs the lines between people we actually know and people we feel like we know.

This is also where we learn about and practice Lateral ReadingIf nothing else, learn about and teach lateral reading. This is a totally transferable skill that is used by professional fact-checkers and will serve our students (and everyone, really) well in our increasingly online worlds, even as we ramp up interactions with generative AI.

One of my favorite lateral reading thought experiments with our students is talking to them about how to lateral read a social media post. People speak with such confidence on TikTok, YouTube, etc... how do you know you can trust what they say on a topic? How do you investigate the user behind the username? We've gotten to put this into action in a really cool assignment for World Literature I (the brainchild of my colleague, Kirsten Reed) wherein students are actually required to use some type of social media as a source. The feedback from the students has been overwhelmingly positive, and it has the twin benefits of transferring this skill to their real online lives and forcing them to understand how to construct a properly formatted citation by hand (I have yet to find an automation for generating a citation from a TikTok URL). 

Relevance. 

What is it? (I always start with that question to make sure we've operating from the same base). How do we determine what is relevant? This question is definitely geared more toward school-based research (some of what we felt was missing from the SIFT approach), but it also offers an opportunity to talk about how the current iteration of the internet works. 

I saw a demonstration of Google searching at a professional workshop a few years back that Blew. My. Mind. So of course I replicate it for my students whenever I get a chance! I usually do this in Google News, but a regular old Google search works too.
  1. Perform a search while logged into my work account. 
  2. Open a window in another browser and perform the same search while logged into my personal Google account. 
  3. Open an Incognito window and perform the same search without being logged in to a Google account. 
  4. Show all 3 windows side-by-side.  The results are always different, though sometimes the differences are subtler than others.

This is a segue into a discussion about the algorithms that personalize everything we experience online, from our Google searches to our social media feeds to our streaming media recommendations. We talk about the dangers of echo chambers, how important it is to recognize our personal biases, and some strategies for doing good research in spite of this (using library resources, naturally, but also using Incognito windows and comparing search results with friends).

That's just level one of Relevance. 

Level two is mostly subconscious: if I'm writing about high-speed rail, do a search for "trains", and there's a result about training animals, my brain filters that out for me automatically.

Level three is more intentional: I'm writing about high-speed rail, do a search for "trains," and have to choose between an article about existing train systems in the US and existing train systems in Japan-- I'll probably need to look into those sources to decide which one is more relevant to my topic. This is a great place to dive into search strategies that can help dial in those investigations to return a limited number of results that are most relevant to your topic. (This is really a whole lesson in and of itself.)

Level four is what students seem to struggle with the most: they have a good source, but they need to pull out the most relevant quote to support their argument. As this is a little more English teacher territory, I usually do a short practice with my students so they know where it fits, then move on to the next letter. 

Intention.

What do we mean by Intention? How do we determine the Intention of whoever created this information/media? And why does Intention matter? 

Most of my upper-level students have encountered the media intention (or purpose) acronym PIE- Persuade, Inform, Entertain- but I like to use the News Literacy Project's breakdown of six intentions to dig a little deeper. (You will need to create a free account to access the link above.) NLP proposes that the six primary intentions of media are
  • to Document (raw footage or audio with little to no commentary; primary source)
  • to Inform ("news" in the traditional sense; straight-forward reporting with some added context)
  • to Entertain (self-explanatory; sometimes overlaps with media in other categories)
  • to Persuade (to change someone's mind)
  • to Sell (to persuade you to part with your money)
  • to Provoke (to persuade you to react strongly; to persuade you to take a certain action)

This is a great place to talk about misinformation, disinformation, and malinformation (and to once again plug the benefits of using vetted library resources) and tie back to any classroom discussions students may have had about propaganda. If the intention behind media or information is difficult to determine, this is also a good place to call back to and practice lateral reading. 

Additionally, we can talk about what types of sources are appropriate for school research when, covering the differences between Scholarly vs. Trade vs. Popular sources, and when it might make sense to use social media in school projects-- for example, you would not cite an Instagram post, even one meant to inform, as factual evidence on its own, but you could cite it as an example of the cultural opinion on a topic. 

Finally, we come to 

Timeliness.

What do we mean by Timeliness? How do we determine publication date, and when/why does it matter?
 
Nadia and I felt it was important to include a piece related to publication date (the C in the ol' CRAAP test) because our students are assigned a variety of projects-- some are historical, in which case students need to be cognizant of whether their sources are primary sources from the time in question or secondary (or even tertiary) reports and analyses, and some are focused on current events, in which case our students need information that may change literally minute-to-minute depending on their topics.
 
This explicit reminder to check the publication date seemed too important to let go of. How many times have you encountered an article shared by a friend on Facebook accompanied by an outraged caption and a call to action, only to realize the article was published four years ago? Surely I can't be alone in that. This is the easiest piece of evaluation to accomplish.

Conclusion

I have been covering CRIT with students for a year and a half now, and they seem receptive to it. My English teachers love it, and we are working on making it an acronym that travels outside of library lessons to infiltrate classroom vocabulary as well. We also have a handy CRIT graphic organizer (I LOVE a good graphic organizer) for students to use as they are researching (one page covers a single source). I am always looking to update and improve, and I'm sure this concept will expand as we use more and more generative AI, but this is working for us for now. 

Hopefully this was helpful! Feel free to copy & modify any of the resources linked below as needed, and leave suggestions in the comments!

 

No comments:

Post a Comment