CRIT: An updated approach to Information Evaluation for Research

 I have always been passionate about information literacy and getting the facts straight. I used to get in trouble as a kid for talking back, which was usually just me trying to correct an adult's factual error. In high school, I wrote a strongly-worded letter to one of our local journalists because he misidentifed turtles as amphibians in a piece about the Teenage Mutant Ninja Turtles. I screamed myself hoarse when people in powerful positions started using phrases like "alternative facts". In an ideal world, I would teach a compulsory course at my high school that is solely devoted to information evaluation. So when, in the fall of 2022, I learned about the shortcomings of that long-touted librarian tool, the CRAAP test, I went straight to work with my colleague Nadia to come up with something that would be more effective, but still suit our students' needs.

 

We combed through a number of alternative methods for information evaluation, including Mike Caufield's SIFT (the four moves), the book Developing Digital Detectives, the book Fact vs. Fiction, and lessons from The News Literacy Project and the Digital Inquiry Group's (formerly Stanford History Education Group) Civic Online Reasoning (COR) course. These are all excellent resources, but none of them independently satisfied everything we were looking for. We liked the acronym aspect of CRAAP, and felt that SIFT was good for general info evaluation but didn't quite tick all the boxes we wanted for school-related research. So we invented... 

 

4 letters that pack a punch. I always start CRIT lessons with a terrible joke-- "this will help you think CRIT...ically about the information and media you encounter"-- that will hopefully help my students remember to apply this lens to all the media the come across, whether it's for school or not. Then we dive in and break down each letter. 

 

Full disclosure: I usually only get 90 minutes to do what I'm about to describe (we have block scheduling). Ideally, we would break this lesson down and take at least one class per letter so we would have ample time to practice each skill, but c'est la vie. Maybe next year. 

Credibility. 

What is it? How do you determine whether a source is credible? This is a crucial skill for both academic research and personal knowledge building. Here we have conversations about 

• web domains ( .orgs have been available for anyone to purchase since 2019, y'all, and top-level domains [TLDs] are an ever changing beast)
• the open web versus databases (Check out Dr. Kristen Mattson's blog post "Academic Databases are the Netflix for Nerds!" if you struggle to teach this concept to your students) 
  
• how human psychology predisposes us to trust information shared by people we know, and how social media blurs the lines between people we actually know and people we feel like we know.

This is also where we learn about and practice Lateral Reading.  If nothing else, learn about and teach lateral reading. This is a totally transferable skill that is used by professional fact-checkers and will serve our students (and everyone, really) well in our increasingly online worlds, even as we ramp up interactions with generative AI.

One of my favorite lateral reading thought experiments with our students is talking to them about how to lateral read a social media post. People speak with such confidence on TikTok, YouTube, etc... how do you know you can trust what they say on a topic? How do you investigate the user behind the username? We've gotten to put this into action in a really cool assignment for World Literature I (the brainchild of my colleague, Kirsten Reed) wherein students are actually required to use some type of social media as a source. The feedback from the students has been overwhelmingly positive, and it has the twin benefits of transferring this skill to their real online lives and forcing them to understand how to construct a properly formatted citation by hand (I have yet to find an automation for generating a citation from a TikTok URL). 

Relevance. 

What is it? (I always start with that question to make sure we've operating from the same base). How do we determine what is relevant? This question is definitely geared more toward school-based research (some of what we felt was missing from the SIFT approach), but it also offers an opportunity to talk about how the current iteration of the internet works. 

I saw a demonstration of Google searching at a professional workshop a few years back that Blew. My. Mind. So of course I replicate it for my students whenever I get a chance! I usually do this in Google News, but a regular old Google search works too.

1. Perform a search while logged into my work account. 
2. Open a window in another browser and perform the same search while logged into my personal Google account. 
3. Open an Incognito window and perform the same search without being logged in to a Google account. 
4. Show all 3 windows side-by-side.  The results are always different, though sometimes the differences are subtler than others.

This is a segue into a discussion about the algorithms that personalize everything we experience online, from our Google searches to our social media feeds to our streaming media recommendations. We talk about the dangers of echo chambers, how important it is to recognize our personal biases, and some strategies for doing good research in spite of this (using library resources, naturally, but also using Incognito windows and comparing search results with friends).

That's just level one of Relevance. 

Level two is mostly subconscious: if I'm writing about high-speed rail, do a search for "trains", and there's a result about training animals, my brain filters that out for me automatically.

Level three is more intentional: I'm writing about high-speed rail, do a search for "trains," and have to choose between an article about existing train systems in the US and existing train systems in Japan-- I'll probably need to look into those sources to decide which one is more relevant to my topic. This is a great place to dive into search strategies that can help dial in those investigations to return a limited number of results that are most relevant to your topic. (This is really a whole lesson in and of itself.)

Level four is what students seem to struggle with the most: they have a good source, but they need to pull out the most relevant quote to support their argument. As this is a little more English teacher territory, I usually do a short practice with my students so they know where it fits, then move on to the next letter. 

Intention.

What do we mean by Intention? How do we determine the Intention of whoever created this information/media? And why does Intention matter? 

Most of my upper-level students have encountered the media intention (or purpose) acronym PIE- Persuade, Inform, Entertain- but I like to use the News Literacy Project's breakdown of six intentions to dig a little deeper. (You will need to create a free account to access the link above.) NLP proposes that the six primary intentions of media are

• to Document (raw footage or audio with little to no commentary; primary source)
• to Inform ("news" in the traditional sense; straight-forward reporting with some added context)
• to Entertain (self-explanatory; sometimes overlaps with media in other categories)
• to Persuade (to change someone's mind)
• to Sell (to persuade you to part with your money)
• to Provoke (to persuade you to react strongly; to persuade you to take a certain action)

This is a great place to talk about misinformation, disinformation, and malinformation (and to once again plug the benefits of using vetted library resources) and tie back to any classroom discussions students may have had about propaganda. If the intention behind media or information is difficult to determine, this is also a good place to call back to and practice lateral reading. 

Additionally, we can talk about what types of sources are appropriate for school research when, covering the differences between Scholarly vs. Trade vs. Popular sources, and when it might make sense to use social media in school projects-- for example, you would not cite an Instagram post, even one meant to inform, as factual evidence on its own, but you could cite it as an example of the cultural opinion on a topic. 

Finally, we come to 

Timeliness.

What do we mean by Timeliness? How do we determine publication date, and when/why does it matter?

 

Nadia and I felt it was important to include a piece related to publication date (the C in the ol' CRAAP test) because our students are assigned a variety of projects-- some are historical, in which case students need to be cognizant of whether their sources are primary sources from the time in question or secondary (or even tertiary) reports and analyses, and some are focused on current events, in which case our students need information that may change literally minute-to-minute depending on their topics.

 

This explicit reminder to check the publication date seemed too important to let go of. How many times have you encountered an article shared by a friend on Facebook accompanied by an outraged caption and a call to action, only to realize the article was published four years ago? Surely I can't be alone in that. This is the easiest piece of evaluation to accomplish.

Conclusion

I have been covering CRIT with students for a year and a half now, and they seem receptive to it. My English teachers love it, and we are working on making it an acronym that travels outside of library lessons to infiltrate classroom vocabulary as well. We also have a handy CRIT graphic organizer (I LOVE a good graphic organizer) for students to use as they are researching (one page covers a single source). I am always looking to update and improve, and I'm sure this concept will expand as we use more and more generative AI, but this is working for us for now. 

Hopefully this was helpful! Feel free to copy & modify any of the resources linked below as needed, and leave suggestions in the comments!

• CRIT lesson slides
• CRIT graphic organizer 
• Search strategies overview & graphic organizer
  

 

Station Rotation for Secondary Research

I mentioned in my first blog post that my best PD session of 2018 was an hour spent listening to two of my colleagues talk about using Station Rotation in secondary classrooms. Here, I'll explain why it was so darn good.

Being married to a former elementary teacher, I am no stranger to the concept of Station Rotation (or "Centers," as they're calling it in the elementary world these days). I had even been to a session on Station Rotation prior to this one, which is part of the reason I signed up for Katie & Cassie's:I knew there was a way I could use stations in the library, but I didn't quite feel like I could crack it. I hoped listening to them and giving myself another hour to sit with the idea would shake something loose-- and boy, did it! This is how my Station Rotation Research (SRR) approach was born.

As a high school library media specialist, I spend a good deal of time talking to classes about research. How to determine whether sources are credible and sufficiently academic, how to cite sources, how to navigate databases, how to use the library catalog to *gasp* find a book on a given subject.... The sessions are never quite the same, always tailored ever so slightly to meet the needs of a given research or inquiry project. What they all used to have in common, though, was a largely lecture-based approach that resulted in my unending frustration (answering the same questions during independent work time, watching students ignore everything I had said and pull information off a Google results page, etc.).

Enter my Station Rotation breakthough, which I have used (so far) with Health classes researching STDs (one of my favorite projects of the year) and English 10 classes doing Decades research. It goes like this:

• Before I meet with the classes, I pull relevant books and have them on a cart so they're handy.*
• I do a whole-class spiel that includes information on how to locate & navigate our databases, how to generate (or find) citations, and anything else that is relevant to the project (sometimes talking about the differences between databases & websites, primary & secondary sources, etc.). 
• I physically break the class into three groups and have each group rotate through each of three stations. These take one of two forms:
• a database station, a website station**, and a book station; or
• a digital station, a print station, and an instructional station to learn a new digital tool such as a particular presentation software
• Each group gets 10-15 minutes at each of the three stations, and they must conduct their research using the station's designated resource type. With the time remaining at the end of the class, they are free to conduct the rest of their research using whichever resources best meet their needs.***
• I do make them get up and physically move from station to station (in spite of their protestations). I think the act of moving helps get the blood flowing and the brain working, and it helps them mentally transition from one type of resource to the next. 
• Limiting their time and frequently reminding them that their time is limited seems to help keep them more on task.
• I circulate through the stations as I am able to help guide their research.

Thanks to this approach, I have seen an huge increase in students using books for their research. Last year my carefully pre-selected books stayed ignored on their cart while students opted for easily searchable databases (or worse, unfocused Google searches). This year I've actually had students ask if they can keep using the books, and some have even confessed that they prefer book research! You could have knocked me over with a feather. 

You better believe with student responses like that I'm going to keep doing this. 

Some possible future tweaks/goals:

• students at the book station using that time to actually search for their own books using the library catalog (for more individualized research projects).
• students at the website station completing a source evaluation exercise (maybe filling out a CRAAP graphic organizer) and learning about advanced Google search techniques.
• students at the database station learning which databases are best for what types of research and determining what best meets their specific research needs instead of my telling them which databases to go to.

Bring on the new semester!

*I was already in the habit of pre-selecting books for these projects.
**Many of our classes use pathfinders of predetermined web sources
***We have 4 90-minute blocks a day, so I'm fortunate enough to have time to do all of this in one class period. If we were on a regular period schedule, I would probably do the class spiel on day one and the stations on day two.