Skip to content

Key humanitarian terms

Here are the top results for key terms in the English and Spanish humanitarian corpora, broken down by ngram (the number of words per term), from 1gram to 5gram. The tables offer a summary of the most common concepts mentioned by humanitarians, and the treemap charts offer more detailed data.

Contents

These tables include the top 100 key terms for each of the statistics described below: up to 500 rows, removing duplicates. This is done for each ngram size, offering several thousand of the top terms of varying length. It's not a complete list, and each statistic has its limits, but it's a good starting point for identifying the concepts that make up the core of the humanitarian domain.

Understanding the data

Each table has a few columns to sort data by:

  • lemma: the lemma (canonical form) of the term's headword (its syntactic root)
  • value: the full term
  • k1: keyness score with a smoothing value of 1
  • k100: keyness score with a smoothing value of 100
  • k10000: keyness score with a smoothing value of 10,000
  • kendall_tau: the strength of the term's trend over time, from 2000 through 2024
    • positive numbers indicate an upward trend
    • negative numbers indicate an downward trend
  • frequency: the absolute frequency (number of occurrences of the term)

Value includes terms collected from our 2024 humanitarian corpora, which include most of ReliefWeb's content. The values are extracted with Sketch Engine term grammars.

Lemma is determined by term dependency parsing with Stanza's models.

Keyness scores use an equation to identify the "most important" terms. Term frequencies are compared in the humanitarian corpus with a more domain-independent corpus - here we use dumps of Wikipedia's 2024 English and Spanish content. Terms that appear often in humanitarian texts but infrequently in Wikipedia, relative to corpus size, are scored higher for keyness. Multiple keyness calculations are made with different smoothing scores, which tend to rank terms differently.

Kendall's Tau is calculated to determine the strength of frequency trends over time, implemented as described by Hilpert and Gries (2009).

Data filtering

The results shown here have been filtered to focus on the most relevant terms, excluding:

  • Very short words (3 characters or less)
  • URIs
  • Symbols
  • Possessive headwords
  • A few false positives, like email and párr
  • All uppercase or title case
    • Eliminates extraneous content, like proper nouns, but also removes important acronyms (IDP) and valid uppercase terms like COVID-19
  • Case-insensitive duplicates