Alan Leal

Teaching

Llama for economists

January 2025.

Llama is a generative language model released by Meta. Unlike paid services such as ChatGPT, Gemini, Claude, Copilot, or Grok, Llama can be downloaded and run on a personal computer. Running it locally avoids per-request API costs and the input and output token limits of commercial services. A 250-page PDF, for example, may exceed the input limit of a commercial service, and long answers may exceed its output limit.

This guide applies a local Llama installation to a task that would quickly hit those limits. Section 1 covers installation, section 2 the classification task and the R code, and section 3 the results. All analyses ran on a MacBook Air M1 with 8 GB of RAM and a 256 GB NVMe SSD.

1. Installation

# Llama installation guide

## Requirements
- Operating system: Windows 10/11, macOS, or Linux
- At least 16 GB of RAM (32 GB recommended)
- At least 10 GB of free disk space
- Python 3.9 or later
- Git

## Step 1: Prepare the environment
mkdir llama-project
cd llama-project
python -m venv llama-env
# Windows
llama-env\Scripts\activate
# macOS/Linux
source llama-env/bin/activate

## Step 2: Clone the repository
git clone https://github.com/facebookresearch/llama.git
cd llama

## Step 3: Install dependencies
pip install -r requirements.txt
pip install torch torchvision torchaudio

## Step 4: Download the model
# Request access to the models on the Meta AI website.
# After approval, download the model and extract it to ./models.

## Step 5: Configure
# config.json
{
    "model_path": "./models/llama-2-7b",
    "max_seq_len": 512,
    "max_batch_size": 32
}

## Step 6: Test
python3 example.py

## Troubleshooting
# CUDA not found: check for a compatible NVIDIA GPU, install current CUDA drivers,
#   and reinstall PyTorch with CUDA support.
# Out of memory: reduce the batch size, close other programs, or use a smaller model.
# Import errors: activate the virtual environment and reinstall the dependencies.

2. Classifying studies that use quantitative trade models

2.1. Sample

The task asks two questions about each paper: does it mention the exact-hat algebra method, and does it use it? The sample consists of the first 15 valid results of a Google Scholar search for “quantitative trade model.” The papers are listed at the end of this guide. The same procedure scales to samples of 100 or 200 papers.

2.2. R code

The script reads each PDF, tokenizes it, sends it to Llama with the query through the Ollama API, and organizes the response.

# Install required packages
required_packages <- c("pdftools", "httr", "jsonlite", "stringr", "dplyr", "tidyr")
new_packages <- required_packages[!(required_packages %in% installed.packages()[,"Package"])]
if(length(new_packages)) install.packages(new_packages)

library(pdftools)   # read PDFs
library(httr)       # HTTP requests
library(jsonlite)   # JSON
library(stringr)    # strings
library(dplyr)      # data manipulation
library(tidyr)      # data tidying

# Preprocess text: lowercase, remove special characters, normalize spaces
preprocess_text <- function(text) {
  text <- tolower(text)
  text <- stringr::str_replace_all(text, "[^[:alnum:][:space:]]", " ")
  text <- stringr::str_squish(text)
  return(text)
}

# Tokenize text
tokenize_text <- function(text) {
  tokens <- unlist(stringr::str_split(text, "\\s+"))
  return(tokens)
}

# Check whether the Ollama service is running
check_ollama_connection <- function() {
  tryCatch({
    response <- httr::GET("http://localhost:11434/api/tags")
    return(httr::status_code(response) == 200)
  }, error = function(e) {
    return(FALSE)
  })
}

# Classify a document through the Ollama API
classify_document <- function(text) {
  prompt <- "INSTRUCTION:
YOU MUST ANSWER EXACTLY IN THIS FORMAT:
FORMAT:

First line: exacthat (YES or NO)
Second line: exacthatuse (YES or NO)
Third line onwards: text of maximum 150 words justifying the answers

VALID RESPONSE EXAMPLE:
YES
NO
The article mentions the exact-hat algebra method in its literature review, citing it as one of the existing approaches for equilibrium analysis. However, the article does not apply this method in its empirical analysis, opting instead to use a different approach based on spatial econometrics.
RULES:

Answer if the text mentions any of these variations: exacthat algebra, exact-hat algebra, exact hat algebra, or any combination of these terms with different spacing or hyphenation
Answer if the text effectively uses any of the above variations of exact hat algebra as an analytical method
Justify your answers in up to 150 words

TEXT:
"

  # Limit text length and combine with the prompt
  text_limited <- substr(text, 1, 100000)
  full_prompt <- paste0(prompt, text_limited)

  url <- "http://localhost:11434/api/generate"
  body <- list(
    model = "llama3.2:latest",
    prompt = full_prompt,
    stream = FALSE
  )

  tryCatch({
    response <- httr::POST(
      url,
      body = jsonlite::toJSON(body, auto_unbox = TRUE),
      httr::add_headers("Content-Type" = "application/json")
    )

    if (httr::status_code(response) == 200) {
      result <- httr::content(response, "parsed")$response
      print(paste("Raw response:", result))

      lines <- unlist(strsplit(result, "\n"))
      exacthat <- trimws(toupper(lines[1])) == "YES"
      exacthatuse <- trimws(toupper(lines[2])) == "YES"
      explanation <- paste(lines[-(1:2)], collapse = "\n")

      return(list(
        exacthat = exacthat,
        exacthatuse = exacthatuse,
        explanation = explanation
      ))
    } else {
      return(list(
        exacthat = FALSE,
        exacthatuse = FALSE,
        explanation = "Error: API request failed",
        error = "API request failed"
      ))
    }
  }, error = function(e) {
    return(list(
      exacthat = FALSE,
      exacthatuse = FALSE,
      explanation = paste("Error:", as.character(e)),
      error = as.character(e)
    ))
  })
}

# Process a single PDF
process_document <- function(file_path) {
  tryCatch({
    if (!check_ollama_connection()) {
      stop("Could not connect to Ollama. Check that the service is running.")
    }
    text <- pdftools::pdf_text(file_path)
    text <- paste(text, collapse = " ")
    processed_text <- preprocess_text(text)
    tokens <- tokenize_text(processed_text)
    classification <- classify_document(processed_text)

    return(list(
      file_path = file_path,
      exacthat = classification$exacthat,
      exacthatuse = classification$exacthatuse,
      explanation = classification$explanation,
      token_count = length(tokens),
      sample_tokens = head(tokens, 10),
      status = "success"
    ))
  }, error = function(e) {
    return(list(
      file_path = file_path,
      status = "error",
      error_message = as.character(e)
    ))
  })
}

# Process several PDFs and optionally save a CSV
process_multiple_documents <- function(file_paths, output_file = NULL) {
  results <- lapply(file_paths, process_document)

  results_df <- bind_rows(lapply(results, function(x) {
    if (x$status == "success") {
      data.frame(
        file_path = x$file_path,
        status = x$status,
        exacthat = as.numeric(x$exacthat),
        exacthatuse = as.numeric(x$exacthatuse),
        explanation = x$explanation,
        token_count = x$token_count,
        sample_tokens = paste(x$sample_tokens, collapse = ", ")
      )
    } else {
      data.frame(
        file_path = x$file_path,
        status = x$status,
        exacthat = NA,
        exacthatuse = NA,
        explanation = NA,
        token_count = NA,
        sample_tokens = NA,
        error_message = x$error_message
      )
    }
  }))

  if (!is.null(output_file)) {
    write.csv(results_df, output_file, row.names = FALSE)
  }
  return(results_df)
}

# Example: replace the path with the folder that holds the PDFs
files <- list.files("path_to_pdf_files", full.names = TRUE)
results <- process_multiple_documents(files, "analysis_results.csv")
print(results)

3. Results

3.1. Example response

The response for Mei (2020):

exacthat: 0
exacthatuse: 0

The text appears to be discussing various aspects of international trade,
economic policy, and game theory. Here are some possible summary points:

1. Trade models: The text discusses different types of trade models, including
   infinite punishment, finite punishment, and punishment with finite periods.
2. Nash equilibrium: The authors discuss the concept of Nash equilibrium in
   repeated games, where countries cooperate on tariffs to maximize their welfare.
3. Sustainability of cooperative equilibria: The text explores the conditions
   under which cooperative equilibria are sustainable, including the minimum
   patience parameter and the length of the punishment phase.
4. Tariff data: The authors use various sources of tariff data, including the
   Global Trade Analysis Project (GTAP) database and the US export data.
5. Trade policy analysis: The text discusses the importance of quantitative
   trade policy analysis in understanding the impact of trade policies on
   economic outcomes.

The model classified the paper correctly on both questions, but the justification is a generic summary rather than a reasoned answer to the prompt.

4. Conclusion

The exercise extracts structured information from a set of papers in a few minutes. It can serve as a first filter in a literature review, before the slower manual reading.

References

  1. Adao, R., Costinot, A., and Donaldson, D. (2022). Putting quantitative trade models to the test: evidence from the Trump tariffs. Mimeo.
  2. Bekkers, E., and Koopman, R. B. (2022). Simulating the trade effects of the COVID-19 pandemic: scenario analysis based on quantitative trade modelling. The World Economy, 45(2), 445–467.
  3. Bekkers, E., and Rojas-Romagosa, H. (2019). The welfare effects of free trade agreements in quantitative trade models: a comparison of studies about Transatlantic Trade and Investment Partnership. The World Economy, 42(1), 87–121.
  4. Blank, S., Egger, P. H., Merlo, V., and Wamser, G. (2022). A structural quantitative analysis of services trade de-liberalization. Journal of International Economics, 137, 103605.
  5. Che, Z., Che, J., and Wang, S. (2024). The gains from changes in internal trade costs: a quantitative analysis of China. Journal of Regional Science.
  6. Egger, P. H., and Erhardt, K. (2024). Heterogeneous effects of tariff and nontariff trade-policy barriers in quantitative general equilibrium. Quantitative Economics, 15(2), 453–487.
  7. Felbermayr, G., Gröschl, J. K., and Heiland, I. (2018). Undoing Europe in a new quantitative trade model. Ifo Working Paper 250.
  8. Huang, X., and Chen, X. (2022). A quantitative model of international trade based on deep neural network. Computational Intelligence and Neuroscience, 2022(1), 9811358.
  9. Krebs, O., and Pflüger, M. (2018). How deep is your love? A quantitative spatial analysis of the transatlantic trade partnership. Review of International Economics, 26(1), 171–222.
  10. Lee, K. Y. (2018). A quantitative trade model with unemployment. KIEP Research Paper, Working Paper 18-04.
  11. Mei, Y. (2020). Sustainable cooperation in international trade: a quantitative analysis. Journal of International Economics, 123, 103305.
  12. Ritel, M. (2022). A quantitative analysis of trade cooperation over three decades. Available at SSRN 4252542.
  13. Thoenig, M. (2023). Trade policy in the shadow of war: a quantitative toolkit for geoeconomics. Centre for Economic Policy Research.
  14. Walter, T. (2022). Applied policy research through the lens of new quantitative trade models.
  15. Yu, Z., Han, J., Shi, X., and Yang, Y. (2024). Estimates of the trade and global value chain (GVC) effects of China’s Pilot Free Trade Zones: a research based on the quantitative trade model. Journal of the Asia Pacific Economy, 1–48.