New task for ChatGPT: Deciphering messages from the Fed
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Eniola Odetunde/Axios
Attention fellow Federal Reserve watchers: ChatGPT can apparently read between the lines of crucial — and sometimes challenging-to-parse — communications from central bankers.
Why it matters: That is the upshot of a recent paper by two Fed staffers who ran several tests to determine whether the artificial intelligence tool could accurately decipher hawkish signals from dovish ones. According to their findings, the answer is yes.
How it works: Richmond Fed researchers Anne Lundgaard Hansen and Sophia Kazinnik randomly selected hundreds of sentences from Fed policy statements published between 2010 and 2020.
- Each sentence was assigned a label — like "hawkish" or "mostly dovish" or "neutral." The team then evaluated how the chatbot's classifications compared to those human classifications.
What they're saying: Hansen and Kazinnik found that GPT models "demonstrate a strong performance in classifying Fedspeak sentences." The researchers also said that the GPT "successfully justifies its classifications, and furthermore is very similar to the reasoning provided by a human reviewer."
- They found that GPT was better at the task than older language models, including a popular one introduced by Google in 2018.
Yes, but: The researchers warn that AI technology "may still misclassify sentences or fail to capture nuances that a human evaluator with domain expertise might capture."
- "Thus, while GPT models may not be able to fully replace human evaluators, they can serve as a highly valuable tool for assisting researchers and analysts in this domain."
