Top Books on Natural Language Processing

Natural Language Processing, or NLP for short, is the study of computational methods for working with speech and text data.

The field is dominated by the statistical paradigm and machine learning methods are used for developing predictive models.

In this post, you will discover the top books that you can read to get started with natural language processing.

After reading this post, you will know:

  • The top books for practical natural language processing.
  • The top textbooks for the theoretical foundations of natural language processing.
  • The NLP books I have on my shelf.

Let’s get started.

Top Practical Books on Natural Language Processing

As practitioners, we do not always have to grab for a textbook when getting started on a new topic.

Code examples in the book are in the Python programming language.

Although there are fewer practical books on NLP than textbooks, I have tried to pick the top 3 books that will help you get started and bring NLP method to your machine learning project.

1. Natural Language Processing with Python

Written by Steven Bird, Ewan Klein and Edward Loper.

This book provides an introduction to NLP using the Python stack for practitioners.

The book focuses on using the NLTK Python library, which is very popular for common NLP tasks.

Contents include:

  1. Language Processing and Python
  2. Accessing Text Corpora and Lexical Resources
  3. Processing Raw Text
  4. Writing Structured Programs
  5. Categorizing and Tagging Words
  6. Learning to Classify Text
  7. Extracting Information from Text
  8. Analyzing Sentence Structure
  9. Building Feature-Based GRammars
  10. Analyzing the Meaning of Sentences
  11. Managing Linguistic Data

This book is perfect if you are looking at getting into classical NLP using the go-to NLTK platform.

Resources

2. Taming Text

This book provides an introduction to a suite of different NLP tools and problems, such as Apache Solr, Apache OpenNLP, and Apache Mahout.

Amazon Image

Code examples are in Java.

It may be more suited to developers getting started with larger enterprise-grade NLP tools on work projects.

Written by Grant Ingersoll, Thomas Morton and Drew Farris.

Notably, Grant Ingersoll is a cofounder of the Apache Mahout project.

Contents include:

  1. Getting Started Taming Text
  2. Foundations of Taming Text
  3. Searching
  4. Fuzzy String Matching
  5. Identifying People, Places and Things
  6. Clustering Text
  7. Classification, Categorization and Tagging
  8. Building an Example Question Answering System
  9. Untaming Text: Exploring the Next Frontier

Resources

3. Text Mining with R

Written by Julia Silge and David Robinson.

Amazon Image

This book demonstrates statistical natural language processing methods on a range of modern applications.

Code examples are in R.

Code focuses on the “tidy” principles by Hadley Wickham (paper) and the tidytext package by the authors.

Of the three books, this is the most recently published and has a more practical and modern feel to the demonstrations.

Contents include:

  1. The Tidy Text Format
  2. Sentiment Analysis with Tidy Data
  3. Analyzing word and Document Frequency: tf-idf
  4. Relationships Between Words: N-grams and Correlations
  5. Converting to and from Nontidy Formats
  6. Topic Modeling
  7. Case Study: Comparing Twitter Archives
  8. Case Study: Mining NASA Metadata
  9. Case Study: Analyzing Usenet Text

Resources

Do you know of other great practical books on natural language processing?
Let me know in the comments.

Top Textbooks on Natural Language Processing

There are a ton of textbooks on natural language processing and on specific sub-topics.

In this section, I have tried to focus on what I (and consensus) seems to see as the best books on the topic for beginners, e.g. undergraduate or graduate students and practitioners looking to step deeper into the theory.

I have tried to pick a mix of general NLP books as well as books on highly studied topics like translation and speech.

The first two books in this section are essentially cannon for NLP students.

1. Foundations of Statistical Natural Language Processing

Written by Christopher Manning and Hinrich Schütze.

Amazon Image

Notably, Christopher Manning teaches NLP at Stanford and is behind the CS224n: Natural Language Processing with Deep Learning course.

This book provides an introduction to statistical methods for natural language processing covering both the required linguistics and the newer (at the time, circa 1999) statistical methods.

This book provides a strong foundation to better grasp the newer methods and encodings.

Contents include:

  1. Introduction
  2. Mathematical Foundations
  3. Linguistic Essentials
  4. Corpus-Based Work
  5. Collocations
  6. Statistical Inference: n-gram Models over Sparse Data
  7. Word Sense Disambiguation
  8. Lexical Acquisition
  9. Markov Models
  10. Part-of-Speech Tagging
  11. Probabilistic Context Free Grammars
  12. Probabilistic Parsing
  13. Statistical Alignment and Machine Translation
  14. Clustering
  15. Topics in Information Retrieval
  16. Text Categorization

Resources

2. Speech and Language Processing

Written by Daniel Jurafsky and James Martin.

Amazon Image

This book provides coverage of NLP from both speech and text perspectives with a strong focus on applications (one in each chapter).

Coverage of the topic feels exhaustive.

Contents include:

  1. Introduction
  2. Regular Expressions and Automata
  3. Words and Transducers
  4. N-grams
  5. Part-of-Speech Tagging
  6. Hidden Markov and Maximum Entropy Models
  7. Phonetics
  8. Speech Synthesis
  9. Automatic Speech Recognition
  10. Speech Recognition: Advanced Topics
  11. Computational Phonology
  12. Formal Grammars of English
  13. Syntactic Parsing
  14. Statistical Parsing
  15. Features and Unification
  16. Language and Complexity
  17. The Representation of Meaning
  18. Computational Semantics
  19. Lexical Semantics
  20. Computational Lexical Semantics
  21. Computational Discourse
  22. Information Extraction
  23. Question Answering and Summarization
  24. Dialog and Conversational Agents
  25. Machine Translation

Resources

4. Statistical Machine Translation

Written by Philipp Koehn.

Amazon Image

This book provides an introduction to the topic of statistical machine translation, a s subfield of NLP.

Contents include:

  1. Introduction
  2. Words, Sentences, Corpa
  3. Probability Theory
  4. Word-Based Models
  5. Phrase-Based Models
  6. Decoding
  7. Language Models
  8. Evaluation
  9. Discriminative Training
  10. Integrating Linguistic Information
  11. Tree-Based Methods

Resources

5. Statistical Methods for Speech Recognition

Written by Frederick Jelinek.

Amazon Image

This book provides an introduction to the topic of statistical speech recognition, another subfield of NLP that saw an overhaul in the 1990s with statistical approaches.

Contents Include

  1. The Speech Recognition Problem
  2. Hidden Markov Models
  3. The Acoustic Model
  4. Basic Language Modeling
  5. The Viterbi Search
  6. Hypothesis Search on a Tree and the Fast Match
  7. Elements of Information Theory
  8. The Complexity of Tasks – The Quality of Language Models
  9. The Expectation-Maximization Algorithm and Its Consequences
  10. Decision Trees and Tree Language Models
  11. Phonetics from Orthography: Spelling-to-Base Form Mappings
  12. Triphones and Allophones
  13. Maximum Entropy Probability Estimation and Language Models
  14. Tree Applications of Maximum Entropy Estimation to Language Modeling
  15. Estimation of Probabilities from Counts and the Back-Off Method

Resources

NLP Books that I Own

I like to have a mixture of practical and reference texts on my shelf.

The hard part of NLP (for me) is simply the large number of sub-problems and the specialized terminology and theory used.

For this reason I have the following 3 NLP textbooks on my shelf:

I also really like the look of:

I recommend choosing the NLP books that are right for you and your needs or project.

Let me know which books you chose or own.
Leave a comment below.

Further Reading

This section provides more resources on the topic if you are looking go deeper.

Top NLP Books

Quora

Summary

In this post, you discovered the top books on natural language processing.

Specifically, you learned:

  • The top books for practical natural language processing.
  • The top textbooks for the theoretical foundations of natural language processing
  • The NLP books I have on my shelf.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.