Natural Language Processing (NLP)

Intersection of computer science, machine learning, and linguistics
Focus on computational understanding, interpretation, and generation of human language

Natural Language Processing (NLP)

Key aspects of NLP include:

Text analysis and understanding
Language generation
Machine translation
Sentiment analysis
Speech recognition
Question answering systems
Text summarization
Named entity recognition

Installing spacy

/path/to/bin/python3 -m pip install -U pip setuptools wheel
/path/to/bin/python3 -m pip install -U spacy
/path/to/bin/python3 -m spacy download en_core_web_sm

Example code for spacy

import spacy

def main():
    nlp = spacy.load("en_core_web_sm")
    text = "This is a test sentence so that we can see the parts of speech and dependencies"
    doc = nlp(text)

    for token in doc:
        print(token.text, token.pos_, token.dep_)

main()

Challenge

What is the most frequent noun in a movie script of your choice?

Choose a movie script to work on, tag the script with part of speech labels, and count the nouns. Which is the most frequent noun in the script you chose?

How do you visualize your results?