16 things you can do with Natural Language Processing

16 things you can do with Natural Language Processing

The world of Natural Language Processing (NLP) has been hotting up in recent years. In 2018, the field was up-ended by a new NLP architecture called a ‘transformer’. Transformers represented a real step-forward in the way machines can understand text in context. And, building on top of this, in 2020 OpenAI released their GPT-3 model - a transformer trained on an eye-wateringly large amount and breadth of text. It shows an impressive ability to understand and generate natural language.

I’m writing this post to give a non-technical overview and examples of the types of things can be done with NLP. It’s worth noting that many of these areas are pretty immature and the performance of these models isn’t as reliable as their demos might suggest.

If I’m missing anything, please feel free to get in touch.

  1. Automatic text summarisation
  2. Question answering
  3. Information extraction
  4. Conversational bots
  5. Text classification
  6. Language identification
  7. Machine translation
  8. Write code using natural language
  9. Get an explanation of some code in natural language
  10. Create non-language content based on a description
  11. Get a description of visual content
  12. Generative text
  13. Autocorrect
  14. Paraphrase detection
  15. Word sense disambiguation
  16. Textual entailment


1) Automatic text summarisation

What?

The process of taking one or multiple text documents and creating a summary that represents the most important or relevant information from the original text. These summaries can either be “generic” (a general overview of the original text) or “query relevant” (a summary that only focuses on the text that is relevant to an inputted topic). The summarisation process is either extractive (directly reproducing parts of the source text word-for-word) or abstractive (forming an internal semantic representation of the original content and using this to write the summary from scratch).

Try it out for yourself


2) Question answering

What?

Tools that can answer questions that are asked in natural language.

It bases its answers on either text it saw during training or on some text you provide to it at the same time as asking the question. As with text summarisation, the answering process can either be extractive (directly quoting the source text) or abstractive (writing the answer based on an internal semantic representation of the original content).

Example: Semantic search


3) Information extraction

What?

The task of automatically extracting structured information from unstructured or semi-structured text documents.

Information extraction can facilitate further computation to be done on the previously unstructured data (e.g. analysing incoming customer support tickets to automatically identify order numbers, customer names). There are two main types of Information extraction: Named entity recognition and Relation extraction.

Named entity recognition allows you to identify all entities of a predefined category (e.g. Extract all cities; or extract all company names).

Relation extraction builds on top of named entity recognition. In addition to finding the entities, it allows you to detect the semantic relationships between them (e.g. Extract all countries and their capital cities; or extract all companies and the year they were founded in).

Example: From Natural Language to a Financial Statement

Try it out for yourself


4) Conversational bots

What?

Normal chatbots that are common on company websites / apps. They can hold conversations, answer your questions or carry out simple tasks on your behalf (e.g. changing a setting in your account, placing an order or scheduling a meeting for you).


5) Text classification

What?

The process of sorting pieces of text into one or more predefined categories. Examples of how this can be used include:

  • Text sentiment classification;
  • Email urgency detection ;
  • Spam filters;
  • Determining whether the author is making a claim or not - as the first step in fact-checking;
  • Analysing trends in social media monitoring.

Try it out for yourself


6) Language identification

What?

A tool to determine the language of some inputted text. It is effectively a form of text classification.

Try it out for yourself


7) Machine translation

What?

Something that can translate inputted text from one language (or style of language) to another.

Examples of “translating” from one form of English to another include:

  • Translating complex “legalese” to English that can be understood by a 2nd grader;
  • Rewriting your email in a less aggressive version of English.

Example 1: From legalese to plain English

Example 2: Make your email sound less aggressive

Try it out for yourself


8) Write code using natural language

What?

Describe what you are trying to achieve, and let the AI draft the code for you (e.g. HTML, CSS, SQL query and Linux commands).

At present the tools that can do this are imperfect and can only really be used to write a first draft that you would need to review.

Example 1: Write Front End code with natural language

Example 2: Write SQL queries with natural language


9) Get an explanation of some code in natural language

What?

Something that explains to you what a piece of code is doing (e.g. Python, SQL).

Example: Describe what a SQL query is doing


10) Create non-language content based on a description

What?

Tools that can create content (such as photograph-like images, user interface designs or music notation) from a written description.

Example 1: Create photo-like images:

Example 2: Create UI Design:


11) Get a description of visual content

What?

The process of using AI to analyse content (such as images or videos) and then add appropriate captions to them.

Example: Adding a descriptive caption to an image


12) Generative text

What?

Tools that can generate text based on some user input. Example applications include:

  • Autocomplete a word or sentence that you have started to write;
  • Writing a passage of text based on an inputted theme or question;
  • Drafting an email based on key points;
  • Writing poetry or song lyrics based on a suggested song title.

Example: Create an email reply from simple bullet point

Try it out for yourself


13) Autocorrect

What?

The process of checking text for spelling and grammatical errors, general tone or other imperfections and suggesting improvements that can be made.

Try it out for yourself


14) Paraphrase detection

What?

A technique that is used to determine whether a pair of sentences have the same meaning or intent.

The types of application this can be used on include:

  • Detecting if someone else has previously asked a semantically equivalent question, in an online forum;
  • Detect plagiarism.

Try it out for yourself


15) Word sense disambiguation

What?

This is the process of determining the intended meaning of a word that has many definitions. An example of such a word is “pen” which can be used as a noun to refer to five different things (including a writing implement, a livestock enclosure and a female swan).

Example 1: A contextual dictionary

Try it out for yourself


16) Textual entailment

What?

A technique that evaluates a pair of sentences and then determines whether:

  • The facts in the first sentence necessarily imply the facts in the second;
  • The facts in the second sentence contract those in the first;
  • Or whether the two sentences are unrelated.

Textual entailment entailment is not the same as pure logical entailment – it has a more relaxed definition. In textual entailment “A entails B” if, typically, a human reading A would infer that B is most likely true.

Try it out for yourself