AI Tagging

Your one-stop-shop to tag text data

When dealing with free text data, there is always a chance that one piece of text refers to a variety of topics. Identification of all these topics is an important step when transforming raw text data into insight.

The AI Tagging workflow comprises multiple steps designed in a way that you can:

  • Extract topics with zero coding - the topics can then be used as tags/code frames
  • Apply a human lens, modifying the extracted tags - i.e. relying on your knowledge of the field to generate the final tag list (i.e. code frame)
  • Apply the final tags (i.e. code frames) to your dataset.

Relevance AI provides you with a great user interface for all the steps above. All you need to do is follow the setup wizard; zero coding is required!

📘

AI Tagging

This workflow extracts the topics from text data and allows you to finalize them based on your domain knowledge. Then tags are applied to the whole dataset. Results can be viewed on the Explorer App.

Note: Theme identification in tagging is of one-to-many grouping type.

This means all entries in a dataset are processed. Tags/code frames are extracted. Each entry is tagged with relevant tags based on conceptual similarity. For example:

Sydney's weather and landscape is amazing => tags = Sydney, weather, landscape

How to run AI Tagging

This workflow is composed of three main steps:

  1. Generate tag suggestion (i.e. topics)
  2. Review suggested tags / tagging
  3. Fine-tune (refine) tagging

1. Generate Tag Suggestions

Once you have uploaded your data, select your dataset and click on "AI Tagging" under workflows.

Relevance AI - Access to AI-TaggingRelevance AI - Access to AI-Tagging

Relevance AI - Access to AI-Tagging

As the first step in the setup, you will be asked to specify the text field you wish to analyze.

Relevance AI - AI Tagging workflow setupRelevance AI - AI Tagging workflow setup

Relevance AI - AI Tagging workflow setup

Next, create the tag list. You can

  • include any tags (code frames) you have in mind (optional). You need to hit enter after typing each tag or separate them by comma. It is also possible to copy/paste them from a text file composed of comma separated values. Have a look at tagging best practice guide
  • use "suggest tags" which provides you with two options.
    1. Most frequent words -> A simple count based suggestion excluding words like "the", "is", "were" or "a"
    2. AI suggestion for candidate tags -> This is based on complex AI algorithm and can take up to one hour.

Make sure to refine the suggested tag list using your domain knowledge. The list of tags can be modified:

  • to delete a tag use the x next to it
  • to add more tags, simply type in new ones.

When you are are happy with the list of suggested tags click on "Get started". You will be notified of the progress via a screen similar to the image below. Note that this is a complex process and depending on the size of dataset it might take up to a few hours. You can continue with your other tasks in the meantime.

Relevance AI - AI Tagging workflow setupRelevance AI - AI Tagging workflow setup

Relevance AI - AI Tagging workflow setup

2. Review Tagging

When Step 1 is finished, click on "Review results of tagging". This will open a new page where you can:

  • see the tag list on the left and documents labeled with each tag on the right
  • use search to list documents including a certain phrase

  • Add new tags to a dataset
    When presented with the tags and their corresponding documents, simply type in new tags on the specified place on the top left. This will create a to-add list. Add as many tags as you have in mind, then click on Apply changes. This will re-execute the tagging process on your dataset to apply the changes. Note that you can apply tag addition and removal simultaneously as shown in the image above. Have a look at tagging best practice guide
  • Remove tags from a dataset
    When presented with the tags and their corresponding documents, simply click on the x next to the tags you wish to remove. This will add them to the remove-list. Next, click on Apply changes. This will re-execute the tagging process on your dataset to apply the changes. Note that you can apply tag addition and removal simultaneously as shown in the image above.
Relevance AI - AI Tagging workflow - Add tags to or remove tags from a datasetRelevance AI - AI Tagging workflow - Add tags to or remove tags from a dataset

Relevance AI - AI Tagging workflow - Add tags to or remove tags from a dataset

3. Fine-tune / Refine Tagging

Modify tags assigned to a document / text response
When presented with the tag list as shown below, click on Fine tune tagging

Relevance AI - AI Tagging workflow - refine taggingRelevance AI - AI Tagging workflow - refine tagging

Relevance AI - AI Tagging workflow - refine tagging

This will activate a new window where you can:

Relevance AI - AI Tagging workflow - refine taggingRelevance AI - AI Tagging workflow - refine tagging

Relevance AI - AI Tagging workflow - refine tagging

  1. See which documents are assigned a specific tag - i.e. when clicking on the tag from the menu on the left
  2. Approve good tagging results by clicking on the "thumbs up" icon, under both "Most confident" and "Least confident" tabs
  3. Reject weak tagging results by clicking on the X icon, under both "Most confident" and "Least confident" tabs
  4. Add more documents/responses to a tag (i.e. label more documents with the selected tag) using "Suggested to add" tab
  5. Remove documents/responses from a tag (i.e. remove the selected tag from tags assigned to a document) using "Suggested to remove" tab.

Note 1: Suggestion to add and remove are affected by your list of approved tags and list of removed tags respectively. We recommend 5 cases of approval and 5 cases of rejection where possible. However, one approval and one rejection is the minimum requirement for populating the two lists, suggested to add and suggested to remove respectively.

Note 2: The search bar helps listing documents including certain words and tagged with the selected tag.

Note 3: Clicking on each document / response will show you the full text responses as well as all tags assigned to it. You can remove tags by clicking on the x, or type in new tags and hit "Save changes".

Relevance AI - AI Tagging workflow - review a document tagsRelevance AI - AI Tagging workflow - review a document tags

Relevance AI - AI Tagging workflow - review a document tags

Best Practice

What is a good tag?

  • Is the tag precise?
    This means that the tag properly conveys the intended topic and minimises opportunity for ambiguity. There are several ways that a tag can be not precise. For example:

    • “Pricing/dollars, incl. deals”
      • Could be split into 2 separate tags: “Pricing” and “Deals”
    • “Convenience of Site” → “Close Proximity”
    • "sydney" -> "Sydney" (Capital letters matter!) and similarly "Product Condition" -> "product condition" will help the AI
    • "Shopping Experience" could instead be broken down into different components that make up the shopping experience such as "checkout experience", "parking lot", "friendly staff".
  • Is the tag concise?
    This means that this is not an incredibly wordy tag. A tag should not have unnecessary words where possible as this can dilute meaning.
    For example:

    • “Good Customer Service” → “Customer Service” (unless you are specifically looking for the good component of the word. This is important because good customer service will not tag customer service but customer service will tag good customer service).