Text Classification Using Support Vector Machines (SVM)

There are many different algorithms we can choose from when doing text classification with machine learning. One of those is Support Vector Machines (or SVM). In this article, we will explore the advantages of using support vector machines in text classification and will help you get started with SVM-based models in MonkeyLearn.

From Texts to Vectors

Support vector machines is an algorithm that determines the best decision boundary between vectors that belong to a given group (or category) and vectors that do not belong to it. That’s it. It can be applied to any kind of vectors which encode any kind of data. This means that in order to leverage the power of svm text classification, texts have to be transformed into vectors.

Now, what are vectors? Vectors are (sometimes huge) lists of numbers which represent a set of coordinates in some space. So, when SVM determines the decision boundary we mentioned above, SVM decides where to draw the best “line” (or the best hyperplane) that divides the space into two subspaces: one for the vectors which belong to the given category and one for the vectors which do not belong to it.

So, provided we can find vector representations which encode as much information from our texts as possible, we will be able to apply the SVM algorithm to text classification problems and obtain very good results.

Say, for example, the blue circles in the graph below are representations of training texts which talk about the Pricing of a SaaS Product and the red triangles are representations of training texts which do not talk about that. What would the decision boundary for the Pricing category look like?

Representations of training texts

The best decision boundary would look like this:

Best decision boundary in SVM

Now that the algorithm has determined the decision boundary for the category you want to analyze, you only have to obtain the representations of all of the texts you would like to classify and check what side of the boundary those representations fall into.

There’s a great many ways of encoding texts in vectors. If you want to learn more about some of them, read this. You can also learn a lot more about support vector machines and kernel functions here.

Creating a Text Classifier with SVM

Creating a text classifier using SVM is easy and straightforward with MonkeyLearn. First of all, sign in to the platform and then click on create a model. You will be prompted to choose the model type you would like to create. Let’s choose Classifier:

Creating a text classifier with support vector machines

Now, you will have to choose the type of classification task you would like to perform. In this mini tutorial, we are going to show you how to create a model to classify the topics being dealt with in texts from hotel reviews, so let’s choose Topic Classification. However, bear in mind that text classification using SVM can be just as good for other tasks as well, such as sentiment analysis or intent classification:

Choosing topic classification

Now it’s time to import your data:

Import your training data to the classifier

Since we’ll be using a pre-existing dataset, let’s choose Data Library and download the sample dataset for hotel reviews:

Downloading a sample dataset for hotel reviews

Then, go back to the previous screen and click the CSV card. The screen below will appear:

Uploading a CSV file with the sample dataset

Once we’ve chosen our CSV file with the sample dataset, a screen like the one below will appear with a preview of the data, let’s click Continue:

Hotel reviews dataset preview

The next step is to define the tags we want to use in our classifier. Let’s define a few tags like Location, Comfort & Facilities, and Staff:

Define the tags for your classifier

Now, it’s time to tag data to train our classifier. By tagging some examples, SVM will learn that for a particular input (text), we expect a particular output:

Tag data for training your classifier

Once you have finished taking care of your training data, you will have to name your classifier before you can keep training it, start using it, or change its settings. Type some descriptive name in the textbox and click Finish:

Name your classifier

Since MonkeyLearn uses SVM as the default classification algorithm, you won’t need to change your classifier’s advanced settings at this point unless you would like to make some other adjustments. Just give it a try, go to Run and try it out. Here’s an SVM text classification example from hotel review:

Testing your classifier

Chances are that some results are not as good as you expect, especially if you have not uploaded a lot of training data. Don’t worry! If this happens, go to Build > Data (see screenshot below) and try uploading more data, tagging it, and trying the classifier again until you get the results you expect:

Uploading more training data to the classifier

Final words

Using SVM classifiers for text classification tasks might be a really good idea, especially if the training data available is not much (~ a couple of thousand tagged samples). Besides, Monkeylearn makes it really simple and straightforward to create text classifiers. Within minutes, you will be able to get really good results out of custom texts classifiers using SVM which will give you great new insights on your data. Why don't you give it a try?

Start using Text Classification today!

Automate business processes and save hours of manual data processing.