0

I'm working with textual data from medical field. I have a list of words and I want to build an algorithm that can classify each word into one or more given categories, like

  • Medicine_Name
  • Disease_Name
  • Body_Part

etc.

Please give me some ideas to implement it, specifically how to prepare the training example and implement the training phase.

DaL
  • 2,663
  • 13
  • 13
Koushik Khan
  • 11
  • 1
  • 2

1 Answers1

2

The task you are asking about is Named entity recognition

A good way to identify a word category is by using patterns for the context of the word.

In your case you can use patterns like:

  • "I took X" for a medicine name
  • "I had pain in my X" for a body part
  • "I suffered from X" for a disease

The easiest and fastest way is to discuss the pattern with a domain expert and do a sanity check on the data you have.

A more systematic approach is to deduce the pattern from your data. I guess you can get a list of samples of the items. The technical details depend on the patterns you use but for this example let consider tokenization. Take the context of k-words next to the items on the list. Compare it to the context of other words. Contexts that are related to your items are suitable as patterns.

Bare in mind that these pattern will have very different precision and recall and they will lead to errors. It is important that you will have a labeled dataset so you will be able to estimate the patterns performance.

It will be fairly easy to get pretty good classification. It might be very hard to get a very good classification.

DaL
  • 2,663
  • 13
  • 13