Week 9

My task: To test the training model on the Biomedical Evaluation dataset, and record my findings.

For the first run through the training model, I just ran the model as it was to see my beginning results.

Run #1

  • Training: TRAIN_biomedical_fullarticles_version2.csv
  • Testing: TRAIN_biomedical_abstracts_version2.csv
  • Changes: N/A
  • Accuracy: 0.932271923174

Then, I attempted to test the code on the Biomedical Evaluation data. Initially, the code for this wouldn’t even run. I kept getting a “utf8″ error. Upon research, I learned that this meant that the training model could not interpret some of the characters in the Biomedical data set, which in some way is to be expected because there are many different and complex symbols used within the medical field. Because those symbols are not really essential to the labeling of the sentences as certain or uncertain, we can simply ignore those complex characters. Adding ” encoding=’latin-1′ ” to the end of the lines that read each data file does exactly that.

Run #2

  • Training: TRAIN_biomedical_fullarticles_version2.csv
  • Testing: Biomedical_EVAL_version2.csv
  • Change: added ” encoding=’latin-1′ ” to the end of each csv.read() line
  • Accuracy: 0.0

Surprisingly, I got a 0% accuracy measurement. This means that the model I trained didn’t predict any of the sentences in the Biomedical Evaluation dataset correctly. I did some research, talked my mentor. Turns out, the model was reading all of the “tabs” in the code, which was the spacing I chose to separate each sentence with its certain/uncertain label, as a series of commas. Thus, the data wasn’t being interpreted correctly. So I fixed it, and ran it on the original training and testing set to see if it worked.

Run #3

  • Training: TRAIN_biomedical_fullarticles_version2.csv
  • Testing: TRAIN_biomedical_abstracts_version2.csv
  • Changes: N/A
  • Accuracy: 0.934967568023

Success! Now, let’s try the Biomedical Evaluation set…

Run #4

  • Training: TRAIN_biomedical_fullarticles_version2.csv
  • Testing: Biomedical_EVAL_version2.csv
  • Change: N/A
  • Accuracy: 0.923046172297

Success!! I got a 92% accuracy measurement on the Biomedical Evaluation dataset, which means the 92% of its sentences were predicted correctly as either certain or uncertain!

Here is the final code: