论文部分内容阅读
As a measurement to the degree of machine’s understanding of a piece of text,machine reading comprehension(MRC)requires the model to answer the questions based on a piece of context.Over the past few years,more and more powerful models have been proposed based on various deep learning techniques.The MRC models based on deep learning is powerful and effective,but most of them are focusing on changing the neural network structure.As an essential part of question answering systems,even minor changes in word representation may lead to substantial performance differences in question answering models.At the same time that the deep learning method is booming,word represen-tation has also made great progress.Global matrix factorization methods are good at leverage statistical information efficiently,while local context window methods have better performance on the analogy task.But these kinds of methods suffer significant drawbacks,global matrix factorization methods tend to indicate a sub-optimal vector space model,so that they may perform poorly on the word analogy task.Though local context window methods perform better than global matrix factorization methods on the word analogy task,they are weak in taking advantage of the corpus statistics since they are trained on local windows instead of on global co-occurrence counts.There have also been attempts to combine both of the two methods mentioned above,such as Glo Ve,which have achieved great improvements,but still cannot make use of the semantic information in the corpus.Recent works indicate that both adjusting the objective function of the training algorithm and relation-specific augmentation of the co-occurrence matrix can improve the word embeddings’quality.However,the above methods are effective only on par-ticular constructing embedding methods.The retrofitting method[1]can be applied to update word embeddings as a post-processing step.It works by running belief propaga-tion on a relational information graph constructed from semantic lexicons.This makes retrofitting can be applied to almost any kind of pre-trained word embeddings.We propose a method to enhance the question answering model by introducing semantic information to word embedding.In our model,we add the retrofitting process to the embedding layer to transform the word embeddings.Then,we do our experiments us-ing PPDB,Word Net,and Frame Net on Glo Ve and Word2Vec.The results indicate that the retrofitted word embeddings can improve the performance of the chosen question answering model.We propose a method to enhance the question answering model by introducing se-mantic information to word embedding.In our model,we add the retrofitting process to the embedding layer to transform the word embeddings.We use Word2Vec and Glo Ve as original word embeddings,and retrofit them with PPDB,Word Net and Frame Net,separately.We choose reinforced mnemonic reader as the question answering model to be improved and switch its embedding layer to the retrofitting embedding layer.The results on SQu AD dataset indicate that the retrofitted word embeddings can improve the performance of the chosen question answering model.