Inducing Semantic Representations From Text

Enabling a machine to understand language − that is, to process any input text or utterance and be able to answer questions or perform actions on its basis − is one of the main goals of natural language processing. Numerous practical applications could take advantage of language understanding, for example, to extract actionable knowledge, answer questions, or summarize events based on the semantic content of a text. And understanding is arguable unattainable without using some form of semantic representation which provides an abstraction from specifics of lexical (words) and syntactic realizations. Inducing these representations of meaning for a natural language text is the focus of my research.

You can watch a video covering some of our recent work (or this older but shorter one) or have a look in the publications below.


Research Subtopics

Unsupervised and semisupervised learning of semantic frames and scripts

Most current statistical methods for semantic parsing rely on vast text specifically annotated for the task. These resources are scarce, expensive to create and even the largest of them tend to have low coverage. Moreover, representations these resources provide may not be appropriate for the tasks in question. In my recent work, we investigate models which either do not use any labeled data, induce indirect supervision signal (e.g., from more abundant monolingual or multilingual parallel data), or rely on a combination of labeled and unlabeled data. In much of this research we focus on induction of semantic roles and frames (identfication of underlying events or relations, along with discovery of their properties or participants), as well as construction of probabilistic semantic scripts (i.e. learning how events are organized into higher order activities).

A frame-semantic representation for a sentence Lansky left Australia to study the piano at the Royal College of Music:

There are 2 frames in this example (Deparing and Education), and for each of them there are three semantic roles (also known as frame elements) realized.

keywords: unsupervised semantic parsing, unsupervised semantic role labeling, Bayesian models, learning semantics from parallel data, multilingual, crosslingual induction, semi-supervised semantic parsing.


Semantic analysis for low-resource languages

We develop methods for transferring resources (and models) from resource-rich languages (for which semantic resources are available) to resource-poor languages (i.e. language for which there is no or very little data):

keywords: crosslingual transfer, annotation projection, word embedding, cross-lingual word embeddings.

Joint modeling of syntax and semantics

The recovery of the meaning of text requires structured analysis of both its grammar and its semantics. However, interactions between syntactic and semantics structures are complex and not well understood. We developed a loosely coupled architecture for syntax and semantics (weakly-synchronized derivations) and applying the ISBN latent variable model to automatically discover appropriate features. The results for this approach are among the best of single-model systems on joint parsing of syntax and shallow semantics (predicting predicate-argument structure, called semantic role labeling). We achieved the 3rd best results overall (out of 14 systems) in the CoNLL 2009 competition (7 languages) with no language specific features and the same set of model parameters.

If you are interested in the topic (enabling semantic analysis of low-resource languages), you may want to watch Part 3 of a tutorial Martha Palmer, Shumin Wu and I gave at NAACL 2013.

keywords: joint model of syntax and semantics, latent variable models, distributed representations, deep learning for semantics, deep learning for SRL.