We propose a novel statistical machine translation approach to automatically generate Chinese couplets and Chinese poetry, one of the most important Chinese cultural heritage. For Chinese couplets, the system takes as input the first sentence and generates as output an N-best list of second sentences using a phrase-based SMT model. A comprehensive evaluation using both human judgments and BLEU scores has been conducted and the results demonstrate that this approach is very successful. We then extended this approach to generate classic Chinese poetry using the quatrain as a case study. Given a few keywords describing a user's intention, a statistical model is used to generate the first sentence. Then a phrase-based SMT model is used to generate the other three quatrain sentences one by one. Evaluation using human judgment over individual lines as well as the quality of the generated poem as a whole demonstrates promising results. Please visit http://duilian.msra.cn for the Chinese couplet generation system based on this research.

Approximate nearest neighbor search is an optimization problem for finding closest items, and has many applications in multimedia search, information retrieval, and computer vision. There are various solutions, including index structures for and compact codes. Indexing algorithms speed up the search by pruning a large number of points that are very likely to be farther from the query, and compact coding algorithms perform linear scan search with reducing the similarity computation cost between each item and the query. In this talk, I will introduce the compression-based compact coding algorithms. Then I will present our recently-proposed new algorithm that has a better balance among search efficiency, search accuracy and storage cost.

In this talk, we explore a novel bilingual word alignment approach based on DNN (Deep Neural Network) and describe in detail how we adapt and extend the CD-DNN-HMM (Dahl et al., 2012) method introduced in speech recognition to the HMM-based word alignment model, in which bilingual word embedding is discriminatively learnt to capture lexical translation information, and surrounding words are leveraged to model context information in bilingual sentences. While being capable to model the rich bilingual correspondence, our method generates a very compact model with much fewer parameters. Experiments on a large scale English-Chinese word alignment task show that the proposed method outperforms the HMM and IBM model 4 baselines by 2 points in F-score.

Download Slide

Contemporary video search and categorization are non-trivial tasks due to the massively increasing amount and content variety of videos. We put forward the study of visual saliency models in video. Such a model is employed to identify salient objects from the image background. Starting from the observation that motion information in video often attracts more human attention compared to static images, we devise a region contrast based saliency detection model using spatial- temporal cues (RCST). We introduce and study four saliency principles to realize the RCST. This generalizes the previous static image for saliency computational model to video. We conduct experiments on a publicly available video segmentation database where our method significantly outperforms seven state-of-the-art methods with respect to PR curve, ROC curve and visual comparison.

Abstract: Human reasoning is a beautiful puzzle: it is productive, extending to new situations without bound, and it is uncertain, dealing gracefully with noisy evidence and shades of belief. Reasoning is also a key window on cognition, providing some of our best evidence about the structures that underly everyday, commonsense thought. I will argue that the core representations that support reasoning can be understood as a 'probabilistic language of thought', and that reasoning is an approximation to probabilistic inference. I will illustrate this claim with examples of reasoning about games and property induction. However, I will suggest that the bridge between the probabilistic language of thought and empirical data is a firm understanding of natural language. I will sketch a model of natural language pragmatics and semantics, and describe experimental evidence from communication games and quantity implicature. I will then describe how we can extend this framework to encompass vague language, focusing on scalar adjectives (like "tall"). I will conclude by coming back to reasoning, explaining two puzzling patterns: the sorites paradox and the effect of additional premises when reasoning general-to-specific. Time permitting, I will discuss possible process-level implementations in a short coda.