HighMMT: Towards Modality and Task Generalization for High-Modality Representation Learning
Paul Pu Liang, Yiwei Lyu, Xiang Fan, Shentong Mo, Dani Yogatama, Louis-Philippe Morency, and Ruslan Salakhutdinov.
TMLR 2023.
Questions Are All You Need to Train a Dense Passage Retriever
Devendra Singh Sachan, Mike Lewis, Dani Yogatama, Luke Zettlemoyer, Joelle Pineau, and Manzil Zaheer.
TACL 2023.
Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?
Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Won Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Q. Tran, Dani Yogatama, and Donald Metzler.
arXiv 2022.
Language Models Can See: Plugging Visual Controls in Text Generation
Yixuan Su, Tian Lan, Yahui Liu, Fangyu Liu, Dani Yogatama, Yan Wang, Lingpeng Kong, and Nigel Collier.
arXiv 2022.
Emergent Abilities of Large Language Models
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus.
TMLR 2022.
A Contrastive Framework for Neural Text Generation
Yixuan Su, Tian Lan, Yan Wang, Dani Yogatama, Lingpeng Kong, and Nigel Collier.
NeurIPS 2022.
ABC: Attention with Bounded-memory Control
Hao Peng, Jungo Kasai, Nikolaos Pappas, Dani Yogatama, Zhaofeng Wu, Lingpeng Kong, Roy Schwartz, and Noah A. Smith.
ACL 2022.
Relational Memory Augmented Language Models
Qi Liu, Dani Yogatama, and Phil Blunsom.
TACL 2022.
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, and Donald Metzler.
ICLR 2022.
Balancing Average and Worst-case Accuracy in Multitask Learning
Paul Michel, Sebastian Ruder, and Dani Yogatama.
arXiv 2021.
End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering
Devendra Singh Sachan, Siva Reddy, William Hamilton, Chris Dyer, and Dani Yogatama.
NeurIPS 2021.
Pitfalls of Static Language Modelling
Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Sebastian Ruder, Dani Yogatama, Kris Cao, Tomas Kocisky, Susannah Young, and Phil Blunsom.
NeurIPS 2021 spotlight presentation.
LiRo: Benchmark and Leaderboard for Romanian Language Tasks
Stefan Daniel Dumitrescu, Petru Rebeja, Beata Lorincz, Mihaela Gaman, Andrei Avram, Mihai Ilie, Andrei Pruteanu, Adriana Stan, Lorena Rosia, Cristina Iacobescu, Luciana Morogan, George Dima, Gabriel Marchidan, Traian Rebedea, Madalina Chitez, Dani Yogatama, Sebastian Ruder, Radu Tudor Ionescu, Razvan Pascanu, and Viorica Patraucean.
NeurIPS 2021 Datasets and Benchmarks Track.
Finetuning Pretrained Transformers into RNNs
Jungo Kasai, Hao Peng, Yizhe Zhang, Dani Yogatama, Gabriel Ilharco, Nikolaos Pappas, Yi Mao, Weizhu Chen, and Noah A. Smith.
EMNLP 2021.
Adaptive Semiparametric Language Models
Dani Yogatama, Cyprien de Masson d'Autume, and Lingpeng Kong.
TACL 2021.
Random Feature Attention
Hao Peng, Nikolaos Pappas, Dani Yogatama, Roy Schwartz, Noah A. Smith, and Lingpeng Kong.
ICLR 2021 spotlight presentation.
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth, Johannes Welbl, Jack Rae, Vishal Maini, Dani Yogatama, and Pushmeet Kohli.
Findings of EMNLP 2020.
Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Adhiguna Kuncoro*, Lingpeng Kong*, Daniel Fried, Dani Yogatama, Laura Rimell, Chris Dyer, and Phil Blunsom.
TACL 2020.
On the Cross-lingual Transferability of Monolingual Representations
Mikel Artetxe, Sebastian Ruder, and Dani Yogatama.
ACL 2020.
Download XQuAD dataset.
A Call for More Rigor in Unsupervised Cross-lingual Learning
Mikel Artetxe, Sebastian Ruder, Dani Yogatama, Gorka Labaka, and Eneko Agirre.
ACL 2020.
A Mutual Information Maximization Perspective of Language Representation Learning
Lingpeng Kong, Cyprien de Masson d'Autume, Wang Ling, Lei Yu, Zihang Dai, and Dani Yogatama.
ICLR 2020 spotlight presentation.
On Memory in Human and Artificial Language Processing Systems
Aida Nematzadeh, Sebastian Ruder, and Dani Yogatama.
ICLR 2020 Workshop on Bridging AI and Cognitive Science.
Modelling Latent Skills for Multitask Language Generation
Kris Cao and Dani Yogatama.
arXiv 2020.
Episodic Memory in Lifelong Language Learning
Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama.
NeurIPS 2019.
Grandmaster Level in StarCraft II Using Multi-agent Reinforcement Learning
Oriol Vinyals, Igor Babuschkin, Wojtek Czarnecki, Michael Mathieu, Andrew Dudzik, Junyoung Chung, David H. Choi, Richard Powell, Timo Ewalds, Petko Georgiev, Junhyuk Oh, Dan Horgan, Manuel Kroiss, Ivo Danihelka, Aja Huang, Laurent Sifre, Trevor Cai, John Agapiou, Max Jaderberg, Alexander S. Vezhnevets, Rémi Leblond, Toby Pohlen, Valentin Dalibard, David Budden, Yury Sulsky, James Molloy, Tom L. Paine, Caglar Gulcehre, Ziyu Wang, Tobias Pfaff, Yuhuai Wu, Roman Ring, Dani Yogatama, Dario Wunsch, Katrina McKinney, Oliver Smith, Tom Schaul, Timothy Lillicrap, Koray Kavukcuoglu, Demis Hassabis, Chris Apps, and David Silver.
Nature 2019.
Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation
Po-Sen Huang, Robert Stanforth, Johannes Welbl, Chris Dyer, Dani Yogatama, Sven Gowal, Krishnamurthy Dvijotham, and Pushmeet Kohli.
EMNLP 2019.
Variational Smoothing in Recurrent Neural Network Language Models
Lingpeng Kong, Gabor Melis, Wang Ling, Lei Yu, and Dani Yogatama.
ICLR 2019.
Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs
Jean Maillard, Stephen Clark, and Dani Yogatama.
Journal of Natural Language Engineering.
Learning and Evaluating General Linguistic Intelligence
Dani Yogatama, Cyprien de Masson d'Autume, Jerome Connor, Tomas Kocisky, Mike Chrzanowski, Lingpeng Kong, Angeliki Lazaridou, Wang Ling, Lei Yu, Chris Dyer, and Phil Blunsom.
arXiv 2019.
LSTMs Can Learn Syntax-Sensitive Dependencies Well, but Modeling Structure Makes Them Better
Adhiguna Kuncoro, Chris Dyer, John Hale, Dani Yogatama, Stephen Clark, and Phil Blunsom.
ACL 2018.
Memory Architectures in Recurrent Neural Network Language Models
Dani Yogatama, Yishu Miao, Gabor Melis, Wang Ling, Adhiguna Kuncoro, Chris Dyer, and Phil Blunsom.
ICLR 2018.
Program Induction by Rationale Generation:Learning to Solve and Explain Algebraic Word Problems
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom.
ACL 2017.
Learning to Compose Words into Sentences with Reinforcement Learning
Dani Yogatama, Phil Blunsom, Chris Dyer, Edward Grefenstette, and Wang Ling.
ICLR 2017.
Generative and Discriminative Text Classification with Recurrent Neural Networks
Dani Yogatama, Chris Dyer, Wang Ling, and Phil Blunsom.
arXiv 2017.
Lookahead Convolution Layer for Unidirectional Recurrent Neural Networks
Chong Wang*, Dani Yogatama*, Adam Coates, Tony Han, Awni Hannun, and Bo Xiao.
ICLR Workshop 2016.
End-to-End Speech Recognition in English and Mandarin
Dario Amodei, Rishita Anubhai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Jingdong Chen, Mike Chrzanowski, Adam Coates, Greg Diamos, Erich Elsen, Jesse Engel, Linxi Fan, Christopher Fougner, Awni Y. Hannun, Billy Jun, Tony Han, Patrick LeGresley, Xiangang Li, Libby Lin, Sharan Narang, Andrew Y. Ng, Sherjil Ozair, Ryan Prenger, Sheng Qian, Jonathan Raiman, Sanjeev Satheesh, David Seetapun, Shubho Sengupta, Chong Wang, Yi Wang, Zhiqian Wang, Bo Xiao, Yan Xie, Dani Yogatama, Jun Zhan, Zhenyao Zhu.
ICML 2016.
A Sparse and Adaptive Prior for Time-Dependent Model Parameters
Dani Yogatama, Bryan R. Routledge, and Noah A. Smith.
NIPS Time Series Workshop 2015.
Extractive Summarization by Maximizing Semantic Volume
Dani Yogatama, Fei Liu, and Noah A. Smith.
EMNLP (short) 2015.
Bayesian Optimization of Text Representations
Dani Yogatama, Lingpeng Kong and Noah A. Smith.
EMNLP (short) 2015.
Learning Word Representations with Hierarchical Sparse Coding (supplementary)
Dani Yogatama, Manaal Faruqui, Chris Dyer, and Noah A. Smith.
ICML 2015 (previous version in NIPS Deep Learning and Representation Learning Workshop 2014).
Embedding Methods for Fine Grained Entity Type Classification
Dani Yogatama, Dan Gillick, and Nevena Lazic.
ACL (short) 2015.
Sparse Binary Word Vector Representations
Manaal Faruqui, Yulia Tsvetkov, Dani Yogatama, Chris Dyer, and Noah A. Smith.
ACL 2015.
Making the Most of Bag of Words: Sentence Regularization with Alternating Direction Method of Multipliers
Dani Yogatama and Noah A. Smith.
ICML 2014.
Linguistic Structured Sparsity in Text Categorization
Dani Yogatama and Noah A. Smith.
ACL 2014.
Errata (thanks to Adam Teichert):
- In the caption of Figure 2, N=8 instead of N=7.
- In Footnote 7, for the Brown cluster regularizer, G=V-1, but L varies depending on the Brown clusters used.
Efficient Transfer Learning Method for Automatic Hyperparameter Tuning
Dani Yogatama and Gideon Mann.
AISTATS 2014.
Dynamic Language Models for Streaming Text
Dani Yogatama, Chong Wang, Bryan R. Routledge, Noah A. Smith, and Eric P. Xing.
TACL 2014.
Structured Sparsity in Natural Language Processing: Models, Algorithms, and Applications
Andre F. T. Martins, Dani Yogatama, Noah A. Smith, and Mario A. T. Figueiredo.
Tutorial at EACL 2014.
A Penny for your Tweets: Campaign Contributions and Capitol Hill Microblogs
Tae Yano, Dani Yogatama, and Noah A. Smith.
ICWSM 2013.
A Probabilistic Model for Canonicalizing Named Entity Mentions
Dani Yogatama, Yanchuan Sim, and Noah A. Smith.
ACL 2012.
Predicting a Scientific Community’s Response to an Article
Dani Yogatama, Michael Heilman, Brendan O’Connor, Chris Dyer, Bryan R. Routledge, and Noah A. Smith.
EMNLP 2011.
Longer CMU tech report version: Predicting Responses and Discovering Social Factors in Scientific Literature.
Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments
Kevin Gimpel, Nathan Schneider, Brendan O’Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith.
ACL (short) 2011.
Code and data.
Multilingual Spectral Clustering Using Document Similarity Propagation
Dani Yogatama and Kumiko Tanaka-Ishii.
EMNLP 2009.
Sparse Models of Natural Language Text
Dani Yogatama.
Ph.D. thesis, 2015, Carnegie Mellon University.
Clustering multilingual documents by estimating text-to-text semantic relatedness
文書間の意味的関係性の推定に基づく多言語文書クラスタリング
Dani Yogatama.
M.S. thesis, 2010, The University of Tokyo.