Artificial Intelligence

The study of systems that behave intelligently, artificial intelligence includes several key areas where our faculty are recognized leaders: computer vision, machine listening, natural language processing, machine learning and robotics.

Computer vision systems can understand images and video, for example, building extensive geometric and physical models of cities from video, or warning construction workers about nearby dangers. Natural language processing systems understand written and spoken language; possibilities include automatic translation of text from one language to another, or understanding text on Wikipedia to produce knowledge about the world. Machine listening systems understand audio signals, with applications like speech recognition, acoustic monitoring, or transcribing polyphonic music automatically. Crucial to modern artificial intelligence, machine learning methods exploit examples in order to adjust systems to work as effectively as possible. Robotics puts artificial intelligence into practice using machines that perceive and interact with the physical world.

Strengths and Impact

The AI group at Illinois is strong, diverse, and growing. It combines expertise in core strengths with promising new research directions.

In machine learning, AI group faculty are studying theoretical foundations of deep and reinforcement learning; developing novel models and algorithms for deep neural networks, federated and distributed learning; as well as investigating issues related to scalability, security, privacy, and fairness of learning systems. Computer vision faculty are developing novel approaches for 2D and 3D scene understanding from still images and video; joint understanding of images and language; low-shot learning (recognition of rare or previously unseen categories); transfer learning and domain adaptation (adapting pre-trained systems to a changing data distribution); and image generation and editing approaches based on generative neural networks. Natural language processing faculty are working on topics such as grounded language understanding, information extraction and text mining, and knowledge-driven natural language generation for applications such as scientific discovery. Machine listening faculty are working on sound and speech understanding, source separation, and enhancement, as well as applications in music and computing. Robotics faculty are developing novel planning algorithms for grasping, locomotion, and navigation; investigating multi-robot systems; as well as pursuing high-impact applications of robotics to medicine, agriculture, home care, and autonomous driving.

The excellence and impact of the AI group’s research has been recognized by a number of awards, including NSF CAREER (Amato, Hauser, Hockenmaier, Hoiem, Ji, Koyejo, Lazebnik, Smaragdis, Telgarsky), Sloan Research Fellowship (Hoiem, Koyejo, Lazebnik), Microsoft Research Faculty Fellowship (Lazebnik), AFOSR Young Investigator (Chowdhary), IEEE PAMI Significant Young Researcher Award (Hoiem), MIT TR-35 (Li, Smaragdis), Intel Rising Star Award (Li), “Young Scientist” selected by World Economic Forum (Ji), “AI’s Top 10 to Watch” Award by IEEE Intelligent Systems (Ji), ACM Fellow (Amato, Forsyth, Warnow), IEEE Fellow (Amato, Forsyth, Lazebnik, Smaragdis), IEEE Technical Achievement Award (Forsyth), and Packard Fellowship (Warnow).

In the last few years, AI group members received a number of best paper awards, including: IEEE Signal Processing Society Best Paper Award (Smaragdis, 2018 and 2020), IEEE MLSP Best Paper Award (Smaragdis, 2017), Best Demo Paper Award at the 58th Annual Meeting of the Association for Computational Linguistics (Ji, 2020).

AI group research has led to a number of startups. Derek Hoiem is co-founder and Chief Science Officer of Reconstruct, which visually documents construction sites, matching images to plans and analyzing productivity and risk for delay. Girish Chowdhary  is co-founder and CTO of EarthSense, a startup creating machine learning and robotics solutions for agriculture, whose work was featured in a 2020 New York Times article. David Forsyth advises a number of startups focusing on augmented reality and image synthesis, including Lightform, Revery, and Depix.

AI faculty are playing key roles in two $20 million AI institutes recently funded by the National Science Foundation and the U.S. Department of Agriculture’s National Institute of Food and Agriculture. The AI Institute for Future Agricultural Resilience, Management, and Sustainability (AIFARMS), led by Vikram Adve from CS, features Romit Chowdhary as Associate Director of Research, with other investigators including Alexander Schwing, Katherine Driggs-Campbell, Indranil Gupta, Kris Hauser, Julia Hockenmaier, Heng Ji, Sanmi Koyejo, and Paris Smaragdis. The AI Institute for Molecular Discovery, Synthetic Strategy, and Manufacturing, led by Huimin Zhao from Chemical Engineering, involves Heng Ji and Jian Peng as investigators.

Seminars

Faculty & Affiliate Faculty

Robot Motion and Task Planning, Multi-Agent Systems, Crowd Simulation

Machine Learning Methods for Imaging Science, Image Reconstruction, Deep Learning for Inverse Problems

Machine Learning, Learning Theory, Optimization, Generative Models, Sequential Decision Making, Physics-Guided Machine Learning, Differential Privacy

Motion Planning and Control

Machine Learning, Natural Language Processing, AI Applications, Data Management Support for AI

Control, Autonomy and Decision Making, Vision and LIDAR Based Perception, GPS Denied Navigation

Computational Statistics, Reproducible Research, Statistics Education, Machine Learning

Social Network Analysis, Natural Language Processing, Machine Learning

Signal Processing, Computational Imaging, Machine Perception, Data Science

Autonomous Vehicles, Validating Autonomous Systems, Interactive Control Policies for Intelligent Systems in Multi-Agent Settings

Computational Linguistics

Computer Vision, Object Recognition, Scene Understanding

Computational Linguistics

Computer Vision Analytics for Building and Construction Performance Monitoring

Computer Vision, Machine Learning, Motion Analysis, Robotics

Computer Vision, Robotics, Machine Learning

Conversational AI and Natural Language Processing

Machine Learning, Natural Language-Based Text Analysis, Text Summarization

Motion Planning, Optimal Control, Integrated Planning and Learning, Robot Systems

Natural Language Processing, Computational Linguistics 

Computer Vision, Object Recognition, Spatial Understanding, Scene Interpretation 

Probabilistic Graphical Models; Deep Learning; Data Science; Health Analytics; Safety, Reliablity and Security of Autonomous Systems; Reinforcement Learning

ML4Code, ML interpretability, testing, and debugging

Natural Language Processing, especially on Information Extraction and Knowledge-driven Natural Language Generation, Text Mining, Knowledge Graph Construction for Scientific Discovery

Reinforcement Learning, Machine Learning, Sample Complexity Analyses

Analytics with Machine Learning, Databases with Machine Learning, Machine Learning Security, Machine Learning + Cryptography 

HCI for ML, AI Explainability

Cyberinfrastructure for Machine Learning, Machine Learning Systems Research, Deep Learning Applications

Systems for Machine Learning, Machine Learning for Systems

Computer Vision, Scene Understanding, Visual Learning, Vision and Language

Adversarial Machine Learning, Robust Learning

Artificial Intelligence

Cyberinfrastructure for Digital Preservation, Auto-Curation, and Managing Unstructured Digital Collections 

AI for Audio; Model Compression; Personalized AI; Signal Separation, Enhancement, and Coding

Motion Planning and Control, Autonomous Robots

Natural Language Processing, Machine Learning, Large Language Models, AI for Science

Machine Learning and Optimization

Computer Vision, Ego4D, VR/AR, Mobile Health, Health AI, Machine Learning, Developmental Machine Learning, Behavioral Imaging

Field Robotics, Autonomous Systems Engineering, Machine Perception, Computer Vision

Machine Translation, Computational Morphology & Syntax

Machine Learning, Computer Vision

Certified Artificial Intelligence, Adversarial Robustness, Neural Network Verification, Safe Deep Learning

Machine Learning for Audio, Speech and Music; Signal Processing; Source Separation; Sound Recognition and Classification

Deep Learning for Drug Discovery, Clinical Trial Optimization, Computational Phenotyping, Clinical Predictive Modeling, Mobile Health and Health Monitoring, Tensor Factorization, and Graph Mining

Deep Learning Theory

Explainable AI, Fairness in AI, Adversarial Maching Learning

Conversational AI

Computer Vision, Robotics

Computer Vision, Machine Learning, Meta-Learning, Robotics

Machine Learning in Computational Genomics, Ensemble Methods, Statistical Estimation

Machine Learning for Graphs and Databases, Foundation Models, Large Language Models, Generative AI, AI Agents

Efficient DL/AI Systems and Algorithms, Parallel Computing and Runtime, Natural Language Processing, AI for Science

Machine Learning Theory and Applications, Optimization, Reinforcement Learning, Robustness, Generative AI, Large Language Models

Machine Learning, Representation Learning, Algorithmic Fairness, Probabilistic Models

Adjunct Faculty

Machine Learning, Automatic Reasoning

Machine Learning, Neuroimaging, Biomedical Imaging

Machine Learning, Natural Language Processing, Knowledge Representation, Reasoning 

Related News