CAREER: Spatial-Temporal Imitation Learning

Award number: NSF IIS-1942680
Duration (expected): 07/01/2020-06/30/2025


Team:

PI: Prof. Yanhua Li
PhD Students: Xin Zhang, and Menghai Pan.

Abstract:

(Project Goal, Research Challenges, and Broader Impacts)

Humans make daily decisions based on their own "strategies" (such as taxi drivers' passenger-seeking processes and commuters' transit mode choices). Understanding and incorporating human decision-making strategies will bring significant benefits to the growing gig-worker population and transportation marketplace. For example, learning the decision-making strategies from taxi drivers, personal vehicle drivers, and urban commuters can facilitate the service providers (e.g., taxi/ride-hailing companies) to better serve the passengers, and enable the urban planners to design better road networks and transit routes to meet the needs of urban travelers. The goal of this project is to develop, implement, and evaluate a unified framework to learn the decision-making strategies of human agents from their generated mobility data, with applications to explain and incentivize their decisions to promote individual and societal well-being. Moreover, in this project, the investigator will integrate research, education, and outreach by developing new courses for both undergraduate and graduate students, reaching out to K-12 students, and engaging women and underrepresented minorities.

There are several technical challenges to learn human decision-making strategies from their mobility data: Human decision-making strategies may vary over time and space, i.e., spatial-temporal dynamics challenge. The mobility data collected may cover only a part of the spatial regions and time periods, i.e., spatial-temporal sparsity challenge. Most human agents are not "experts", thus the generated mobility data are noisy and uncertain, i.e., non-expert challenge. A large number of human agents interact with each other when making decisions, i.e., interaction and scalability challenge. Human decisions are governed by many (sometimes hidden) factors, which make it hard to infer explainable information from their decision-making strategies, i.e., explainability challenge. Human agents have diverse reactions to offered incentives, making it hard to design targeted incentive mechanisms to consider both agents' inherent decision-making strategies and their online feedback, i.e., incentive design challenge. This project will address these research challenges, and include the development of a novel spatial-temporal imitation learning framework for learning decision-making strategies from individual and interactive human agents, and an interactive system that provides human agents with explainable learning results and online incentives to promote their decision-making strategies. The spatial-temporal imitation learning framework and associated algorithms have the potential to be transformative in both the data and urban sciences by enabling efficient and accurate discovery of human decision-making strategies from their mobility data.


Publications:

[Conference papers], [Journal papers]
  1. [NeurIPS'20] Xin Zhang, Yanhua Li, Ziming Zhang, Zhi-Li Zhang,
    f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning. [PDF][NeurIPS Link] [arXiv]
    Thirty-fourth Conference on Neural Information Processing Systems, Virtual Conference, December 6 - 12, 2020, (20.0%=1900/9454 Acceptance Ratio)
  2. [NeurIPS'20] Qiong Wu, Felix Wong, Zhenming Liu, Yanhua Li, Varun Kanade,
    Adaptive Reduced Rank Regression. [PDF] [NeurIPS Link] [arXiv]
    Thirty-fourth Conference on Neural Information Processing Systems, Virtual Conference, December 6 - 12, 2020, (20.0%=1900/9454 Acceptance Ratio)
  3. [KDD'20] Menghai Pan, Weixiao Huang, Yanhua Li, Xun Zhou, Jun Luo,
    xGAIL: Explainable Generative Adversarial Imitation Learningfor Explainable Human Decision Analysis. [PDF] [Link]
    the 26th SIGKDD conference on Knowledge Discovery and Data Mining, San Diego, CA, August 23 - 27, 2020, (16.8%=216/1279 Acceptance Ratio)
  4. [KDD'20] Yingxue Zhang, Yanhua Li, Xun Zhou, Xiangnan Kong and Jun Luo,
    Curb-GAN: Conditional Urban Traffic Estimation through Spatio-Temporal Generative Adversarial Networks [PDF] [Link]
    the 26th SIGKDD conference on Knowledge Discovery and Data Mining, San Diego, CA, August 23 - 27, 2020, (16.8%=216/1279 Acceptance Ratio)
  5. [KDD'20] Huimin Ren*, Menghai Pan*, Yanhua Li, Xun Zhou and Jun Luo,
    ST-SiameseNet: Spatio-Temporal Siamese Networks for Human Mobility Signature Identification [PDF] [Link]
    the 26th SIGKDD conference on Knowledge Discovery and Data Mining, San Diego, CA, August 23 - 27, 2020, (16.8%=216/1279 Acceptance Ratio)
  6. [ICDM'20] Xin Zhang, Yanhua Li, Xun Zhou, Ziming Zhang, Jun Luo,
    TrajGAIL: Trajectory Generative Adversarial Imitation Learning for Long-term Decision Analysis. [PDF]
    IEEE International Conference on Data Mining, (9.8%=91/930 Full paper acceptance ratio), Sorrento, Italy, Nov. 17-20, 2020.
  7. [ICDM'20] Yingxue Zhang, Yanhua Li, Xun Zhou, Jun Luo,
    cST-ML: Continuous Spatial-Temporal Meta-Learning for Traffic Dynamics Prediction. [PDF]
    IEEE International Conference on Data Mining, (19.7%=183/930 Short paper acceptance ratio), Sorrento, Italy, Nov. 17-20, 2020.
  8. [SIGSPATIAL GIS'20] Menghai Pan, Weixiao Huang, Yanhua Li, Xun Zhou, Zhenming Liu, Jie Bao, Yu Zheng and Jun Luo,
    Is Reinforcement Learning the Choice of Human Learners? A Case Study of Taxi Drivers. [PDF]
    28th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Nov. 5 - Nov. 8, 2020, Seattle, WA, USA. (33/149=22.1% Full paper acceptance ratio.)
  9. [SIGSPATIAL GIS'20] Han Bao, Xun Zhou, Yingxue Zhang, Yanhua Li and Yiqun Xie,
    COVID-GAN: Estimating Human Mobility Responses to COVID-19 Pandemic through Spatio-Temporal Conditional Generative Adversarial Networks. [PDF]
    28th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Nov. 5 - Nov. 8, 2020, Seattle, WA, USA. (33/149=22.1% Full paper acceptance ratio.)
  10. [SIGSPATIAL GIS'20] Yichen Ding, Xun Zhou, Han Bao, Yanhua Li, Cara Hamann, Steven Spears, and Zhuoning Yuan,
    Cycling-Net: A Deep Learning Approach to Predicting Cyclist Behaviors from Geo-Referenced Egocentric Video Data. [PDF]
    28th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Nov. 5 - Nov. 8, 2020, Seattle, WA, USA. (33/149=22.1% Full paper acceptance ratio.)
  11. [TBD] Menghai Pan, Yanhua Li, Zhi-Li Zhang, Jun Luo,
    SCCS: Smart Cloud Commuting System with Shared Autonomous Vehicles [PDF]
    IEEE Transactions on Big Data, Accepted for publication, 2020.
  12. [TBD] Xin Zhang, Yanhua Li, Xun Zhou, Jun Luo,
    cGAIL: Conditional Generative Adversarial Imitation Learning—An Application in Taxi Drivers’ Strategy Learning [PDF]
    IEEE Transactions on Big Data, Accepted for publication, 2020.

  13. Acknowledgement:
    This material is based upon work supported by the National Science Foundation under Grant No. (NSF grant 1942680).
    Disclaimer:
    Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.