Projects (ML/AI directions)

  • Heterogeneous Graph Neural Network for incomplete multimodal data analysis
  • Context-guided Meta-Learning Approach for Few-shot Learning Across Modalities
  • Generalizing Topological Structure of Task in Hybrid Few-shot Classification
  • Deep Reinforcement Learning for Human-like Car-Following System
    • Built an actor-critic reinforcement learning framework to learn an optimal car-following behavior from empirical data; implemented deep deterministic policy gradient algorithm to learn the continuous-control policy network.
    • Tools: Python, PyTorch
  • GNNMutation: A Mutation Tool for Graph Neural Networks
    • Proposed a novel mutator to inject faults into GNN models, including eight node/edge-level mutation operators.
    • Tools: Python, PyTorch
  • Controllable Text Generation via Generative Adversarial Network
    • Implemented a conditional GAN to generate realistic-looking sentences whose attributes (e.g., emotion in sentences) can be controlled; collected social media texts and split them by emotion annotations; trained the RNN-based generator and discriminator adversarially, using the idea of reinforcement learning to maximize the rewards from discriminator.
    • Tools: Python, PyTorch, Tensorflow

Projects (CV/CG directions)

  • Artwork Generation for 3D Scene Models based on Computer Vision & Graphics
    • Studied human knowledge-guided neural style transfer, focusing on improving the illusion of space in generated images by simulating how artists use their skills to observe and reproduce a 3D scene (e.g., geometric structures, lighting and shallow); also studied 3D non-photorealistic rendering based on the neural style transfer paradigm.
    • Created a 3D-2D dataset, including 3D models rendered by multiple types of lighting (using Autodesk Maya), 2D photos annotated by lighting and segmentation (by Photoshop and Matlab), and a hand-drawn stylistic material for testing (by CorelPainter).
    • Proposed an illumination-guided deep alignment method based on CNN, Lighting Path Expression, and PatchMatch (Keras, Python).
  • Traffic-scene Image Enhancement
    • Proposed a fast, detail-enhanced, and halo-free method to simultaneously correct the over- and under-exposure problem in LDR images, which widely exists in traffic-scene images in our smart-vehicle vision system.
  • Efficient Human Action Recognition based on Video-Compression Domain
    • Extracted motion vectors (MV) and DCT from MPEG-4 video bitstreams.
    • Proposed to fast detect Spatial-Temporal Interest Points from video bitstream using MV and DCT, instead of from the decoded video, and then formed action features using BoW and GMM.
    • Trained traditional ML classifiers-decision tree, Naive Bayes, and SVM.
    • Tools: Matlab, C++, OpenCV, ffmpeg, Linux.

Projects (Robotics direction)

  • TurtleBot Autonomous Security Guard
    • Built an autonomous framework on TurtleBot to act as a human security guard—wandering, finding AprilTag targets, approaching each target, aiming and then shooting the target with a motorized toy gun installed on TurtleBot.
    • Developed the target searching/ranking, goal-position and gun-pitch calculation, and go-to-goal functions.
    • Tools: Robotic Operating System (ROS), C++, Python, Linux
  • Drone Vision-guided Autonomous Navigation & Search-and-rescue System
    • Centered around a mission making the drone to complete a search-and-rescue task. Participants are tasked with building an app that enables a drone to autonomously take-off from a moving vehicle, collect data in a survivors searching area, and finally track and land on the moving vehicle.
    • Developed a real-time vision-based module on drone system for Object Detection and 3D Localization, computing the 3D poses of objects from 2D images using real-time camera gimbal, Homography and 3D Transformation (C++).
    • Developed a PID-based autonomous tracking-and-landing module for landing on a moving vehicle, and developed the workflow module to manage the multiple modules simultaneously running on the drone (ROS, Python, Linux).
    • Won the 4th place from 130+ international teams in “2016 DJI Developer Challenge”, NY, USA (as a team of three).
    • Tools: Robotic Operating System (ROS), C++, Python, Linux

Back to CV.