Scene Caching for Video Capture Data Reduction

Key words: Machine Vision, Data Reduction, Intelligent Sampling, Motion Compensation, Motion Segmentation, Background Modeling, Encoder-Decoder

• Developed an intelligent Scene Cache that uses heterogeneous spatial capture, temporal buffering and multiround capture to reduce the amount of data captured and transferred without accuracy reduction.
• Used motion compensation, motion segmentation and background modeling algorithms to determine regions to transmit to the remote inference server.
• Developed a encoder-decoder pair to process low resolution images on the camera and reconstruct the transmitted data into a high resolution image on the server.

Faculty Mentors: Robert Dick

Patent:

Multiscale Deep Attention-Sampling Model

Key words: Multiple Instance Learning, Attention Sampling, Multiresolution, Object Detection

Existing deep architectures cannot operate on very large signals such as megapixel images due to computational and memory constraints. To tackle this limitation, Angelos Katharopoulos and Franc ̧ois Fleuret propose a fully differentiable end-to-end trainable model that samples and pro- cesses only a fraction of the full resolution input image. They refer to their method as attention sampling and it can process images of several megapixels with a standard single GPU setup. However, we found their method was only able to learn the distribution of simple, small objects because their architecture is trained with a small, fixed-size patch configuration. Our work introduces a multiresolution architecuture, combine it with attention sampling methods and change the way to sample patches from the multiscale sample space so that our network is able to learn the attention distrbution of complicated, size-variant objects like vehicle without large inference overhead.

Faculty Mentors: Robert Dick

Web Automation based on Programming-by-Demonstration

Key words: Programming-by-Demonstration, Web Automation, Recorder, Domain Specific Language

Problem Description: Web automation is the concept of letting software robots perform actions, tasks, and processes on a web browser or web application. In daily life, when people interact with web pages, including filling out forms or fields, scraping data from web pages, clicking buttons and elements or shifting among pages, they often have to do repetitive operations. Our goal is to automate those repetitive tasks. The idea is, when a user is manually performing the task, our tool is able to record and analyze these actions and suggest next action that the user is likelyto perform. The user can simply approve our suggestion or manually correct the action if our suggestion is wrong.

Contribution:

Faculty Mentor: Xinyu Wang

EECS 499 Directed Research: Final Report

Prototype of Autonomous Excavator with Obstacle Avoidance

Sponsored by BuilderX, Inc.

Key words: Embedded System, Computer Vision, ROS, Binocular Camera, Point Cloud Fusion, Object Detection

• Designed 3D vision algorithm to capture, fuse point cloud data of the surroundings and detect obstacle.
• Co-Designed collision-free path planning algorithm to move the excavator arms to the desired location.
• Co-Designed our ROS environment to make connection among all of our components so that the system integates the hardware and software parts well.

Map-based Scavenger Hunt Game App

Key words: Swift, Django, PostgreSQL, Google Map API, AR, Frontend, Backend, Project Management, Model-View-Controller

• led a 5-student team to conduct product design, code development and project management.
• Programmed an iOS app where users can play, design and publish their own scavenger hunt games.
• Implemented most of front-end features including map localization, drawing routes and marking spots using Google Map API, interacting with AR, game pause/continue, game creation and uploading and App navigation

Github link:

Network File System

EECS 482

Key words: Operating Systems, Networking, Server, Multithreaded, File System

Designed and implemented a multi-threaded, secure network file server. Clients that use the file server can interact with it via network messages. This project required understanding of hierarchical file systems, socket programming, client-server systems, remote code execution, and security protocols. This was the fourth project in EECS 482: Introduction to Operating Systems. Source available on request to non-students for personal evaluation, it is unavailable to students due to the University of Michigan Engineering Honor Code.

Client/Server Side Dynamic Website (Instagram)

EECS 485

Key words: Python, Flask, JavaScript, React, SQL, Asynchronous