This is an undergraduate second year course covering the
fundamentals of human-computer interaction. The meta-objective
is to have computer science students think more broadly about
"last-inch connectivity": the efficiency of information transfer
between the users' minds and the computing interfaces. This
includes developing awareness of the importance of usability,
and having a keen perspective of the input-output interfacing
between computers and human (input:senses, output:muscles).
Other aspects of the course includes UI and UX design,
prototyping, familiarity with human and machine capabilities, as well as
analysis of underlying "protocols" when humans communicate with or
manipulate interfaces.
CE7491 Advanced Digital Image Processing
This is a graduate level course covering different aspects
of image processing, computer vision and machine learning (in
the context of computer vision). As no prior experience is
required, the fundamental theories of image formation, spatial
and frequency domain processing, and imaging geometry are
covered, although at a fairly rapid pace. In addition, deep
learning frameworks are discussed as is popularly used in the
latest computer vision research.
Student Projects / Fun Projects – Highlights
Recent Final Year Project Showcase
(2020-2021)
Seeing Sounds Wu Haoran
Severe hearing loss may mean that it is difficult to know when sounds occur, much less tell
where they are coming from. This project investigates how a smartphone can be paired
with a microphone array that sense sound directions. When calibrated to the smartphone camera for
see-through augmented reality, the combined system allows objects and regions emitting sounds to be
highlighted on the screen. This allows sounds to be "seen".
HDR Panoramas in Mobile Phones Leroy Low
When taking multiple photos to create wide or 360° panoramas, the camera adjusts the gain
automatically (auto-gain) to adapt to the brightness in different directions. Directly stitching these
images together meant the high dynamic range (HDR) is lost, even if the images were originally HDR.
In this project, a system is created that automatically "undid" the auto-gain when
stitching the images. This allows HDR panoramas to be created easily on a phone, including to be used
for environment lighting in computer graphics.
Pipeline for 3D Reconstruction and Augmented Reality
(2017)
Exploratory implementation of a semi-automated pipeline to evaluate
the quality limits of video see-through augmented reality. One component
involves the 3D multiview stereo reconstruction of a bear ragdoll based on
multiple images taken from different angles, the computed mesh of which is
subsequently manually rigged for animation. This is integrated with egomotion
tracking in video and a captured HDRI for illumination in rendering..
Multiview 3D Reconstruction and Fly-through of Moonriver Lodge, Cameron Highlands, Malaysia
(2010, 2017)
In 2010, using a compact camera I had taken 419 photos of
Moonriver
Lodge, an eco-friendly farmstay near Cameron Highlands, Malaysia. Separately, I
also captured 195 photos of the "Sensory
Waterfall" that was located on its premises. By processing these
images through bundle adjustment (to recover the original camera
poses) and dense multiview matching, 3D mesh models of the two
scenes were reconstructed. The reconstruction were redone in 2017
based on updates to state-of-the-art algorithms. The fly-throughs are shown below.
SmartSpace with Projector-Camera Systems
(2004-2009)
SmartSpace is an interior space within the SCSE Media & Interactive
Computing Lab (previously CeMNet – the Centre for Multimedia & Network
Technology) that is instrumented with multiple
casually angled projectors and cameras By intelligently using these
projector-camera systems (or PROCAMS), we can make all physical
surfaces (e,g, furniture, walls) interactive and also create very
interesting effects. Multiple undergraduate final-year projects were
done using PROCAMS, with selected ones highlighted below.
SmartSpace Projects Sampler (2004-2006)
SCSE undergraduate project demos in SmartSpace in 2006, mainly involving interactive
computer vision and PROCAMS.
SmartSpace portion in SCSE publicity video (2006)
This is a short footage from SCSE's publicity video in 2006,
featuring SmartSpace projects.
Light Fantastic (2005)
This is an
undergraduate FYP demonstrating how a projector-camera system
can be combined with normal furniture and objects to create
interesting effects of adding virtual light sources to the
actual space itself.
Silica Graphics (2009, 2015)
This were
undergraduate FYPs, in which a real sandbox was augmented with a
PROCAMS to enhance interactive capability when playing with
sand.
The clip on the upper right was taken during the NTU
Discover Engineering Exhibition in 2009. Alternatively, see this
other more comprehensive video.
The clip on the lower right was a
follow-on project in 2015 that significantly increased speed and functionality.
Melded Reality (2007)
This is an
undergraduate FYP in which a PROCAMS was used to make walls,
furniture and objects become interactive to a laser pointer, in
a form of "melded reality".
Student Projects in the Singapore-MIT Alliance
Computer Science Programme
(2004)
This is a ChannelNewsAsia report on Masters level student group projects developed
in 2004 as part of SMA5508 Pervasive Human-Centric Computing, a
course within the Singapore-MIT Alliance Computer Science Programme.
I co-taught this course together with Larry Rudolph. The equipment
used were Compaq iPAQs
(these were PDAs) together with the
BackPAQs
developed in Compaq's Cambridge Research Lab that included cameras
and accelerometers, which were novelties at that time – this was way
before smartphones appeared!
