Picking up where I left off 3 years ago with this year’s UIST
highlight reel. As expected, the research creatively applied interesting
principles, but many applications were adorably contrived. Also, I miss academia!
WhichFingers / characterizing end-to-end latency
Both papers used a finger-mounted vibration sensor made from a piezoelectric
polymer, and implemented simple finger tap detection. One showed how this
method could be used for measuring latency using a blinking screen. They did a
great job of measuring latency at every stage.
The other paper had vibration sensors for each finger, and did time-based
correlation to determine which finger tapped a screen. They had a 30ms window
within which you had to fall to have unambiguous classification. Interesting:
they also moved their vibration sensors up to the ring position (third phalanx),
and seemed to also get pretty good results. Also nice is that they open sourced
everything, and that the whole system is cheaper than $35.
Carpacio: unobtrusive passenger & driver ID
The cheesily named project served to repurpose capacitative screens to
differentiate between drivers and passengers. Motivated by driver distraction,
cars are now shipping with on-screen UIs that are locked until you answer an
onscreen question: are you a driver or passenger? This is counter-productive.
Edward Jay Wang from UW showed how the car system can identify who is
interacting with it automatically. Both seats are instrumented with signal
transmitters, transmitting a unique signal through each of the sitting people,
and the screen is instrumented with a receiver. The rest is signal correlation.
Then they did validation and got really good results (for research): > 99%
accuracy.
Zooming out for a second, I really wish that this wasn’t a problem that needed
to be solved. Physical controls work better in a car, and allow the driver to
interact with them more easily.
Grabity: ungrounded haptics for grasping
This project is an end-to-end look at an ungrounded device that provided haptic
feedback for grasping an object with your hand. They detailed each stage:
grabbing it, picking it up, feeling its weight, then moving it around and
feeling its inertia.
Weight is perceived by multiple senses in the human body: muscle spindles (eg.
in biceps), golgi tendons (eg. in elbow), mechanoreceptors sensing shear (eg. in
palm and fingers). Their idea: only use mechanoreceptors in hand, which lets you
simulate the equivalent of about 15g of weight. It was cool that they compared
the sensation to an actual weight with real users. I’ve seen prior work which
relies on asymmetric vibration to create virtual forces through skin
displacement. A few limitations: it looks pretty wacky, and users thought that
the constant vibration was annoying, but pretty cool tech demo.
Dodecapen: 6DOF Pose Tracking System
This project puts a dodecahedron-shaped AR marker on the end of a regular pen. A
cube didn’t work because of pose ambiguity, and other Platonic solids have
triangular faces that are relatively small in area.
3D printing a perfect dodecahedron is hard, so they needed a calibration step
for it. The pen-tip also needs to be calibrated, since pens vary in size and
shape. Also the paper surface itself had to be calibrated. That’s a lot of
calibrations! They used a mocap system with 8 markers to create ground truth.
Looked at the mean shortest distance and saw mm-level precision. They claim that
precision is comparable to a mocap system with 10 active cameras.
I also tried the demo, which worked very well. One caveat is that it relies on a
global shutter camera, so not sure how well it generalizes to smartphones yet.
Another big limitation here is that the dodecahedron needs to be in the camera’s
FOV.
Inviso: 3D design environment to create sonic VR
Anil Camci showed a web-based GUI to define point sources as well as regions for
playing back ambisonic soundscapes. Point sources could have multiple cones that
you could attach with a graphical tool, and also adjust their direction, angles,
and amplitude. Each point source, and the observer could also be animated over a
trajectory. You could then play it back through the Web Audio API. Most of the
design is done from a top-down view, since this is a good match to human
auditory perception, which is much better at lateral than vertical
source differentiation.
In the end he showed multiple examples of people using the system and creating
complex auditory scenes, including overlapping zones, and animated, multi-coned
point sources. Seems to be quite efficient for building up immersive auditory
environments.
I encouraged them to recreate classic spatial audio demos, like my favorite
barbershop example. The editor is available online, and
just recently open sourced.
SoundCraft: smart watch interaction using hand generated acoustics
This project features a microphone array that fits onto a watch form factor.
They can localize sounds made by the other hand against itself (what is the
sound of one hand clapping?) or against another surface. Localization gives you
angles relative to the watch (angular resolution ~15 deg).
The applications were kind of weak, but having a small aperture (1/2 inch
between microphones) and wearable form factor was an interesting idea. They also
showed tracking the lateral position of a finger on the same hand as the watch,
and using the watch as an audio marker for an AR application.
Data storage & interaction using magnetized fabric
Justin Chan from UW showed what can be achieved when conductive fabric (woven
out of conductive thread) is magnetized using a permanent magnet. First they
explored various properties of the medium, like how long it stays magnetized
(28% decrease per week), how far the field can be detected from (up to 1cm), how
its properties vary with weave pattern (better for dense patterns), and whether
magnetization decreases from washing (not very much). The basic idea is that you
can encode data onto a conductive thread by having conductive segments that can
be magnetized interleaved with insulated segments. A binary string is then
encoded by mapping 0 to one polarity and 1 to the other.
They also showed use cases, such as writing and reading data from the clothing,
or even engraving data images (smallest pixel using a regular loom was 1.8cm^2)
onto clothing. They also showed some interactive examples where the clothing can
be read by a magnetometer on a phone or watch. This is interesting especially
because it doesn’t require any batteries, and can be achieved using
off-the-shelf embroidery machines.
iSoft: soft touch and stretch sensor
This project uses electrical impedance tomography (EIT, which I know nothing
about, and wasn’t explained) to detect touch and stretch in (1D) in a special
elastomer which is designed to be affordable. Discrete contact accuracy was
really good: 96%, using a small dataset. Distance error was ~10% of sensor size.
Applications (as usual, weak): a bunch of instrumented objects (lamp, cup,
clothing, travel pillow, etc). Limitations: hard to make it big, hard to make it
arbitrarily shaped, doesn’t support multidimensional stretching.
Mutual human actuation
I missed most of the VR sessions because parallel tracks were more interesting,
but caught this one because it was super clever. Previously, Haptic Turk used
support people to lift and lower the player immersed in VR. TurkDeck was an
environment for players that is constructed by support people outside VR.
However, previous research found that players enjoyed the experience more than
the support people, so Mutual Turk (presented here) made all participants into
players. Two people are in VR, but they are playing different games that are
complimentary in force (eg. A is fishing, B is flying a kite.)
This generalizes to two things: 1. shared props through which users exchange
forces, and 2. synchronization of state between the people involved. They had a
few other examples, like having one player fan a campfire, which created wind
for the other player in the middle of a storm. As far as practicality, obviously
this requires multiple people to be co-present, and also requires multiple
players to synchronize up to the action sequence.
Demos, Posters, Contests
Apparently you can print piezoelectric elements. I tried a set that emits low
frequency vibrations for haptic feedback. But you need a special print process
(won’t work on an inkjet).
Artem Dementyev built a robot that walks along human skin using pneumatics.
Supposed applications in health care.
Pepper’s cone was a great illusion from Steve Seitz’s student. Nice improvement
over the usual ghost setup, which has four discrete viewing points, where here
you can pick any point in a circle.
Gierad showed a poster about a multimodal sensing platform that plugs right into
the wall and which can track all sorts of activity in the house, using high
frequency accelerometers, microphones, and other sensors.
Hanchuan Li from UW showed a poster summarizing some of his PhD projects,
including passive sensing, IR based user localization, and various power
harvesting techniques.
Jihyeon Janel Lee showed her poster which automatically created accompanying
visualizations for travel podcasts via geographic entity extraction.
The good folks at i.am showed a way of making TTS sound better by extracting
contours from speech spoken by a person onto a TTS-synthesized example.
The student design contest featured creative ideas implemented with the help of
an Arduino Braccio robot arm. One project required multiple people to operate
one arm. Each person had to do a gesture to operate one degree of freedom of the
arm. Another project involved two front body-mounted robot arms that fed the
other person. Yet another featured an improv show where an actor operated a sock
puppet, and a robot arm spoke lines that were crowdsourced by the audience.
Keynotes…
Gabriella Coleman kicked off UIST 2017 with a slightly off-topic, but
nevertheless interesting (and timely) keynote, summarizing hacker culture from a
historical and anthropological perspective. Much of it was review for me, but I
also learned a fair amount:
- The origin of “hacker” comes from the Tech Model Railroad Club from MIT. Now
James’ fascination with model railroads is crystallizing. - Some of the earlier phreakers were blind, and phreaking was empowering. Some
were (sometimes additionally) gifted with perfect pitch, and could whistle the
correct frequency to emulate phone network signals. - The speaker’s personal beliefs were very anti-US, and she was very aligned
with the hackers. Perhaps it is not coincidental that she is at McGill now.
Upon receiving a lasting impact award, Ivan Popyrev gave a short summary of his
storied career. I liked his frankness on the topic of VR: “With every VR fad,
the next fad is the AR fad”. At one point he also suggested that we need to
break away from “heavy reliance on rich visual feedback and undivided attention
from the user” that are common in today’s UIs. I completely agree.
I liked his succinct vision: “How can we make the whole world augmented and
interactive?”, which he then elaborates on as having two parts:
- Augment physical, tactile & proprioceptive channels of human augmentation & actuation.
- Deliver novel UI at scale by integrating them into everyday physical objects.
I should come up with one for my HCI work.
Niki Kittur closed off the conference with a summary of his crowdsourcing work,
and I was quite glad to see CrowdForge prominently mentioned in the beginning.
He did a good job weaving a coherent narrative throughout the last five years of
his career, which covered disparate topics such as peer production,
crowdsourcing and web browsing. I often wonder how much of that storytelling is
premeditated, and how much of it is done post-hoc. He showed off some neat
projects, including Alloy (name perhaps forge-related?) and KA, as well as
Iota, a task-oriented browser inspired by V. Bush’s vision of people
leaving trails of knowledge for one another.
I skipped the last few UISTs, and it was nice to say hi to some familiar faces,
especially from CMU. I also met some great people, including Danny Fan from an
audio startup in LA, some talented grad students and fortuitously
re-connected with old acquaintances (Hi Trevor) who are apparently also based in
Seattle.
via : Boris Smus
