Wattom: Ambient Eco-feedback with Mid-air Input

This paper presents Wattom, a highly interactive ambient eco-feedback smart plug that aims to promote a more sustainable use of electricity in the home. This paper describes our latest implementation of the Wattom plug, and three system applications. The first enables Wattom to power a connected device, and provide real-time feedback on the amount of electricity in the grid from renewable sources. The second enables users to schedule power events from their smart watches. Finally, the third application uses non-intrusive load monitoring (NILM) to provide users with personal consumption information on multiple devices connected to a single Wattom plug. The paper concludes by presenting insights into the development and use of various iterations of the Wattom plug.


I. INTRODUCTION AND RELATED WORK
The issue of sustainability is one of the most pressing concerns of our time, one that requires multi-disciplinary efforts from areas such as engineering, psychology, design, and economics. Energy consumption at home is a particularly interesting area of study, as it accounts for close to 15% of all energy consumption in the world -a value that is expected to continue increase in the near future [1]. Approaches that deal with energy consumption at home are normally described as eco-feedback technologies, and are defined by any technology that feedbacks on individual or group/family behaviors with the goal of reducing the environmental footprint of their activities [2]. These technologies tend to provide users with numerical or graphical information [3]; and represent their consumption in Watts, financial cost, or through metaphors such as a digital fish tank whose cleanliness is tied to users' energy use [4]. While valuable, this information has been criticized for being detached from the user's behaviour driving the consumption, or from the resources behind the energy produced [5].
An emergent field that addresses this limitation is described as ambient eco-feedback. These systems often rely on tangible artifacts that alter their form to directly and immediately inform the user of their energy consumption [6]. One of the early examples of this is the Power-Aware Cord [7], a power cord that displays a light animation representing real-time energy consumption. Other examples include Heller et al.'s [8], [9] work on a approach that displayed similar information This work was supported by the Fundação para a Ciência e a Tecnologia (FCT, UID/EEA/50009/2013). but on a wall socket, and bespoke tangible artifacts [10]- [12] and smart lighting systems [13] that behave as ambient displays. And while these prototypes have been shown to be quite effective at educating users on their energy use [14], some have pointed that the data they present is quite coarse and that they offer a very limited interaction space, ultimately limiting the engagement between users and the environmental impact of their actions [15].
We present our latest implementation of Wattom, a smart plug inspired by the PowerSocket [9] that provides users with direct and touchless controls over the energy information being displayed. This is supported through motion matching [16]- [18], an interaction paradigm where users interact by tracking a system's different animations with their hands (tracked through an inertial measurement unit -IMU -available on any smart watch). Compared to other gestural and touchless interaction techniques [19]- [21], motion matching is less likely to be affected by the 'Midas touch' (i.e., false positives), has a radically simple recognition pipeline that does not need training, and does not require users to learn and memorize specific gestures [22]. Further, our IMU-based approach also introduces several benefits over traditional computer vision implementations, including a camera's finite fieldof-view (FOV) and its susceptibility to environmental factors (including occlusion), and understandable privacy concerns when used in the home [23].
The remainder of the papers describes the latest Wattom implementation, and three applications that enable users to switch off appliances at a distance (and schedule these events), consult the current amount of renewables in the grid, and check on the consumption of different appliances and users.

II. WATTOM
Wattom is a energy aware smart plug supporting direct and mid-air input, and inspired by ambient eco-feedback systems such as the Power-Aware Cord [7] and PowerSocket [9]. The coupling approach described in the latter two demos, and where a smart watch application and the Wattom plug are momentarily paired to allow users to schedule events or retrieve consumption information is inspired by previous work that uses synchronous movements to: pair devices [24]; authenticate users [25]; and identify a pointing apparatus [26].  1. After a Wattom plug is selected, it displays two targets moving in opposing directions. The first enables users to turn a connected appliance on or off, here represented as a white LED to indicate the appliance's current state (i.e., off). The second, in purple, accesses the disaggregating consumption demo. The user is turning on the kettle by the movement of the white LED with its hand (tracked through a smart watch). Finally, Wattom's green background (unused LEDs) indicates to the user that this is a great time to brew some tea, as close to 50% of the electricity in the grid was produced by renewable sources.

A. Implementation
Each Wattom employs a Schuko plug 1 housed in a 13 (w) × 12 (h) × 9cm wooden case and connected to a standard wall socket. Guided by the form factor of the round plug, this is surrounded by 24 RGB LEDs (24-bit color), equally spaced around a 7cm ring (diameter). In addition to the LEDs, Wattom houses a small single-board computer (Raspberry Pi 3 Model B+ 2 ) that deals with the program logic, LED and server access, and energy management and appliance control (using an ADC, a 10A relay, and a ASC712 current sensor). In addition to energy management, Wattom also keeps track of power events representing power changes, which indicate a change in state in an appliance (e.g., a a kettle automatically turning off). This information is processed using a non-intrusive load monitoring (NILM) algorithm [27] that enables Wattom to automatically identify which appliance is responsible for which power event. Details and source code can be found at: anonymous sub..

B. Server-side
Each Wattom plug communicates wirelessly over a local network with a Node.js 3 server responsible for keeping track of the state of every LED in the system. Only two Wattom plugs were built, but the server is designed to automatically support multiple concurrent plugs sharing the same network. The web server also provides third-parties with an API for LED management, appliance control (on/off), and to query each Wattom plug on their energy consumption. Finally, the server is capable of accessing online services such as data on the amount of renewables currently powering the grid.

C. Interaction
Direct interaction with Wattom is supported through motion matching [17], an interaction paradigm where interface elements move in continuous and distinct trajectories, and where users engage with these by tracking their movement with their bodies (e.g., eyes [28], head [29], hands [22]). This is exemplified in Figure 1, where a user interacts with Wattom by tracking a moving blue light with its hands. This user input is captured using most forms of wrist-mounted IMUs (e.g., smart watch, fitness tracker), and represented as Euler angles (yaw and pitch) -an approach first introduced in [30]. Visually, our system is inspired by [31], where interface elements are orbited by one or more moving targets. The lack of a graphical displays means these moving targets are displayed for the first time using a ring of LEDs. Due to the resolution of the ring used in our implementation (24 LEDs), each Wattom plug can display up to 12 concurrent targets that move in clockwise or counter-clockwise motions -these share the same speed to minimize overlapping.
Target selection works as follows: a bespoke Android Wear application runs in both the user's smart watch and smart phone. As in [30], the interaction starts after a user performs a flick of the wrist (the only gesture that needs to be learned and memorized) that triggers the start of the target movement in all local plugs. This movement, i.e., LED state (0 to 23) and direction (0 or 1), is conveyed to users' mobile apps through the Wattom server. The user's mobile app is then responsible for: (1) mapping target movements to a Cartesian plane (x-and y-positions); (2) receiving yaw and pitch values from the user's wrist-worn device; and (3), running continuous Pearson's correlations between input and output data. If four consecutive correlation results between user input and a target movement are over 0.8 in both axis (x-yaw and y-pitch), the target is selected and the user gets confirmation of this through haptic feedback on their watch. These correlations take place 60ms intervals, using the latest 40 data points (rolling window), and these activities take place at 25Hz. Lastly, every 300ms the users' mobile devices poll the web server for changes in the LEDs, and update their Cartesian-mapping if necessary. Multiple concurrent users can be connected to the server.
1) Powering devices and environmental awareness: The first of the three demos built as third-party applications to our server's API enables users to remotely turn a connected appliance on or off. This is done through a target in blue if the appliance on, or white if not (see Figure 1). As in [7], [9], the speed of this target and color of the remaining (unused) LEDs changes according to the amount of renewables present in the grid at the moment of interaction. This was implemented by averaging the value of renewables in the grid in hidden for anonymous submission for the past three years (approx. 25%). As such, the target speed and background colors would Fig. 2. Wattom can automatically power appliances at specific times of day. This schedule is specified through a bespoke mobile application where the user can define its start and end times (left). Afterwards (right), all local Wattom plugs display a single moving target that is either white (add the schedule to the plug) or red (remove the schedule from the plug, if already present). Fig. 3. Wattom's NILM capabilities enables the user to interact with various concurrent appliances connected to a single plug. In this example the lamp and kettle are represented by a blue and red targets, respectively (top). Upon selection, Wattom displays a plot with the consumption of the appropriate appliance in the user's smart watch (bottom). range from 100°/s and green, to 180°/s and red to represent renewable quotas from 0 to 50%, respectively. The goal of this functionality is to not only inform users of the impact of the activity they are about to engage in, but also facilitate or hinder that engagement (i.e., faster targets are harder to select). Finally, this demo could be easily expanded by applying the same speed/color mapping to the plug's, household's or neighbourhood's average electrical consumption.
2) Scheduling power events: Our second demo expands the functionality above by allowing users to schedule when to turn an appliance on or off. The goal is to provide users with a finer control over their electrical consumption, such as planning certain activities when the electrical costs are excepted to be lower (e.g., night plans [32]), or for when the user expects more electrical availability from photo-voltaic installations onsite [33]. To engage with this functionality, users first start the scheduling application on their smart watches, and then select a start and end period for their intended activity (see Figure  2). This triggers all local plugs to display a moving white target (or red if another activity is already planned for the same time). The user is then free to select as many plugs as it wishes. Upon selection a plug stops displaying its moving target, and the remaining plugs do the same after 10 seconds of no activity. Finally, a schedule can be removed from a plug by selecting its red moving target.
3) Disaggregating consumption: The third and final relies demo on Wattom's NILM capabilities to distinguish between multiple concurrent appliances connected to a single plug. Wattom displays a colored target per appliance (up to 12 appliances), which upon selection displays the latest energy consumption of that device as a colored plot on the user's smart watch (see Figure 3). This data can also represent a single user's consumption of that appliance, as Wattom also stores a unique ID representing each user's smart watch (the one used to power or disable the appliance using one of the two demos described above). Other visualizations could be easily explored, such as how much of the user's activity with a particular appliance were powered by renewables, or a comparison of use across users in a household (e.g., who is responsible for the of the electrical heater in a dorm).

III. CONCLUSION AND FUTURE WORK
This paper presented the latest implementation Wattom, an interactive smart plug with mid-air. This work was motivated by a need to have a tight coupling between users' energy use, its source, and its visualization. Through the use of motion of matching, Wattom was designed to not only facilitate touchless and direct control over electric appliances, but also invite users to better engage with their energy use. Because Wattom affords a public API, we hope our prototype can inspire and support future demos and applications, including outside of the real of sustainability (e.g., the LEDs could be used to display a connected appliance's charging and battery state). We highlight the following as pressing future work, a long-term, in-the-wild study of Wattom's: effectiveness (e.g., savings, behaviour change); practicality (e.g., plug visibility, multiuser scenarios); social acceptability (e.g., input movements in shared spaces); and overall user satisfaction.