Ubiquitous computing/IoT and architectural design, essay #1

GE estimates that the "Industrial Internet" has the potential to add $10 to $15 trillion to global GDP and Cisco increased to $19 trillion its forecast for the economic value created by the "Internet of Everything" in the year 2020. It is said that the largest growth in the history of humans. However, there have been little estimate about the impacts on future of architectural and urban design in the context of traditional architecture history and theory. Together with my architectural academic and professional career, I would like to research and develop this topic at the Interactive Telecommunications Program(ITP) at NYU, in USA. This essay will include my perspectives from studying and working in architectural design. 

What is Internet of Things? 

Internet of Things is any physical objects that connects to the Internet. Gradually, things around us have been connected to the Internet. In 1990, there were 300,000 desktop computers connected to the Internet around the world and the number soared to 300 millions in 2000. Today, in 2016, more than 2 billion mobile phones are connected to the Internet. 

Many tend to overlook about IoT that it is essentially a small computer. It is not just an electronic products connected to the Internet but it has high performance microcontrollers inside which allow the users to manipulate many complicated things. For instance, Philips Hue light is not only a light that the users can turn on and off around the world, yet also allows to manipulate the color, brightness, flashing patterns, group the different lights, and schedules. It also generates tons of data, difficult to process and analyse, named 'Big Data'. 

In four years time, it is anticipated that "13 billion kettles, fridges, freezers, TVs, thermostats, security cameras, lights, smoke detectors, and other things in your home connected to the Internet" according to Visual Capitalist. . My main interests about this epic change are two: how our lives with IoT will require readdress spatial design and how the data IoT generates will be meaningfully used in designing. 

My basic observation about IoT

Take GE's WINK Light's bulb as an example, the light bulb can be connected and manipulated by smart phone.

WINK's hub and the instructions

WINK's hub and the instructions

 

At ITP, in the class led by Tom Igoe called Understanding Networks, we had a chance to tested some IoT products and I chose to user test GE's WINK light, a connected light bulb. The basic way to use the bulb is to install an mobile app, a controller of this light on our smartphone (they provide an app) and the app allows us to control turn on or off, brightness, and schedules. 

diagram of GE light's system drawn by Saki Hayashi

diagram of GE light's system drawn by Saki Hayashi

Although both the mobile phone and the bulb are in our house, the lighting command is sent to the API, the command automatically route to the particular bulb.  

After testing it, I realised two main problems about this product: the controller switch, and the parameter controls. Since we have used lights with such comfortable stitch mounted on the wall, it had been very difficult for me to change the habit from turning on the little switch next to the door to through my smart phone before opening the door. Without thinking, my hand automatically turned on the switch. In addition, when I turned on the light with the conventional switch, (it turns on with it anyway), indeed, I did not recognise until the time I needed to turn off the switch in that I did not want to go around all the bulbs in my room one by one and I'm usually in the bed.

A hundred of icons of the connected device apps on your home screens as well as emails, social media, ticketing, music, books, etc may annoy you to search them and eventually, you may demand another controller that designed for the lights. While some states that all the controllers will be physical, others think voice control device will be our future. 

Both GE's WINK and Philips' Hue lights allow other companies' voice recognition devices to control the lighting system such as Apple computer's Siri, Amazon's Echo, Google's Google Home,  Logitech's Harmony, Google's NEST, and Samsung's SmartThings. Some of them records all the conversation around the product(in a house), analyse the words, store them, sell the data for the marketing and raised privacy issues. 

It took me more than two weeks to adjust myself to use my mobile phone to control it. It may be quite hard for some, especially elderly people to change their behaviours who are not familiar with PC, and smart phone interface. IoT's devices require a number of this sorts of behaviour change. If a fridge is connected, we may need to register food we buy to the fridge, set a set of toast before go to bed. We may need to stand by the fridge long time ever before and the toaster has to be placed on where the wifi reaches.

Objects talk

Another notable characteristic of IoT is they can talk each other. For example, when you turn off your alarm in the morning, the alarm can possibly turn on the light and coffee maker as well. The fridge will send you a text message that your coffee milk will expire tomorrow. It may tell you where the food come from and how safe it can be. You are connected to your fridge while working outside and through your mobile phone and gain the information of the food around the world. When you come back to your house, the lights welcome you(light up) by knowing you are close to the house with GPS system on your mobile phone. I believe that the user experience of the space will become more and more aware of objects than ever before. 

The feature offers another level of design opportunity. The chatty objects can offer different ways to send a message to users as well as sending notifications and switch the other devices on. Google's NEST, their air quality monitor devise allows users to connect with Philips' Hue light. When NEST detect a carbon monoxide emergency, they turn the Hue light Red color.  There are a number of different ways to create a new environment, atmospheric messages through lights, sounds, touch and may be smells. This is a new spatial design even though the designers design an object and if architects would like to truly control their design, in the future, architects would need to develop architectural languages that integrate the new devices.

 

IoT and Big Data

One of most interesting factor of Internet of Things we would have a hard time to recognise without specific knowledge, which I'd like to value this essay for architects, is it generates 'big data'. Every time the users utilise an IoT device, it generates data that become big data over time. For instance, when a man turn on a connected light, he generates a data such as brightness, the colour range he used, the time and date he lit it up. This is generally archived and utilised in various ways, including optimisations, marketing, advertisement, and academic and business researches. The data is often sold to third parties.

 

Smart Home and City

IoT is often mentioned in conjunction with smart house and city that they make the our energy and time use efficient. Today, houses, offices and schools started to install devices which read energy usage and consumption so that the users can revise what makes energy waste and efficient. By connecting different gadgets, the users can control use of energy more precisely and efficient. 

One of my projects at ITP is an air quality monitor specifically for asthma patients called 'Networked Weather Modules'. This project started from our teammate's real experience that she has been suffering from asthma. 

The project name called Networked Weather Modules' collaborated with another two class mates, Federico Burch, and Michelle Chandra at ITP monitors the air quality of inside and outside, and automatically turn on and off air cleaner, humidifier, air conditioner and heater before they experience asthma attack. 

The User Research: Our target user group is elderly people who suffer asthma condition with invisible air conditions which affects asthma attacks due to their high death rate. Many people die in US by asthma attach, yet they do not need to die if they use a tool called 'inhaler'. The problem is that asthma patients do not know when asthma attack happens and they often carry the inhaler close to them when they feel like they need in that they never know the invisible air quality condition. 

The Findings by Our Customer Interview:

  • The death by asthma attacks can be prevented by using an inhaler, but people do not carry them all the time as it barely happens and they never sure when it will happen.
  • The asthma attack is caused by humidity, and air quality.
  • The alarming levels of humidity and air quality depend on individuals

Our Solutions:

  1. Create user-friendly air quality monitoring products and alarming system
  2. Create a user interface that displays their outside and inside air conditions
  3. Create a user interface that instruct correctly what to do when it alarms
  4. Create the mechanism that notices to their family and friends
  5. Create the AI mechanism that remembers in what air quality levels the user experiences an asthma attack

 

the prototyping process photo 

the prototyping process photo 

IMG_9996.jpg
The enclosure design for Networked Weather Modules

The enclosure design for Networked Weather Modules

We further wanted to develop the project that attach wearable computer which monitor air quality of the user wherever she or he is. With very short time period of development, unfortunately, we could not test wether it can be successful or not, yet after finishing this project, we found the very similar project called AlerSense and they are accepting pre-order now.  

Array of Things(AoT), Smart Citizen (Citizen Science) and LINK

IoT project is not limited to a smaller scale like our home yet larger cities. A number of organisations both from private and public have developed IoT products as well. Array of Things is "led by Charlie Catlett and researchers from the Urban Center for Computation and Data of the Computation Institute, a joint initiative of Argonne National Laboratory and the University of Chicago. AoT will provide real-time, location-based data about the city’s environment, infrastructure and activity to researchers and the public. This initiative has the potential to allow researchers, policymakers, developers and residents to work together and take specific actions that will make Chicago and other cities healthier, more efficient and more livable. The data will help make Chicago a truly “smart city”. 

In detail, they have developed a hardware device that collects measure temperature, barometric pressure, light, vibration, carbon monoxide, nitrogen dioxide, sulfur dioxide, ozone, ambient sound intensity, pedestrian and vehicle traffic, and surface temperature. Continued research and development will help create sensors to monitor other urban factors of interest such as flooding and standing water, precipitation, wind, and pollutants. They place it all around the city of Chicago and open the data for public so that all researchers and startup companies can use it.

 

Smart Citizen (Citizen Science)

Collecting high-resolution data is can be done by citizens globally not government locally. Smart Citizen is a project initiated by FabLab Barcelona and the Institute for Advanced Architecture of Catalonia. It is a platform that anyone can provide environmental data they gather on a world map. The software and hardware to send the data to the platform is open and anyone can participate in this free. They also sell a kit that beginners and educators can build a sensor with the instructions that it makes the participation easier. This kind of participatory science by citizens often called 'Citizen Science' and in the future, more global citizens will participate in this kind of global platform, environmental issues will be advanced faster and precise which will ultimately contribute to public.

People who believe 'open data' is beneficial are not minority. Alex Sandy Pentland, Professor of Computer Science, MIT, says that "If you have children, you don't want to see them die of an H1N1 pandemic. How are you going to stop that? Well, it turns out that if you can actually watch people's behavior in real time...something that is quite possible today…you can tell when each individual person is getting sick. This means you can actually see the spread of influenza from person to person on an individual level. And if you can see it, you can stop it".

Future of urban studies will, more or less, collaborate with this kinds of data provided by citizens.

The third project is LINKNYC led by the city of NY, CityBridge and a few private companies such as IntersectionQualcomm, and CIVIQ Smartscapes. The city plans to replace phone booth from the modern one to WIFI station that the users of the city can call in US free with an app they released. The fund is supported by the advertisement they projects on surface of the device. Dissimilar to Array of Things and Smart Citizens, this project is not meant to gather and provide data for public. Still, the arrival of this sort of machine that interact with pedestrians will change the dynamics of pedestrians.

MTA, Metropolitan Transportation Authority in NYC has provided bus and some subway tracking data and today, the third party mobile app companies launched mobile app so that the users are able to see where exactly buses arrive in any particular bus stop or some subway lines. Urban living has been connected to the Internet more and more.

 

Our Peer to Peer (P2P) solution to public transport for urban infrastructure

One of my project at ITP called Strategy of the Commons is developed by another two class mates, David Tracy and Daniel Melancon. Under professor Dimitris Papanikolaou who researched Mod system at MIT MediaLab and PhD at Harvard University. We proposed an mobile app that solves a problem of CIti bike's imbalanced bike stock problems.

There are always empty and full bike stations, limiting the users' bike trips, and worse, CitiBike charges if the users cannot return the bike in time. If a user could not find an empty dock, they have to call to CitiBike and continue to look for an empty dock that sometimes far away. Citibike runs tracks and vehicles to move bikes from full to empty stations, not a sustainable solution and cost more. Instead of having the current outdated system that one person sitting in front of PC and call to truck driver to move bikes, we prototyped an mobile app, a game that can balance the stocks by P2P system. The question is, can it be improved by applying a decentralized system architecture model?

The analysis of the users:

Our team, Dan Melancon, David Tracy and I conducted a research about the user types by analyzing the system data Citibike provides monthly[3-5]. The results of the research show that 75 percent of the users are men and the age range is from 30 to 40 years olds in every neighbor. Our assumption was that each group of the station had different range of users in age and gender yet the outcome was almost the same in all the neighbours.

Application of Game Theory:

Our team analyzed the problem in this bike sharing system with the theory of, Tragedy of the commons of Game theory. This economics theory by Garrett Hardin states that the users’ selfish behaviors causes a lack of resources[1], in this case, the bikes and available dock. If every user takes into account a lack of resources, self-organizes and self-balances the system itself, the problems that Citibike sharing system is suffering can be solved. It benefits the entire community. Considering Prisoner’s dilemma[6], even though cooperation will benefit the entire system including the individuals, our rational modern thoughts conflicts with gaining the cooperative advantages. But, if all the users leave the last bike or fill quickly on the empty station, or take the bike from full stations as soon as possible, the system can be balanced. The problem is though how to get rational thinkers of our modern Citibike users to behave in a self-forgetful manner.

Utilizing sense of competition:

Ultimately, our game creation’s main point is to see how/if self-interest agents could behave like self-organized agents which could balance the system by applying rules and incentives without asking them to be altruistic. We think that best ideal solution to this is to always have at least one bike or one dock available in every station. After long brainstorming, we came up with final proposal has two main factors. Firstly, every station will provide points when the riders return their bikes and the points change dynamically according to the number of bikes that remain in the stations. Less bikes there, more points the users can gain. Secondly, when the last bike is taken from any station, everyone’s points are reduced and the points become bonus for the first user who parks a bike in that station. In order to change the behavior of the agents in real-time, we proposed to create a mobile application which visualizes the data of their scores and stocks of the stations.

The play test:

As there was time and physical constrains, our play test simulated the bike sharing system of NYC in a smaller scale on the floor at ITP. We created RFID card readers with LED feedback interface as stations, and a customized website for each player which updates the points, bonus and stock of the stations, and scores and bike stocks of the users in real-time. The players can look at their phones while playing. 

We did the play test five times in total and we slightly changed our system and interface design. However, some of the players only aimed to get bonus and run only between two closer stations. In order to prevent this, we set up missions that they had to go to a bike station the system asked them. That means that they cannot go and get bonus when they want. If they had missions more than half of the times, the players got bored. Indeed, most balanced session was least interesting for the players. 

Unfortunately, we could not have time to develop this further, yet  the success of this game will depend on how players can flexibly go to other stations to gain higher points and bonus without disturbing their study and job. It will not be an easy development, yet it has a greater potential that shapes our future of bike sharing system that can build local bike sharing community with technology. 

The prototype final we created / the mobile interface: data visualisation/  back: 'station' having RFID card reader -we use the same technology of today's CitiBike uses

The prototype final we created / the mobile interface: data visualisation/  back: 'station' having RFID card reader -we use the same technology of today's CitiBike uses

'Compact City' by  George Dantzig and Thomas L. Saaty is a modern icon of urban planning due to its cost, and time efficiency. It is described as city of short distances is an urban planning and urban design concept, which promotes relatively high residential density with mixed land uses. It is based on an efficient public transport system and has an urban layout which – according to its advocates – encourages walking and cycling, low energy consumption and reduced pollution. A large resident population provides opportunities for social interaction as well as a feeling of safety in numbers and 'eyes on the street'.[1] It is also arguably a more sustainable urban settlement type than urban sprawl because it is less dependent on the car, requiring less (and cheaper per capita) infrastructure provision (Williams 2000, cited in Dempsey 2010).[2] However, it is said that when all the areas are easy to access, the price of the real estate boosted up that they eventually kicked poorer people out. Also, Jane Jacobs and her book The Death and Life of Great American Cities (1961), a critique of modernist planning policies claimed by Jacobs to be destroying many existing inner-city communities. Previously, urbanists have suffered to change the city of hardware such as urban infrastructures yet today, we have gained another tool, designing software to change the activities of the crowds. This is one of biggest reasons that I strongly feel architects and urbanists should understand more about technology.  

Today, the crowds get around the city not only for food, drinks and shopping, but for a game that the city becomes a big game ground. Pokemon Go is an augmented reality (AR) game that the users can look up their mobile app , find Pokemons (animation characters) mapped in the real city and catch them to get scores on their mobile phone. Games, and data hugely impact on our public spaces. Previously, urban planning is about hardware, such as public transports, zoning, and public squares etc, which mostly shape the dwellers' activities yet today, an mobile app has a strong power to change the big crowds' activities. 

Pokemon Go's smartphone display showing the virtual character mapped on the real street

Pokemon Go's smartphone display showing the virtual character mapped on the real street

source: http://www.polygon.com/2016/7/18/12215658/pokemon-go-meetup-central-park

source: http://www.polygon.com/2016/7/18/12215658/pokemon-go-meetup-central-park

 

Of course, majority of the data usage and mobile apps today have been making urban living more efficient. Restaurants reviews, busses arrival time tracker, weather forecast apps are to name but a few. There are a number of startup companies are working to make our home and city smarter today.

Machine Learning for IoT

Artificial Intelligence(AI), machine learning program is another noticeable way to use our data and a key to make IoT successful. Like we have already experienced one social media, and shopping websites showing advertisements specifically for you, they store, analyse the data and use it for marketing and advertisements based on your buying and web search history. Once we buy a product in Amazon, they suggest other things we might like. Like your private secretary, it assists us find, and buy what we are looking for. The similar idea and algorithm can run on IoT products through the big data. It stores, analyses and creates new suggestions to you based on your past activities. Albert-László Barabási points out the new era of data analysis that “The difference between human dynamics and data mining boils down to this: Data mining predicts our behaviors based on records of our patterns of activity; we don't even have to understand the origins of the patterns exploited by the algorithm. Students of human dynamics, on the other hand, seek to develop models and theories to explain why, when, and where we do the things we do with some regularity.” ― Albert-László Barabási, Bursts: The Hidden Pattern Behind Everything We Do. 

The conversation between humans and IoT home

As I explained, due to 1. controlling more complicated parameters, 2: experiencing the feedback system running in back based on stored and analyzed data changed our conversation between space and the users. The bottom is a diagram I drew about an example of a conversation taking place between IoT (+smart) light and the users. 

A missing opportunity for designers

When I started to study technology at ITP, I realised that the importance of personal big data, the data of new level of reality, and self. Essentially, generating data means we leave rich and new traces of our existence in digital format, lacking an opportunity to visualise and materialise it.

The power of object

This is my personal story. My stepfather was an almost orphan and his father's parents seemed they cut his family tree. When he was ending his life, my mom showed me a notebook called 'past book' which records ancestry on it in Buddhism, and cried. Until this incident happened, I did not realise 'the power of object', materialising the past. The fact that my stepfather was born from his parents and they had their parents do not change although he did not write them down on a beautifully crafted paper. However, he needed it. It was a way to express his emotions, wish to carry his family tree after his death, and gave him a sense of belongings. It tells stronger story about his life and anchors his existence after passing away for us. 

In architectural and urban design context, this may not be by far different. Aldo Rossi states that "one can say that the city itself is the collective memory of its people, and like memory it is associated with objects and places, The city is the locus of the collective memory". 

Objects and materials are essentially a source of our ego and complete elimination of reconstructing our past memory through objects is impossible. Therefore, I believe that it's is crucial to create objects that powerfully tell our existence. 

My main thesis of IoT and spatial, architectural and urban design is how meaningly visualise and materialise the rich data remaining traces of our existences and I have created two projects, Mnemonic Making/Table(for larger version) and Future Antique Chandelier in Digital Age. 

Mnemonic Making/Table

I created interactive architectural building blocks that memorises past combinations. It memorises, visualises the past combinations, test them how the space are different, and remind the users when they created the same configurations with their past, or other users' one on the screen. As well as it visualises the data, the physical objects lights up and makes sounds, when they find the same configurations

In order to further develop the project in the context of architecture, I studied about 'A Pattern Language' by Christopher Alexander and created five pieces that responds to his theory.

ITP Sprint Show 2014 A boy kept playing with blocks

ITP Sprint Show 2014 A boy kept playing with blocks

In this project, I was planning to create real-time animation that changed the story by changing the configuration of this physical objects, that allows the users to quickly learn spatial language, yet unfortunately, the time is too tight to make it happen. Still, the exhibition at ITP was particularly successful for kids they engaged with this game very well. I won a student competition to exhibit this project in a muserum called BLDG92 in Brooklyn Navy Yard for semi-permanent exhibition. I hope that the project will develop not only for the model scale but also real human scale. 

Future Antique Chandelier in Digital Age

The last project is my thesis at ITP called 'Future Antique Chandelier in Digital Age' , a prototype of future hardware design, sustainable and future electronics of heirloom. It ages and traces the users' existence by electrochemical reactions on the lamp shade according to the data the user create whenever they use the smart light. 

I continued this thesis from my architectural thesis called ' Aesthetically Ageing City' at Architectural Association School of Architecture (the AA). The urban design proposal on top of Troy remains offers residential high-rises the surface ages over time by weathering, which protects the remains from ageing. 

section drawing of Aesthetically Ageing City

section drawing of Aesthetically Ageing City

Traces of time and events that strengthen our sense of security

The problem of IoT is that it's hard to be heirlooms in that they are meant to be upgraded and replaced thus I created the first prototype of connected Future Antique Chandelier and this is the link to my whole presentation at ITP 2015.

My mother never cared about an antique clock which my stepfather bought and winded the screw. After he passed away, though, she has been obsessed with not stopping the clock. Objects are a powerful way to tie with our past our memories and allow us to express holding emotions. I am hoping that this project will contribute to future of our society. 

I have researched, examined and prototyped some solutions to the problems, yet, this entire change is huge, and no one knows how users will shape our future house with connected devices. Some people believe that human life will go back to animism and if we believe what David Rose states, “Any sufficiently advanced technology is indistinguishable from magic” ― David Rose, Enchanted Objects: Design, Human Desire and the Internet of Things, architectural designer may be required to design a house for enchanters. Wether we revise our design for humans or enchanters, our lives are changing dramatically with technology. 

Wireless technologies and user experience design

Lastly, I'd like to highlight user experience design will be very important when we design any space. Like Le Corbusier considered well in the details of hinges, and door handles when he created Vila Savoye, greater architects need to be careful about how all the objects work in relation to the spatial configurations. And when we think about user experience design, a hidden big influence of IoT products can be wireless technologies such as wifi, bluetooth, zigbee (protocols controlling electro magnetic field), rechargeable battery technology, and wireless battery charging technology to name but a few. The users feel a lot stress about wires, cables and position of plugs. 

Thank you very much for providing me such great opportunity to write about this essay. I would like to thank to Union Foundation and I deeply apologise my delay.