Tag Archives: Raspberry Pi

Logs, logic and inspiration: Managing a complex home control setup

You know how it goes.  You start with one or two home control devices.  You find it amazing that you can control them from your phone.  You want more.  And more…

Here’s a quick diagram of my current setup at home.

setup.png

There are at least 77 items for the home control system to control.  Each one has a unique set of capabilities, and inter dependencies with other devices.  Certain groups of these items require different communication protocols, some radio, some infrared, some HTTP and some via a webserver.

As my system has been created from several protocols and brands, I find it engaging and a full-on hobby to ensure they perform perfectly in concert.  Basic scripting has become more in-depth as I attempt to squeeze out the most from every device.  Although adding additional functionality is stimulating for me and ultimately rewarding for me and my flatmate, each iteration adds a new layer of complexity – and like every complex system, the bigger it is, the harder it can fall.

I’ve recently been faced with a problem.  A few times in a row, the Raspberry Pi 2 has frozen overnight. The controls and automatic lighting obviously do not respond, and then something as simple and as taken for granted as getting light and audio into the shower requires scrabbling through phone apps: not good if the water is already running!  Worse, the switches that are supposed to be triggered in the early morning, such as the “it is dawn” variable do not fire.  So with such a complex system how do you diagnose the problem?

Logs

The first answer is logs.  Loads of logs.  Ensure each of your subsystems are writing down what they are doing and just as importantly when they are doing it.  You can then rifle through the logs and find anything that is not behaving as you’d planned.

Inter-dependency diagrams

Okay, so this may be the most geeky thing I have said on this blog so far, but I like to keep diagrams and spreadsheets showing which systems and activities are inter-related.  And in the event of a catastrophic failure, they pay dividends.  You can literally trace your finger through the lines and see which scripts you need to check if something is not working right.  You can also keep track of things like ID codes and group codes for all your connected devices.

Logic

To do this you need to empty the house of unexpected variables (i.e. the rest of the family and pets large enough to trigger any sensors) and then physically run through each process that you think may be causing the problem.  If you are anything like me this usually involves an embarrassing and potentially uncomfortable period where you are remaining totally motionless right in front of a motion sensor to see what happens when the “no movement here” signal is sent.

Inspiration

You may be surprised by the other users’ perception and understanding of your home control system.  Ask other occupants what they think is happening.  At best they could hit the nail on the head, and if not they just may throw something so left-of-centre out there that is provides you with the fresh outlook you need to trace the problem.

Summary

I feel this entry will become outdated very soon.  As consumers we are on the cusp of having our cake and eating it: a fully integrated one-stop solution for home automation that will work seamlessly and without requiring manual programming.  It may even have the ability to provide reasons for failure and suggest ways to work around it, especially if open-source and app-based: a fellow user in the Netherlands could be granted temporary access to help sort out the problem you’re having with your garage door in California.

This new way will remove the complexity involved in getting disparate systems to work together, but will it provide the level of control we ‘first gen’ full home control aficionados will require?  Either way, I’m glad I’ll be able to say “In my day, we had to fumble around to find the solutions to these issues, and sometimes create our own!”

 

 

Writing a home control front end in HTML.

I’ve been asked a few times about my custom front-end for my Domoticz, Hue and Sonos setup, so here are a few HTML snippets and where to put them, assuming you are running Domoticz on your home control server.

Right from the offset I must stress that the Icons I have used are made by various incredibly skilled designers at http://www.flaticon.com/ and therefore this front end cannot be used for any commercial purpose.

I have previously written an extensive post about the continuous development of a home control user interface and have posed a video containing my front end, so this post will not be about the thought process behind creating a UI, rather how I have managed to put one together using HTML.

I must also stress that I am not an expert in baking a web app.  Developers may read my code and scoff at its inefficiency but it works for me, and hopefully can give you some inspiration.

Layout

The layout of all panels follows the same template: A main area where the actual buttons and controls live, changing depending on which screen the user selects; the ‘scenes’ bar, a blue strip towards the bottom of the display which is a kind of ‘quick access’ ribbon for regularly used commands; and a Links bar which permanently shows the pages that can be displayed.

layout

There is also a ‘notifications’ display which shows up just above the scenes bar with information provided by a variable in Domoticz.  This text can be ‘cleared’ by touching it (more on this later).

All screens use variations on the same HTML so not every screen is shown in detail in this post.

Home Screen

Home

Home is where the heart is.  I like the home screen to be simple, uncluttered and good looking.  It’s by far the screen shown most regularly so less is more here.

The interesting part of this screen is the background.  It changes depending on the weather.  First I saved six 2000×600 backgrounds with names ranging from weatherback-rain.png to weatherback-fog.png.  You need cloudy, fog, partlycloudy, rain, snow and sunny.  I set the size of these backgrounds to fit the tablet which the screen is displayed on so you may need to adjust accordingly.

The HTML code to change the background depending on the weather is as follows (change xx, yyyy and zz to the address of your Domoticz server, the port Domoticz is using and the idx code of your weather source in Domoticz):

function updateweather(){

                var forecast

                var xmlhttp = new XMLHttpRequest();

                var url = "http://192.168.1.xx:yyyy/json.htm?type=devices&rid=zz";

                var forecast

                xmlhttp.onreadystatechange = function() {

                 if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {

                 var myArr = JSON.parse(xmlhttp.responseText);

                                                forecast = myArr.result[0].ForecastStr;

                                                console.log("Forecast is " + forecast + ".")

       // myFunction(myArr);

                                                 if (forecast == "Partly Cloudy") {

                                                 document.getElementById("weatherindicator").src = "weatherback-partlycloudy.png";

                                                                }

                                                 if (forecast == "Sunny") {

                                                 document.getElementById("weatherindicator").src = "weatherback-sunny.png";

                                                                }                                                              

                                                if (forecast == "Rain") {

                                                 document.getElementById("weatherindicator").src = "weatherback-rain.png";

                                                                }              

                                                if (forecast == "Fog") {

                                                 document.getElementById("weatherindicator").src = "weatherback-fog.png";

                                                                }                              

                                                if (forecast == "Snow") {

                                                 document.getElementById("weatherindicator").src = "weatherback-snow.png";

                                                                }

                                                if (forecast == "Cloudy") {

                                                 document.getElementById("weatherindicator").src = "weatherback-cloudy.png";

                                                                }                                              

                                                }

                }

xmlhttp.open("GET", url, true);

xmlhttp.send();

setTimeout(updateweather,60000);

}

Notice that the last line of this code sets up the web page to update the weather picture every 60 seconds.  Ok, in the body of your HTML you’ll need something like

<div id="weatherdisplay" align="left" class="weatherback"><img id="weatherindicator" src=""></div>

And you’ll need something like this wherever you save your CSS:

div.weatherback {

    position: fixed;

    top: 0px;

    left: 0px;

    width: 1000px;

                height:600px;

}

Activating Hue scenes

The API for Hue is relatively easy to use.  I used the API to save scenes to each room and then can recall them from the press of an icon on the scenes bar.  The HTML at the scene bar is easy:

<a href="javascript:;" onClick="groupscene(1,7);switchoff(24)"><img src="scene-cinema.png" width="150" height="150" border="0"></a>

The button called scene-cinema.png does two things actually, sets the group 1 to scene 7 and then switches of a switch in Domoticz.  Let’s see the code for each of these functions:

function groupscene(group,scene){

                execute('PUT', 'http://192.168.1.aa/api/bbbbbbb/groups/'+group+'/action', '{"scene":"'+scene+'"}');

}

Change aa to the address of your Hue bridge and bbbbbb to the name of the developer (if you followed the Hue API instructions from the Hue website this might be “newdeveloper”.  If your scene does not change straight away then your control panel might not be authorised to the Hue bridge.  If this is the case, press the button on the Hue bridge and then try again a couple of times.

function switchon(devicecode){

                execute('PUT', 'http://192.168.1.xx:yyyy/json.htm?type=command&param=switchlight&idx='+devicecode+'&switchcmd=On', '');

}

 

function switchoff(devicecode){

                execute('PUT', 'http://192.168.1.xx:yyyy/json.htm?type=command&param=switchlight&idx='+devicecode+'&switchcmd=Off', '');

}

 

function toggle(devicecode){

                execute('PUT', 'http://192.168.1.xx:yyyy/json.htm?type=command&param=switchlight&idx='+devicecode+'&switchcmd=Toggle', '');

}

 

function dim(devicecode,dimlevel){

                execute('PUT', 'http://192.168.1.xx:yyyy/json.htm?type=command&param=switchlight&idx='+devicecode+'&switchcmd=Set%20Level&level='+dimlevel, '');

}

The above codes (again change xx to the address of your Domoticz server and yyyy to the port number) are all similar. Switchon, switchoff do what they say on the tin.  Toggle changes the state of an on/off switch and then dim sets a certain switch you specify (idx) to the dim level you select (dimlevel).

While we’re here, here’s a list of the other Hue functions I put into the home control system:

function lightoff(light){

                execute('PUT', 'http://192.168.1.aa/api/bbbbbb/lights/'+light+'/state', '{"on":false}');

}

 

function lightmax(light){

                execute('PUT', 'http://192.168.1.aa/api/bbbbbb/lights/'+light+'/state', '{"on":true,"bri":255,"sat":0,"hue":0}');

}

 

function briup(group){

                execute('PUT', 'http://192.168.1.aa/api/bbbbbb/groups/'+group+'/action', '{"bri_inc":40}');

}

 

function bridn(group){

                execute('PUT', 'http://192.168.1.aa/api/bbbbbb/groups/'+group+'/action', '{"bri_inc":-40}');

}

 

function groupcontrol(group,hue,bri,sat){

                execute('PUT', 'http://192.168.1.aa/api/bbbbbb/groups/'+group+'/action', '{"on":true,"bri":'+bri+',"sat":'+sat+',"hue":'+hue+'}');

}

 

function groupscene(group,scene){

                execute('PUT', 'http://192.168.1.aa/api/bbbbbb/groups/'+group+'/action', '{"scene":"'+scene+'"}');

}

So these are the main components of the home page – and here’s a snippet of how I go the inside and outside temperature (change zz to the idx of your temperature sensor):

function updateintemp(){

                var instatus

                var xmlhttp = new XMLHttpRequest();

                var url = "http://192.168.1.xx:yyyy/json.htm?type=devices&rid=zz";

                xmlhttp.onreadystatechange = function() {

                if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {

               var myArr = JSON.parse(xmlhttp.responseText);

                                                instatus =  "Inside: " + myArr.result[0].Data;

                                                console.log(instatus)

                                                document.getElementById("intemp").innerHTML = instatus

                }

                }

xmlhttp.open("GET", url, true);

xmlhttp.send();

setTimeout(updateintemp,20000);

}

You need a div with id “intemp” positioned where you like, and again the last part of the above code sets up the web app to update the temperature each 20 seconds.

Devices screen

devices

The devices screen updates the icons with a green bar when the switch is on and a grey bar when off.  I save two identical png pictures, one with -on.png at the end and one with -off.png.  I enter this HTML repeatedly, changing the id and the source of the picture for each button:

<a href="javascript:;" onClick=”toggle(xx)"><img src="chesterlampoff.png" width="125" height="125" hspace="5" vspace="5" border="0" id="chesterlamp"></a>

Change xx to the idx of the device in Domoticz (you can find the idx of the device in the ‘Devices’ tab of the Domoticz interface.

Change the id=”chesterlamp” to id=”whateveryourdeviceiscalled” then change the img src to the ‘off’ picture for your device.

In the code part you need the following:

function updatedevice(idx,location,onimage,offimage){

                console.log("checking status of idx "+idx)

                var xmlhttp = new XMLHttpRequest();

                var url = "http://192.168.1.94:8080/json.htm?type=devices&rid="+idx;

                var onoff

                xmlhttp.onreadystatechange = function() {

                if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {

               var myArr = JSON.parse(xmlhttp.responseText);

                                                onoff = myArr.result[0].Status;

       // myFunction(myArr);

                }

                                if (onoff == "On") {

                                document.getElementById(location).src = onimage;

                                }

                                if (onoff == "Off") {

                                document.getElementById(location).src = offimage;

                                }

                                if (onoff == "Open") {

                                document.getElementById(location).src = onimage;

                                }

                                if (onoff == "Closed") {

                                document.getElementById(location).src = offimage;

                                }

                }

xmlhttp.open("GET", url, true);

xmlhttp.send();

}

Then another function where you will put all the code to tell the web app to update the icons depending on how Domoticz reports the switch (on or off, or even open or closed if you’re using door sensors too):

window.setInterval(function(){

                updatedevice(187,'chesterlamp',"chesterlampon.png","chesterlampoff.png");

                updatedevice(132,'washingmachine',"washingmachineon.png","washingmachineoff.png");

                countup();

                updatenotification(11)

                }, 1000);

I’ve put two switches here, Chester’s lamp and the washing machine.  You can add as many switches as you like here, as long as they have been set up in the body of the HTML.

There are two other functions that are called each second too: countup() and updatenotification(11)

Automatically reverting to the Home Screen

If you want the screen to revert to home after two minutes of inactivity, you can use this code:

First put

var ticker = 0;

at the start of your code block, then put:

function countup(){

                ticker=ticker+1

                console.log("Ticker is " + ticker);

                if (ticker>120) {

                                console.log("Moving to index...")

                                MM_goToURL('self','index.htm');

                                }

                }

This means that once the variable ‘ticker’ has incremented to 120, the screen will go to the page called index.htm.  If you use this code, remember to put this at the end of each function:

ticker = 0;

This will reset the timer so that another 2 minutes have been added to the time before the page will switch to the Home Screen.

Notifications from Domoticz

I have set up the screens to show a strip which indicates what Domoticz is up to.  Some of my LUA scripts in Domoticz update a variable with a string of text in English to tell the user what Domoticz is doing.  This could be confirmation that a switch has been turned on/off or it could report is something has been triggered automatically.

First, create a string variable in Domoticz called LastEvent and then note down its idx.  In the below case the idx is 11 (and there’s already a string update in there too, yours will be empty when you first set it up).

variables

Back to the home control HTML:

function updatenotification(idx){

                console.log("checking status of idx "+idx)

                var xmlhttp = new XMLHttpRequest();

                var url = "http://192.168.1.xx:yyyy/json.htm?type=command&param=getuservariable&idx="+idx;

                var textentry

                xmlhttp.onreadystatechange = function() {

                if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {

               var myArr = JSON.parse(xmlhttp.responseText);

                                                textentry = myArr.result[0].Value;

       // myFunction(myArr);




                                document.getElementById("notification").innerHTML = textentry;

                                }

                                }

xmlhttp.open("GET", url, true);

xmlhttp.send();

}

And:

function clearnotification(idx){

                execute('PUT', "http://192.168.1.94:8080/json.htm?type=command&param=updateuservariable&idx="+idx+"&vname=LastEvent&vtype=2&vvalue=%00", '');

}

And in the body of the HTML (so once the text is clicked the variable in Domoticz is reset to null):

Something like this for the CSS: div.notificationpane {     position: fixed;     bottom: 250px;     left: 0px;                 width: 100%;                 font-size: 30pt;                 background-color: #333333;                 text-indent: 25px;                 opacity: 0.6; }

Then in the LUA script, when you want to notify the user about something, you can add a line like this:

commandArray['Variable:LastEvent'] = tostring(os.date("%H") .. ':' .. os.date("%M") .. ' Rear balcony door opened, lights on for 15 minutes.')

This adds the time and the text to the variable, which then almost immediately pops up on the Home Control screen until it is clicked.

Where to store your HTML

When you have created your masterpiece, you can save it in a new folder of your choice below the /domoticz/www/ folder.  You can use something like WinSCP to create a folder and then transfer all your files in one go.

Then, when you usually navigate to 192.168.1.1:8080 to go to the Domoticz home screen, add / then the name of your folder then /index.htm or whatever your home screen address is.

Summary

Due to the jerry-rigged nature of my HTML code I am not going to publish it in its entirety.  I also don’t know the ins and outs of using the flaticons.com icons and publishing them directly.  Hopefully, however, this post will give you some inspiration to write your own home control front end.

Designing a user interface

Introduction

Something that I give a lot of thought regarding my home control set up to is the interface.  I have changed the ‘look and feel’ of the interface tens of times in the last few years (much to Chester’s frustration) and to be honest I don’t think I’ve ever found the ‘holy grail’.  But what makes a good home control interface and how would we recognise one?  I explore these questions with my guide to designing and implementing your own home control interface.

Step 1: Who will be using your interface, and on what?

One of the first things to think about is which devices will be displaying the interface.  In my household the interface is accessed via a fixed large tablet, a computer screen and smaller smartphone screens.  Therefore I may think about designing two types of interface, maybe that look and behave very similarly, one portrait (for holding a smartphone) and one landscape (for the computer screen and the permanent tablet on the wall).  I have designed two types in the past, but using a 100% web interface means that the content can adapt to multiple display types, therefore saving time and resources.

Step 2: Look at what is out there for inspiration (but don’t get too depressed with what you find).

If you search the internet for images of ‘home control interfaces’, you’ll see a few common themes.  I think the main thing that hits me is that although millions of pounds have been spent on perfectly attuning user interfaces in smartphones and computers (for example with the upcoming Windows 10 operating system), home automation interface design seems to have remained the same since its introduction. Without wishing to offend anyone here, I get the distinct impression that the interface is last on the development list.  It seems to me that the hardware takes priority, and then the interface is designed by someone in the existing team who really did not want the job.  I honestly believe this to be the main reason why home automation has not yet become mainstream.  It’s not the hardware: it’s the interface.

If you look hard enough however, you’ll find some clear and well-designed interfaces out there.  The companies that have chosen to invest in design are clearly the market leaders, and therefore can afford to spent time and money developing a look and feel that homeowners can use and appreciate.

What is the one design element that proves to me that the interface has been developed professionally?  Space.  Too many home control screens are jammed full of buttons, sliders and album art.  It’s hard to know where to look, and although this may be something of a visual trophy for the geek community – think how confusing this will look to others in your household.  This leads nicely on to my next step…

Step 3: Think about the order of things.  How will your commands ‘flow’?

As far as I am concerned, there are two main ways to organise your home control screens: by location or by function.  Organising by location means that you gather all your controls for one room onto one screen, so for the living room you’d have all the lights, the dehumidifier, the air conditioner and the TV controls all together.  Conversely, organising by function means that you’d put the controls for all your lights on one screen, heating and cooling on another screen, security on another screen etc.

I have created dozens of versions of both, and I personally prefer the latter.  For me it means that the user starts from the home screen with a definite decision on what they are intending to do.  The interface becomes less of a show-home piece and more of a practical and logical day-to-day assistant.

There is a balance to be achieved between the number of options displayed on the screen at one time and the number of selections required to perform an action.  Instead of scrolling through one giant page of options, or having to reduce button size down to miniscule proportions, it may be better to increase the number of screens and create a ‘flow’ for the user.  So instead of selecting ‘Audio’ and immediately being presented with a screen full of options, it may be better to then ask ‘Where do you want to the audio to play?’ and further ‘What kind of audio?’.

flowexample

Step 4: Write it down!  Plan your screens.

I have wasted literally days of my life not following this simple step.  Plan your screens on paper.  Falling out from this step will be all the buttons you will need for each screen.  Don’t worry about what the screens will look like yet, just think about what options you want to show.

Navigation is a consideration here.  How will your users know where they are?  If you provide ‘breadcrumbs’ on your display (Home > Audio > Living Room) you’ll need to reserve the same area on each screen for that.  To get back to the main screen, will the ‘home’ button be in the same place for each screen?

Step 5: Get ‘the look’ and stick to it.

Decide on a look and feel for your interface.  You may choose to compliment your home décor or to stand out as an ultramodern feature.  Either way, this step requires as much external input as possible.  If you’ve found some designs that you liked from step 2, speak to your users about which ones they prefer, and what they’d like to see.

When you’ve selected a look – stick to it.  By that I mean don’t be tempted to bend your own rules.  Future-proof your design.  For each of your screens ensure that there will be scope to add new devices in the future with ease.  Don’t make your icons or buttons so intricate that you’ll not be able to match them easily.

If each of your buttons contains an icon and some text, make sure the icons are roughly the same size and the text is the same size and font.  Little things like ensuring there is consistent capitalisation across your interface can make a huge difference.  “Living Room Lights On” next to “Living room lights OFF” leaves an unprofessional air.

hc-fsexample

After years of designing interfaces for my own home, I have some pearls of wisdom for you here.  The first is to use perfectly square, flat buttons.  These look modern and will not go out of fashion (if you are reading this in 2020 and flat buttons are out of fashion, then I apologise).  Another reason I prefer perfectly squared icons, is that depending on what interface controller you use, you may be disappointed at how the controller renders curved corners, and your beautiful vision becomes a pixelated mess.  The bigger the button, the better.  Our cleaner visits once a week, and if she needs to use her glasses to see an option, I change the button immediately.  Larger buttons also means that fewer options can fit on each screen, and I am all for that.

Get each button you need from step 4 designed now.  I would recommend you put a selection of these buttons in one mocked-up screen to ensure they are the right size and they look right.  Choose unobtrusive wallpaper or a solid inconspicuous colour for the display.

Step 6: Think about feedback.

The devices your user will be controlling may not be in the same room as the interface, so remember that they may need reassuring that what they have requested has been carried out.  There are a few ways to achieve this.

A button which changes its look depending on the state of the device is one way to go.  This approach is more common in consumer installations and requires constant sensing (called ‘polling’ in home automation terms) to ensure that all interfaces ‘keep up’ with the device’s status.  Don’t forget that someone could switch on a device from their smartphone, so then the corresponding button will need to update on all screens to reflect this.   One such controller that is capable of keeping track of devices is Openremote – more on this in step 8.

Another idea is that a pop-up box could appear to confirm your selection, for example confirming “Fan in Living Room has been switched on.”

The panel itself could speak to you.  I have elected to use this as my primary feedback method in my latest interface design.  The display is not obstructed by pop-ups, and the user can walk away from the panel and will still hear the announcement.

Feedback should also be provided automatically, for example when a security alert is activated or when a user needs to do something that cannot be completed automatically (such as a reminder to close a door).  Users could be anywhere when this feedback needs to be given, so thinking beyond the physical interface of a screen, what other devices could be used?  In our home, we have Sonos speakers in every room, so urgent feedback can be given via spoken word through them.  For smoke alarms triggered by Nest, the Hue lights in each room will flash momentarily, alerting occupants that action needs to be taken.  I also use Notify My Android (NMA) via Domoticz to keep me informed via short ‘texts’.

Step 7: Remember the ‘automation’ in ‘home automation’.

Advanced controls that only you will use can be tucked away at the ‘back’ of your interface.  Settings and variables can be as hidden away as you like, followed by screens for ‘fine tuning’ settings for devices that other users will probably never use.  It’s sometimes useful to control one individual lamp in the living room, to set brightness, colour and saturation until it’s just perfect.  But how often do you actually do that? It would be wiser to initially show a list of pre-set scenes to be selected from, and then have the option to move to a different screen to change individual lights if required.

Just because your system can do something, don’t give it pride of place and assume that every user will want to do it.  I have been guilty of re-building my interface around that a newly-purchased piece of hardware, just to show it off its new features.  This is a waste of time and ultimately confuses other users.

Step 8: Where to host your interface?

This depends on your home control set up.  I have used a few different downloadable options here, one being Openremote which is a program that runs on your computer or Raspberry Pi and ‘serves’ up information to panels around the home.  By polling your home control hardware at intervals you can select, Openremote can keep track of your devices.

Domoticz also has its own user interface that can be used on a plethora of screen sizes, however although I absolutely adore Domoticz and use it as the basis of all my home control activities, I think that the user interface is – dare I say it – where there has been less development.  By no means do I think this is a let-down however, as I would not have been able to thoroughly automate our home without Domoticz.

Another route – which is the one I have taken – is to design a web-based interface from scratch.  As Domoticz has a ‘www’ folder that can serve up any html document in that folder, I don’t need a separate web server.  I just upload all the html screens into a sub-folder of the Domoticz/www folder and then navigate to that page from any panel in the home network.  Here’s the secret.  Controlling Domoticz devices from a web-page is really easy.  You just have to navigate to a specific address and Domoticz will turn on a device.  You can get more information about that on the Domoticz API/JSON URLs page.

The only thing I had to work out for myself is – when I navigate to a URL asking Domoticz to turn on a device, the screen shows a ‘result page’.  The user would have to then click the browser’s ‘back’ button to get away from this result page and then back to the interface.  To solve this, I use a very small frame at the bottom of the screen.  When the user clicks a selection in the main screen, the link is followed in the small frame at the bottom, so as not to disrupt the main window.

Step 9: Testing and Rollout

Before announcing to your loved ones that you have created a masterpiece, I would encourage you to test every function on every page of your home control interface, especially those functions which will affect many devices in the home.  As an example, choose a time when no-one else is around and activate your ‘leave’ function and see if it does actually behave in the way you expect.  It’s too late when your family is ready to leave and you proudly press the button, only for the whole system to crash.  Trust in the system as a whole can also be eroded in this way.

Personally, I have the flat to myself every other Friday, only having to navigate around our cat.  I use this opportunity to test new functions I have added.  Our neighbours must think the flat goes crazy every two weeks, but it’s worth it.

Once everything is ready, you can add links to your interface on every device that will access it.  You can now go back to step one to restart the process.  I say this with confidence because as I stated at the start of this article, there is no ‘perfect’ user interface and there are always improvements to be made.  I truly believe that I will always be in pursuit of user interface perfection.

Homemade disco using Sonos and Hue

I’ve used Hue Disco and other apps to sync up my Philips Hue lights with my music, but what I really wanted was something that I could start during a party, and just leave to get on with it.  I wanted the lights to change reasonably often, but I wanted them to respond to the type of music being played.

This program is written in Python 3 and uses the Echonest database to get the energy and danceability, as well as the tempo of the currently plying track from the Sonos player (obtained by this great program: node-sonos-http-api.  The program then selects a brightness and saturation range and randomly changes all the hue lights you specify.  The lights change in a frequency relating to the tempo of the track playing.

The program must be started AFTER you’ve started playing a track on the Sonos, and then continuously runs until you pause the Sonos player, and then raises the lights to a medium white light, and then the program quits.

I’ve got this program running on my Raspberry Pi, and am really pleased with it, obviously more tinkering can be done, but I’ve not seen any similar apps that do this.  I have a Domoticz ‘switch’ that can start running this program, and then to end the program, you just pause playback.  But you could just run the program from the command line each time you want to have a disco!

Best used when you have a night of partying, with a large random playlist going.  Don’t forget that as long as the player you specify in the variables is part of the playing ‘group’, you can play music to different rooms and include those Hue bulbs into the list too!

You can download the code here: Hue-BR-Analyse

Here’s the code:

import urllib
from urllib import request 
from random import randint 
import base64,requests,json,time,datetime

"""
Hue Sonos Analyser by Harry Plant 2015

This program changes the hue, brightness and tempo of Philips Hue lights depending on the energy and danceability of the track,
as reported by Echonest.

Requirements:

*Philips Hue bridge and at least one Hue Light (http://www2.meethue.com/en-gb/)
*At least one Sonos player (http://www.sonos.com/)
*Python 3 (I have it installed on my Raspberry Pi)
*Echonest api key (obtainable at https://developer.echonest.com/ - click 'Get an API Key')
*node-sonos-http-api (https://github.com/jishi/node-sonos-http-api)

change the following variables to match your setup:

zone is the name of the sonos player you will be listening to
my_list is an array of the hue lights you want to control
sonosinfo is the address of the node-sonos-http-api server
hue_bridge is the address and key that you use to control the hue
echonest_apikey is the key you obtained from echonest

I have used this program with success (great parties) on multiple occasions but do not accept 
responsibility for any loss or damage caused by using this code.

"""

zone = 'Kitchen'
my_list = ['3', '4', '5', '7', '15', '17', '19', '20']
sonosinfo = 'http://192.168.1.100:5005/'
hue_bridge = 'http://192.168.1.101/api/newdeveloper/'
echonest_apikey = '02MKNPMXXXXXXXXXX'

"""
MAIN PROGRAM BELOW, YOU DON'T NEED TO CHANGE ANYTHING FROM HERE
"""

currenttrack = 'No track'
min_bri = 100
max_bri = 100
secs_waited = 0
secs_to_wait = 10
sat = 100
tt = 10
trackfound = 1
std_assumptions = 0

for r in range(1,999999):
 
 secs_waited = secs_waited + 1

 req = request.urlopen(sonosinfo + zone + '/state')
 encoding = req.headers.get_content_charset()
 obj = json.loads(req.read().decode(encoding))
 """
 print(json.dumps(obj,indent=4))
 """
 if obj['zoneState'] == 'PAUSED_PLAYBACK' or obj['zoneState'] == 'STOPPED':
 print('Player has stopped playing. Returning lights to white and exiting program.')
 my_list_len = len(my_list)
 for i in range(0,my_list_len):
 hue = 10000
 bri = 200
 sat = 50
 tt = 50
 payload = '{"on":true,"sat":' + str(sat) + ',"bri":' + str(bri) + ',"hue":' + str(hue) + ',"transitiontime":' +str(tt) + '}'
 """
 print(payload)
 """
 r = requests.put(hue_bridge + "lights/" + my_list[i] + "/state",data=payload)

 req = request.urlopen('http://192.168.1.94:8080/json.htm?type=command&param=switchlight&idx=252&switchcmd=Off')
 exit()

 track = obj['currentTrack']['title']
 artist = obj['currentTrack']['artist'] 
 elapsedtime = obj['elapsedTime']
 playing = obj['playerState']

 if track == currenttrack:
 """
 print('Waited ' + str(secs_waited) + ' out of ' + str(secs_to_wait))
 """
 if track != currenttrack:
 std_assumptions = 0
 print('This is a new track!')
 print('Now playing : ' + track + ' by '+ artist + ".")
 requeststring = 'http://developer.echonest.com/api/v4/song/search?api_key=' + echonest_apikey + '&format=json&artist=' + urllib.parse.quote(artist) + '&title=' + urllib.parse.quote(track)
 """
 print(requeststring)
 """
 req = request.urlopen(requeststring)
 encoding = req.headers.get_content_charset()
 obj = json.loads(req.read().decode(encoding))
 """
 print(obj)
 """
 if not obj['response']['songs']:
 std_assumptions = 1 
 print('Song not in database - using standard assumptions')
 
 if std_assumptions == 0:
 songid = obj['response']['songs'][0]['id']

 """
 print(songid)
 """

 requeststring = 'http://developer.echonest.com/api/v4/song/profile?api_key=' + echonest_apikey + '&id=' + songid + '&bucket=audio_summary'
 """
 print(requeststring)
 """
 req = request.urlopen(requeststring)
 encoding = req.headers.get_content_charset()
 obj = json.loads(req.read().decode(encoding))
 if not obj['response']['songs'][0]['audio_summary']:
 print('Although song was in database, there is no energy and danceability data. Using standard assumptions for this track.')
 std_assumptions = 1

 if std_assumptions == 0:

 print(obj)
 
 dancepercent = int((obj['response']['songs'][0]['audio_summary']['danceability'])*100)
 energypercent = int((obj['response']['songs'][0]['audio_summary']['energy'])*100)
 pulserate = obj['response']['songs'][0]['audio_summary']['tempo']
 print('Danceability is '+str(dancepercent) + '% and energy is ' + str(energypercent) + '% with a tempo of '+str(pulserate) +'bpm.')
 currenttrack = track

 if std_assumptions == 1:
 
 dancepercent = 50
 energypercent = 50
 pulserate = 100
 print('Danceability is '+str(dancepercent) + '% and energy is ' + str(energypercent) + '% with a tempo of '+str(pulserate) +'bpm.')
 currenttrack = track

 if pulserate &lt; 100:
 secs_to_wait = 20
 tt = 20
 
 if pulserate &gt; 99 and pulserate &lt; 120:
 secs_to_wait = 6
 tt = 10
 
 if pulserate &gt; 119 and pulserate &lt; 160:
 secs_to_wait = 4
 tt = 7

 if pulserate &gt; 159:
 secs_to_wait = 2
 tt = 5
 """
 
 secs_to_wait = int((pulserate/60)*2)
 """
 if energypercent &lt; 20:
 sat = 50
 
 if energypercent &gt; 19 and energypercent &lt; 40:
 sat = 100

 if energypercent &gt; 39 and energypercent &lt; 60:
 sat = 120

 if energypercent &gt; 59 and energypercent &lt; 80:
 sat = 170 

 if energypercent &gt; 59 and energypercent &lt; 80:
 sat = 200

 if energypercent &gt; 79:
 sat = 255

 if dancepercent &lt; 20:
 max_bri = 60
 min_bri = 40
 
 if dancepercent &gt; 19 and dancepercent &lt; 40:
 max_bri = 100
 min_bri = 80

 if dancepercent &gt; 39 and dancepercent &lt; 60:
 max_bri = 150
 min_bri = 80

 if dancepercent &gt; 59 and dancepercent &lt; 80:
 max_bri = 200
 min_bri = 50 

 if dancepercent &gt; 59 and dancepercent &lt; 80:
 max_bri = 255
 min_bri = 50

 if dancepercent &gt; 79:
 max_bri = 255
 min_bri = 10

 print('Changing hue lights now.')
 my_list_len = len(my_list)
 for i in range(0,my_list_len):
 hue = randint(0,65000)
 bri = randint(min_bri,max_bri)
 payload = '{"on":true,"sat":' + str(sat) + ',"bri":' + str(bri) + ',"hue":' + str(hue) + ',"transitiontime":' +str(tt) + '}'
 """
 print(payload)
 """
 r = requests.put(hue_bridge + "lights/" + my_list[i] + "/state",data=payload)
 secs_waited = 0
 
 if secs_waited &gt;= secs_to_wait:
 print('Changing hue lights now.')
 my_list_len = len(my_list)
 for i in range(0,my_list_len):
 hue = randint(0,65000)
 bri = randint(min_bri,max_bri)
 payload = '{"on":true,"sat":' + str(sat) + ',"bri":' + str(bri) + ',"hue":' + str(hue) + ',"transitiontime":' +str(tt) + '}'
 """
 print(payload)
 """
 r = requests.put(hue_bridge + "lights/" + my_list[i] + "/state",data=payload)
 secs_waited = 0

 time.sleep(1)
 



Why Domoticz is a no-brainer

When I first decided to connect all my devices centrally, I looked around the internet for a solution.  There are many options to choose from, almost too many.  Some you pay for, others you don’t.  Some come with a ‘hub’ that you connect to your router and the hub sends out radio and network commands to your devices.  Others rely on external hardware that you purchase separately.  There are some really good paid services out there, but I wanted to try it on the cheap.  That’s when I found Domoticz.

rfxcom433Domoticz is a small-footprint program that you can install on pretty much any computer, but if you want to connect to your devices via radio waves, to send on/off/dim signals, and receive temperature or movement sensor data, you’ll need to buy a separate piece of hardware.  I bought an rfxcom rfxtrx433.  It’s a matchbox-sized box with a stubby aerial sticking out of the top.  You connect it to your computer via USB.

At first we used a Windows PC that was left on all day every day as our ‘home server’ (one of the terms used to describe the ‘brain’ of the home control system).  It worked reasonably well, and Domoticz almost never crashed.  The thing was, we were probably using far more power than we needed to.  So I looked at using the Raspberry Pi, a small and cheap computer.From the Domoticz website you can download an image to transfer to the Raspberry Pi via an SD card.  This image contains all you need to start using Domoticz straight away, right from the second you plug in the Raspberry Pi.  This makes life sooo much easier.  You can even plug in the rfxtrx433 into one of the Pi’s USB ports and it will start working straight away.  It’s that simple.

raspi

domoticz-screenshotThen, the exciting part.  When you type in the Pi’s network address (eg 192.168.1.34) and add on the port number (the default of the Pi when Domoticz is first installed is 8080), you’ll see an unpopulated screen with no devices.  Adding devices is done in quite an innovative way: devices that send information routinely, like temperature sensors for example, appear in the ‘unused devices’ tab and with a few clicks they can be added to your dashboard.  Adding other hardware that uses network commands, such as the Philips Hue Bridge, is done via the Hardware Setup page.

domoticz-blockyAfter some or all of your devices are added, you can create decisions that Domoticz can make on your behalf.  You can make these as sophisticated or easy as you want: and there are two ways of programming these decisions.  The easy way is using ‘blocky’.  This is a drag-and-drop way to quickly build up a program.  A great feature of blocky is that the blocks you drag around will only fit together if they can interact in the way you’re trying to get them to interact.  The more advanced way to tell Domoticz what to do is to edit and create mini-programs that run either when a device changes state, or every minute.

This is a really quick summary of what Domoticz can do, I would advise that you visit the website and forum, grab a Raspberry Pi, install the Domoticz image onto an SD card and have a go yourself.  You’ll be amazed at what a free piece of software can achieve to realise your home control dreams.

I’ll be posting a lot more about Domoticz.  After all, pretty much everything in our flat is controlled by it one way or another.

The bits and bobs what we have

Here’s a starter for 10.  An initial list of the items I’ll be talking about on this blog.  This is not exhaustive, even though I feel exhausted writing this down!

We’ve collected these various disparate systems over the past few years.  It’s my job to make them speak to each other.  Like some kind of flamboyant interpreter at an important European parliament discussion.

Sensors

3 x Oregon Scientific thermo/hygro sensors to monitor temperature and humidity throughout the flat, and outside on one of the balconies.  They send messages to the computer, roughly once a minute, giving up to date temperature and humidity data.  They also include in the message, a request for batteries to be changed.  That’s polite, isn’t it?

Available from: John Lewis

4 x LightwaveRF door sensors to determine if doors in the flat are open or closed – these send one signal when the contacts are moved apart – door open, and one signal when the contacts come back together – door closed. Open, closed.

Website: LightwaveRF

3 x LightwaveRF PIR sensors. These send out a signal when they detect movement, and then another signal when there is no longer any movement.  You can set how long the sensors wait before checking if there is no movement by selecting a time period using a switch on the back.  This can be from 5 seconds, to 10 minutes.  To literally never.

Website: LightwaveRF

Plugs

10+ LightwaveRF plug-in units.  These units do one of two things, depending on which ones you buy.  One type is ON/OFF which means that a signal is received and then ‘ping’, the device you’ve plugged in comes on.  Used for things like TVs or Microwaves.  The other type is DIMMER which means that not only does the device plugged in switches on and off when a command is received, but also that the device can dim from between 0% and 100% brightness.  Or it should go down to 0%, but doesn’t.  More on that in another post.  You really should not use dimmer switches to control the power level of a Microwave.

The plug-in units are really, really useful, and they were especially so when we rented, because they can be removed from a plug socket when you move, instead of frantically tugging out every electrical outlet in the last 5 minutes of your tenancy.

Website: LightwaveRF

1 x Belkin WeMo plug-in unit. I bought this from Maplin as an impulse buy, as it links with another recent purchase, my Ivee Sleek.  I’ll be doing interesting things with both of these items in the coming weeks.  Mainly when Ivee becomes more than a fancy alarm clock that chirps up randomly whilst you’re watching television.

Lighting

20140706_163537We have 17 Philips Hue bulbs and lightstrips in the flat.  These are what really shook up the home control thang, and made our lives better, more relaxing, and simpler.  Quite simply these are the best home control things I have bought.  Not only do they look good, they also behave 99% of the time, and they can be controlled from practically any computer program.  This is the holy trinity as far as home control goes.  Honestly, I can’t say too many good things about Philips Hue.  More gushing to come.

Where it is physically impossible to get Philips Hue bulbs, we have LEDs.  So the flat is very nearly 100% LED lit.  There are three exceptions in the kitchen where LEDs cannot be used at the moment.  Damn you, kitchen!

We also have a few LED downlighters that can be controlled by the LightwaveRF signals.  These are cute and can be used under shelves to create mood lighting.  The only downside is that they go through AAA batteries like our cat goes through litter.

Heating and Environmental Controls

We have an app-controlled heating system called Hive.  You can control the heating from wherever you can get online with your phone (which means everywhere).

Any flat can get problems with high humidity, so we have a dehumidifier.  It’s quite a cheap one, but I’ve made it very intelligent.  It only comes on when necessary, and only when we’re home.  I’ll blog about that some time.

We also have an air purifier which is also similarly automatically controlled.  The air purifier has a lovely colour changing effect and belches out lavender fragrance.  This is nothing to do with home control, but is very much to do with my other hobby, which is being incredibly and breathtakingly gay.

Controllers

To ensure we can ‘talk’ to the flat, we need to have some kind of way to speak to it.  I can go in and change the code in my control programs, but that isn’t very easy to do for many users, so a pretty looking button or a screen is a good way of doing this.

Many of these controllers are supposed to be ‘paired’ with a specific device to switch them on or off.  I shun this concept as amateurish.  I ‘pair’ the controller with the computer, so that the computer decides what to do when it receives a signal from the controller.  More on this later in the blog.

pad4 x LightwaveRF wall mood switches are used throughout the flat to control lights and moods in various rooms.  These are useful because (a) you can stick them anywhere – so you can place them where a light switch would normally be, and (b) because they are always there.  It’s one thing having a smartphone controlled flat, but what if you don’t have your smartphone with you and all you want to do is turn on a light!  Cue this.

The mood switches have 6 ‘pads’ to press.  Two large ones, marked 1 and 0, and four smaller pads, marked with symbols ‘-‘, ‘–‘, ‘—‘ and a standby icon.

The mood switches are good, and look great, but they have very generic markings on them.  I’ve overlaid a printed design on top of them so that users can tell what will happen when a button is pressed.  The switches also have a delightful blue LED which charmingly illuminates to confirm that a signal is being sent.

4 x LightwaveRF handset controllers.  These are hand-held controllers, comprising of 10 buttons and a four-way switch.  8 of the buttons send out unique signals which change depending on the position of the switch, therefore there are 8 x 4 = 32 unique signals.  The other two buttons send out the same signals regardless of the position of the 4-way switch.  Therefore there are 32 + 2 = 34 different signals that can be sent out from these controllers.  I am no Carol Vorderman but Excel tells me that I’m correct.

Sony Tablet S in charging dock.  This is a pretty standard android tablet which is always standing in its charging dock.  It makes controlling the flat easy, as long as you’re in the living room and you have fingers.  The above controllers are used elsewhere.

Ivee

Ivee was so exciting before I bought her.  She literally took months to enter my life, I was counting the days from when I ordered her from Maplin.  Ivee is supposed to be an always-listening voice-controller for your connected devices.  As it is, I can try and ask her “Hello Ivee.  Turn on the bathroom lights.” and she’ll respond with the phrase “The time in Handsome Eddy, New York, is 12:50am”.

I’m sure she’ll get smarter.

Smartphones.  This one is a no-brainer.  It’s also very impressive to show people.  “Hey, look at this.  I can switch on the fan in my living room from here in the office”.  And then they usually say something like “I’ve left something on the photocopier” and leave.  Quickly.

Audio

20150111_215333We have Sonos everywhere.  Literally everywhere.  As the flat is quite small, we have 3 PLAY:1 speakers, a CONNECT (for broadcast to the bathroom), and a PLAYBAR which is a stereo and a TV soundbar all rolled up in one very “I may look like a draught excluder but actually I cost twice as much as the TV I’m under” package.

The Sony tablet in the living room, and our smartphones can control the Sonos, as well as any of the handsets affixed throughout the flat.

The Brain

The brain of the whole crazy outfit is a teeeeeny tiny little computer called a Raspberry Pi.  This computer, the size of a pack of cards, is useful because it doesn’t use a lot of power (so you can leave it on all the time) and it’s reliable.  I use a program called Domoticz to run everything.  A heck of a lot more on this particular set up later.

The interesting thing about home control is that it can be as simple or as complex as you want to make it.  You can decide ‘I want a lamp in my living room to come on when it gets dark outside’ or ‘I want the flat to email me when the humidity gets stupidly high in there’, or ‘sound an alarm and email me when we’re supposedly out of the flat and a door is opened or a movement sensor is activated’.

With a bit of playing around, the world is your oyster.  Or your home is your servant.