Ghost in the Shell – Fixing your MQTT issues

With the recent push in Home-Assistant to become more point and click and less yaml coding, I thought I would take a look and try and clean up some of my yaml and move over to the integration method. For those of you who have been using Home Assistant for the last 6months to a year, you’ll probably already be using the integrations method, but for those of you who (like myself) have been using this for 4-5 years+ you’ll understand where I’m coming from. One of the things that has sat quietly gnawing away at the back of my mind has been my MQTT set up. I am currently using several MQTT brokers. I use the Mosquitto broker which is built into Home Assistant Core (formerly Hassio), and I am also using CloudMQTT.  I mainly use the local broker (Mosquitto) for all things HA, relying on CloudMQTT when I am using Owntracks for location tracking andf experimenting with other installations (parents house). I felt it preferable to link CloudMQTT to my local node-red installation for passing messages rather than punching an additional hole in my firewall and giving internet access to my local Mosquitto installation.

I am currently running Tasmota on all my flashable devices. I use several Teckin SP22s (under bed lighting, bedside lighting), a Blitzwolf SHP5 (office lamp and headphones / ipad charger), two Sonoff basics (one works as a test mule, the other is for external holiday lights). I control/administer these devices using the TasmoAdmin plugin in Home Assistant Core. Although if truth be told, i usually like to update the firmware on all the devices manually. First flashing the latest minimal build and then flashing the full firmware build directly after it. For ages I was stuck on Tasmota 6.X. You know the saying, if it’s not broke, don’t try and fix it. The only issue was I began to have random ghosting with my installation about a month or so ago. Something I had updated had began to play havoc with my MQTT devices. This occured around the update of HA to 0.109ish. I can’t remember exactly. These were some of the symptoms i was getting:

• Random lights turning on in the evening. There’s nothing more annoying than having your bedside lamp switch on at 4am for no reason at all. Maybe having your partner’s light switch on is worse. I was having my girlfriend moaning at me as well as being woken up.
• Every time I restarted my HA server lights would automatically switch on. No matter what I tried i couldn’t stop it.
• Some of the switches wouldn’t work as reliably and instantly as they had done for ages (a year+) previously.

I tried to fault find the MQTT issue. I didn’t have retain set up in node-red, nor was it in the yaml (default = false). As a last resort, I decided to upgrade the devices to the latest version of Tasmota, thinking that my current firmwares were quite possibly too dated now versus the latest HA and there were incompatibilities.

Using Tasmoadmin, I opened each device and as I mentioned above, uploaded the minimal firmware for a 8.X install, followed directly by the full firmware build 8.3.1. I know the installation instructions tell you to go 6.X, 7.X and then to 8.X, but I started with the sonoff and just skipped straight to 8.X which worked without issue. I did this with the sonoff basic first because if there was an issue, that was the only device I could easily open to get at the pins to manually reflash. After some initial investigation, everything was fine and I upgraded the rest of the devices to the latest firmware. Everything was showing correctly and operating as it should. I thought that would be the end of the ghosting I was seeing in MQTT and decided I’d leave things there. Satisfied that the upgrade had gone through without a hitch and i was now running the latest and greatest.

For the next day or so everything was working as it should. No midnight interuptions, no random behaviour, although everytime I restarted Home Assistant the same lamp kept coming on, much to my annoyance. It was at this point that I decided to try something new. I hashed out the mqtt configuration in switches.yaml and decided to make the plugs discoverable. I also wanted to manually make sure and check that all of the devices had various retain and power options set up correctly, so these were the steps I took. I opened Tasmoadmin tab in Home Assistant and clicked on to one of my tasmota devices. Again, I selected the sonoff basic as the test mule.

Once I brought up the device’s webpage, i entered the console menu.

In order to set the device to be discoverable in Home Assistant’s integrations page, I ran the following command:

SetOption19 1 (you can also just type SO19 1)

This option set the discoverability to On.

Next I wanted to set up a few more options manually to make sure that they were as they should be. I wanted to make sure that in the event of a connection issue, the switch didn’t retain a message. I.e. You switched it on. Then it goes off automatically for what ever reason. Then there’s a connection error, when it reconnects, it still sees that command to switch it on and voila, a ghost command!

switchretain off

Next, following the same lines, I wanted to make sure that we didn’t exhibit similar behaviour with the button or power. I wanted to manually override any existing settings and then clear them again, so I entered the following commands in short succession.

buttonretain on
buttonretain off
powerretain on

I focused on doing this on the tasmota page directly, before messing around with Home Assistant. When I went to the MQTT integration in HA, sure enough there was my sonoff basic waiting to be adopted. I clicked to add it to lovelace and there it was. I made sure to keep the name of it exactly as it was before so that I didn’t need to change anything in lovelace or Node-red (where all my automations reside). I repeated this exercise with all my other Tasmota devices. Within 10 minutes they were all completely integrated. If anything they were integrated better than ever before. Previously I had just added the switch manually and had been using it as is. Now not only did I have power on and off, but a whole host of power monitoring options available instantly. I didn’t expect this and was impressed with the ease in which it appeared. I monitored it for a few days and again, everything was working perfectly. No more ghosting, and just regular automations working as they should. Well almost.

 

The midnight light ups had stopped already, but i still had this one annoying behaviour everytime I restarted Home Assistant. The bedside lights kept switching on. This in itself was annoying, but something I could live with, if it wasn’t for the fact my girlfriend kept teasing me that I didn’t know what I was doing after previously announcing everything was sorted (for the second time!). A bit of googling later, and I found what I think was the problem. Apparently this one device seems to have retained a command for power on in it’s memory. I don’t know if this is usual, or an anomaly. After all, none of the other devices showed this behaviour. Anyways, in order to fix it, i needed to manually wipe the command from its memory. Enter Tasmota Manager!

I downloaded and installed Tasmota Device Manager. I wasn’t quite sure what to expect. It seemed quite intuitive. I entered my broker’s options (mosquitto- host, port, user, pw etc). Within seconds, all my devices listed.

I selected the offending switch, and with the click of a right mouse button brought up the options.

Tasmota Device Manager Menu

Clear retained

Clear backlog

I basically did both of these in succession. I restarted Home Assistant and voila… the bed light stayed off. I then proceeded to repeat the same commands on each device in the list. Two weeks later, not a single misfire, ghost command. I’ve restarted HA more times than I can remember and not once did that light come on! Not only that, but this Tasmota Device Manager allows me to interact directly with the devices without needing to go into their webpages individually. I’m a huge fan. Hopefully if you’ve been suffering from similar issues with MQTT this can help you too! Pop a note in the comments if this helped you, and if you’re interested in supporting this blog in any way shape or form, consider having a look at my Smarthome Book.

Cheers!