For those of you with Youtube channels it’s natural to want to know how you’re getting on to be able to check up on statistics like subscribers and total numbers of viewers etc. In this tutorial, I’m going to show you how you can scrape the data and bring it into Home-Assistant but then also how we can use Node-red to make it somewhat more useable. After all, how many of you are glued to your Home-Assistant UIs! This method in principle should work with any website that doesn’t need authentication i.e. a login.
In this instance we’re going to use Socialblade.com. Social blade can be thought of as an amalgamator of statistics for various platforms. At the time of writing, it handles twitch, twitter, youtube and instagram among others. Firstly we are going to use the scrape sensor in home-assistant to effectively mine the data we require on socialblade.com and create a sensor out of it.
Go to socialblade and find the channel you’re looking for. In this example, I will use my own here. I’m not particularly big into video and prefer writing as you can tell, so don’t judge me 🙂
The link to my channel is here. Next we want to see the top line stats just under the cover picture. You can see the following:
What we want to do is hover over the statistic we’re interested in. In my case it’s subscribers and right click the value. Choose inspect in your browser’s menu (hopefully you’ve already seen my privacy video and you’re all using Ungoogled Chromium :)). You should see something similar to below. What we want here is the id=”xxxxx” part for the scrape sensor.
Let’s repeat for the number of views:
Let’s move over to Home-Assistant now and open up your sensors.yaml file (if you’re not using a separate yaml file for sensors, then this bit needs to go under the sensors category in your configuration.yaml file). We need to paste in the following code (I’m using a screenshot because it clearly shows the indentations):
You can use whatever names you like for the sensor. For “resource” you will need to paste in the socialblade link for your own channel or whatever channel you’re interested in. You can see under “select” where we paste in the value we’re interested in displaying. We use the “headers” section so that social blade thinks we’re actually browsing the website as a person and not automatically scraping (as bots can get blocked). Once all done, save the file and restart Home-Assistant. You should now have two new sensors (sensor.views and sensor.subscribers). You can add these to lovelace as you would normally.
So far so good? So we have the data in Home-Assistant, but now it’s time to do something with it. Enter Node-red! I’m going to be building on some earlier topics I’ve discussed previously so if you’re relatively new to Node-red I’d suggest you have a flick back through some of my prior posts including the 3 part beginner’s guide I wrote a few months back. Part 1 is here. I’ll link other relevant posts as we go.
My goals here are as follows:
- Recieve a notification on my phone via telegram (I have a channel with my home) everytime the subscriber count changes (hopefully increases!).
- I want to be able to ask the home via telegram what are my current youtube stats and have it respond.
- I want to be able to ask the home via voice what are my current youtube stats and be notified accordingly.
Let’s get the data into a debug window first so we can play with the flows. I will use a check state node and manual inject initially so that we can manually trigger things as opposed to actually waiting and then testing everytime the subscriber count naturally increases. We’d be here all month otherwise. I’ll deal with just subscribers initially as it’s easier. For those of you who haven’t set up Telegram yet in Node-red, see here for the guide. If you’d sooner use Signal, then see here. I’ll stick with Telegram.
We inject a simple state check node, set the msg.payload accordingly, and output it to the telegram sender node. (note i’ve added the state_changed node at the same time, because ultimately, as this sensor changes, it will send the flow automatically which is what we want.
By now you should have a telegram message in your chatroom telling you “YouTube Subscriber Count: XX”. If you haven’t check back over your flow and see my previous telegram tutorial to check your chatroom is correctly configured. Next we want to be able to interact with the chatroom and ask accordingly.
We add a telegram receiver node and configure it. Now when we type /yt into the chatroom, the current number of subscribers is returned.
Next I want to add the number of views to the mix. This involves playing with a second msg.payload so now we need to use msg.topic to be able to differentiate and combine the two. Consider the following flow:
What we are doing, in essence is reading the current state of the sensors (manually triggered), changing the msg.topic of the outcome to subs and views, and then joining both payloads accordingly into one msg.payload. We then format that for telegram and send it on.
The resulting message in Telegram is:
Now we can hook the /yt receiver node (trigger) to the start, aswell as the sensor.subscribers state change node and we’re practically done. Note I’m only concerned with subscribers firing off the flow, not the sensor.views. I’m sure it would get pretty tiresome if everytime someone watched a video on the channel you got alerted. I’ve used a link node (in/out) to neaten things up. These are very easy to set up. The final flow should look something like this:
To go a step further, I set up “stats” on both Google Home and Alexa using the third party bridge I’ve been using. Links to how to set up Alexa and Googlehome entites are here. The Flow now looks like this, and every time I ask either of them to turn on stats, it fires the telegram flow.
You can see why I wanted to use a link node. Imagine if you were looking at the other platforms instead and rather than have just views and subs, maybe you wanted (twitter) following, followers, likes, tweets etc. You can see things would get out of hand quite quickly.Without the link node, it would look like this:
At this point, I was happy with what I’d set up. I could go a step further and actually have the voice assistant read out the stats to me, but frankly speaking, hearing Alexa or Google Home speaking to me for that long as they listed both subscribers and view numbers would irritate, so I stopped there. That’s the beauty of Node-red, you can take it as far as you want to go!
If your interested in sharing your own solutions, tips and tricks with like minded people perhaps you’d consider joining our facebook group. The aim of this group will hopefully be more show and tell rather than support, but that’s not to say we can’t lend a helping hand!
https://www.facebook.com/groups/386238285944105
Thanks for letting me know. Haven’t upgraded to the latest and greatest yet, but do you think it matters that I am not suing the socialblade integration per se but the “naked” scrape sensor? When I get round to updating HA I’ll revert to see if the scrape works, but in that link there, it refers to
sensor:
– platform: socialblade
where as I am going outright for – platform: scrape
?
Just FYI – SocialBlade has been removed from Home Assistant – due to it still scraping web page for data.https://github.com/home-assistant/core/pull/48677
FYI I updated to the latest version of home-assistant this morning (2021.4) and the scrape sensor solution I wrote about is still working, so for now we’re good 🙂
my ip address just got blocked using this method.
How long had it been running for? Mine is still working ok? Am just trying to understand has something changed and I should expect mine to stop working or if there’s something specific to your usage case.
i only had it running for about 2 days
I think you must have been super unlucky or your “scrape rate” was too aggressive. I’m still using it now without issue.