Have Access From Anywhere - Home Automation - Part II

Access Your Stronghold

Part II - Lets automate everything!

Having automated, smart home is good, having it available anywhere in the world is better. Exposing your home through the internet has two major advantages. First one, it enables you to check what’s going on in your home whenever you wish to. The second one, is an ability to receive web-hooks from systems in the cloud, like IFTTT, making our automation easier to write and maintain.

To get the Controller, your hub controlling everything, available on the internet one has two options. You can reconfigure the router a little and use static IP address or a Dynamic DNS service. Alternatively, if you are limited by your internet provider, you can use a tunneling service like ngrok.

Access your Raspberry Pi over the internet

If you can, the fastest option to make your Controller available over the internet, is to use your IP address and reconfiguring your router. To do so, you need to enable port forwarding on your router of both 21 and 443 to the Raspberry. If you don’t have a static IP, like most users, you can use a free Dynamic DNS service like DuckDNS. The service provides a DNS record in their own domain (*.duckdns.org), with an ability to update it easily, using cron for instance. To have your own domain name, using this setup, simply add CNAME record pointing to the selected subdomain.


If you can’t do that for some reason (for example your internet provider is blocking incoming connections) you are left with the second option. To expose the Raspberry in a network provider independent way you need to use a tunneling service, like ngrok for example.

In this article I will cover both options, feel free to choose the one that suits you best.

Using Router Using Ngrok

Exposing the Raspberry through the Router

Majority of routers do support port forwarding, but that’s not the only way we can force router to expose the Raspberry. An alternative is to set up a DMZ zone, which allows you to forward whole traffic to the Controller. Doing so it’s certainly easier, but exposes all ports. To increase security we will start by installing and configuring a firewall. The fastest way to do so is to use ufw - an Uncomplicated Firewall. Let’s log in to the Raspberry through ssh we configured in the part I and execute these commands, installing the firewall and exposing http(s) and ssh.

pi@raspberrypi:~ $ sudo apt-get install ufw -y
pi@raspberrypi:~ $ sudo ufw allow ssh
pi@raspberrypi:~ $ sudo ufw allow http
pi@raspberrypi:~ $ sudo ufw allow https
pi@raspberrypi:~ $ sudo ufw enable
pi@raspberrypi:~ $ sudo ufw status verbose

Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW IN    Anywhere                  
80/tcp                     ALLOW IN    Anywhere                  
443/tcp                    ALLOW IN    Anywhere                  
22/tcp (v6)                ALLOW IN    Anywhere (v6)             
80/tcp (v6)                ALLOW IN    Anywhere (v6)             
443/tcp (v6)               ALLOW IN    Anywhere (v6)             

Then, login to the DuckDNS service and reserve an address. If you have your own domain address, you can setup a CNAME record pointing to the one in duckdns. Let’s assume that your address is home.example.com.

If you can, consider donating the DuckDNS service, to help the guys cover their infrastructure bills.

The only thing left to do on the device, is to set up a cron that will be keeping our entry with IP in DuckDNS up to date.

pi@raspberrypi:~ $ sudo su
root@raspberrypi:~ $ cat <<EOF > /usr/local/bin/duckdns
echo url="https://www.duckdns.org/update?domains={YOUR_SUBDOMAIN_HERE}&token={YOUR_TOKEN_HERE}&ip=" | curl -k -o /var/log/duck.log -K -
root@raspberrypi:~ $ sudo chmod 700 /usr/local/bin/duckdns
root@raspberrypi:~ $ crontab -l | { cat; echo "*/5 * * * * /usr/local/bin/duckdns >/dev/null 2>&1"; } | crontab -  
root@raspberrypi:~ $ exit

Having that configured you can go ahead and setup either port forwarding of 21 and 443 or a DMZ on your router. Finally, check if you can access the device over the internet:

ssh pi@home.example.com #where home.example.com is your domain

Exposing the Raspberry through the Ngrok

If you exposed the Raspberry using the first option skip this part.

To set up ngrok, you first need to sign-up into the service. The free, basic plan should be enough to get you started, but I recommend going Pro, as it enables you to have both reserved TCP addresses and a custom domain. It’s a bit steep, but allows you to have two online processes, which means you can share it with your fellow automation maniac. If not, Basic plan allows you to have reserved domain, which is enough to enable HTTP access to your home.

Alternative solution is to buy a small server in the cloud (you can fine cheapest one from 3 up to 5 bucks a month) and use open-source reverse proxy software, like FRP for example.

Having an account in ngrok, if you can, go ahead to the reserved page and reserve a domain, home.example.com in my case, as I own a domain. Ngrok will create a CNAME target for this domain to setup, something like s00kkkim.cname.eu.ngrok.io. Having the target, configure it in your domain using a CNAME record. Then, do the same with Reserved TCP Addresses if your pricing allows you to do so.

We will start with installing ngrok using link to linux arm distribution from Download page. Let’s log in into the Raspberry through ssh.

pi@raspberrypi:~ $ wget https://bin.equinox.io/c/4444AAAABBB/ngrok-stable-linux-arm.zip # Link from download page here
pi@raspberrypi:~ $ unzip ngrok-stable-linux-arm.zip 
pi@raspberrypi:~ $ sudo mv ngrok /usr/local/bin/
pi@raspberrypi:~ $ rm ngrok-stable-linux-arm.zip
pi@raspberrypi:~ $ mkdir .ngrok2
pi@raspberrypi:~ $ cat <<EOF > /home/pi/.ngrok2/ngrok.yml
authtoken: 0000000000i1oVxSFbL8i_33H6vcYJUoH0000000000 # Your ngrok authtoken

region: eu

    proto: tls
    addr: 443
    hostname: home.example.com
    inspect: false

    inspect: false
    proto: tcp
    addr: 22
    remote_addr: 2.tcp.eu.ngrok.io:11300

Having ngrok installed we have to start it. We will setup a system service, configured to start ngrok automatically when system loads. systemd, new service manager is available on Raspbian, so we can go ahead and write a service descriptor file.

pi@raspberrypi:~ $ sudo su
root@raspberrypi:~ $ sudo cat <<EOF > /lib/systemd/system/ngrok.service

Description=Ngrok service

ExecStart=/usr/local/bin/ngrok start --all --config /home/pi/.ngrok2/ngrok.yml --log=stdout

root@raspberrypi:~ $ exit

pi@raspberrypi:~ $ sudo chmod 644 /lib/systemd/system/ngrok.service
pi@raspberrypi:~ $ sudo systemctl daemon-reload
pi@raspberrypi:~ $ sudo systemctl enable ngrok.service
pi@raspberrypi:~ $ sudo systemctl start ngrok.service

After creating service descriptor we need to reload the service daemon, enable the service to be run at boot time and finally start it.

Exposing Https

Lots of existing smart-home devices allows you to integrate with them using a web-hook. Starting with smart assistants like Google Home, to even bulbs, mostly through the excellent IFTTT platform. We need to expose our Raspberry to the internet using Https protocol, to be able to consume those web-hooks.


For that, we are going to use the Let’s Encrypt, a free, automated, and open certificate authority. On the 443 port we will run a nginx as a proxy terminating the ssl connections, forwarding to the 8080 port, which will be used by our, custom application controlling the devices.

Obtaining the first certificate

To obtain a certificate from Let’s Encrypt we need to install a certbot. The fastest way to do that is by using certbot-auto wrapper.

pi@raspberrypi:~ $ wget https://dl.eff.org/certbot-auto
pi@raspberrypi:~ $ chmod a+x certbot-auto
pi@raspberrypi:~ $ sudo mv certbot-auto /usr/local/bin

Having certbot-auto installed we need to install and configure nginx. We will configure a basic server serving content from /var/www/html directory, which will be used to obtain a certificate.

pi@raspberrypi:~ $ sudo apt-get update
pi@raspberrypi:~ $ sudo apt-get install nginx -y
pi@raspberrypi:~ $ sudo su
root@raspberrypi:~ $ sudo cat <<EOF > /etc/nginx/sites-enabled/default 
server {
        listen 80;

        root /var/www/html;

        index index.html index.htm index.nginx-debian.html;

        server_name home.example.com;

        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                try_files \$uri \$uri/ =404;
root@raspberrypi:~ $ exit
pi@raspberrypi:~ $ sudo systemctl restart nginx

If you used ngrok to expose your Raspberry we need to reconfigure it to obtain the first certificate and switch the ngrok configuration back.

# Only if you used the ngrok
pi@raspberrypi:~ $ mv /home/pi/.ngrok2/ngrok.yml /home/pi/.ngrok2/ngrok2.yml 
pi@raspberrypi:~ $ sudo su
root@raspberrypi:~ $ cat <<EOF > /home/pi/.ngrok2/ngrok.yml
authtoken: 0000000000i1oVxSFbL8i_33H6vcYJUoH0000000000

region: eu

    proto: http
    addr: 80
    hostname: home.example.com
    inspect: false
root@raspberrypi:~ $ exit
pi@raspberrypi:~ $ sudo systemctl restart ngrok.service

With this setup everything is ready, and we can go ahead to obtain first certificate, answering the usual questions, using auth schema with static file server hosting files in /var/www/html.

pi@raspberrypi:~ $ sudo certbot-auto certonly --webroot -w /var/www/html -d home.example.com #  

And again, if you used ngrok we need to clean things up, reconfiguring everything back.

# Only if you used the ngrok
pi@raspberrypi:~ $ mv /home/pi/.ngrok2/ngrok2.yml /home/pi/.ngrok2/ngrok.yml 
pi@raspberrypi:~ $ sudo systemctl restart ngrok.service

Proxy Setup and Certificate Auto-Renewal

The final nginx configuration will consist of two servers. First, available on port 80 will serve permanent redirect to https. Second, is the ssl-terminating proxy from 443 to 8080 on localhost, also used for certificate renewal.

pi@raspberrypi:~ $ sudo su
root@raspberrypi:~ $ cat <<EOF > /etc/nginx/sites-enabled/default 
server {
    listen 80;
    return 301 https://\$host\$request_uri;

server {

    listen 443;
    server_name home.example.com;

    ssl_certificate           /etc/letsencrypt/live/home.example.com/fullchain.pem;
    ssl_certificate_key       /etc/letsencrypt/live/home.example.com/privkey.pem;

    ssl on;
    ssl_session_cache  builtin:1000  shared:SSL:10m;
    ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
    ssl_prefer_server_ciphers on;

    access_log            /var/log/nginx/access.access.log;
    location /.well-known/ {
        alias /var/www/html/.well-known/;
        try_files \$uri \$uri/ =404;
    location / {

      proxy_set_header        Host \$host;
      proxy_set_header        X-Real-IP \$remote_addr;
      proxy_set_header        X-Forwarded-For \$proxy_add_x_forwarded_for;
      proxy_set_header        X-Forwarded-Proto \$scheme;

      # Fix the “It appears that your reverse proxy set up is broken" error.
      proxy_read_timeout  90;

      proxy_redirect https://home.example.com;
root@raspberrypi:~ $ exit
pi@raspberrypi:~ $ sudo systemctl restart nginx

To renew a certificate we will use a cron, running every night, checking if certificate needs to be renew, obtaining a new one if need be. Thankfully majority of the fuss is already done by the certbot, so we just need to run it correctly.

pi@raspberrypi:~ $ sudo su
root@raspberrypi:~ $ cat <<EOF > /usr/local/bin/renew
/usr/local/bin/certbot-auto renew >/var/log/renew.log 2>&1
root@raspberrypi:~ $ crontab -l | { cat; echo "2 3 * * * /usr/local/bin/renew"; } | crontab -  
root@raspberrypi:~ $ exit

As everything seems ready we can finally test it. To do so, lets quickly spin up a http server on a 8080 port.

pi@raspberrypi:~ $ while { echo -en "HTTP/1.1 200 OK\r\nConnection: keep-alive\r\n\r\nOK\r\n"; } \
                    | nc -l -q 1 8080; do echo "==="; done


In this part we exposed in a secure manner our Raspberry through the internet. The next part will cover setup, configuration and securing of the Controller service.

Part I Part III