To get the Controller, your hub controlling everything, available on the internet one has two options. You can reconfigure the router a little and use static IP address or a Dynamic DNS service. Alternatively, if you are limited by your internet provider, you can use a tunneling service like ngrok.
Access your Raspberry Pi over the internet
If you can, the fastest option to make your Controller available over the internet,
is to use your IP address and reconfiguring your router.
To do so, you need to enable port forwarding on your router of both 21 and 443 to the Raspberry.
If you don’t have a static IP, like most users, you can use a free Dynamic DNS service like DuckDNS.
The service provides a DNS record in their own domain (*.duckdns.org), with an ability to update it easily, using cron for instance.
To have your own domain name, using this setup, simply add CNAME record pointing to the selected subdomain.
If you can’t do that for some reason (for example your internet provider is blocking incoming connections) you are left with the second option. To expose the Raspberry in a network provider independent way you need to use a tunneling service, like ngrok for example.
In this article I will cover both options, feel free to choose the one that suits you best.
Exposing the Raspberry through the Router
Majority of routers do support port forwarding,
but that’s not the only way we can force router to expose the Raspberry. An alternative
is to set up a DMZ zone, which
allows you to forward whole traffic to the Controller. Doing so
it’s certainly easier, but exposes all ports. To increase security we
will start by installing and configuring a firewall. The fastest way to do so
is to use ufw - an Uncomplicated Firewall.
Let’s log in to the Raspberry through ssh we configured in the part I
and execute these commands, installing the firewall and exposing http(s) and ssh.
pi@raspberrypi:~ $ sudo apt-get install ufw -y
pi@raspberrypi:~ $ sudo ufw allow ssh
pi@raspberrypi:~ $ sudo ufw allow http
pi@raspberrypi:~ $ sudo ufw allow https
pi@raspberrypi:~ $ sudo ufw enable
pi@raspberrypi:~ $ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip
To Action From
-- ------ ----
22/tcp ALLOW IN Anywhere
80/tcp ALLOW IN Anywhere
443/tcp ALLOW IN Anywhere
22/tcp (v6) ALLOW IN Anywhere (v6)
80/tcp (v6) ALLOW IN Anywhere (v6)
443/tcp (v6) ALLOW IN Anywhere (v6)
Then, login to the DuckDNS service and reserve an address.
If you have your own domain address, you can setup a CNAME record pointing to
the one in duckdns. Let’s assume that your address is home.example.com.
If you can, consider donating the DuckDNS service, to help the guys cover their infrastructure bills.
The only thing left to do on the device, is to set up a cron that will be keeping our entry with IP in DuckDNS up to date.
pi@raspberrypi:~ $ sudo su
root@raspberrypi:~ $ cat <<EOF > /usr/local/bin/duckdns
#!/bin/sh
echo url="https://www.duckdns.org/update?domains={YOUR_SUBDOMAIN_HERE}&token={YOUR_TOKEN_HERE}&ip=" | curl -k -o /var/log/duck.log -K -
EOF
root@raspberrypi:~ $ sudo chmod 700 /usr/local/bin/duckdns
root@raspberrypi:~ $ crontab -l | { cat; echo "*/5 * * * * /usr/local/bin/duckdns >/dev/null 2>&1"; } | crontab -
root@raspberrypi:~ $ exit
Having that configured you can go ahead and setup either port forwarding of 21 and 443
or a DMZ on your router. Finally, check if you can access the device over the internet:
ssh pi@home.example.com #where home.example.com is your domain
Exposing the Raspberry through the Ngrok
If you exposed the Raspberry using the first option skip this part.
To set up ngrok, you first need to sign-up into the service. The free, basic plan should be enough to get you started, but I recommend going Pro, as it enables you to have both reserved TCP addresses and a custom domain. It’s a bit steep, but allows you to have two online processes, which means you can share it with your fellow automation maniac. If not, Basic plan allows you to have reserved domain, which is enough to enable HTTP access to your home.
Alternative solution is to buy a small server in the cloud (you can fine cheapest one from 3 up to 5 bucks a month) and use open-source reverse proxy software, like FRP for example.
Having an account in ngrok, if you can, go ahead to the reserved page and reserve a domain,
home.example.com in my case, as I own a domain. Ngrok will create a CNAME target for this domain to setup,
something like s00kkkim.cname.eu.ngrok.io. Having the target, configure it in your domain using a CNAME record.
Then, do the same with Reserved TCP Addresses if your pricing allows you to do so.
We will start with installing ngrok using link to linux arm distribution from Download page. Let’s log in into the Raspberry through ssh.
pi@raspberrypi:~ $ wget https://bin.equinox.io/c/4444AAAABBB/ngrok-stable-linux-arm.zip # Link from download page here
pi@raspberrypi:~ $ unzip ngrok-stable-linux-arm.zip
pi@raspberrypi:~ $ sudo mv ngrok /usr/local/bin/
pi@raspberrypi:~ $ rm ngrok-stable-linux-arm.zip
pi@raspberrypi:~ $ mkdir .ngrok2
pi@raspberrypi:~ $ cat <<EOF > /home/pi/.ngrok2/ngrok.yml
authtoken: 0000000000i1oVxSFbL8i_33H6vcYJUoH0000000000 # Your ngrok authtoken
region: eu
tunnels:
http:
proto: tls
addr: 443
hostname: home.example.com
inspect: false
tcp:
inspect: false
proto: tcp
addr: 22
remote_addr: 2.tcp.eu.ngrok.io:11300
EOF
Having ngrok installed we have to start it. We will setup a system service,
configured to start ngrok automatically when system loads.
systemd, new service manager is available on Raspbian, so we can
go ahead and write a service descriptor file.
pi@raspberrypi:~ $ sudo su
root@raspberrypi:~ $ sudo cat <<EOF > /lib/systemd/system/ngrok.service
[Unit]
Description=Ngrok service
After=multi-user.target
[Service]
Type=simple
ExecStart=/usr/local/bin/ngrok start --all --config /home/pi/.ngrok2/ngrok.yml --log=stdout
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target
EOF
root@raspberrypi:~ $ exit
pi@raspberrypi:~ $ sudo chmod 644 /lib/systemd/system/ngrok.service
pi@raspberrypi:~ $ sudo systemctl daemon-reload
pi@raspberrypi:~ $ sudo systemctl enable ngrok.service
pi@raspberrypi:~ $ sudo systemctl start ngrok.service
After creating service descriptor we need to reload the service daemon, enable the service to be run at boot time and finally start it.
Exposing Https
Lots of existing smart-home devices allows you to integrate with them using a web-hook. Starting with smart assistants like Google Home, to even bulbs, mostly through the excellent IFTTT platform. We need to expose our Raspberry to the internet using Https protocol, to be able to consume those web-hooks.
For that, we are going to use the Let’s Encrypt,
a free, automated, and open certificate authority. On the 443 port we will run
a nginx as a proxy terminating the ssl connections, forwarding to the 8080 port,
which will be used by our, custom application controlling the devices.
Obtaining the first certificate
To obtain a certificate from Let’s Encrypt we need to install a certbot.
The fastest way to do that is by using certbot-auto wrapper.
pi@raspberrypi:~ $ wget https://dl.eff.org/certbot-auto
pi@raspberrypi:~ $ chmod a+x certbot-auto
pi@raspberrypi:~ $ sudo mv certbot-auto /usr/local/bin
Having certbot-auto installed we need to install and configure nginx.
We will configure a basic server serving content from /var/www/html directory,
which will be used to obtain a certificate.
pi@raspberrypi:~ $ sudo apt-get update
pi@raspberrypi:~ $ sudo apt-get install nginx -y
pi@raspberrypi:~ $ sudo su
root@raspberrypi:~ $ sudo cat <<EOF > /etc/nginx/sites-enabled/default
server {
listen 80;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name home.example.com;
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files \$uri \$uri/ =404;
}
}
EOF
root@raspberrypi:~ $ exit
pi@raspberrypi:~ $ sudo systemctl restart nginx
If you used ngrok to expose your Raspberry we need to reconfigure it to
obtain the first certificate and switch the ngrok configuration back.
# Only if you used the ngrok
pi@raspberrypi:~ $ mv /home/pi/.ngrok2/ngrok.yml /home/pi/.ngrok2/ngrok2.yml
pi@raspberrypi:~ $ sudo su
root@raspberrypi:~ $ cat <<EOF > /home/pi/.ngrok2/ngrok.yml
authtoken: 0000000000i1oVxSFbL8i_33H6vcYJUoH0000000000
region: eu
tunnels:
http:
proto: http
addr: 80
hostname: home.example.com
inspect: false
EOF
root@raspberrypi:~ $ exit
pi@raspberrypi:~ $ sudo systemctl restart ngrok.service
With this setup everything is ready, and we can go ahead to obtain first certificate, answering the usual questions, using auth schema with static file server hosting files in /var/www/html.
pi@raspberrypi:~ $ sudo certbot-auto certonly --webroot -w /var/www/html -d home.example.com #
And again, if you used ngrok we need to clean things up, reconfiguring everything back.
# Only if you used the ngrok
pi@raspberrypi:~ $ mv /home/pi/.ngrok2/ngrok2.yml /home/pi/.ngrok2/ngrok.yml
pi@raspberrypi:~ $ sudo systemctl restart ngrok.service
Proxy Setup and Certificate Auto-Renewal
The final nginx configuration will consist of two servers. First,
available on port 80 will serve permanent redirect to https. Second,
is the ssl-terminating proxy from 443 to 8080 on localhost, also
used for certificate renewal.
pi@raspberrypi:~ $ sudo su
root@raspberrypi:~ $ cat <<EOF > /etc/nginx/sites-enabled/default
server {
listen 80;
return 301 https://\$host\$request_uri;
}
server {
listen 443;
server_name home.example.com;
ssl_certificate /etc/letsencrypt/live/home.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/home.example.com/privkey.pem;
ssl on;
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/access.access.log;
location /.well-known/ {
alias /var/www/html/.well-known/;
try_files \$uri \$uri/ =404;
}
location / {
proxy_set_header Host \$host;
proxy_set_header X-Real-IP \$remote_addr;
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto \$scheme;
# Fix the “It appears that your reverse proxy set up is broken" error.
proxy_pass http://127.0.0.1:8080;
proxy_read_timeout 90;
proxy_redirect http://127.0.0.1:8080 https://home.example.com;
}
}
EOF
root@raspberrypi:~ $ exit
pi@raspberrypi:~ $ sudo systemctl restart nginx
To renew a certificate we will use a cron, running every night, checking if
certificate needs to be renew, obtaining a new one if need be.
Thankfully majority of the fuss is already done by the certbot, so we just need to run
it correctly.
pi@raspberrypi:~ $ sudo su
root@raspberrypi:~ $ cat <<EOF > /usr/local/bin/renew
#!/bin/sh
/usr/local/bin/certbot-auto renew >/var/log/renew.log 2>&1
EOF
root@raspberrypi:~ $ crontab -l | { cat; echo "2 3 * * * /usr/local/bin/renew"; } | crontab -
root@raspberrypi:~ $ exit
As everything seems ready we can finally test it. To do so, lets quickly spin up a http server on a 8080 port.
pi@raspberrypi:~ $ while { echo -en "HTTP/1.1 200 OK\r\nConnection: keep-alive\r\n\r\nOK\r\n"; } \
| nc -l -q 1 8080; do echo "==="; done
Summary
In this part we exposed in a secure manner our Raspberry through the internet. The next part will cover setup, configuration and securing of the Controller service.