Murphy's Law of Self-Hosting: My Blog Crashed on Day One

I love programming so much, the thrills, the joys and the absolute agony of having something work when you run it on your own, publish it for the first time, and then get confident enough to leave on holiday to a country 16 hours away.

Yep, on day 1 of my blog being up, my entire server crashed on the precipice of me announcing it.

Let me walk you through how I went about debugging and bringing it back up remotely from the Netherlands, and why having a secure remote connection to your server is so important!

In this post, I'll show (and hopefully teach you a little) about some painless ways you can remotely access your servers from anywhere in the world, for completely free!

How do I access my server remotely

Now of course, in the self-hosted world there are several ways you can access your own servers, for instance the most known (and unsecure) common way to expose a website to the internet is to open a port on your home Wi-Fi to the broader internet. This basically allows anyone with access to your public IP address to simply enter your IP and the accompanying port and have access to the site! Most routers these days make this a pretty simple task to do as well, all you have to do is go to the port forwarding section on your router and enter the local IP of your device and the port you want to forward. For instance, my ghost server runs on port 2368 so I find the IP of the device running it, and set this in my router `http://192.168.68.77:2368`.

Great! This means we have the website exposed and all, right? It's public to the Internet, so we can access it from anywhere in the world?

Well, yes technically you can, but this also means that anyone that has this IP can, there is no way to prevent someone from quickly taking this IP, logging into your server (with your super secure password set to @password123 of course) and start setting that machine to mine crypto or something else disgustingly useless.

Instead, I use this amazing software known as Tailscale.

Tailscale? Another weird engineering named software?

Yep, it's named like we're genetically cloning whale tails to build bridges, but it's actually a lot cooler than that! (debatable I know, I would also like to see how we clone whale tails, but that's for another day)

Tailscale in the most basic sense is just a VPN a Virtual Private Network, which basically, when we connect to it, sends all our network traffic through another computer in another location, making it so that all our web-browsing can't easily be traced back to our network, but instead all of our traffic will be seen as if that computer in another location is doing the searching (this can be quite good if you're privacy conscious about Jeff Bezos knowing your shoe size).

However, it adds a little spin on the VPN concept, where usually we pay to use a computer we have absolutely no relation to or ownership of, Tailscale allows us to attach a VPN client to any of our own personal and private servers and computers! This means that we can instead connect to the Tailscale network, and access or send traffic through any of the computers (in this case we can refer to them as nodes) that we have connected to our own Tailscale network!

Tailscale will handle all the complicated setup for the encryption and security purposes, you just need to sign in to their service and they handle the rest! And it's completely free up to 1000 devices which is absolutely insane for personal use

If you have more than 1000 devices that you can connect a VPN to, there's a 99% chance you will be the first to fall when AI takes over, you have been warned.

Ok ok, I'm getting ahead of myself, let me give you a quick little tutorial on how you can set this software up on your own devices!

A quick setup of Tailscale on any device!

The first thing you can do is go to the Tailscale website and create an account! They have a lot of SSO compatibility, so you can use something simple like your Google account or GitHub to sign in!

After that is done, go to the admin dashboard and click on the add device button in the machine's tab

Or sometimes at the bottom they'll even give you this neat infographic in case you want to set up your node using something like Docker instead, completely up to you! I will be using Linux for this tutorial as it's what I base all of my servers on

As I chose to set up a Linux server, I was greeted by this page

There are a lot of options on this site when it comes to how you want to set up a Node, but I won't be going over things like an Ephemeral connection in this post, instead the only things I want you to focus on are the authentication key and the install script button!

The authentication key is quite self-explanatory, it will determine how long the encryption key it puts on your server is valid for, you may have to occasionally renew this, and because I'm usually quite lazy I put this on the 90-day limit, to renew a key is also quite simple as well (Tailscale makes so much of this stuff painless!) just use the command:

tailscale up --force-reauth

Crazy simple right?

Anyway, after you set up how long you want the authentication key to stay active for, you can click on the generate an installation script, it will give you the commands that will launch a pre-configured install script + a unique authentication key for that instance used to register it to your private network! Just copy and paste this into your terminal, and it'll do the rest!

After the Tailscale instance is installed

There are just a few simple commands you will need to use in order to get the node up and running, and then it's pretty much hands off!

The first command I use is:

sudo tailscale up

This command activates the node, to allow me to connect it to the rest of the network, I can then use:

tailscale status

To list all the nodes on my network! Including their IP's! (I have cut that off from the image, nice try though)

Now when I instantiate a node in the network, I have to give it a name, usually it will default to the name you give your system when you first set it up (the system itself not Tailscale), this name will be what I use to remotely access the machine itself, it's super simple! In this case the server that is running all of my projects is called the homelab so in order to access this server remotely all I have to do (on Linux) is:

ssh homelab

On Windows systems, in order to SSH you may need to install some external services to make it easier to do (this is one of the many reasons I don't like programming on Windows), you can check out this tutorial from Microsoft for more information

This will attempt to SSH me in as root (which for this debugging purpose I do want) but if you wanted to access the server under a different user, you'd do it just like you would with a normal IP!

Such as:

ssh noah@homelab

And there we have it, I'm in!

However, even though I can run the restart commands and diagnosis via the terminal, I prefer to do it via the UI (sue me), so in order to access the UI, I need to have Tailscale port forward the port the UI lives on!

Yes, yes I know I just had an entire paragraph about this previously about how port-forwarding is not secure, and you should avoid it at all costs (and you still should) but because Tailscale handles a lot of the security issues I mentioned before and this port forwarding will stop the moment I exit my terminal, I believe this is a good compromise for this sort of luxury.

Now Proxmox UI is served via an unsecure localhost address and does not have any SSL certificates (used for accessing via HTTPS), so I will need to port forward an insecure IP at port 8006 for the Proxmox UI to show up, this can be done with this simple command:

tailscale serve https+insecure://localhost:8006

This will make it so that I can now access the Proxmox dashboard from the node's IP Address that Tailscale assigned to it! (this is a public IP address, but only someone who has logged into the Tailscale account and has access to the VPN network can access this IP's contents)

Aaaand, the first thing I see is that my main homelab node on Proxmox has died somewhat mysteriously, and is refusing to start again naturally (yay!), the only reasonable way I have found to solve this is by simply restarting that node, it'll auto-start all my containers again and it'll be all good!

Now unfortunately, this is not the first time this has happened to this node... so what I'm going to do for the sake of my sanity, is migrate this blog (as well as the way we host it to the internet) to my seemingly more stable minilab node (the aforementioned Hermes node), which is done as simply as just clicking migrate and...

Boom! It simply restarts and is on the new Proxmox node! Dang, that was actually a lot simpler than it should be, lol.

Wrapping things up

Now, I've managed to turn this bit of pain into a learning experience and tutorial for you (hopefully)!

I knew when I started this journey of self-hosting that I would experience hiccups and plenty of headaches, but I didn't expect to be able to write about it already so soon 😅.

The key-takeaways from this for me is, that you should always have a reliable and secure way to access your services and or devices while you're away, you can add failsafes and fallbacks, but you can never fully prepare for the random things that can happen to your own services while you're away, I hope you do try out Tailscale for yourself, or even something alternative to it, there's a lot out there!