For the past couple of years I have been working on a fairly extensive project. I have been attempting to replace the major tech companies (e.g. Apple, Google e.t.c.) services with equivalents I run myself. I have a few reasons for doing this, but primarily it is about privacy and security. I haven’t always been this way. I remember using Skype when it was still in beta, Facebook when you had to have a *.edu email address to register and Gmail when it was invite-only. However, the revelations of the past few years around data breaches and how these companies use your data have made me weary.

As a disclaimer, I would say that should anyone wishing to embark on this journey should not take this on all at once. This is a big job. Consider why you use all of the services that the large tech companies offer you. Convenience, I suspect, played a significant role in your decision. Moving these services to equivalents you control moves away from this convenience. The trade off is, as ever with self-hosted applications, an increase in security.

These are the services I have set up over the past few years and found work really well as replacements for iCloud/Google offerings;

They are other programs I run that function as pseudo-cloud services, but they are probably best discussing in another post.

So how do you get started?

Having a basic understanding of Linux definitely helped me get started learning this stuff. Additionally, I took on each project one by one and didn’t start all at once. The first thing you will need is your own server. You have a few options here. If you have a spare old computer, raspberry pi or even old laptop you have almost everything you need already!

Virtualisation

Virtualisation is probably a good idea also. This comes in many flavours, but is essentially running an operating system as a program. This means that should something go wrong you can simply flush the operating system and start over without messing up your entire system. There are a few different options here, but simplifying hugely you have;

virtualbox-screengrab

  • Virtualbox (above) is an Opensource, multi platform programme from which you can host a number of guest operating systems virtually. I ran this initially with four Virtual Machines doing different things. It’s not a bad solution, but I found it a bit clunky and resource intensive at times.

  • LXC Containers - I run these for almost all of my services. Rather than a sandboxed operating system like a Virtual Machine in Virtualbox, containers share the kernel of the host machine. Effectively they run almost exactly the same to a basic end user. I really like the LXD tools Ubuntu comes with to manage containers. The majority of my services run in this way.

  • Docker/Kubernetes - For me this seems like one of the more complex solutions, and not really an entrypoint into running your own services. Docker is a little like LXC containers, except that rather than running a guest operating system, it is simply sandboxing the application being virtualised only. This make Docker super fast. Kubernetes is a way of managing Docker containers, ideally over multiple machines to form a pool. These technologies are super fashionable at the moment, but perhaps are not the best for someone looking to get started with self-hosted services. The jargon alone is a fairly large barrier.

Hardware

If you want to have total control over your entire setup, you might want to consider running your machine at home. This means you need a server! In reality a server is any computer that serves content. Dedicated servers used in industry tend to have features like motherboards designed to run 24/7, ECC ram support and cases which can be mounted in a rack. I have never felt the need to use machines like this, but second hand ones can be bought on eBay fairly easily. My personal recommendation is either a Raspberry Pi or equivalent if you only wish to run one or two services, or an Intel NUC if you want something a bit more powerful. My issues with rack servers is they tend to be HOT and LOUD.

synology

A Synology NAS is also a great potential first server. It works really great as a NAS, but has a really handy App-store like feature where compatible software can be installed and run with a couple of clicks. There is also a Synology suite of apps for iOS & Android.

If you don’t want to worry about maintaining your own hardware it is possible to rent it instead! There are many companies to do this, but famous basic Virtual Private Server (VPS) providers include Hetzner and KimSufi. This can be a great option as it’s likely that their ability to keep a machine up and running is probably slightly greater than yours.

Backups

Have a strategy! A few years back I had hard drive failures on 2 of the four drives in a Synology NAS unit. Whenever I turned the thing on to try and get data off it, it would copy about 100gb and then crash. I spent nearly a week just turning the thing on, copying data off, having it crash and then repeating. I didn’t lose anything hugely important, but it did get to a final crash before the data became completely inaccessible. Since then I have taken a pretty conservative approach to backing things up. Thankfully it is relatively straightforward to backup from VirtualBox as it can export the machines as a single file. Snapshots are also available if you mess up a setting and want to revery to a prior configuration. LXD can also take snapshots and export machines with just a couple of commands.

First Projects

So which projects are the best to get started with?

Personally I think that the most straightforward is Plex. Fair warning though; if you want it to transcode your media so it is playable on any device, a Raspberry Pi is not likely to be powerful enough. Barring the hardware requirements, the install is very straightforward and, as a piece of software, showcases the power of self-hosted applications really well. I love that it grabs artwork and information about media. I have been in hotels on the other side of the world and simply streamed all my media from my server to whatever device I have at the time. It’s really impressive.

After that I think it’s a great idea to try and install Wordpress. I don’t necessarily think Wordpress is the greatest bit of software, but it is a great introduction to LAMP. I wrote about it here. Certainly when I have had students learning about infrastructure I start them with a Wordpress install and they tend to then move on to other web applications with decent basic knowledge and understanding.

Resources

There are some great resources out there to learn from. Outside of the Github repositories for the various projects, communities like r/selfhosted are really great. Really, just searching for set up guides will often yield a how-to from blogs or tech news sites.

With every scandal that hits the news about the big tech companies I feel a little better in knowing that I am doing my best in not using their services. I like that I can control what my data is doing to a great extent. It has taken me a while to get everything working the way I want, but the knowledge I feel I have received from doing this is really important. Next time you find yourself frustrated with iCloud or wondering who else has access to your data, perhaps it might be time to investigate replacing these services yourself.