Over the past couple of years, I have iterated over my homelab, which I covered briefly in another post1.
That the "homelab" isn't really used for anything educational is not under discussion here. It is used to store, backup and serve content, primarily movies, T.V. shows and music. The nature of the media is not the topic of this post and never will be.
To some of you, this setup may seem excessive. I recommend that you amuse yourselves over at /r/cordcutters, /r/datahoarder and /r/homelab first and then revisit this post. In my journey for finding the perfect setup, I have found people with expertise and passion that inspired me to build the setup I have today. With what I have, I can stream my content to remote devices anywhere in the world, watch basic cable and have all my media consolidated in one place. It is worth noting that this isn't the most economical of setups, though the rationale behind most hardware choices is to ensure longevity and reliability. The upfront cost of acquiring hardware necessary for the build might seem prohibitive and rightly so; I use server-grade components meant to run 24/7, which comes at a price.
Having said that, in the long run, you will have on-demand access to a library of content that is curated by you, is of the highest quality that internet has to offer and at a monthly2 cost that is still less than any bundled garbage that your cable television provider offers.
I will cover different aspects of the setup, put an dollar estimate against each one and try to point out which pieces are optional.
NAS (Network Attached Storage)
At the time of writing this post, the hardware I list below is still perfectly functional and has been for the last 2 years. Take the list of hardware and the corresponding prices with a grain of salt; you can definitely find better deals on your favorite computer hardware websites. For the motherboard, RAM and processor, I went with brand new parts. You may find success with refurbished or used parts; I encourage you to use your imagination.
The goal of my setup was to have a place to store my media, serve it and then consume it. To that end, I started with a NAS or Network Attached Storage. The choice between readymade NAS systems such as the ones manufactured by Synology or Drobo and creating your own is primarily driven by cost, willingness to spend time configuring the system to do exactly what you want. I want maximum control over the hardware and the software of the NAS, which is why I chose to build my own.
An important thing to consider is what you are going to use the NAS for the most. If you plan on streaming a lot of HD video from your NAS, you may need more than the Atom/ARM processors that most readymade NAS systems come equipped with. For transcoding3 HD video over the network, an Intel Core i3 or an i5 processor will offer vast performance gains over embedded processors.
If you plan on using the NAS for just storing and backing up data, then readymade solution is better suited to your needs.
Homelab setup as of August 24, 2016
|The NAS hardware consists of:|
|4U rackmount enclosure||Norco 4220 20 hot-swap bays||~$340 with shipping|
|Power Supply||Corsair 750W||~$85|
|Server grade motherboard||Supermicro X9SCM||~$180|
|Dual-core Intel Processor||Intel Core i3-3240||~$140|
|2x RAID controller cards†||IBM ServeRAID M1015||~$170|
|16GB ECC RAM||Kingston ECC RAM||~$135|
|10x 4TB HDD†||Western Digital Reds||~$1500|
|2x SAS to SATA breakout cables||Monoprice SAS to SATA breakout cables||~$36|
|~$2640, all things considered.|
† Each of the RAID controllers has 2 SAS ports; with a SAS to SATA breakout cable, each port supports 4 drives, so one card will support 8 HDDs. I have 10 HDDs so I need 2 cards. You can definitely save money here by starting small.
IBM ServeRAID M1015 Cards
You do have to flash the firmware on the RAID controller cards to LSI firmware6 so that they appear as LSI Logic 9211.8i cards to Freenas. Most RAID controller cards add their own layer of abstraction over the HDDs that are connected to them. Freenas likes to be directly interfaced with the HDDs and many times, the proprietary software abstraction offered by the RAID controller cards can cause problems, and even loss of data.
By cross-flashing the IBM card's firmware to LSI Logic 9211.8i firmware, Freenas sees the card as a simple HBA (Host Bus Adapter) minus the abstraction.
Enclosure for your NAS
The enclosure for the NAS is highly subjective. I am not a huge fan of putting cases on the floor, so I went with a rackmount enclosure. The 2016 Norco 4220 enclosures are quite nice and have fixed problems previously reported with their backplanes. Depending on your preference for hot-swap bays, number of hard drives you wish to have and future expandability, you can opt for a smaller case with fewer hot-swap bays and save money there.
That said, I did replace the stock fans that came with the enclosure with Arctic Cooler fans, which are a lot quieter and offer sufficient cooling.
Closing thoughts on NAS Hardware
I also purchased an open rack for mounting all the equipment, so that all of the hardware has a place to live and there aren't cables running all over the place. This is entirely optional however, since this consideration is dictated by the space you have at your disposal. There are other pieces of hardware that I left out, such as a switch, firewall appliance, a UPS and a base station.
I highly recommend investing in a UPS, since power spikes or surges could damage the HDDs in the NAS and lead to data corruption, data loss or worse, mechanical failure of the HDD.
|Other hardware in my setup|
|1U 700VA UPS||Cyberpower OR700LCDRM1U||~$170|
|1U Firewall Appliance||Motorola Symbol||~$75|
|24 port Gigabit Switch||D-Link DGS 1024-D||~$130|
|WiFi Base Station||Apple Airport Extreme Base Station||~$179|
Assuming that you have acquired the hardware for the NAS, the next step is to install and configure the software. We begin with the OS (operating system.) There are as many, many choices; Freenas, Nas4Free, any flavor of Linux or even Windows Server and Mac OS. I went with Freenas owing to the fact that it is a stable NAS OS, based on FreeBSD and comes with ZFS7 support out of the box, along with many plugins that I need for media automation.
The next important piece is your media automation, which is powered by usenet. Usenet is one of the oldest existing means of exchanging information; I implore you to do your own research on usenet's history.
There are three distinct aspects of media automation—the search/cataloguing software, usenet servers and an indexer. You will have to pay a subscription fee for a decent usenet server and a couple of indexers. Usenet servers keep files for a certain period, known as retention. Most competent servers will offer 1800-2000 days of retention and offer 99% NZB completion, which is another way of saying that if a file is corrupt, the server offers enough redundant pieces to complete it.
|Usenet servers||Astraweb and Giganews||~$20/mo|
|Usenet indexers||omgwtfnzbs, nzbgeek||~$10-$12 each for a VIP membership|
|Total estimate with a $60/mo Internet connection†: ~$80|
† A basic 250Mbps internet plan from Comcast, Cablevision or Time Warner will run you anywhere between $60-$90 depending on the area and the device rental fee, taxes et al.
Usenet servers host the files you need to download, usenet indexers maintain an index of those files and the cataloguing software searches for the files and automatically downloads them. My setup includes Freenas running plugins to look for movies, TV shows and music and download them as they become available.
|The software on the server side is:|
|Couchpotato, Sonarr, Headphones||for automating media downloads|
|Sabnzbd and Transmission||plugins for downloading media|
|Plex Server plugin||for metadata scraping and serving content|
Freenas comes with support for all of the above in the form of plugins, and it also makes configuring these plugins very straightforward. You can set up Couchpotato, Sonarr and Headphones to connect to the indexers and then send matches to Sabnzbd and Transmission. You can also configure post-processing and metadata scraping; set up notifications so that the plugins can inform you when the media has been found, downloaded and post-processed. Furthermore, you can also have these plugins send notifications to Plex so that the library is always kept updated.
I have set up Pushover, and configured all the media automation plugins to send me notifications to my phone so I know if a movie/show/album has been downloaded.
|The hardware on the frontend/consumption side:|
|HTPC (Home Theater Personal Computer)||sourced from Amazon|
|HD Homerun Extend for OTA cable||Amazon|
|Amazon Basics Leaf Antenna||Amazon|
|Vizio M50-C1 4K 50" Television||Amazon|
|Software on the client side|
|Windows 10 Home Edition|
|Kodi (Jarvis)||HTPC frontend software|
|Steam||Video game cataloguing software|
Glossary of terms
NAS Network Attached Storage. This is a computer or a headless server with hard drives available over the local area network. To other computers on the network, this appears as a shared hard drive.
Homelab A term used to describe the cumulative hardware; including and not limited to a NAS, a firewall, a switch, a virtualization server, a packet filter and many others. This is usually done by networking or IT professionals to practice for their network certification examinations.
HTPC Home Theater Personal Computer. This is a computer; either custom-built or pre-built and is suited to be put in the living room, connected to the TV set or the projector with the sole intention of media consumption. To that end, there are many HTPCs that come in a small form factor, which allows them to be put out of view.
NZB NZB is an XML-based file format that is used to download files from usenet. It describes the data to be downloaded and where to get the different pieces from. Since headers are not downloaded, when you download data using NZBs the transfer process utilizes bandwidth in an efficient manner.
Compared to Verizon's Basic Triple Play Plan as of April 2016, which starts at $69.99 and after adding just HBO (which is not a part of their original bundle. Worth noting that Starz, Showtime and Epix are also sold separately) and device rental/taxes/fees comes to $106.99 for the first year, and $126.99 after that. ↩