Post Archive

2020 8
2019 2
2018 3
2017 19
2016 6

2020

Installing Paperless in Docker - Second Steps

This page is a followup to my Installing Paperless in Docker article.

Installing Paperless in Docker

I’m writing this in response to this Paperless issue. The goal is to list out the complete installation procedure to install Paperless using Docker.

House Network Upgrade

During the quarantine, the whole family is back in the house. After receiving some complaints about the WiFi coverage, I decided to overhaul the networking gear we had.

Making Ethernet Cables

I actually started this a long while back and never finished. I bought a box of bulk Cat6 cable and some RJ-45 connectors and I borrowed a friend’s crimper tool.

June Food

So I wanted to share some recipes for food I’ve recently made.

Home Assistant in Docker with PicoTTS

One issue I discovered is that the official Home Assistant docker image does not support the PicoTTS component. The image simply does not have the PicoTTS program.

Home Automation and Docker

This is something I had put off for a while. I won’t go into what Docker is or why it’s awesome, that’s an easy Google.

I wanted to incorporate my garage doors into my Home Automation system. With a simple wall button that causes the garage door to open and close, surely I could build something that does this and is controllable by Home Assistant.

2019

ZFS NAS Attempt #2

Ditching FreeNAS

Zigbee2MQTT Part 3 - Antenna Modification & Zigbee Router

I was having some issues with some Zigbee devices not reporting and going “offline” for long periods of time. I also noticed that these devices were physically far away from my CC2531 stick.

2018

Proxmox Server - Part 3 - Storage Upgrade

So instead of using a NAS as my primary bulk-data storage location, I decided to add bulk-storage to my Proxmox server itself.

Home Automation - Part 1

I am diving deeper into a smaller project that I started earlier. First, I wanted a way to be able to control the lights in my house. Particularly the outdoor lights. and second, I wanted something that would handle turning these lights on and off automatically.

Magic Mirror #2 - PIR Sensor

After building a larger Magic Mirror, I thought it would be very cool if the Magic Mirror application would normally be off, but turn on when somebody stands in front of it.

2017

Low Powered Cluster Project, Part 3

: The three (working) nodes, the T410 laptop, and the Raspberry Pi firewall

: Side view

: More of the networking is visible here (but not everything)

Building a NAS Part 8

Status Update

MSR Whisperlite

Backpacking - Appalachian Trail from HWY 19E to Carver’s Gap

: A view near Doll Flats

: The trail near Doll Flats. Sometimes the trees would open up

: AT trail through Doll Flats

: The trail is very rocky sometimes. (Yes, that's the trail)

: A view after the treeline going up Big Hump Mountain

: Looking at the trail going up Big Hump Mountain

: Our campsite on top of Big Hump Mountain

: Our tent on top of Big Hump Mountain

: Eating oatmeal in the tent

: On the other side of Big Hump Mountain

: Looking back towards Big Hump Mountain

: The trail on the way to Little Hump Mountain

: The AT near Little Hump Mountain

: Looking down at Overmountain Shelter

: Clouds rolling in

: Same location, more to the right

Magic Mirror #2

After I completed my first Magic Mirror project, I really wanted a larger one. The first one was kind of a proof-of-concept. Now that I successfully built it, I wanted to make a better one. A bigger mirror, a better frame, an actual Raspberry Pi. So with the knowledge I gained making my first, I started planning my second Magic Mirror.

Building a NAS Part 7

Implementing Snapshots

Building a NAS Part 6

Backup Software

Magic Mirror #1

Magic Mirror 2 (the software)

Black Bean Orzo Salad

I got this recipe from a friend (Shoutout to Aaron) who, I think, got it from his mother. It taken from a whole sheet of vegetarian recipes that I’ve included below.

What I’ve Been Listening To - March 2017

What I Played

Undocumented Projects

I haven’t gotten around to writing about everything I have going on. So this will be a brief list of some other things I’m doing.

Building a NAS Part 5

FreeNAS Impressions

Low Powered Cluster Project Part 2

HDMI Dummy Plugs

Superbowl Food

I have a guilty pleasure of making party/snack foods. I can’t help it. It’s fun.

Low Powered Cluster Project Part 1

So this project wasn’t planned out, it was just a vague idea, then I saw somebody selling these computers on Ebay for cheap, I bought some and the project started.

What I’ve Been Listening To - January 2017

What I Played

My House’s Network

I thought I would document how my house’s network is set up. It’s nothing special or interesting except for the whole thing is relatively cheap.

Building a NAS Part 4

Installation

So I have my HPE Microserver ready to go. I’ve already installed the Xeon E3-1225 v1 in it and disabled two cores (2 enabled). I’ve also installed an 8 GB stick of RAM, bringing the total to 12 GB.

Today in the mail, I received five 3 TB Seagate Constellation ES.2 hard drives, two 16 GB Transcend USB 3.0 flash drives, and two long ethernet cables.

The two 16 GB Transcend flash drives will be used to store the OS (FreeNAS). FreeNAS allows itself to be installed on to two flash drives so if one fails, the other can be used. I figured this was a small cost for additional redundancy.

So far, my costs have are:

HP Microserver Generation 8 - $350
Xeon E3-1225 v1 - $50
8 GB Micron DDR3-1600 unbuffered ECC - $45
five 3 TB Seagate Constellation ES.2 - $385
two 16 GB Transcend USB 3.0 flash drives - $25

Total is $855. My original budget was $800. I went over mostly because of the HP Microserver purchase. I made that a little on a whim, I think I could have found a better price if I shopped around more. No regrets though. $55 to get something that you really want is worth it to me.

Enough words. Pictures!

: The Microserver and drives

: The Microserver with case removed

: Closeup of the Seagate Constellation ES.2 hard drive

: Front of the Microserver, showing the HDD caddy

: Installing FreeNAS

Including a SSD

Wells-Family-Vegetarian-Recipes-Final The finished product I have an old 60 GB OCZ Vertex SSD laying around doing nothing so I thought it would be a good idea to throw it in the Microserver. It might sound like a good idea to use the SSD as a SLOG or L2ARC (caches) for the ZFS array, but the FreeNAS Documentation warns that this will only help in some specific cases. And I’m certainly not going to have heavy synchronous writes nor have a working data set that exceeds the RAM in the machine.

What I want to use it for is storage for any virtual machines or containers that I want to have in the future. I have no plans to set up anything using the SSD now, but I did want to go ahead and get it installed.

The optical drive bay in the top of the machine is the perfect place to mount the SSD. Only a 4 pin floppy drive power cable is available in the top bay, so I dug through my old stuff to find a Molex splitter (so this can be installed in between an existing Molex connection) and a Molex to SATA power adapter. I will connect the SATA data cable to the Microserver’s single internal (3GB/s) port intended for the optical drive.

: The Molex to SATA adapter I put together

: The optical drive bay were the SSD will go

: The SSD installed in the optical drive bay

Building a NAS Part 3

This is an entry in my Building a NAS project.

The Hard Drives

I made the final decision on my hard drive choice and configuration.

I purchased five 3 TB Seagate Constellation ES.2 drives, model: ST33000650NS.

I got them from the big Ebay seller GoHardDrive. These drives are ‘new’ old stock, meaning that they are new and unused drives that did not sell. They are not under warranty from Seagate, but GoHardDrive (the seller) offers a 3 year warranty. Each drive cost $77.

I decided on ‘new’ old stock because of price. A 2 TB drive runs about $50-55 and a 3 TB drive runs about $70-80. ‘Normal’ new 2 - 3 TB drives are either not manufactured (?) or are more significantly more expensive ($100+). The only notable exception is Toshiba’s P300 drive, which runs about $80 for 3 TB. I did consider that drive, but decided against it because I could get a longer warranty period as well as a enterprise-class drive for about the same price.

I expect reliability of these drives be nearly as good as any new drive. These are unused enterprise drives. Enterprise drives are designed for heavy use and long life and their zero use means they have their entire life ahead of them.

I settled on this particular drive because the seller offered a 3 year warranty on them. 1 year is standard and 2 years is occasionally offered. But a 3 year warranty means I can go to the seller for a replacement if one fails for the next three years.

Due to the redundant design of my storage, I originally was comfortable with not having a warranty. If a drive died, I would simply replace the drive. The discovery of drives that have some warranty was an added bonus.

Had I not chosen this particular drive, I would have most likely picked a 3 TB Hitachi Ultrastar 7k4000 or 7K4000 series drive. It’s a ‘new’ old stock enterprise drive made by Hitachi. I chose this drive for the same reasons I picked the Seagate drive. The Seagate drive simply had a longer warranty period.

The Array Configuration

For the array, I have decided on a 3 way mirror.

I will then add the fourth drive to the array, have it mirror all the data, then remove the fourth drive and store it in a remote location. I will then add/mirror/remove the fifth drive and store it locally.

I will add/sync/remove the local drive on some regular schedule, probably monthly. I will also add/sync/remove the remote drive on a less-frequent-but-regular schedule, probably once or twice a year.

This gives me an array that can suffer two hard drives failing and still be operational. In the case that three drives fail, I have a local copy of all my data. Only the last month’s of data would be lost. (My data won’t change too much and my desktop computers may have a copy of the most recent data.)

And in the event of some large disaster where all three drives in the array and my local backup are destroyed, I still have a copy of my data stored remotely. I will likely lose some data, but I will surely have most of it. Especially considering the magnitude of an event that would cause such a situation.

2016

Community College

I believe that community college should be the default ‘thing’ to do after high school for most people.

Most people have no idea what they want to do with their life at the ripe age of eighteen. Instead of perusing an expensive university degree and assuming all the associated debt, I think that people should go to community college and pursue a degree there.

Community college is cheap. I can’t speak for every place, but the school I go to (Guilford Technical Community College) is $76 per credit hour. A 60 credit hour degree costs a meager $4650. Add $1500 for textbook costs, transportation, and other costs, and you’re still only at $6000 for a 2 year degree.

Going to community college gives a person time to decide what direction the person wants to go in. Even if someone takes out a loan for the complete amount and even if they end up completing a degree they have no interest in, the person will only be $6k in debt. This degree costs less than tuition for one semester at most universities.

In addition, community college also gives skills to people. People can go to a community college and get a Associate Degree in two years. This makes skills accessible more people. Sure, it’s not a Bachelor’s from big State University, but it’s a degree showing the holder has a set of skills.

Another upside is the ability to easily transfer to a University to complete a Bachelor’s degree. I’m not sure if this happens in every state, my college offers six degrees that any public NC university will accept. This give people that chose community college a route to continue their education. (And save money since they would transfer as a junior.)

Those skills at that low cost is what makes community college a really good decision for a lot of people. I really believe many people would be better served by a community college rather than a university.

Instead of pushing university on all the high school graduates out there, we should push them to community college. The highest performing, motivated graduates should absolutely be encouraged to go to a university. But I’m sure most graduates aren’t super dedicated students that know how they want with their life. Community college gives these students a useful skill for employment, or an avenue to continue to a Bachelor’s, all while costing very little.

Postscript: I also want to mention another path which is going to community college part time while working a part time job. I think spending four years working while completing an Associate’s Degree is a great path that is very doable for the majority of people. Completing the degree is obviously helpful for getting better employment. Working while completing the degree allows the student to gain work experience, which is valuable, even if it’s not in the student’s intended field. Working also allows the student to earn money to offset costs, such as housing and food.

There is absolutely nothing wrong with finishing an Associate’s Degree at age 22. Especially combined with four years of work experience.

Sometimes I think people consider anything less than Bachelor’s at 22 as failure. I have personally thought that and realized how wrong I was. Life is not a race and you are not behind. Your life is about you and you should always look for ways to better yourself.

In May 2017, I graduated from Guilford Technical Community College with an Associate of Science at the age of 25.

I Use PGP

OpenPGP is a standard for encrypting and verifying the integrity of data. This means that you can use it to transform data into something that only a single person can read. You can also use it to prove that somebody wrote something and that something has not been modified.

For example, you can use my public key (below) to encrypt a message that only I can decrypt and read. You can send the message over any medium and anyone who reads the message will not be able to know what it says.

I have also signed this paragraph with my private key. This processes creates a digital signature that only I can create. You can use my public key (below) to verify that the signature came me. If anyone modified this text in any way, the signature would not verify.

Feel free to verify this text or send me encrypted message.

Here’s the signature of the above message. (Not including this line.)

My public key:

Building a NAS Part 2

This is an entry in my Building a NAS project.

The Hardware

I have decided to run FreeNAS as my OS on my NAS. This requires 8 GB of RAM and ECC ram is highly recommended. (Required or not is debated) I also need the ability to plug in 4 SATA hard drives. I want the system to use as little power as possible.

Some other, smaller features that I really would like is to have is USB 3.0, removable drive bays, and AES-NI support on the CPU.

DIY ITX build

One idea I considered was to build a small ITX system myself.

Case: Cooler Master Elite 110 ($30)
PSU: Seasonic G Series 360W ($60)
Motherboard / CPU: Asrock N3150-ITX ($75)
RAM: 8 GB DDR3L ($30)

Total is $200. A pretty decent setup. The case is limited to three non-removable drive bays. No ECC RAM support. I could most likely add these features for additional costs, but if I spend more money, I could get a:

HP Microserver Gen 8

I saw this product and it seemed to fit my needs very well. It’s an ITX cube case with a 4 removable drive bays, a dual-core Celeron G1610T, and 4 GB of ECC DDR3.

It was a little expensive at $350, but I decided to splurge and get it. My reasoning was that it was only $100 more than my ITX design, and it was designed specifically to do exactly what I wanted.

Memory Upgrade

The first issue with the stock Microserver was that it only had 4 GB of RAM. FreeNAS requires 8 GB. I went to my trusty source of computer hardware (Ebay) and bought a single 8 GB stick of DDR3-1600 that was both ECC and was unbuffered. Unbuffered ECC ram seems to be much less common than registered ECC memory, and consequently, more expensive. I paid $45 for the 8 GB of memory, giving me a total of 12 GB of RAM.

There are only two memory slots in the HP Microserver and only 16 GB is supported. (This is apparently a chipset limitation, 32 GB total is supported, but requires 4 DIMMS) I picked an 8 GB stick so that if I ever wanted to upgrade to the full 16 GB, all I would need is another 8 GB stick.

CPU Upgrade

I also decided to upgrade the Celeron CPU. Primarily because virtualization was a possibility in the future (after I get the NAS stuff set up) and also because drive encryption was a feature I wanted. The Celeron does not support VT-d (the ability to pass devices straight to a VM) nor AES-NI.

A caveat of upgrading the CPU is the cooling situation in the Microserver. There is a small all-aluminum heatsink on the CPU and the only airflow is created by the rear 120 mm fan. I’m sure it’s fine for a 35 W CPU, but adding a CPU with a higher TDP may create a cooling issue.

Ideally, some kind of low-voltage Socket 1155 Xeon would be the best choice. However, I realized that the Xeons that fit the bill were going for a bit more than $100 at the cheapest. I also found that the absolute cheapest option was a 90 W TDP Xeon E3-1220 v1, which could be had for $50.

I had to make some trade-offs. I really wanted the features the Xeons brought, not their processing power. I was also a little budget constrained after splurging on the Microserver. So I decided to purchase a full TDP Xeon and disable two cores. This would (hopefully) half the TDP. The new 40 W TDP was still higher than the stock CPU’s TDP of 35 W, but I am pretty confident it will work just fine. I don’t plan to run the CPU at 100% for any extended time anyway.

I ended up finding a Xeon E3-1225 v1 for $50. This is the same CPU as a E3-1220 v1 except for the E3-1225 v1 has integrated video and a 15W higher TDP. The Microserver has integrated video built into the motherboard, so the on-CPU video won’t be used. This (hopefully) lowers the TDP to 80 W. I also disabled 2 cores, which (hopefully) brings the effective TDP down to 40 W.

Conclusion

I have the HP Microserver in hand and have upgraded the memory to 12 GB and the CPU to a Xeon E3-1225 v1 with two disabled cores. I am very happy with my decision so far. The Microserver looks very capable and very small, which is exactly what I wanted.

I am still deciding on the hard drive config. I am still leaning of a 3 drive mirror of ‘new’ 2TB drives from Ebay.

Postscript:

Apparently, I failed to take any pictures of the Microserver on this day. D’oh! Here are (some terrible) pictures of the CPU and the memory stick I added.

: Xeon CPU

: 8GB Memory Stick

: 8GB Memory Stick

Building a NAS Part 1

The Problem

I am sick of losing data. I really value the data I have and I simply do not want to lose it.

So I have decided to build a NAS to consolidate and most importantly replicate my data.

The Solution

At minimum, I want some kind of network-accessible storage array that can withstand one disk failure without loss in uptime and then back up that array locally and remotely.

This strategy addresses disk failure, array failure (electrical surge, theft), catastrophic disaster (fire).

My data needs are modest. I have less than 1 TB of data that needs to be backed up. This amount is unlikely to grow. I estimate that 2 TB is the most I will ever need to store. Accessibility of the data is not terribly important, so uptime is not critical.

I then decided one some kind of low powered computer that could support an array of 2-4 disks. I also decided to use the FreeNAS operating system.

I am still debating the array configuration and hard drive choice.

The Drives

‘New’ 2TB enterprise drives with a one year seller warranty for approximately $50 each on Ebay. These drives are apparently new drives that did not sell. They are warranted by the seller, not the manufacturer. I have considered these drives because they are cheap, have some warranty, and my backup plan allows for unreliable drives. If the array fails, I have all my data locally backed up.
3 TB Toshiba P300 with a 2 year manufacturer warranty. These drives are new, regular desktop drives commonly available for $80 each from places like Newegg.

I have not considered new NAS designed drives (eg WD Red) because of expense. A new 2TB WD Red costs $90 and a new 3 TB model runs $105. Their only advantage is NAS optimized firmware and a 3 year warranty. The problem is that these drives command at least a 25% price premium. I’m sure these drives are better choices, but I don’t think their advantages are worth the 25% premium in my situation. (Low usage, highly redundant, not important uptime)

The Array

I have considered a few options:

3 drive mirror of ‘new’ 2TB disks. $150, 2 TB capacity, 2 disk failure tolerance
3 drive mirror of 3 TB Toshiba disks. $240, 3 TB capacity, 2 disk failure tolerance
4 drive Z2 array of ‘new’ 2 TB disks. $200, 4 TB capacity, 2 disk failure

I have eliminated a 2 drive mirror from consideration because I can achieve 2 disk failure tolerance for $150, which is in my budget and very much worth it. If I can afford it, there is no reason to save money to drop to 1 disk failure tolerance.

I have also eliminated a 4 drive stripe of (2 drive) mirrors. (Aka Raid 10) This array achieves the same capacity of a Raid Z2 array, but with the caveat that after a single disk dies, the other drive in the mirror cannot also fail. Either drive in the other mirror can fail, but not not the mirror’s partner. A Z2 array can withstand any second disk failure. Now the odds of the other mirrored disk failing is pretty small, but a Z2 array achieves better fault tolerance at no additional cost. I’m sure there’s some performance difference, but I really doubt I will see the difference between the two array types in my usage scenario.

Performance of the different arrays is something I do not fully understand. I am ultimately limited by gigabit ethernet. Also, this array will only be used for backup and storage purposes. IO performance is not critical. As long as I can get acceptable performance out of gigabit ethernet, I will be happy. I am willing to sacrifice a little performance for redundancy, but performance still needs to be acceptable.

The only benefit of a Z2 drive array seems to be capacity. Capacity above 2 TB is not hugely beneficial to me. Sequential performance seems to be improved in a Z2 array compared to a mirror, but hard drives can achieve near 100 MB/s performance and gigabit ethernet is limited to 125 MB/s. Gaining performance beyond a single hard rive won’t be hugely beneficial to me either.

Conclusion (for now)

I am leaning on a 3 drive mirror of the ‘new’ 2 TB enterprise drives. The drives are cheap and reasonably reliable since they are unused and have a 1 year seller warranty. If one fails outside the 1 year window, $50 is not a huge cost to pay. A 3 drive mirror seems like the best option that meets my reliability requirement while having the highest cost to performance ratio.

Hello World!

So I have decided to make a blog to record some of the stuff I have going on in my life. This blog will mostly document thoughts/ideas I have, my (mostly computer) projects, and my general day-to-day life. (Also check out my twitter @max_farrior) My goal is to make at least two posts a week and I will probably have more in the beginning as I post things that I’ve already started.

A little about this blog. Purely for the educational value, I have decided to host this blog on my own. I have a VPS set up to run Apache. The blog is a static website generated using Jekyll. This will definitely be a work in progress and I will probably never be done with making changes.