This covers some additional topics about running Paperless, that are not required, but may be very useful.

Generally speaking, before you run these commands, you want to cd into the location where you installed Paperless. (In the first article, it’s cd /opt/paperless/)

Additional Docker Commands

Paperless can be started by running:

# docker-compose up -d

Similarly, Paperless can be stopped by running:

# docker-compose down

And naturally, there’s also:

# docker-compose restart

Another useful one is reading the logs of the container:

# docker-compose logs

That command spits everything out in the screen. To view it in a program where you can scroll through it, run:

# docker-compose logs | less

Press “q” to quit.

You can also save the logs to a file, and use less to view that file.

# docker-compose logs > paperless.logs
# less paperless.logs

Please note that if you run that first command a second time, everything in the paperless.logs file will be overwritten. Usually, that’s fine, but if you want to save the logs, change the file name in the command.

Additional docker-compose.yml Options

The docker-compose.yml file primarily controls how docker-compose will run Paperless.

If you want Paperless to start automatically when the computer starts, remove the “# ” at the beginning of Line 7 and Line 35. Delete the “#” sign and the space after it.

Additional docker-compose.env Options

The docker-compose.env file primarily controls how Paperless will run. Think of it as being the primary Paperless configuration file. I’ve found the following options to be very useful.

You can add these lines at the end of the file, each option on a new line.

• PAPERLESS_DISABLE_LOGIN=true

This option will remove the login page from Paperless. I trust everybody on my network with access to Paperless, so it was annoying to log in every time.

• PAPERLESS_INLINE_DOC=true

By default, when you click a document in Paperless, Paperless will have your browser download the file to your computer. 95% of the time, I want to look at the document, not download it. Setting this option will have Paperless display the document in your browser.

• PAPERLESS_DATE_ORDER=MDY

Paperless will look through documents to find dates. However, it’s default date format is DD/MM/YYYY. Here in the US, the date is typically written as MM/DD/YYYY. This was a huge help when implementing Paperless.

Working with Docker

When you make changes to the above configuration files, you need to restart Paperless to have the changes go into effect. See the docker-compose restart command above.

This will all be based on the official Paperless Docker installation documentation. I intend this to be used as a reference, it can be modified to suit your own needs.

Assumptions

• You’re running Debian or Ubuntu and you’ve already installed docker, docker-compose, and git. If you’re not sure, run apt install docker docker-compose git

• Paperless works by watching a folder for files and processing any files that are placed in that folder. It makes sense that this folder would be shared in a local network, so that a scanner could save scanned documents to this share.

• I will be assuming that this share is on the computer running Paperless located in /mnt/sambashare/paperless-consume-directory.

• You are the root user.

• Paperless will be installed to /opt/paperless.

Download the Paperless application

I like to keep my docker-compose files in subdirectories of /opt. You can put them anywhere you want.

At a basic level, git clone just downloads a git repository. In this case, the git repo contains the Paperless application.

# cd /opt

# git clone https://github.com/the-paperless-project/paperless.git

# cd /opt/paperless

Paperless comes with example files for docker-compose. We will now copy the files that have the .example suffix, and create the real files, which will not have that suffix.

# cp docker-compose.yml.example docker-compose.yml

# cp docker-compose.env.example docker-compose.env

The docker-compose.yml file is what the docker-compose program will read to set up Paperless. It’s a docker configuration file to run Paperless. The docker-compose.env file is just a file that defines some variables that control how Paperless runs. Paperless decided to have a separate file for this.

Configure Paperless

Paperless is configured by editing the files we just created.

To make the changes, we will use the command-line text editor nano. It’s very simple and intuitive, it behaves exactly how you would expect. To save and quit, press Ctl + x, then y, and last enter.

First, let’s edit the docker-compose.yml file.

# nano docker-compose.yml

Now set the location of the Paperless consume directory. Edit Line 22 and Line 44. You will delete the “./consume” and replace it with “/mnt/sambashare/paperless-consume-directory”. The final line will look like:

            # You have to adapt the local path you want the consumption
            # directory to mount to by modifying the part before the ':'.
            - /mnt/sambashare/paperless-consume-directory:/consume

Save and quit. Now we we will edit the docker-compose.env file.

# nano docker-compose.env

You probably want to change the time zone on Line 15. Remove the # from the line and replace America/Los_Angeles with your own value. All available values can be found on this Wikipedia page.

Save and quit.

Starting Paperless with Docker

Now that the Paperless configuration is complete, we will now run Paperless with Docker.

Whenever you use any docker-compose command, docker-compose will be reading the docker-compose.yml file in your current directory. So in our case, any time you use docker-compose with Paperless, you need to first make sure you are in the /opt/paperless directory.

# cd /opt/paperless

Once in this directory, you can start Paperless by running:

# docker-compose up -d

When you run this for the first time, you are going to witness some of the magic of Docker. An image is going to be built that’s already configured with everything Paperless needs to run. You don’t have to do any of those steps, you’re watching Docker do it for you. All you have to do is set set configuration options and Docker handles setting up the application.

Please note that I did get what looks like an error when running the above command. It appears that the installation hangs, but just give it a few minutes. In my testing, it the installation did not fail.

Installing dependencies from Pipfile.lock (6754ac)…
Failed to load paths: /bin/sh: /root/.local/share/virtualenvs/paperless-fScSATYQ/bin/python: not found

In the end, you should see:

Creating paperless_webserver_1 ... done
Creating paperless_consumer_1  ... done

Paperless should now be running. You can check by opening a web browser, and navigating to “http://<server_IP>:8000”. Replace “<server_IP>” with the actual IP of the machine running Paperless.

You should see the Paperless login page!

However, the administrator password has not been set. Go back to your SSH session, and run the following command:

# docker-compose run --rm webserver createsuperuser

That command will prompt you for a user name, an optional email address, and password.

After you run that command, you should be able to log in to Paperless.

Testing Paperless Consumption

The first big checkpoint is getting logged into Paperless. The next big checkpoint is making sure it can read and process files.

I made the assumption that you’ve already set up the network share. However you have that set up, put an image you would like to be processed in the paperless consumption directory. In this article, I’ve assumed that to be /mnt/sambashare/paperless-consumption-directory.

In my assumption, I’m using a SMB share with Samba. I then point my scanner to save files to that SMB network share, in the paperless-consumption-directory folder.

Once the file to be processed is in the consumption directory, Paperless should see it, and start processing it, which involves using an OCR program to try to parse the text. Give Paperless a couple of minutes, and your file should A) disappear from the consumption directory and B) appear in the Documents section of Paperless.

If this happens, then you are all set!

Final Words

After you get everything working, I would encourage you to read this followup article where I go over some extra useful commands and configuration options.

If something isn’t working, the docker-compose logs command would be useful to find errors.

All of this may look really complex, but once you have a basic understanding of shell commands and docker-compose, I promise you everything here is fairly basic. It looks like a lot because I go into detail about every step. It really just boils down to git clone, edit the docker-compose files, and run using docker-compose.

Before the upgrades, there was an Ubiquiti EdgeRouter X, and Ubiquiti AC-Pro, and a Netgear gigabit switch. Fairly simple, everything worked.

The first task was to address the WiFi coverage issue by getting more access points. I got two Ubiquiti AC-Lite units, and I positioned them on the edges of the house, covering rooms where people spent time. The existing AC-Pro is very centrally located.

I had to be very careful choosing channels in the 2.4 GHz spectrum. I’ve talked about it in other articles, I have a Zigbee wireless network that’s used for some home automation devices. The Zigbee protocol also operates in the 2.4 GHz spectrum… Because of this, I’ve chosen to not use WiFi channel 11, and put my Zigbee network on channel 24. In order to use non-overlapping channals for my WAPs, I’m limited to a 20MHz wide Channel 1 and a 20 MHz Channel 6. With all these restrictions, I set my central AC-Pro to use Channel 6, and the two AC-Lite units (near the edges of the house) will both use Channel 1. I’m hoping that they’re far enough away that they do not interfere.

Another point I’d like to make is that Ubiquiti has created a web application named “UniFi” which is used to manage the whole line of UniFi products. This application is awesome. It’s so cool to be able to manage all three access points from one single interface. The downside is that the application needs to run on a computer, ideally something that’s on 24/7. If you’re like me, that’s no issue, but for less technical people that don’t have datacenters in their house, that may be inconvenient / challenging.

While using the UniFi application, I couldn’t help but to notice the pages I couldn’t use because I didn’t have a router in the UniFi line. (The EdgeRouter X is in the EdgeMax line.) Purely to make management easier, I decided to change routers to the UniFi Security Gateway. There was nothing wrong with the ERX, it just didn’t work with the slick UniFi application.

However, I realized that the VLANs I was using for different WiFi SSIDs would be an issue with the USG. The ERX had 4 LAN ports (all of which support VLANs) so plugging in a switch and 3 WAPs was not a progblem. The USG only has 2 LAN ports. And if I plug the WAPs into the Netgear (dumb) switch, I’d lose the VLAN tags… So I got a managed switch… in the UniFi produt line. The UniFi Switch-8-60W supports VLANS, PoE, and integrates into the UniFi application for management.

The installation and configuration went very smoothly. The USG is now the primary router. That connects to the UniFi switch. The three WAPs are all connected and powered by the switch. And all five devices are completely controlled by the UniFi application.

I hate to sound like a shill, but the Ubiquiti UniFi gear is slick.

For some reason I wasn’t very successful. I don’t remember exactly what happened, but I remember it being a lot more work than I thought, I had to return the crimper and I never came back to this.

Well, on a whim, I bought a crimper and a cable tester. This time around, it really wasn’t too bad. Tedious, yes, but I was able to manage. Things went much more smoothly after after the first three or four, I found a rhythm.

From last time, I already had a bunch (quantity sixteen) of roughly ten foot cables that were unterminated. Plus a few other smaller lengths. I went ahead and put connectors on all the cable I had. Each cable takes about eight to ten minutes to complete. I dedicated about 45 minutes a day for about a week to get everything done.

Hopefully, I won’t need to buy Ethernet cables any time soon.

Hummus

I’ve had this hummus recipie infographic saved for a while. I made the Classic and Tapanade versions.

Both turned out pretty good and were not difficult. I do recommend using a food processor, and not a blender. Ahem.. a friend told me that a blender would have trouble mixing everything because the the mixture is so thick. You don’t want the bottom half to be mixing while the top half is stuck in place..

Vegetarian Sandwich

Not sure what to call this. I found the idea from [this Reddit comment]. I had already made some hummus, so I made the sandwich. It’s bread, hummus, cucumber, black olives, bean sprouts, black pepper, a drizzle of olive oil, and feta cheese.

I was a little unsure about how it would taste, but it was quite delicious. I had it for lunch several times.

Chorizo Burgers

This is something else I found in a Reddit comment. Mix in Chorizo with your ground beef, then grill it.

The best ground beef for brugers is 80% lean because you want there to be some moisture (and flavor) in the high-heat grill. I also use the Mexican-style chorizo (brand: La Banderita) which is intended to be removed from its casing and cooked. Its essentially ground sausage (rather than the typical sausage link).

I didn’t measure exactly, but I estimate that I used a 75% ground beef and 25% chorizo mixture. I also tried a roughly 67% ground beef and 33% chorizo mix, and that worked well too. No preference, but I wouldn’t go higher than that because the chorizo has fat in it too, and it got pretty fatty.

Granola

Came across this granola recipe and decided to give it a go.

First off, this makes a lot of granola. I filled a 2+ liter container. In hindsight, I would use 6 cups of oats. The rolled oats come in 12 cup containers at the grocery store near me, so that nicely makes two servings.

Also, you can completely improvise on the ingredients. I added more nuts as well as chia seeds, and dried mango and dried cranberries. And Trader Joe’s has a great selection of nuts as well as dried and freeze-dried fruits.

This turned out really great, it’s crunchy, sweet, and filling. It’s not hard to make and there’s no added sugar, just what’s naturally in the fruit. Two thumbs up on this recipe.

I’ve also had great results mixing this granola into nonfat Greek yogurt. Greek yogurt also has a lot of protein in it, something I didn’t realize.

I moved to Docker so I could more easily add new programs to my Home Automation setup. In particular, I wanted to try out Zwave2Mqtt. And after realizing that I’d need to not only install the program, but also the Open-Zwave library.. which could only be done by downloading the source, compiling, and adding some paths to an environmental variable…

All that is completely possible, but it’s a lot of moving pieces. If I needed to update that library, I’d have to get the source and compile again. It’s just a very manual process.

Add that on top of updating the Zwave2Mqtt program itself, which has its own process. And that’s on top of every other piece of my HA setup, which are installed in various places across the system. It was just a lot of manual work, and I was getting tired of it.

Enter Docker

Moving to Docker would simplify a lot of that. Each program lives in its own container and the container has everything it needs to run. To update, just get a newer version of the container.

I’m also going to use docker-compose, which will allow me to define the configuration of many Docker containers, and bring them all up and down together.

Implementing this wasn’t too bad. There were definitely some snags and some learning that had to be done, but no major issues. See my post on getting PicoTTS installed.

I started with Zwave2Mqtt, and carefully moved each other HA piece to a container. This mostly involved stopping the existing service, and adding more lines to my docker-compose.yml file, and tweaking it for my particular environment (and figuring out why the container didn’t start, or why it couln’t access some other container).

Results

There’s not much to show you. Here’s my docker-compose.yml file that I’m using.

$ docker ps

CONTAINER ID        IMAGE                              COMMAND                  CREATED             STATUS                   PORTS                    NAMES
ffffffffffff        homeassistant-maxmods:latest       "/init"                  3 weeks ago         Up 3 hours (healthy)                              homeassistant
ffffffffffff        carldebilly/zigbee2mqttassistant   "dotnet Zigbee2MqttA…"   3 weeks ago         Up 3 hours               0.0.0.0:8092->80/tcp     zigbee2mqttAssistant
ffffffffffff        mariadb                            "docker-entrypoint.s…"   3 weeks ago         Up 3 hours               0.0.0.0:3306->3306/tcp   mariadb
ffffffffffff        robertslando/zwave2mqtt:latest     "docker-entrypoint.s…"   3 weeks ago         Up 3 hours (healthy)   0.0.0.0:8091->8091/tcp   zwave2mqtt
ffffffffffff        koenkk/zigbee2mqtt                 "./run.sh"               3 weeks ago         Up 3 hours                                        zigbee2mqtt

The real advantage is how easily I can add and update my programs. All the dependency resolution and environment setup is handled by Docker (in the images). All I have to do is add new containers and configure them in my docker-compose.yml file.

This is best demonstrated with how I set up Zwave2Mqtt. All I had to do was add the docker-compose lines, adjust the path to my USB Zwave stick, and then start the container. That’s it. Brand new application to do something new, set up just like that.

In a similar vein, after seeing Zwave2Mqtt’s nice web interface, I really wanted something similar for zigbee2mqtt. Turns out somebody already made one, Zigbee2Mqtt Assistant. Installation was literally just adding some lines to my docker-compose file. Start the container you can access the web interface.

I’m really happy with how this turned out. My Home Automation stack is much simpler to set up and manage, and future growth will be much easier (NodeRED, I see you).

On a side note, this seems to be a known issue, but this has not been addressed.

Custom Image Creation

To get PicoTTS working, I need to add the picotts program to the Home Assistant image, resulting in my own Home Assistant image. This was surprisingly easy.

First, I created a file: homeassistant-picotts.dockerfile.

# More info here https://www.reddit.com/r/docker/comments/c8jela/can_we_add_packages_to_a_docker_container/

# Start with the official Home Assistant image
FROM homeassistant/home-assistant:latest

# Install PicoTTS
RUN apk add --no-cache picotts --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing

And then build your own image.

$ docker build -f homeassistant-picotts.dockerfile -t homeassistant-picotts .

This will create the homeassistant-picotts Docker image, which you can use in your docker-compose.yml file instead of the official Home Assistant image.

Updating Home Assistant

You will need to do this every time you update Home Assistant. Super inconvenient, but luckily, updating a Docker container is trivially easy, so adding a step isn’t the worst.

Well I can :D

So I have a Chamberlain 1000SDR manufactured in 2001. After some investigation, I found that there’s normally ~16 VDC between the two wires that go the the wall button. Pressing the wall button causes this to drop to zero, and when that happens, the garage door opens.

Hardware

I thought this would be a perfect use of a ESP8266. I still have a few NodMCU units on hand that would be perfect.

To achieve some electrical isolation, I figured it would be best to use a relay to trigger the garage door. I got a module with two relays because I have two garage doors to control.

I also ordered two generic reed switches so I could detect if each garage door was closed or not. In hindsight, it would have been better to use two reed switches per door. This would allow me to detect both “fully closed” and “fully open”. (With one reed switch, I can only detect “fully closed” or “not fully closed”. That might mean fully open, or stuck halfway.)

Everything is wired to the NodeMCU. I soldered directly to pins because I thought it would be secure than using jumper cables. The relays are connected to pins D1 and D2, and the reed switches are connected to D5 and D6. Power for the relay module comes directly from the Vin of the NodeMCU.

One good idea I had was to use quick-connect connectors on the wires going to the relays and reed switches. This lets me work on the opener unit without dealing with long wires to the other components.

Nothing secures the components inside the enclosure box. The unit is reasonably held together by the white wires tied together. I wouldn’t call it secure, but it’s secure enough to be handled. Between that and the fact there’s not much space in the enclosure box, everything stays together, there’s not much wiggle room or opportunity for something to be pulled apart.

Software

I’ve great experiences with ESPHome for ESP8266 devices. You tell ESPHome what you want the device to do, ESPHome will then compile a firmware that does all those things. ESPHome was designed to make ESP8266 devices interface with Home Assistant. It makes the software side really simple, you don’t have to dive into code, you don’t have to figure out how to compile and upload the firmware.

MQTT is the default way for to communicate. ESPHome reports the the status of the relays and reed witches. ESPHome also handles working with Home Assistant’s MQTT Auto-Discovery feature, so everything show up with no configuration necessary.

Full ESPHome configuration available here.

Garage Installation

I bought some generic “security wire” to connect the relays and reed switches. The “security wire” is a plain cable with two 22 AWG wires in it. I run one cable to each garage door motor unit and one wire to each reed sensor.

No snags on this step. Just measuring/cutting the wires and mounting the sensors, and connecting to the garage door motor unit.

I was fortunate that there was a spare outlet available, I didn’t have to worry about finding a 5 VDC source.

Home Assistant

Using the Lovelace picture-entity and custom:button-card cards, I created a nice looking control interface.

- type: vertical-stack
  cards:
    - type: picture-entity
      title: Garage Door Primary
      entity: binary_sensor.garage_door_primary_sensor
      state_image:
        "on": /local/garage_door_primary_open.jpg
        "off": /local/garage_door_primary_closed.jpg
    - type: custom:button-card
      entity: switch.garage_door_primary_switch
      show_state: false
      name: '[[[ if (states["binary_sensor.garage_door_primary_sensor"].state == "on" ) return "Close Garage"; else return "Open Garage" ]]]'
      icon: '[[[ if (states["binary_sensor.garage_door_primary_sensor"].state == "on" ) return "mdi:garage-variant"; else return "mdi:garage-open-variant" ]]]'
      state:
        - value: 'on'
          icon: mdi:garage-alert
          color: rgb(255, 0, 0)

Video

Coming soon…

Automation Ideas

• If garage door is open for 10 minutes and nobody is home,
  • Close the garage
  • Send push notifications
• If garage door is open for more than 30 minutes and somebody is home,
  • Send push notifications to everybody (or only people that are home)
  • (Alternate) Say a TTS message over the house intercom

I decided to abandon using FreeNAS.

The main reason is that I want to use Sanoid to create and manage ZFS snapshots. So it seems the most reasonable way to do this is to run Sanoid in a jail. The dealbreaker came when I tried to pass all the datasets to the jail. It did not work. I could not access all my datasets from the jail. And, I did this a while ago, so I don’t remember the exact problems, but I spent a tremendous amount of effort on getting this working, and I was not able to. I also found no information on the Internet on what the problem could have been. I mean I found nothing. My biggest hunch I found was that the issue was caused by an 88-character limitation on the path to the dataset. If that is the problem, I think that’s a dumb limitation. Why have a limitation at all, and if there has to be one, why not make it something huge, like a million characters or something? Whatever the reason, I think it’s dumb.

A minor girpe I have with FreeNAS is the direction the projet is going. FreeNAS includes a feature to be able to run VMs. From what I have read, this is complex and causes a lot of issues, because creating a hypervisor is not simple. It seems like FreeNAS is trying to create an “all-in-one” product like Unraid, where you have one server that manages the storage, and can run various applications in jails and in VMs. IMO, that’s not what I want in a storage applicance. I just want storage. If I wanted a hyperivsor, I’d get an actual hypervisor. I like the ability of having jails (a default FreeBSD feature) to be able to run a few programs/scripts, but I don’t think major programs and infrastructure should be running on my storage. And the bigger issue is that FreeNAS devs are spending time developing a hypervisor, which has been the source of a lot of issues. I wish they would just focus on ZFS storage. Again, minor gripe.

Last, I want to say that I don’t dislike FreeNAS at all, I still think it’s great. It just doesn’t do what I want.

My Revised Storage Needs

With the creating of my Proxmox server, this box is now purely backup storage. I have no need for any type of performance. I just need ZFS storage, and a way to access that storage on my network. I also don’t have any particular capacity goal. As long as it can hold around 4 TB, it can work.

Hardware Changes

First, I took out the OCZ Vertex (1) and replaced it with some 128 GB OEM Samsung (PM851?) SSD. I made the swap because the Vertex is super old, and I the Samsung SSD was just sitting around.

Next, I took out the Xeon 1221 v1 CPU and replaced it with the stock Pentium G1610T. Again, no need for performnace, the stock CPU will draw less power.

I will also re-use the 3 TB Dell-branded Seagate Constallation ES.2 hard drive. I have five total, I will use four in the box, and the fifth will be a cold spare.

Software

My only requirements were to be able to run ZFs, be accessible over the network, and run Sanoid. I setteled on Debian, because it can easily do all that.

I installed Debian on the SSD, installed ZFS, and installed and configured Sanoid.

I am much happier with this sofware setup. No need for the complex system that is FreeNAS, this is so much simpler, and does what I want.

Native ZFS Encryption

Since ZFS 0.8, ZFS now has native encryption capabilites. So no need to depend on FreeNAS’s (FreeBSD’s?) GELI encryption, and no need to stack ZFS partition on top of LUKS devices. Now, you can create your ZFS pool, and then create datasets that are encrypted.

CC2531 Antenna Modification

I followed the steps on this Hackaday page.

In short, we cut the original antenna and solder in a antenna connector. With that in place, a larger, external antenna can be used instead of relying on the small built-in antenna.

Zigbee Network Changes

To also help with the range, I followed some steps on this page.

First, using a USB extension cable to move the CC2531 away from the computer. Then I changed the channel of the Zigbee network to 24. (All my Wifi networks used either channel 1 or channel 6.) Changing the channel required repairing ALL DEVICES, which was a PITA.

CC2530 Router

I found out it’s possible to create a Zigbee router, using another radio similar to the CC2531. This radio can operate without a computer, and just relays signals for oter devices. The page on the different types of Zigbee devices is worth reading.

It also bring the benefit of extending the 20 device limit of the CC2531, which I am at or very near. I hope this would really help with extending the range of the far-away devices, by acting as a middle point between the device and the CC2531.

I decided to use the CC2530+CC2591 because it is more powerful, and only costs ~8 USD more.

A future improvement is to use a larger antenna. I left room to accomade this. I also used a liberal amount of hot glue over the exposed wires of the (120 VAC) extension cord. I definitely want them to be secured and isolated.

Results

Honestly, the results were underwhelming. First, I didn’t take proper “before” measurements, so I don’t have hard numbers to use for comparison. But from observation, I noticed that I don’t have any more dropouts anymore. So I think there was definitely an improvement.