Poor man’s NAS with Raspberry PI 3

Disclaimer: Raspberry PI is using the Micro SD Card to keep the OS files (I’m using Raspbian Jessie) and unfortunately the card storage solution isn’t 100% reliable because of card data corruption. It’s happening rarely, but is happening. Details how to minimize the risks here. All my important files are stored on an external 3.5 HDD with own power source.

The goal:

  1. Easy to access external 2TB drive from intranet via samba
  2. Easy to access my external 2TB drive from internet via owncloud (alternative: scp)
  3. Run the nginx web server secured, pointing to custom domain (eg. https://www.emilian.co.uk) and reverse proxy to few custom urls (details below)
  4. Run plex server (https://plex.emilian.co.uk)
  5. Run subsonic server (https://music.emilian.co.uk)
  6. Run deluge UI (https://deluge.emilian.co.uk) with custom authentication
  7. Dynamic DNS client (http://freedns.afraid.org)

Initial steps

  1. Download Raspbian Lite and unzip to your home directory
  2. Insert a new Micro SD Card into your computer and write the Raspbian image to your card. I’m using OSX so the quick steps are:

We’re ready to unmount the card from your computer and install it in the raspberry pi. Please make sure you connected the network (I prefer wired but wireless is available too) and your Raspberry Pi is powered by min 2.5A (I’m using a cell phone quick charger and that could explain why I’m having card corruption issues). You can find a genuine power supply here.

We’re ready to ssh on our raspberry pi. I’m using a static IP based on MAC address so it’s easy for me to ssh on that IP. There are plenty of tools to scan your network and find the ip of your raspberry pi (a quick one is to use nmap -v -sn 192.168.0.0/24). My IP is 192.168.0.112.

I’m connecting to my raspberry pi and first I change the default password and create a new user. Then I run raspi-conf to customize the current installation (eg. timezone, setting hostname, set locale (en_US.UTF-8),  expand the boot partition)

I’m adding my user to sudo group

Fix mirror for apt-get (Optional)

If apt mirror is causing issues when doing the update the quick workaround I’m applying is:

1.Open /etc/hosts file with sudo rights $sudo nano /etc/hosts

2.Paste the following lines at the end of /etc/hosts file.

 

The next step is to ssh with my new user, update my apt repos and start installing packages.

Updating Firmware (Optional)

Your Raspbian installation also included a pretty recent copy of the Raspberry Pi firmware. However, sometimes there are important updates, so I recommend to use the latest firmare version:

Mount external drive

In order to achieve the goal #1 we’ll need to opt for an external HDD drive (I’m using a 2 TB drive, external power supply) formatted as ext4. You can find your external drive by running below command:

Add below line into /etc/fstab file to mount your external drive automatically:

Mount entry in the “/etc/fstab” file shall make this mount permanent across reboot. Reboot and to check if the external disk is mounted in the /nas location.

Now we have assigned a fixed mount point for the external disk drive, which is available under “/nas” directory in our raspberry pi device.

In case our external connected usb disk takes a long time to initiate this automount might fail and you may need to initiate the system mount at a later time. Adding an entry to /etc/rc.local can do the trick. Add below lines before exit line:

Install and configure samba

We’ll use the folder “/nas” where our 2TB external drive is mounted. We’ll share “/nas” using samba. To do this, we need a samba account in our raspberry Pi. We’ll use the user account “iulian” in this case. Enter the following command and type in a password to set the samba password for “iulian” user:

The password you set for your samba share will not affect your login to Raspberry Pi over ssh.

Let’s backup the original smb.config file.

Add the following section in the smb.conf file:

Restart samba service:

Check if samba service is enabled on startup, so that the service starts on reboot (usually it is)

The “[ + ]” indicates the service is active on startup.

You can mount your /nas folder to your local computer and check if the share works as you expect.

Installing nginx and configure reverse proxy

Nginx is a light and modern web server and fits perfectly our raspberry pi configuration.

You should be able to see nginx is up and running by opening your local browser with http://192.168.0.112 (raspberry pi assigned ip).

Install PHP 7 (optional)

Raspbian being based on Debian Jessie ships with PHP 5.6 by default. But we want to run the latest PHP version. To install PHP 7 on Raspbian we must switch to the testing branch of Raspbian, commonly known by the codename stretch.

Add below line at the bottom of the file:

By adding this all installs or updates will default to use the newer versions of files available in the stretch release which is not considered 100% stable. To prevent that, we’ll pin all packages to use the jessie release with a higher priority by default. To do this create a preferences file:

Now we’ll need to configure nginx in order to support our needs.

 

Links of the week

  1. git standup -d 1000 via https://github.com/kamranahmedse/git-standup
  2. Free ssl with let’s encrypt with nginx https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-16-04
  3. Still valid https://codahale.com/how-to-safely-store-a-password/
  4. http://www.adhikary.net/post/cygwin-zsh-solarized-apt-cyg/
  5. https://github.com/powerline/fonts
  6. https://blog.alexellis.io/jenkins-2-0-first-impressions/
  7. http://rustamagasanov.com/blog/2016/02/23/dockerize-your-project-part-1-registry-and-jenkins-setup/
  8. http://rustamagasanov.com/blog/2016/02/24/dockerize-your-project-part-2-building-rails-and-postgres-images-with-docker-compose/
  9. https://blog.florianlopes.io/host-multiple-websites-on-single-host-docker/
  10. https://blog.codecentric.de/en/2015/10/continuous-integration-platform-using-docker-container-jenkins-sonarqube-nexus-gitlab/
  11. https://discourse.osmc.tv/t/how-to-install-couchpotato-and-sickrage-on-raspberry-pi/10788/16
  12. http://immutables.pl/2015/12/12/tmux-powerline-zsh-osx/

Configure OSMC via VNC

When running KODI (OSMC) on the Raspberry PI it’s always handy to set in front of your desktop and adjust/install stuff.
So I end up setting up the vnc server on my Raspberry PI.

You’ll need VNC Viewer to connect to your fresh Raspberry PI VNC server.

But first, let’s connect to the raspberry pi instance via ssh (ssh osmc@raspberry_ip) and update it:

then you will need to install:

We’ll need to change the refresh rate so we’re going to edit the main.c file before compiling it:

We’re looking for

to be replaced with:

Then we’ll compile:

Last step before connecting is:

You need to keep the process running:then start your VNC viewer and use below url to control your raspberry PI.

Enjoy.

Links of the week

  1. ECMAScript 6: https://kangax.github.io/compat-table/es6/ -> Compatibility of your javascript with ES6. I use Google Chrome Canary (currently 97% compatible).
  2. http://scottjehl.github.io/picturefill/ -> picture polyfill. Deliver appropriate image size based on client conditions (screensize, viewport, etc).
  3. lsof -i :22
  4. SSL is not an option. How to achieve A+ score https://michael.lustfield.net/nginx/getting-a-perfect-ssl-labs-score
  5. Convert crt to pfx: openssl pkcs12 -export -out domain.name.pfx -inkey domain.name.key -in domain.name.crt>

iTerm3 and docker startup

If you open iTerm2 and try to run docker ps you’ll probably receive below error:

Cannot connect to the Docker daemon. Is the docker daemon running on this host?

The solution to use iTerm2 instead of Docker QuickStart Terminal is relatively simple:

That will generate below output:

You then need to run:

The output of it is:

All you need to do now is to run:

That’s all 😉

 

Node.JS Module Patterns

1. The simplest module

2.Module Patterns
2.1. Define a global

Node: Don’t pollute the global space

2.2. Export an anonymous function

2.3. Export a named function

2.4. Export an anonymous object

2.5. Export a named object

2.6. Export an anonymous prototype

2.7. Export a named prototype

3.Summary:

  • Named exports – allow you to export multiple ‘things’ from a single module
  • Anonymous exports – simpler client interface

http://www.commonjs.org/specs/modules/1.0/

Manage multiple versions of nodejs with nvm

I’m using nodejs as my primary development language and usually I deploy my projects in docker containers. There days I realised my development nodejs version is not the latest one.

So I was looking to update it really easy with the option to choose between the current version (reminds me on .net framework version via app.pool or php versioning).

Nvm seems the perfect tool to do the job:

Now using node v6.1.0 (npm v3.8.6)

default -> 6.1 (-> v6.1.0)

The output was v6.1.0

You can check the installed node.js versions with:

 

Docker Engine and DockerCompose on OSX – configure iterm2

After you install docker toolbox via

just add below on your iterm console and add support for zsh

 

MongoDB – Security

By default security in mongoDB is turned off. You can enable it by using mongod –auth or add the key into mongodb.config

If you try any commands from mongodb, the result will look like below:

However you have access to admin database.

or you can try to start the shell from command line:

Type of users (clients):

  • “admin” users:
    • can do administration
    • created in the “admin” database
    • can access all databases
  • regular users:
    • access a specific database
    • read/write or readOnly

To alter the roles:

Available roles:

  • read
  • readWrite
  • dbAdmin
  • userAdmin
  • clusterAdmin
  • readAnyDatabase
  • readWriteAnyDatabase
  • dbAdminAnyDatabase
  • userAdminAnyDatabase

To secure cluster mongodb you can enable Mongodb authentication and authorization with –keyFile flag. When using –keyFile with a replica set, database contents are sent over the network between mongod nodes unencrypted.

The keyfile must be present on all members of replicaset.

Login and test the replica set authentication:

Starting MongoDB 2.6 –auth is implied by –keyFile.

Additional resources:

  1. https://docs.mongodb.org/manual/security/
  2. https://docs.mongodb.org/manual/tutorial/configure-ssl/
  3. https://docs.mongodb.org/manual/reference/built-in-roles/

Docker private registry

Docker even has a public registry called Docker Hub to store Docker images. While Docker lets you upload your Docker creations to their Docker Hub for free, anything you upload is also public. This might not be the best option when you work on your projects or inside your organization.

So I wanted to spin up my own private docker registry based on … docker. Actually docker-compose as I need to components for my repo: the docker server itself and the nginx server acting as a proxy to handle authentication and ssl.

Below the docker-compose.yml file I’m using to spin up the environment:

On my host I created two directories to handle data images and nginx configuration.

In nginx directory I added the configuration file nginx.conf together with the ssl certificate.

The registry.password file is used by nginx to keep users and passwords for our registry.

Initially created with

you can strip -c and add additional users.

You can spin up the docker registry with:

If everything is ok, you can check if platform is up:

If you want to start docker-registry as a service you need to create the Upstart script:

Let’s test our new Upstart script by running:

Feel free to check the log files:

Your private repository is accessible via https://registry.iulitab.co.uk:5443/v2/

Using our private docker registry:

You can login to your registry with:

From your client machine, create a small empty image to push to our new registry. We will create a simple image based on the ubuntu image from Docker Hub

Inside the container create a dummy directory (eg. my_application) and commit the change:

then tag the image:

Note that you are using the local name of the image first, then the tag you want to add to it. The tag does not use https://, just the domain, port, and image name.

Now we can push that image to our registry. This time we’re using the tag name only:

Similar you can pull images from your private registry:

or

Happy Dockering!