Archive for skills

Tiny Computer Equals Great Geek Fun

For years I’ve been fascinated by the small microcontrollers and computers available on the market. The Arduino was interesting but limited. It seemed like a hardware component that I just didn’t have a project for. The Raspberry PI, marketed as a learning system, was even more interesting. It gave the hobbyist (and later the professional) a full Linux computer in a tiny form factor. Still, it originally was close to US$50 which made it less attractive for projects that weren’t super serious or that you couldn’t simply run on an existing desktop/laptop or in a virtual machine. And, for a while it seemed as if there was more interest in making these small boards more powerful instead of less expensive. The BeagleBone Black and Intel Galileo, while popular with hobbyists, are also more powerful and hence more expensive.

The latest spate of tiny computers, however, are looking for the sweet spot between a microcontroller and a powerful, albeit it small, computer selling in the US$35 to US$50 range. A sub-$10 product makes it affordable in quantity to hobbyists and professionals prototyping Internet of Things and other applications where cost and a small form factor matter. Two recent inexpensive tiny board computers that have emerged were the C.H.I.P. from Next Thing Co. ($9 through the Kickstarter project) and Raspberry Pi’s Zero ($5 list). I hate the latter’s name. It makes it sound like it’s missing something useful. That said, neither is a super powerful board that can compete with the BeagleBones, Intel Galileo, or even the latest Raspberry Pi 2 B. But that’s the point. They are meant for stripped down products where cost matters but a standard environment (Linux in both cases) is desired.

I went with the C.H.I.P. since it was out first and I like the “working out of a garage” aesthetic. It also came with the OS preloaded (a big time saver), didn’t need an SD card like all Raspeberry Pi products, there were connectors instead of bare holes for the various I/O headers, and had Wi-Fi on Board. The onboard Wi-Fi made it especially easier to get a project started. Much less time is spent just getting the C.H.I.P. up and running than the Raspberry Pi Zero. Also, when you up in all the parts you need for the Zero such as the SD card, USB network or Wi_fi adapter, etc. the C.H.I.P. is actually the less expensive product.

To test out the utility of these small boards I decided on a project that was not a simple controller/sensor type but also not a complicated memory hog either. The project I settled on was a small private cloud file sync-and-share system. Think of it as on-premises Dropbox for the home office. Since there would never be a lot of users, the server portion didn’t need a ton of memory and the Wi-Fi capabilities meant it could be located anywhere I wanted it. Aside from the C.H.I.P and a SATA drive enclosure, the final product would be made from spare parts lying around my office. The total cost was under US$20.

Getting Started

The hardware setup was simple. It was:

  • The C.H.I.P. board, acquired through the Kickstarter project. It arrived when Next Thing said it would and worked right out of the box.
  • A SATA disk powered enclosure from Orico complete with cables and sourced from Amazon. It also had a switch to shut off the drive without cycling the power.
  • A 250G 2.5 ” SATA disk salvaged from a laptop
  • A USB-Micro USB with a plug for standard power salvaged from an old smartphone. This provides power through the USB On The Go (OTG) port.
  • A USB Hub. This was a passive not a powered hub. I have a powered hub, as it turns out, I didn’t need it.

To get started I also needed a USB mouse and keyboard to initially test and configure the C.H.I.P. The C.H.I.P worked right out of the box and connected to my home network easily.

Figure 1: Physical Setup of File Sync-and-Share Server:

Creating a Headless Computer

The board comes with a rather stripped down version of Debian Linux, which meant that I would have to load a variety of packages later. The most difficult part of the configuration was shutting off the GUI. X-Windows with a lightweight interface, while not the memory pig of the Windows 10 or Mac OS X, is not insubstantial. Most of the time, I would never be seeing the GUI anyway and preferred headless operation. There are a number of ways to boot without the GUI in Linux and most didn’t work. Advice from a fellow C.H.I.P owner did present a workable solution in that I needed to set the C.H.I.P into multi user target model using the following:

systemctl set-default multi-user.target

The other most common way of disabling the GUI was to reflash the C.H.I.P with a version of Debian that lacked the GUI. However, I knew I might want to turn the GUI back on and that was possible with this method by setting the default target mode to graphical.target. Now I had a headless GUI.

Mounting the USB Drive

Linux users who have only experienced a desktop environment such as Ubuntu will wonder why mounting the USB drive requires aby effort at all. That’s because there are programs that are part of most desktop distributions that automatically mount a USB drive the second it is detected. It doesn’t work that way in the stripped down, GUI-less version of Linux such as the C.H.I.P’s. When the drive is detected by the operating system, it will assign each partition on the disk a device identifier. Typically, this is something like /dev/sda1. However, that is not always the case. That means looking at the system messages generated during the mounting of the disk and verifying the device identifier. A good way to do that is with the following:

dmesg | tail

or

dmesg | grep sd

After that it is a simple matter of mounting the drive using the mount command as such:

mount /dev/sd1 /mnt/CHIP01

The disk always has to be mounted at the same mount point in order for the file services to work in the future. The mounted disk has to be a reliably predictable path.

Loading the File Sync-and-Share Software

There were two paths I could have taken for developing the file-sync-and-share software. First, I could have written the code myself which would have yielded a rudimentary system in a few months. The other, more palatable, option was to use already available open source software and save myself the effort. Not being a moron, I chose the latter. Since I cover collaboration software for a living I had a product in mind. I chose ownCloud from the company of the same name. Like so many “open source” companies, they have a community edition and an enterprise edition. ownCloud provides easy to understand installation instructions on their website.

Before loading the software, there were two tasks that need to be performed

  1. Find the Debian version you are working with. The command cat /etc/debian_version does this. Apparently the C.H.I.P. ships with Debian 8.2.
  2. Load locales. Locals are needed for localization and the only locales that the C.H.I.P comes preinstalled with are C and POSIX. This is unlike most Linux server and desktop versions I’m familiar with. ownCloud wants there to be one of the UTF locales installed. It’s simple enough once you know how. Finding how was a bit trickier.

    sudo apt-get install locales en_US.utf8

    sudo dpkg-reconfigure locales en_US.utf8

After that, it was simply a matter of following the instructions. I had already installed the Apache web server but the ownCloud download probably would have done so for me, recognizing it as a dependency like it did for MySQL. Accessing the web app at http://<chip name>/owncloud starts the login process and helps set up the admin account.

From here on, it was a matter of configuration from within the ownCloud application’s admin menus. I loaded a number of preexisting ownCloud apps but the one that truly mattered was the External Storage app. Without it, the USB drive can’t be recognized and the 4GB storage in the C.H.I.P won’t last very long. Once that was complete, it was a matter of configuring shares on the drive. The External Storage app also allows an administrator to configure cloud file services from Amazon, Google, and Dropbox as well as network drives and FTP. Configuring the external cloud file services is much harder than it should be though. You have to go through the process of creating authentication tokens yourself rather than have the software do it for you. If this was easier to use, ownCloud would make a great single front-end to all types of file stores, internal, public cloud, or private cloud.

The User Experience

The ownCloud user experience is much like one would expect from a file-sync-and-share system. Users can see folders with files in them. With the installation of certain apps, files can be viewed, created, and edited. There is also a desktop application that performs the actual file synchronization, similar to the ones Dropbox, Box, OneDrive, etc.

Figure 2: ownCloud Web Interface

Future Work

As it stands, my homegrown, file sync-and-share server works fine. I mostly use it for holding onto WIP (work in progress) that I don’t want to backup to a permanent backup storage device or a cloud device but would be pretty upset if I lost somehow. For that it’s fine and better than a typical windows file share. I can access these files on my mobile devices as long as I’m within my home network. There is a mobile app that ownCloud sells but that would have limited utility unless I exposed the server to the outside world. From a security perspective, it’s not hardened enough for that (yet).

To Do

There are still a number of things to do before this can be considered completely finished.

  1. Examine ways to mount the USB drive automatically. The tried and true method of altering the /etc/fstab file doesn’t seem to work but I may have not configured it correctly. The real challenge is that the initial device identifier is not always the same. This is a case where “usually” is not good enough.
  2. Reconfigure SMB/CIFS. I would love to mount one of my other network storage devices as an external drive. This would allow one modern interface for all of my network file storage devices. At the moment, the SMB/CIFS package is either missing or misconfigured. There is some chatter on the Next Thing forums about problems with SMB/CIFS. It’s the next thing to suss out.
  3. Theme it! The ownCloud software is written in PHP, in which I am well versed. Changing theme elements is sometimes a matter of altering code, others simple changes in configuration files. Since the default theme is pretty good, this has a very low priority.

Problems

The Next Thing C.H.I.P is a neat and powerful little board. It is not, however, perfect. For a rev 1.0 product, there are relatively few problems but there are not zero problems. Here’s the one’s I found so far:

  1. Power problems when plugging in USB devices without a hub. I found, on numerous occasions that the C.H.I.P. did not want to boot when there was a USB device plugged to it’s one non-OTG USB port. In several cases, when I plugged a USB device directly into the C.H.I.P. , it killed the power for the entire board. For some reason, plugging USB devices into a passive hub and plug the hub into the C.H.I.P. alleviates the problem.
  2. It takes roughly 5 to 7 minutes without power in order to reboot. No matter how one reboots – using shutdown -r, shutdown -h, or pulling the power cable, the C.H.I.P. won’t reboot until the power has been disconnected for 5 to 7 minutes. There’s a lot of speculation in the forums but this issue combined with the USB power problems makes me think there is a general power problem with the board. I’m betting we will see either a hardware or firmware fix for this in the future but that’s just conjecture.
  3. Can’t boot to the GUI without a keyboard and mouse. This is another widely but not universally encountered problem with the C.H.I.P. When operating with the GUI, the board will not boot when there is no keyboard and mouse. I get that without a keyboard and mouse, the GUI is pretty useless but it shouldn’t cause the device to hang.
  4. Not enough memory for large files. This is not a problem with the C.H.I.P. per se but with running ownCloud on it. When transferring large files, some of the data is held in a cache in RAM. Since the board only has 512MB of RAM and half of it is taken up with the operating system, Apache web server, MySql database, and the ownCloud software, there’s not enough memory left over to transfer large files. I can configure it for larger files but can’t really upload or sync more than about a 1G file. That’s fine most of the time – 1G is actually a pretty big file – but not quite big enough for backing up rich media files such as music and video.

Conclusion

My description of the building the file-sync-and-share system may appear linear but I assure you the process was not. Finding out how various aspects of the system worked (or didn’t work) took a lot of trial and error. The user forums on the Next Thing site were immensely helpful. Ultimately, this is not a toy a for lay person. It takes reasonable technical chops to put this together but that, of course, is the fun of it all.

IEEE Consultants Network of Western NY

I spoke at the IEEE Consultants Network of Western NY on Monday November 10, 2014. My topic was “New Tech but Old Skills”. I looked at three related trends – The Internet of Things, Big Data, and Analytics – and identified some skills that many engineers and computer scientists already have that relate to them.

Below is a presentation of my main talking points in PDF format.

New Tech but Old Skills – IEEE Consultant Group 10-14-14