Friday, August 15, 2014

API Endpoints for Arduinos

As part of my home automation system, I need to connect "high-level" systems (such as the primary Linux server) down into the underlying hardware systems around the house. The PIC16F688 microcontrollers that run those systems are seriously low-level. Thus, I've chosen to place all of them onto an I2C bus(*) driven by an Arduino. Why? ... the Arduino has enough capability to mount an Ethernet port on the "high" side, and has built-in I2C support for the "low" side. It is a great mapping device between the complex systems, and the hardware systems.

With the hardware selected, and the wiring selected, it came down to protocol. How can that Linux server talk to the Arduino, as a proxy to the individual microcontroller critters splattered across the household? ... Naturally, HTTP was my first choice, as it allows all kinds of languages, libraries, and scripts to perform the work, and even some basic interaction via a full-on browser.

Digging into HTTP servers for the Arduino... Ugh. The landscape is very disappointing. Much of the code is poorly engineered, or the code is designed for serving web pages. During my search, I ran into a colleague's TinyWebServer. I'd call it the best out there for web serving (well-designed and well-coded), but is still overpowered for my purpose: mapping a protocol down to simple hardware interactions.

As a result, I designed a small library to construct a simple API endpoint on the Arduino. The application can register dozens of endpoints to process different types of client requests (62 endpoints are easy, more if you want to seek out the critical edges of allowed-characters in URIs). Each endpoint can accept a number of bytes as parameters, and can return zero, or an unlimited set of bytes to the client.

I have yet to "package" the Endpoint system, so any requests and changes are most welcome! I've got some documentation to write, along with incorporating feedback from others' and my own usage.

Would love to get some feedback! ==>

(*) and yes, I know an I2C bus is designed for 0.5m meter runs, rather than a whole house; bus accelerators, speed compensation, and other approaches "should" manage it. I'll report on my success/failure in a future post.

Friday, August 01, 2014

CanaKit PIC Programmer Reset

Earlier tonite, I scorched one of my 16F688 PICs in a stupid move. To see if it was recoverable/usable, I dropped the sucker into my CanaKit programmer and ran 'pk2cmd -P' on my Mac to check whether it could see/detect the PIC.

Bad move.

"No PICkit 2 found." ... and the red BUSY light just started flashing.

It took some research because CanaKit has almost zero documentation. The short answer is the programmer is a clone of Microchip's PICkit 2 Development Programmer/Debugger. The red flashy means the board couldn't load its firmware.


Download the V2.32 firmware from the above page. Unzip the file. Then load it onto the board:

$ pk2cmd -D/path/to/firmware/PK2V023200.hex

It'll load the firmware, verify it, then reset the device. No more angry LED!

(I hope those who run into a similar problem will find this blog post, to more quickly reach a solution)

Friday, June 06, 2014

LED lighting control boards

Woot! I just ordered a set of PCBs for lighting control in the new house. Wifey would kill me if we had no lights, so the priority kinda jumped on this one ... (*Wifey comments:  I wouldn't kill you.  However, I consider it prudent to go to Home Depot and buy some cheapo mechanical switches just in case)

One of the more unique things about my (upcoming) house is that the bulk of our lighting are LED 24 VDC cans/pots/recessed lamps(*). Ceilings fans, chandeliers, and watercloset light/fans are standard high-voltage AC, but the bulk of our electrical fixtures are using these nifty low-voltage lamps. One neat thing is that the lamps don't require AC/DC stepdown transformers, so their cost is dropped, their reliability increases, and their heatprint is lowered.

The wiring becomes both simpler, and more complex in ways, and possibly cheaper depending on your choices. Simpler because you can run standard 14 gauge stranded speaker wire (cheap; flexible around those corners) instead of the heavier gauge Romex stuff. The lower gauge wire can save you quite a bit on copper costs, and you can lose the heavy insulation and concerns of high-voltage wires in your walls. But much more complicated because you need special hardware to run them ... a repair is no longer just a run to your local Home Depot.

In my case, much of the wiring in the house has been "home-run" back to my server room, so I ended up doubling my copper/install costs. All the control is localized to that room, which also means I don't have actual "switches" in my house, but just sensors (see Capacitive Touch Wall Switches). This leads to "how do I control these lighting circuits?"

Thus, my custom PWM Control Boards. They have an embedded microcontroller which can "do anything" with the nine (9) output channels. A higher-level "upstream" controller will tell it what actions to perform, when, and how, communicating via TTL Serial or I2C or what-have-you. The boards sink current, at whatever voltage (so I will also use these for my 12VDC LED light strips).

I've sent the PCBs off to ITEAD for production. Between now and when they arrive, I'll finish the microcontroller work that I've been working on. Assemble some parts from Mouser, plug it all together, and LIGHTS!


Will report back, as I make progress...

(*) I decided not to name/link the manufacturer until I get experience and form an opinion. Just email me to ask, if you're interested.

Monday, September 09, 2013

Capacitive Sensing Wall Switches

For my new house, I wanted to go with an unobtrusive, "no controls" style. There are no thermostats, no pool controller, no security panels, and no standard toggle/paddle switches for the lights. Instead, I'm going for a designer style piece of colored glass with capacitive touch sensors behind it. Think of your smart phone's touch capability, and that will be my light switches.

The "switches" are really just sensors, as the signals are delivered to the main house controller where the actual lighting management occurs. The sensor pad can detect touches, gestures, and multi-taps to provide different lighting requests, based on time of day and ambient light. The switches also include optional hookups to an IR/motion sensor, temperature measurement, and all switches have RGB backlighting.

All my work will be Open Source, so I've started the documentation for the wall switches. I have a couple custom PCBs that I'm working on, and (eventually) a whole mess of microcontroller code for the PIC16F688.

I'll keep updating the wiki pages, and committing changes to my repository. I don't have commit emails going anywhere, but Google Code supports feeds for wiki and source changes if you'd like to track the changes. Of course, I'll keep posting here when interesting milestones occur!

Monday, August 12, 2013

Bluetooth Household

For my home automation project, I chose to go with a fully-wired approach since I'm building from scratch. I get to run wires wherever needed, providing (hopefully) more reliability and an ability to troubleshoot problems. Certainly, the parts cost will be much lower than an RF solution.

But with that said, I met Eric Migicovsky a couple weeks ago (Founder/CEO of Pebble). He came up with a great idea: use a Pebble watch as a control mechanism. Sure, I'll have phones, tablets, infrared remotes, and various sensors... but something on my wrist? Always handy? Very cool idea! With multiple Bluetooth base stations, I can even detect signal strength and triangulate a user's position in the house, in order to provide context-sensitive menus and button controls. If you're in the home theater, then "Pause" is going to be a handy watch button, when that drink needs a refill! Given that I'm writing the app, I can even provide Wifey with her own customized watch experience.

To that end, I started doing some research on Bluetooth, and on the Pebble SDK. The first thing to pop up was the need to use Bluetooth Low Energy (aka Bluetooth 4.0, BLE, or Bluetooth Smart [Ready]) rather than the older Bluetooth 2.x or 3.x protocols. BLE allows for interactions without pairing, which is important for roaming about the house, with multiple base stations. The Pebble hardware supports BLE, but it seems that the SDK doesn't (yet) allow for applications to deliver messages to one/more/available base stations. My hope is to work with the Pebble guys to see where/how to make that available to (my) home automation application.

The second part of the problem is the development of the base stations for my house. There are inexpensive Bluetooth/USB dongles (about US$11) that can speak BLE. I've got a few Raspberry Pi boards around the house, with previously-unused USB ports. A little searching seems to indicate the dongles are supported under Linux.

These dongles seem to present themselves as an HID device (eg. keyboard, mouse, etc), and can be switched to a [Bluetooth] Host Controller Interface (HCI). I haven't dug in deeply on this stuff yet, but I do have a Fitbit dongle on my Mac OS. The Fitbit (Flex) speaks BLE, so it seemed appropriate to experiment with.

Working with HID seemed harsh, until I found hidapi. The API is very clean and simple. As a Python programmer, bindings were the next step. Ran across Cython-HIDAPI, which sucks: forked copy of HIDAPI and heavyweight Cython-based bindings (given the ugly, I'm not gonna provide link-love).

Answer: I wrote a ctypes-based binding for hidapi. My first, undocumented draft landed at just 143 lines of Python. Of course, I've checked it in, along with a sample script.

And after all that, my Fitbit dongle is purely a USB device (calling hid_open() fails). Sigh.

I've got more research to do, and maybe ordering a dongle for experimentation (see Adafruit, or various on Amazon). Maybe I can interact with the Fitbit dongle through USB rather than HID. Who knows. But once I figure the base station thing out, I can track Pebble watches, Fitbits, and other Bluetooth devices throughout my house.

Tuesday, April 16, 2013

Building omxplayer on a Raspberry Pi

The past couple days, I set aside my PIC work and concentrated on setting up a Raspberry Pi ("RPi"). I've got a couple of these, and will use them as video streamers for televisions in my house.

There is quite a bit of documentation for getting an RPi set up, so I won't repeat that here. My current focus is on getting video streaming working. An obvious candidate is RaspBMC, but I was looking for something very bare-bones to simply put a video onto the HDMI output. I ran across PyPlex which seemed right up my alley: Python and effectively an interface-less control of the video.

Yah. Well. Then I look at the setup/build requirements. twisted-web? pexpect? Seriously? Somebody has made this much more complicated than it should be. Whatever happened to just using BaseHTTPServer and the subprocess module?

Digging in, I find it is using omxplayer underneath. No wonder they're using pexpect -- there is a tty/keyboard interface to omxplayer. (of course, pty might be simpler than pexpect, but whatever) So this PyPlex thing starts up a web service and then controls omxplayer via a tty connection. I'm not seeing reliability and responsiveness here. And a lot of code, to boot.

Tearing off another layer of the onion, I start looking at omxplayer. Sigh. Requirements hell yet again. GCC 4.7. Boost. ffmpeg. Oh, and it is generally set up for cross-compilation rather than building on the RPi. This isn't a bad concept in general, as the RPi is no speed demon. But the build only takes a long time because they chose ffmpeg, whereas the Raspbian distribution uses libav. (these two libraries are reasonably similar, as libav forked from ffmpeg rather nastily a couple years ago)

So I'm looking at this giant pile of C++ code with a bunch of crazy requirements, which would take hours to build on my RPi. This is the wonderful state of video on the RPi. Sigh.

Well... I found a post by Keith Wright where he talks about building (a tweaked fork) of omxplayer on Raspbian. Much better, but the instructions still have crazy oddities about reconfiguring RAM, sudo to build in strange filesystem locations, and hey! fun! building ffmpeg from scratch again. Sigh. A guy just can't get any love around here.

Being the good geek that I am... this just wasn't something that I want to put up with. I want to build this sucker on my RPi, using standard tooling and libraries that come on Raspbian.

First up, I started from huceke/omxplayer rather than Keith's because it is much newer. But I did grab the Makefile.include from Keith, as it was sane for building on the RPi. Adjusted some of the paths to point to the installed items. Then, I had to install the following packages on the RPi: libpcre3-dev, libavcodec-dev, libavdevice-dev, libavfilter-dev, libavformat-dev, libboost-dev. As I started working through getting omxplayer built, I ran into a bug in a system header.

In /opt/vc/include/interface/vmcs_host/vcgencmd.h, line 33 needs to be changed to:
#include "interface/vmcs_host/linux/vchost_config.h"
I've filed a pull request to github:raspberrypi/firmware to fix this. Not sure if that is the Right place (that code may come from upstream?), but hopefully somebody will see it.

Next up, I had to hack away, tweak, and otherwise put a bit of pain on the omxplayer sources. Some hacks were easy, but others likely broke stuff (I'm not sure if subtitles work any more). Hard to tell. A/V code is not easy, and not something that I'm familiar with.

You can find all of my changes in my omxplayer fork. Clone that to your RPi, install the necessary packages, and hit "make". No system reconfiguration. No sudo. No hours of ffmpeg building. No GCC 4.7 requirement.

Clone. Make.

Go have fun, and watch some movies!

(my next step is to tear off the user interface bits, and shift to a simpler, pure-C library which I can wrap/control from Python)

Sunday, April 14, 2013

PIC Programming on Mac OS X

Lately, I've begun working on home automation to wire up my entire house with all kinds of goodies. The hobbyist in me, and with an attempt to be frugal, I'm skipping off-the-shelf solutions and building my own. A friend of mine calls me crazy, that it will reduce the value of my house, etc. Whatever. This is some fun stuff!

A big part of these systems is wiring together sensors and activators. You need something to control all of these things. There are a gazillion various solutions, with the obvious ones being an Arduino or Raspberry Pi and their GPIO pins (among many other features they have). I decided on a layered approach with "small brains" connected to the sensor, managing the specifics, then communicating upstream to a larger brain. I'll likely talk about the upstream bits in a later post, but this one is dedicated to the small brain: the PIC 16F688 microcontroller.

I grew up on a 6502 microprocessor. Graduated to a 68000 when it arrived in the Mac 128k. And after that, never really worried about machine/assembly code. As I looked around for microcontrollers, I ran into the 16F688 on SparkFun. This is a crazy chip -- the number of features packed into a tiny 14-pin DIP is simply incredible compared to where I came from. A couple key features that pointed me at this chip: UART on board, and about $2 per part. This allows me to do stuff such as communicate to serial sensors (such as the Zilog ePIR IR sensor), use bit-banging to communicate to I2C sensors (such as the MPR121 capacitive touch sensor), measure voltages for security systems and temperature (TMP36), ... and all in a tiny and cheap package.

Next up is programming the dang thing. I've got a programmer and my Macbook. This post will document the steps needed to get some code running on the PIC. (to help others, and if I have to retrace these steps in the future)

First, you will need the following software packages installed:

  • pk2cmd (download from Microchip)
    This uses a standard configure; make; make install
  • gputils (see their SourceForge project)
    I used "make mac105", then symlink'd "pk2cmd" and "PK2DeviceFile.dat" into my PATH
    Note: I did not setuid-root on the binary (as the docs seem to suggest). It seems to operate fine with my login id.
When I plug my programmer into the USB port and run "pk2cmd -P" it detects my PIC16F688. Woot!

And for a basic "Hello World" to test my setup, I wrote a "blink an LED" program. Download that and type "make install" and it should assemble and load the program onto the PIC sitting in your programmer. Yank out the PIC, wire up RA0 to an LED tied to Vdd (I used a 680Ω resistor), and apply power. The LED should blink at you.

Not that hard!

If it doesn't? Well. Not like you can bring up Visual Studio and debug this thing. The program works for me, and the wiring is dead simple, so I wouldn't know where to point you.

Next up: switch my blinky program to use the chip's Sleep Mode and interrupts [rather than a busy loop]. Less power consumption!