Saturday, August 22, 2015

My Google Code projects have moved

Back in March, Google announced that the project hosting service on Google Code was shutting down. I wrote a post about why/how we started the service. ... But that closure time has arrived.

There are four projects on Google Code that I work on. Here is the disposition of each one:
serf
This has become Apache Serf (this link might not work right now), under the umbrella of the Apache Software Foundation. Justin and I started serf at Apache back in 2003. Two people are not sufficient for an Apache community, so we moved the project out of the ASF. We had a temporary location, but moved it to Google Code's project hosting at the service's launch, where it has resided for almost 10 years. The project now has a good community and is returning to its original home.
(link to: old project site)

pocore
This is a portability library that I started, as a tighter replacement for APR. Haven't worked on it lately, but will get back to it, as I believe it is an interesting and needed library. I've moved it to GitHub.

ezt
This is a very old, very simple yet capable, and mature templating library that I wrote for Python. It is used in many places due to its simplicity and speed. Also moved to GitHub.

gstein
This is my personal repository for random non-project work. I open source everything, even if it might not be packaged perfectly for use. Somebody might find utility in a block of code, so I keep it all open. The code in this repository isn't part of a team effort, so I'm not interested in the tooling over at GitHub. I just want an svn repository to keep history, and to keep it offsite. For this repository, I've chosen Wildbit's beanstalk, and the repository has been published/opened.
(link to: old project site)
I'm sad to see Google Code go away, and I don't consider the above movements ideal. But it's the best I've got right now. Flow with the times...

Saturday, March 14, 2015

Sigh. Google Code project hosting closing down

Google has just let us know that Google Code's project hosting will be shutting down.

On a story over on Ars Technica, there were a lot of misconceptions about why Google chose to provide project hosting. I posted a long comment there, but want to repeat that here for posterity:

As the Engineering Manager behind Google's project hosting's launch, I think some clarifications need to be made here. 
In early 2005, SourceForge was not well-maintained, it was hard to use, and it was the only large hosting site available. Chris and I posed the following question, "what would happen if SourceForge went dark tomorrow". … F/OSS apocalypse. SF would take 10's of thousands of projects down with it. This wasn't too far-fetched, given the funding and team assigned to SourceForge.net at the time. Chris and I explored possibilities: provide Google operational support, machines, or just offer to buy it outright. … Our evaluation was: we didn't need to acquire SourceForge. We just needed to provide an alternative. Provide the community with another basket for their eggs. 
Myself and three highly-talented engineers put together the project hosting from summer 2005, to its launch at OSCON in July 2006. We let SourceForge know late 2005 what we were doing, and they added staff. We couldn't have been happier! … we never set out to kill them. Just to provide safety against a potential catastrophic situation for the F/OSS community. 
Did GitHub provide a better tool? I think so. But recall: that is their business. Google's interest was caretaking for the F/OSS community (much the same as the Google Summer of Code). The project hosting did that for TEN YEARS. 
I'm biased, but call that a success. 
There are many more hosting options today, compared to what the F/OSS ecosystem was dealing with in 2005 and 2006. I'm very sad to see it close down, but I can understand. Google contributes greatly to F/OSS, but what is the incremental value of their project hosting? Fantastic in 2006, but lower today. 
… I hope the above helps to explain where/how Google Code's project hosting came about.

Thursday, January 15, 2015

Disappointing

I've been reading Ars Technica for years. The bulk of what they do: I find awesome.

A recent article used the phrase "Climate Denial" in its title. To me, in terms of the scientific method, there is no such thing as "denial", but simply "critical" or "questioning" or "not convinced". "Skeptical", if you will. All of these labels are fine, as they acknowledge that the hypothesis in question (AGW) is being tested. But "denial" has been used to shut down conversation, as if critical examination is no longer allowed.

So I posted my thoughts, in the forum attached to that article, basically repeating the above.

Ars Technica appears to have disliked my points about questioning. and that falsifiability is no longer applicable to AGW. So they closed my forum post, marking it as "trolling".

The ridiculous thing is that somebody even replied to my post, pointing out "scientific consensus" on Wikipedia, yet that article specifically discusses that certain theories can never be proven. Only disproven (ref: falsifiability, above). So when you find a hypothesis in this pattern... the approach is to disprove.

But nope. Ars Technica shut me down.

I will still read you, Ars. I like your content. But when you shut down discussion? And call it trolling, despite some kind of rational basis, and an attempt at civil discussion?

No. That is wrong, and I have lost respect for what you do.

Sunday, October 05, 2014

Raspberry Pi and (lack of) I2C Repeated Starts

Just spent several hours digging into a communication bug between my Raspberry Pi and a SparkFun MPR121 breakout board. I found two core problems: the MPR121 requiring Repeated Starts in its I2C communication, and the RPi's BCM2835 not implementing them.

MPR121 Requires Repeated Starts


The MPR121 uses I2C to communicate with its host. There are over a hundred registers in the MPR121 that can be read. From a functional standpoint, it would look something like:
uint8 value = read_register(uint7 addr, uint8 which_reg)
On the I2C bus, the bits/frames look something like:
| Start | addr/W | which | R-Start | addr/R | data | Stop |
The Repeated Start allows the host to hold the bus during this "write which register we want, now read it" transaction. If a normal Stop-then-Start sequence was performed, then you have a time where the bus was released, allowing another I2C master to take control. That race condition could allow the other master to change the register-to-read. The original I2C master would then get bad data.

The requirement for a Repeated Start is eminently sensible in a multi-master environment, although the MPR121 documentation does not call out this requirement. My own experimentation and a bit of Google action confirms this. Sensible, yes, but poorly documented. (Though I have to say: the MPR121 doc and application notes are overall very outstanding! Usually, I use their I2C timing diagrams rather than the formal I2C specification)

It is also important to note that most environments are single-master, so a Repeated Start wouldn't be necessary, and the requirement is a potential burden upon the I2C master (compared to a standard write/read pair of operations).

BCM2835 Lack of Repeated Starts


My test code was using the Python "smbus" module to speak I2C to the MPR121 breakout. Everything was quite straight forward, and there are several tutorials and pages on the web to show you how to set up I2C on an RPi, and to use Python to control it.

But when I tried to read the "touched" results for the 10 pads (two bytes), I kept getting the same byte values. The first byte (in register 0x00) has the first eight pads, and the second byte (at 0x01) has the next four. But that second byte was always the same, and changed right along with the first byte as I touched various pads.

After thorough digging through code, trying alternative approaches, oscilloscope review of the I2C bus, etc, it became apparent that upon receiving a Stop condition, the MPR121 would reset the which-register value to 0x00. Thus, I'd tell the MPR121 "read from 0x01. Stop. give me the byte.", and it would always return the value from register 0x00.

After further investigation and reading: the BCM2835 does not implement Repeated Starts. There is no way for it to read from arbitrary registers of the MPR121. Some people have attempted gimmicks around the BCM2835's 10-bit addressing feature, but I'll avoid that.

However, you can read all the sensor data by reading two bytes, starting at register 0x00. I2C supports a block data transfer, so this is quite straightforward. In fact, the host doesn't ever have to say "start reading from 0x00" since that is the default. It can just issue a read for two bytes.

In order for an RPi to read arbitrary registers, it would be necessary to use "bit-banging" on the GPIO pins to manually run through the I2C protocol. Code out there exists, if this feature is needed.

Summary


In my research, I found there are a number of I2C peripherals that require a Repeated Start. Presumably for transaction purposes (as noted above). Most work just fine with a Stop/Start pair, which will work in the typical single master environment.

In my home automation scenario, the host will (always?) be a PIC16F688 microcontroller running my own I2C master code. Needless to say, it will incorporate the Repeated Start capability.

Hopefully, my research will help your own use of an MPR121, or the I2C bus on a Raspberry Pi.

Capacitive Touch Sensors, revisited

Last year, I wrote about my plan to use capacitive touch sensors in my house as "wall switches" rather than what you'd find in any sane person's house.

I got my first batch of sensor pad boards a year ago, but never got around to actually writing a post about them. The rather poor picture to the right shows the pads and the ground-hatch on the back. You can even make out an Apache Subversion revision tag on the silk screen :-)

It has a few problems, however: sizing is incorrect for our 1-gang electrical boxes, it is missing drill/mounting holes, and there are no cutouts so the backlighting can shine through. Whoops! When the new board is designed and back from production, then I'll get some good pictures posted.

So why post now? It's been a year!! ... well, I finally got around to pairing this sucker up with SparkFun's MPR121 breakout board. Connected that to a Raspberry Pi, and started Real Testing. I may have to tweak the pads and traces a bit for Rev2, but it is doing very well for the first iteration.

The largest obstacle to getting this functioning with the MPR121 was failure to use its "Auto Config" feature. Once I let the chip figure out what the hell it is connected to... it worked like a dream.

(see next post, re: problems talking to an MPR121 from a Raspberry Pi)

Friday, August 15, 2014

API Endpoints for Arduinos

As part of my home automation system, I need to connect "high-level" systems (such as the primary Linux server) down into the underlying hardware systems around the house. The PIC16F688 microcontrollers that run those systems are seriously low-level. Thus, I've chosen to place all of them onto an I2C bus(*) driven by an Arduino. Why? ... the Arduino has enough capability to mount an Ethernet port on the "high" side, and has built-in I2C support for the "low" side. It is a great mapping device between the complex systems, and the hardware systems.

With the hardware selected, and the wiring selected, it came down to protocol. How can that Linux server talk to the Arduino, as a proxy to the individual microcontroller critters splattered across the household? ... Naturally, HTTP was my first choice, as it allows all kinds of languages, libraries, and scripts to perform the work, and even some basic interaction via a full-on browser.

Digging into HTTP servers for the Arduino... Ugh. The landscape is very disappointing. Much of the code is poorly engineered, or the code is designed for serving web pages. During my search, I ran into a colleague's TinyWebServer. I'd call it the best out there for web serving (well-designed and well-coded), but is still overpowered for my purpose: mapping a protocol down to simple hardware interactions.

As a result, I designed a small library to construct a simple API endpoint on the Arduino. The application can register dozens of endpoints to process different types of client requests (62 endpoints are easy, more if you want to seek out the critical edges of allowed-characters in URIs). Each endpoint can accept a number of bytes as parameters, and can return zero, or an unlimited set of bytes to the client.

I have yet to "package" the Endpoint system, so any requests and changes are most welcome! I've got some documentation to write, along with incorporating feedback from others' and my own usage.

Would love to get some feedback! ==>  http://goo.gl/O5GJ3g


(*) and yes, I know an I2C bus is designed for 0.5m meter runs, rather than a whole house; bus accelerators, speed compensation, and other approaches "should" manage it. I'll report on my success/failure in a future post.

Friday, August 01, 2014

CanaKit PIC Programmer Reset

Earlier tonite, I scorched one of my 16F688 PICs in a stupid move. To see if it was recoverable/usable, I dropped the sucker into my CanaKit programmer and ran 'pk2cmd -P' on my Mac to check whether it could see/detect the PIC.

Bad move.

"No PICkit 2 found." ... and the red BUSY light just started flashing.

It took some research because CanaKit has almost zero documentation. The short answer is the programmer is a clone of Microchip's PICkit 2 Development Programmer/Debugger. The red flashy means the board couldn't load its firmware.

Solution:

Download the V2.32 firmware from the above page. Unzip the file. Then load it onto the board:

$ pk2cmd -D/path/to/firmware/PK2V023200.hex

It'll load the firmware, verify it, then reset the device. No more angry LED!

(I hope those who run into a similar problem will find this blog post, to more quickly reach a solution)

Friday, June 06, 2014

LED lighting control boards

Woot! I just ordered a set of PCBs for lighting control in the new house. Wifey would kill me if we had no lights, so the priority kinda jumped on this one ... (*Wifey comments:  I wouldn't kill you.  However, I consider it prudent to go to Home Depot and buy some cheapo mechanical switches just in case)

One of the more unique things about my (upcoming) house is that the bulk of our lighting are LED 24 VDC cans/pots/recessed lamps(*). Ceilings fans, chandeliers, and watercloset light/fans are standard high-voltage AC, but the bulk of our electrical fixtures are using these nifty low-voltage lamps. One neat thing is that the lamps don't require AC/DC stepdown transformers, so their cost is dropped, their reliability increases, and their heatprint is lowered.

The wiring becomes both simpler, and more complex in ways, and possibly cheaper depending on your choices. Simpler because you can run standard 14 gauge stranded speaker wire (cheap; flexible around those corners) instead of the heavier gauge Romex stuff. The lower gauge wire can save you quite a bit on copper costs, and you can lose the heavy insulation and concerns of high-voltage wires in your walls. But much more complicated because you need special hardware to run them ... a repair is no longer just a run to your local Home Depot.

In my case, much of the wiring in the house has been "home-run" back to my server room, so I ended up doubling my copper/install costs. All the control is localized to that room, which also means I don't have actual "switches" in my house, but just sensors (see Capacitive Touch Wall Switches). This leads to "how do I control these lighting circuits?"

Thus, my custom PWM Control Boards. They have an embedded microcontroller which can "do anything" with the nine (9) output channels. A higher-level "upstream" controller will tell it what actions to perform, when, and how, communicating via TTL Serial or I2C or what-have-you. The boards sink current, at whatever voltage (so I will also use these for my 12VDC LED light strips).

I've sent the PCBs off to ITEAD for production. Between now and when they arrive, I'll finish the microcontroller work that I've been working on. Assemble some parts from Mouser, plug it all together, and LIGHTS!

Theoretically.

Will report back, as I make progress...


(*) I decided not to name/link the manufacturer until I get experience and form an opinion. Just email me to ask, if you're interested.