PRNG: Pseudo Random Noise Generator

Installing Zabbix on Mac OS (Leopard)

2011-10-31T14:06:00.000-07:00

My friend Sam Ruby dabbles in a lot of technology, and he tends to do writeups on his blog as he experiments with the stuff. I figured to take a page from his book, and share my own issues/troubles getting Zabbix up and running on my MacBook (running Leopard).

I grabbed the 1.8.8 tarball and unpacked it. For my scenario, I needed the server, the agent, and the frontend (but not the proxy). For simplicity in testing, and because I don't need to monitor bunches o' boxes, I decided to go with SQLite for the database. Zabbix uses the standard "configure/make/make install" pattern, so no hassle so far.

Burp. The compilation failed. Investigating, I found that I needed to apply the patch from ZBX-4085. The build completed, so started to look at the frontend.

The Frontend is written in PHP, which is natively available (along with Apache) on my laptop. With some configuration, I got the frontend to load in my browser. There is a click-through GPL license (huh?) and then a really awesome page that checks your setup. I quickly realized that the builtin PHP was not going to work. Sigh.

I've got MacPorts installed on my laptop, so I just continued with that. Homebrew is all the new rage with the kids, but it doesn't have builtin recipes for PHP. There are a few out on the 'net, but I really didn't want to monkey with that stuff.

Lots of packages were needed: php5, php5-gd, php5-mbstring, php5-sockets, php5-sqlite3, sqlite3. A hojillion dependencies were installed, including another copy of Apache (sigh).

Reloading the setup page, it continued to say SQLite wasn't installed. Looking at the frontend source, it was using a function named sqlite3_open(). With some investigation, I found an email describing the SQLite interfaces for PHP. Zabbix was using an unmaintained version. Rather than monkeying with that, I just edited the code to use the preferred PHP SQLite interface, and filed issue ZBX-4289 to push my changes upstream.

Finally, I needed to tweak /opt/local/etc/php5/php.ini for the recommended Zabbix settings (after copying php.ini-development to php.ini). This included some timezone settings, timeouts, upload sizes, etc. The Zabbix setup page is quite good about guiding you here.

So I created my initial SQLite .db file based on the instructions from the manual and pointed the Zabbix configuration page at it (taking a moment to realize it wanted the pathname put into the database field of the form). The test connection worked and then Zabbix saved the configuration file into frontends/php/conf/zabbix.conf.php. It looks like there is a "download" option for that configuration file, which I presume appears when the conf directory is not writeable. The Apache server (running from MacPorts now, using the MacPorts PHP) was running as myself, so it had no problem writing that configuration file.

Next up: wrestling with the zabbix-server. The first annoying problem was that you cannot give it a configuration file in the current directory. It fails trying to lock "." for some dumb reason. Solution: pass an absolute path to the custom configuration file (the default is in /etc or somesuch, which I didn't want to monkey with). Getting the server running was very frustrating because it spawns multiple processes which communicate using shared memory. It kept failing with errors about not being able to allocate the shared memory segments. After some research, I found that Mac OS defaults to some pretty small limits. Given that I wasn't about to reconfigure my kernel (using sysctl and some recipes I found on the web), I went to rejigger all the various cache sizes in the zabbix_server.conf file.

It ended up that I had to drop all the sizes to their minimum 128k setting: CacheSize, HistoryCacheSize, TrendCacheSize, HistoryTextCacheSize. Each were set to 131072. Finally, the server started. Whew.

When I returned to the frontend to "Finish" the installation and bring up the console... it hung. No response from the server. Huge sigh. With a bunch of investigation, I found that something was holding an exclusive lock on the whole damned SQLite file. Nothing else could write to it (and it seems the frontend likes to test its writability by creating/dropping a dummy table).

Fuck. Time to scrap the whole damned "simple SQLite" idea. Fine... I've used MySQL before, so I went with that. Back to MacPorts to install MySQL, the server, and the PHP driver for MySQL. Then I fired it up, created a "zabbix" user, loaded in all the tables, and zapped the zabbix.conf.php file to trigger reconfiguration (after noting to restart Apache to pick up the PHP changes).

The frontend looked happy now, so I tweaked the server's configuration file for MySQL and restarted the server. No workee. Damn. Forgot to reconfigure the server using --with-mysql=/opt/local/lib/mysql5/bin/mysql_config. After reconfiguring, the link failed with unsatisfied references to iconv(), iconv_open(), and iconv_close(). The MySQL interface in the server needs these for some UTF-8 conversions. The builtin Mac OS libiconv should work, but my MacPorts copy of libiconv was interfering, and these functions are named libiconv(), libiconv_open(), and libiconv_close(). My patience was ending, so I was not about to delve into autoconf bullshit and conditional compilation and all that. I simply edited src/libs/zbxcommon/str.c to call the libiconv* versions of the functions. The compile and link succeeded, and I re-installed the newly built server.

Yay! The server restarted, and the website loads up with a nifty little default console.

After a day to get this sucker installed, now I gotta start figuring out how to use it. Oh, joy.

I hope this post will help some future person treading these waters. Good luck!

ps. I may have missed some steps or packages to install or whatever. YMMV, but I think that I've got most of it down. Zabbix is supposed to be some hotness, and I do like its custom agent capability. But hoo-wee. Not a simple package to bring up (I hope it will be easier on a recent Ubuntu, than it was on my creaky Leopard install).

Blast from the past: removing the GIL

2011-08-15T09:54:00.000-07:00

Way back in 1996, I created a patch to remove the GIL from Python's interpreter (version 1.4!). Dave Beazley just picked up the patch and tore it apart, and writing a fantastic blog post. It is quite nostalgic for me, from back in the day when I was working at Microsoft on their electronic commerce efforts.

[ I commented on Dave's post; it provides some context that you may also be interested in reading ]

Open Languages are Not Required

2010-11-29T13:50:00.000-08:00

I just posted again to Apache Asserts on Computerworld UK: Open Languages are Not Required.

And please note that I'm speaking primarily to enterprise (internal) software developers, who are the vast majority of developers on the planet. They shouldn't really have to worry about the language that they use for their development. Having an open language is critical for us FLOSS developers, but that is an entirely separate discussion. (hat tip to webmink, to clarify my point here)

Note: the publish date is wrong (says last month); dunno what's up with that.

Update: corrected link after the publish date was fixed.

Are You An Open Source Friend?

2010-10-29T13:08:00.000-07:00

The Apache Software Foundation was invited to find some people for Computerworld UK to write for a new blog named "Apache Asserts". Myself and a few others were selected to post our thoughts on open source, the enterprise, and whatever else we may find interesting.

My first post has been published... check it out!

Android uses Java? Um... no

2010-08-15T21:39:00.000-07:00

I've seen a lot of misinformation over the weekend, talking about the Oracle/Google lawsuit. Many of these blog posts and article talks about how "Android uses Java". Heh. That simply isn't true.

Android applications are written using the Java programming language. True. But those applications run on the Dalvik virtual machine. Not the Java virtual machine. Source code is owned/copyright by the author and is entirely unbound from any intellectual property concerns based around the syntax/grammar of that language.

Class libraries? Not Java either. Much of the core libraries come from Apache Harmony, and the rest are libraries that Google wrote. Given that Apache was never provided access to the Java Compatibility Kit, Harmony is not labeled as "Java-certified". Also note that Harmony is a clean-room implementation of the Java class libraries.

So, people: stop saying that Android "uses Java". It doesn't.

(obviously, some of these various components may trample on Oracle's patents; I have no idea, and that is an entirely separate question)

Outcomes

2010-08-04T23:46:00.000-07:00

Nights out with a friend can be quite interesting. Especially if they are single and "looking". I've found there are generally three possible outcomes with these nights out:

The Cock-Block.
Your friend is trying to hook up or otherwise get especially friendly with somebody, but you monopolize the "target's" attention in some way to distract them from your friend's intent. Obviously, this outcome is "poor", unless you're some kind of dickhead that doesn't want your friend departing early with the target. Quite selfish, to try and keep them out with you. Of course, there are all sorts of minor rules variants here, that are rather crass: e.g if you're both interested in the target, who steps forward, who holds back? It's simply best to avoid this scenario because it never turns out well.
The Wingman.
Oh yah. We all know this one... the friend who props up the other and makes them ever more desirable. Talk up their strengths, ensure that the person-of-interest gets excited to know more about your friend. This is the ideal outcome, especially if they make some kind of lasting connection.
The Bus-Tosser.
This isn't nearly as bad as the Cock-Block, but your friend isn't going to be all that happy with you. At least for a short while. This is where you think your friend is interested in somebody, so you move into Wingman mode. Provide lots of opportunity for the two to talk and hang out, provide some good commentary, etc. Like any good Wingman would do. But afterwards, you find out your friend was not interested. At all. This is the "thanks for throwing me under the bus" maneuver, putting your friend into harms way. Especially if the purported target is interested and giving undue attention to your friend. ... Thankfully, in the long run, this provides lots of laughable material for how you sucked as a myopic Wingman.

I think the best answer all around is to simply go out and have a great time with your friend. Anything that will involve a possible third person can fall into a poor outcome, or simply distract from an awesome evening with a friend.

svn stash

2010-05-19T11:53:00.000-07:00

This is the third of (at least four) posts in my miniseries about Subversion's next-generation working copy library. See the introduction and what we're doing to fix things.

Once we have this fancy new code, it will provide a stable and robust base for building new features. The DVCS systems have done a great job exploring new areas and needs of version control users. One feature in particular is called stashing or shelving (see "git stash" and "hg shelve").

For those not familiar with the stash concept: consider the scenario where you've been doing some work, and a high-priority bug arrives, needing to be fixed right away. Classically, a Subversion user would check out a fresh working copy, fix the bug, perform the commit, and go back to their work in the original working copy. Instead, when using stash, it takes all of your current work and sets it aside, leaving you with an unchanged working copy, ready for your bug-fix work. After your commit, you retrieve the changes that were stashed. The presumption here, of course, is that stashing is a much faster and simpler operation than setting up a new working copy.

We'll be able implement this feature quite easily using the WC-NG datastore. It will take just a few operations:

preserve all metadata about local changes
place a copy of each locally-modified into pristine storage, recording their SHA-1 key with the stashed metadata
revert all local changes

Since the metadata is recorded in a single SQLite database, step 1 is "simply" some copying of those changes off to a separate set of tables. The pristine storage is a generalized mapping of SHA-1 keys to file contents that we'll be using for storing more things (such as merge sources, pending conflict resolution), so it can easily hold stashed items. And step 3 has been in Subversion for a long time :-)

Recovering the changes from the stash is effectively running a big "svn merge" operation. The merge is required because you may have made other changes to the working copy (your bug-fix) and/or updated to the latest revision.

Other features, such as multiple stashes, management of those changes, applying subsets, and whatnot would be added, too. The feature set has not (yet) been designed, so I have no idea what is required or how we would present this to our users. We'll definitely be looking at git and hg as we explore the needs around stashing/shelving.

"When?" is your next question, I'm sure :-) ... Well, we're releasing WC-NG in Subversion 1.7. That will probably happen this fall. We want to get those changes out the door since that will mark 18 months of development time. WC-NG is a feature in itself, and we want to get it into people's hands without further delays [waiting for additional features]. After that, I'm interested in adding stash support (and a "checkpoint" feature (described in my next post)). So let's say stashing will appear in 1.8 which should be released around this time next year.

Heading to Berlin!

2010-05-05T07:07:00.000-07:00

In June, in Berlin, elego is hosting a "Subversion Day", along with workshops and a hackathon/sprint. And with great thanks to elego, I will be able to attend and contribute to the event. I'll be in Berlin from June 9th through the 14th.

As always, I'm looking forward to meeting up with my fellow Subversion developers, but there are quite a few others in Berlin that I want to spend time with. Torsten, Valerie, Erik -- I'm looking at you! Julian: road trip from Münster? Who else am I missing? Torsten says that he'll arrange for one of the regular Apache-people dinners. Want to join?

WC-NG Changes

2010-04-26T18:19:00.000-07:00

In my last post, I described how libsvn_wc had become brittle and hard to manage. The WC-NG process is working to solve that problem, and though we're not yet done, I believe we're on the right path.

The basic question that needed answering is, "where did we go wrong?" While version control is a hard problem (especially if you version directories!), it does not inherently lead to a brittle library. Somewhere, we had gone wrong in the design, the data model, or simply the implementation.

Before I had started working on the problem (almost) two years ago, one of the Subversion developers (Erik Hülsmann, I believe) laid out his thoughts for a next-generation library. In those notes, he postulated on what I now call the Three-Tree Model:

the tree you checked out
the above tree, plus structural changes (add, delete, move, copy)
the above tree, plus content changes (file edits, property edits)

Any working copy operation generally affects one of these trees. svn update and svn switch work on the first tree. svn add and svn merge modify the second tree. Your editor and svn propset affect the last tree.

This was the key insight. In our "wc-1" implementation, the svn_wc_entry_t structure blended all three trees together. Making a change to that structure could have been operating on any of the three trees depending on its flags. Its checksum field could correspond to a checked-out file, or a locally-copied file. To determine, you had to look at the schedule field and the copied field. And hell will rain upon you, should you mess up the flags or forget to check one.

For WC-NG, we have built a new data storage system with an API designed around this three-tree model. This has isolated our storage mechanism behind a solid encapsulation (wc-1 code had too much knowledge of the old "entries" storage model). Operations are now understandable: "copy nodes in the restructuring tree" instead of "set entry->schedule".

This new storage subsystem could produce an entire post on its own. It is radically different from the prior model (a single .svn subdir at the root of the working copy and SQLite-based storage). This is causing huge challenges in upgrades/migrations to the new format, and backwards compatibility for our classic APIs.

Another radical change was our move to using absolute paths to refer to items. The prior model used an "access baton" which implied a relative directory, along with a path relative to that baton. These relative batons and paths caused enormous problems because it led to the question, "relative to what?" In most cases, the answer was "the operating system's current working directory," which is a terrible basis for a deterministic API. In switching to absolute paths, this rendered the access batons obsolete. Since they were a core part of the public API for libsvn_wc (not to mention the widespread internal changes!), this has had a huge impact on the API and its users (such as Subversion's libsvn_client library and its command-line tools).

These two items (data model and absolute paths) are the core changes in WC-NG. The ripple effect from just these two items is immense. We will need to rewrite almost every one of the 40,000 lines of code in the library. And given our incremental approach, many of those will be changed multiple times. We're a solid year into this (although we saw downtime last fall due to our move to the Apache Software Foundation), and we probably have another several months of basic grunt work ahead of us. Stabilization and testing will put our 1.7 release into late summer or possibly this fall.

I could really go on and on about this stuff, but I hope this post provides some basic background on the WC-NG efforts. Please feel free to post any questions (I have no idea what aspects you may want to hear more about!), and I'll work on answering them.

What is Subversion's WC-NG?

2010-04-13T08:25:00.000-07:00

When I started working on the Subversion project (again) back in August 2008, I wanted to do something that was interesting, technically challenging, and important to the project. For many years, the developers had been complaining about the "working copy" (WC) library. This library was one of the first that we worked on back in June 2000, and had grown (ahem) "organically" over the following eight years. By "organically", I mean it had become a rat's nest of brittle code. Hard to work with, not fun to modify, and difficult as hell to build new features reliably. Over such a lengthy time frame, most actively-developed code tends to end up like this, unless you work real hard against it.

In 2000, we didn't even know all the requirements for the library. Nobody had ever done versioning for directories. Just files. In fact, I think that Subversion may (still) be the only version control system (VCS) out there which treats a directory as a first-class object. It is a very difficult problem, along with being able to work with only pieces of your repository (which leads to "mixed-revision" working copies; something that distributed VCS systems like Git and Mercurial don't have to deal with, much to their enjoyment!).

So we started the library and figured things out as we went. Then it was too slow, so we added stuff to make it work faster. Then we added more features. And revamped some stuff to make it go faster again. More features. And even more.

By this time, the library had become brittle. Adding a feature usually broke something else. There were too many considerations, and internal layering/hiding was not present. Everything could, and did, manipulate a public structure (called svn_wc_entry_t). If you didn't do it right, then something broke. And there was some very deep and hard to understand relationships in the handling of data in that structure. Forward progress was being stifled.

The developers had been talking about fixing the WC library for years, but most of them had other priorities. I had no such baggage, and the WC problem had everything I was looking for: interesting problems to fix, challenging to accomplish, and very important to Subversion's future. Some people had already written up some thoughts on a next generation of the WC library, calling it "WC-NG". After I started digging in, and some other developers joined, the project took on the WC-NG title in earnest and in day-to-day use.

WC-NG is Subversion's name for an entirely new working copy library. We have a new design, and we're incrementally rebuilding the library towards this new design. Due to stringent backwards-compatibility requirements, and the complexity of the system, we cannot simply "rewrite from scratch". This effort is the current focus of our upcoming 1.7 release, and it will provide a Subversion client that will be vastly faster, much more robust and capable, and provide a solid foundation for new features.

In future posts, I'll provide some more detail about WC-NG's design (and how the original WC was broken). I also want to talk about a couple of these new features that will be implemented upon this new foundation. Stay tuned!

Version Control

2010-03-29T17:40:00.000-07:00

Last week, I spent some time in NYC with friends of mine talking about Subversion. The conversion focused around the long-term vision and roadmap. I'll post more on that soon, along with some specific ideas on how I'd like to build some features that other version control (VC) systems have demonstrated as useful and demanded by users.

For this post, I wanted to share a discussion document written by Martin Fowler. This is one of the best, level-headed comparisons between Subversion, Git, and Mercurial. I believe these are the Big Three VC systems that the industry will be using over the next decade, and to see a useful discussion, absent of rhetoric, is very encouraging.

Are Insurers the real Bad Guys?

2010-03-15T04:01:00.000-07:00

"In 2009, the largest 14 insurers had profits of roughly $9 billion; that approached 0.4 percent of total health spending of $2.472 trillion. This hardly explains high health costs." -- Robert J. Samuelson, Washington Post.

yet...
"the five largest health-insurance companies racked up combined profits of $12.2 billion, up 56 percent over 2008" -- Noam N. Levey, The Seattle Times.

Which is right? But then again... who cares? Either value is small compared to the total costs (given by Samuelson, and Wikipedia). So why ostracize and penalize insurers? They're just the middlemen. What am I missing?

My belief is that a lack of cost/value/benefit feedback is the basic problem. There is no pricing-pushback from the consumer, so the prices escalate without control. (you ever see the prices for toothbrush, toothpaste, or slippers in a hospital stay?) We've seen similar problems with higher education where financial aid covered "any" gap between ability to pay, and the amount requested, so the schools simply requested more.

So why all the rhetoric against insurers? Removing all of their profit will reduce overall health care purchasing by less than one percent. It appears they are a scapegoat, to be blamed in lieu of proper analysis and approaches at reducing health care costs.

Columbia Code Camp, and my Subversion talk

2010-01-25T13:58:00.000-08:00

I'm going to be at the Columbia Code Camp this-coming Saturday (January 30th). If you're in the Columbia, SC area, then come find me. Join in the Code Camp, or we can meet in the evening to share some beers.

I'll be talking about the rewrite of the working copy library in Subversion (see the "Rebuilding Subversion's Working Copy Library" on the sessions page).

100k apps ... so what?

2010-01-06T14:26:00.000-08:00

Why do people keep saying that the iPhone App Store has an "advantage" over others because it has 100,000 applications?

Would it still have an advantage at 90,000? 50,000? How many does it really take?

A month or so ago, Apple wiped out all 1000 applications from a single vendor. Did anybody miss those applications? Probably not. So how many other thousands could be wiped out without taking a hit to its success?

Google stopped putting the "pages indexed" on its front page many years ago because it realized a key principle: the value is in the results, not the quantity.

The Android Market is definitely behind -- it is missing some nice applications. But not many! All the apps that I used to have on my iPhone are now available on my Android phone. Thus, Apple's App Store has zero "advantage" for me. How many other people are like me? Or conversely, how many people want and use all of those 100k applications?

I think the conversation should be rephrased into "do the apps exist, that a typical consumer wants?" rather than focusing on a mere count. That is the success of any app store.

How to Screw Your (Open Source Software) Customers

2009-12-13T14:40:00.000-08:00

I've been reading with interest (and amusement) a lot of the brouhaha around Oracle's acquisition of Sun, with specific relation to Sun's ownership of the MySQL database. The first salvo came from Stallman's concerns about MySQL being licensed under the GPL (ha!!) (also see Matt Asay's review of that event). Next up was Monty's discussion of MySQL's licensing, followed by yesterday's request to "save MySQL".

My opinion: the GPL has screwed customers worldwide.

All those customers who believed they were buying an Open Source database were misled. They bought a proprietary database, plain and simple. The GPL version was not available to them because they needed a license that didn't not require them to GPL their own software. They needed a version that could be combined with their own proprietary software.

Monty is entirely correct -- Oracle can totally screw them. It could raise licensing prices or even discontinue that licensing. It can cease development and third-parties could not fork and continue development under the MySQL name (it's trademarked).

But the fault lies in MySQL AB's choice of a business model. They chose to develop the software under GPL and require customers to pay for a less-restrictive license. Now that the software has moved beyond their control, their customers are subject to the whim's of Oracle. I'm making no predictions of Oracle's behavior, but there is no doubt who holds all the cards.

Much of the problem lies in the chosen business model (GPL with pay-for-permissive), but the enabler here is the GPL license. It gives undue control to the copyright owner. When that owner changes hands... the customers lose.

Contrast with a permissive license, such as the Apache License. Absolutely none of this discussion would have happened. Everybody would have rights to use the software, in exactly the same way. Customers could not market under the (MySQL) trademark, but they wouldn't care -- the database is a component in a larger system they are delivering to their own customers. There would not be a need to discuss the internal pieces, though they could still say "this produce includes MySQL(tm) software" and the trademark owner could not do anything about it.

If you are developing Open Source software for use by your customers, and you care about those customers, then choose a permissive license. You have no idea who will own the copyrights in the future (shoot... you might be trying for an acquisition!).

Sure, a permissive license will prevent this kind of business model, but we've now seen the dangers of the model.

Find a better model rather than hanging your customers out to dry.

Update: PJ has written a great article about this topic over at Groklaw

SubConf and ApacheCon

2009-10-26T12:02:00.000-07:00

I'm about to head to the airport, leaving for Munich, Germany. SubConf is happening there this week. Last year, I gave a keynote; this year, I'm not giving any talks, but simply attending to work with the other Subversion committers. CollabNet, elego, and WANdisco have all pitched in to make my trip possible (thanks, guys!). Their businesses are built on Subversion, and getting committers together is a great way to make Subversion better, and grow their business.

I guess that I don't post enough about my hacking here, but Subversion has been occupying all my coding over the past year+. I'm working on rebuilding its "working copy library" -- the library that manages all those ".svn" subdirectories you see when using Subversion. There are several other people in the community that are working on this change, too. It is a huge amount of work, but will make Subversion much, much faster. The new library will appear in Subversion 1.7, which should be released early next year.

After SubConf, then I'm flying off to San Francisco. Going to see a bunch of people over the weekend, bar crawl on Halloween, and then spend next week at ApacheCon in Oakland. This is going to be a big event, as we're celebrating the ten year anniversary of the Apache Software Foundation. Huge milestone!

Posting about the Xbox again

2009-10-23T15:56:00.000-07:00

For a couple years, I posted quite regularly to my Xbox blog. Then... nothing for pretty much the past three years.

Well. I've got my Xbox plugged in again. I've got the time. And I've been playing.

... so I started posting again. Go check it out, if you're a gamer.

OpenBSD and (oops!) NO threads

2009-09-14T08:14:00.000-07:00

Wow.

I just found out that OpenBSD only has user-space threads.

User. Space.

This came up because stdout doesn't work quite right, apparently. My friend says that OpenBSD sets it to non-blocking. In general, non-blocking means "try to write to , and what isn't written... try it later." But every program I've ever seen assumes stdout is blocking, so in the OpenBSD environment the unwritten data will just get dropped on the floor. Answer: build as an unthreaded application so stdout will work appropriately. Are you kidding?

How can an operating system claim to be Modern, yet lack something as basic as kernel threads?

Insanity.

Google Books

2009-08-29T10:05:00.000-07:00

There is an interesting furor that has been building over the past six months about the Google settlement with authors/publishers, which gives them the rights to digitize and make available books on the web. Then we have the Open Content Alliance that is "against" this process. They complain that it is giving Google some kind of monopoly over online books.

Google went through a long lawsuit and then paid a settlement to receive those rights. Why is the OCA is complaining? Why don't they just pay the same license? Why don't they just set up their own online book system?

Pah. That would be too easy. Instead, they want to complain and drag back the one company who is trying to advance online works.

They should step up, not hold back.

I Don't Believe in Obamacare

2009-08-05T05:30:00.000-07:00

For a while now, I've been having issues with the idea of "Big Government" and it taking part in the health industry. There are lots of problems with health in the United States, but government bureaucracy is never a solution.

Consider the simple economic question of "how do we pay for this?" The answer has been various taxes on different entities. But that does not create value, it merely redistributes it. And any process of redistribution has overhead costs, and experience further tells us that government is quite inefficient doing this.

In short, we're simply talking about a productivity hit. Where is the additional value?

At a town hall meeting in Philadelphia, a woman had this to say:

I look at this health care plan and I see nothing that is about health or about care. What I see is a bureaucratic nightmare, senator. Medicaid is broke, Medicare is broke, Social Security is broke and you want us to believe that a government that can't even run a cash for clunkers program is going to run one-seventh of our U.S. economy? No sir, no.

She sums it up quite well. Our government has shown that it does not run programs well. The health industry is a huge part of our economy. I certainly do not trust it with the government.

I believe much of the problem in the health industry is in the supply chain. Huge markups exist at all points along the chain, and those costs are passed along to the insurance companies (not people!). Further, these supply chains and the resulting use and treatments are opaque, and (thus) resistant to careful analysis.

Many years ago, when I was not employed and had no health insurance, I was going to be billed $1500 by a hospital for a treatment. When the doctor found that I was uninsured, it was dropped to $1200. What does that tell us? There are costs in the chain of treatment that exist specifically to be paid by insurers. Rooting out inefficiencies like this would go a long ways to improving our health system today.

The Baen Free Library

2009-06-23T11:24:00.000-07:00

A couple years ago, I was stranded at home with a broken ankle. Since I had just purchased my loft, and was only staying there a couple days a week (the rest of the time at a crashpad near work), I had no cable TV. Shoot... not even a TV. No internet either -- I was "borrowing" connectivity from a neighbor, though it was a crappy connection.

How to pass the time?

I figured reading a book would be good, but I certainly wasn't capable of getting myself to a bookstore. Then I recalled the Baen Free Library. I'd run into it, in some story about how publishing free content can actually help your business. Eric Flint wrote a page about why they are giving away books for free. The man gets it.

So... off I went. Through the crappy connection, I was able to get several books downloaded, and settled in to read them. One of those books was 1632, a story about a West Virginian town picked up, in whole, and thrown back in time and relocated to Germany. It is the first in a long series by Eric Flint. And it is really, really good.

And this is exactly why I say that Jim Baen gets it. After reading all the free books in the series, I went on to purchase another eight books or so. Baen Books made money where they otherwise would not have.

I have recently returned to the B.F.L. in a quest for books for my Kindle 2, which I got back in March. Yes, they offer their books in Kindle format. Lately, I've been reading the books from The Belisarious Series. The first four are available for free in the B.F.L., and the last two are free on Baen's Fifth Imperium website.

I'll end with a note that the B.F.L. is housed on webscription.net, which sells quite large number of books, and all available in Kindle format. There are even some books that Amazon doesn't provide in Kindle format (like the excellent Paladin of Shadows series by John Ringo).

[Update: Eric Flint wrote that web page, not Jim]

My Bacon Number

2009-06-19T18:59:00.000-07:00

No, not the tasty pork product.

The actor...

I've just discovered that Wil Wheaton has a Bacon Number of One, after appearing with him in She's Having a Baby.

Back in 2004, I played poker with Wil after his keynote speech at ApacheCon in Vegas.

Thus... my Bacon Number is Two.

Modern Day Racism

2009-05-13T04:11:00.000-07:00

I twittered a pointer to a story about "Buy Black", and noted that it is just racism under a different label. Twitter is not a very good outlet for really discussing something, though.

I think the key question here is, "should business ownership diversity match the diversity of the population?"

If you answer "yes", then we need to fix these inequities, too:

overrepresentation of Mexican ownership of Mexican restaurants
too many Chinese-owned Chinese restaurants
way too many asian-owned dry cleaners
what's with all the Jewish diamond merchants? Get whitey in there!

I could keep going, but I hope you see my point. Why must business ownership match? Aren't you being racist by only buying from a black-owned business?

Now, granted. I can see supporting black ownership if there is something truly keeping them "down". But I would counter that Obama blows apart the notion of blacks being held back by "The Man". Hell... a black guy is The Man.

If you want to fix an inequity, then your time could be used to fight against the glass ceiling endured by women.

(and before anybody comments: yes, I use "black" just like I'm "white"; most Americans were born in the US and have nothing to do with Africa)

The Licensing Conspiracy Idiots

2009-04-23T12:00:00.000-07:00

When I worked at Google, one of the things that I got to do was choose the licenses that were allowed on Google Code. A quick consultation with Chris DiBona to ensure that I wasn't totally in left field, and that was it. We launched Google Code with seven licenses, and added an eighth (GPLv3) when that license was released.

I'd been observing the use of licenses on the site, and saw that the uptake on Mozilla was dead last. I mean really last. Less than a few percent. After many months of saying "we should axe that license", I actually dug in and killed it off.

And then the conspiracy nutjobs came out of the woodwork. Saying Google had some Master Plan and how it didn't like certain licenses. That it was trying to kill off licenses to further its own ends. Of course, I'd seen this utter nonsense before, so was ready for it.

The simple fact is that my goal was to reduce the number of licenses that people use for their Open Source projects. When you combine multiple software packages, the combinatorics around licensing just becomes insane. "Can we combine these? What are the overall requirements? What notices are necessary? Which pieces do we need to provide source for?" All of these questions become very difficult for the end user, and for packagers/redistributors.

Our job, as authors, is to help these people. Open Source and Free Software should be easy. We should not be wasting so much time on licensing (our own, or the people who consume our software).

In an ideal world, I believe people should choose the Apache License, v2, or the (GNU) General Public License, v3. Pick one based on your philosophy.

Middle-of-the-road licenses like MPL, EPL, and CDDL are wishy-washy. They can't decide to be permissive, or to maintain Freedom. Choose a philosophy.

And the experience on Google Code showed that people simply do not tend towards these wishy-washy licenses. Instead, they are chosen for your depending on whether you play in the Mozilla, Eclipse, or Sun playgrounds. Those licenses are not universal. So I took a step to help in that direction. Worst case, a few percent of projects would be hosted somewhere else with more choice around licensing (such as SourceForge).

The idiots barking about the Affero license totally missed the point. No conspiracy. They just had no numbers, and were fragmenting the licensing market. Go host your projects elsewhere. You never had and never will have an "entitlement" to host on the service of your choice. That is up to the service provider, and Google Code chose the path of limiting license choice.

Anyway. If you want your software to be adopted and used, then pick ALv2 or GPLv3. That keeps things very easy for your users.

Now go write some code.

(after I left, somebody added MPL back, along with EPL and CDDL; ah well... their choice now!)

Europeans Smoke Too Much

2009-04-22T16:17:00.000-07:00

I've become very used to heading into a bar in Europe and wading thru smoke. I keep a machete on hand for just that purpose -- it is that thick.

And here is my obligatory stereotype comment: European is full of smokers. Fine with me. It's the culture. I'm not about to tell anybody to stop. But I am happy to return to the United States where the anti-smoking peeps have dug in and keep the air smoke-free in restaurants and bars and whatnot. Here in EU? I just deal. No complaints.

Today? Oh, dear. I've realized that the people here have elevated smoking to a whole new level. Not quite to Millenium levels, but wow.

I was out walking through the nearby forest / park. Some serious stuff here, with big trails for walking, biking, and even horses. Today, I finally saw some peeps on horses. They're at walking speed on the trail, going the opposite way. And guess what?

They're smoking. On the back of the horse. In the beautiful outdoor forest.

Sigh.

I hope these people will wake up one day and realize the world is a much more beautiful, clean, and clear place, when the haze of their smoke is not filling their lungs and face. I don't mind (lots of fresh air out there), but I feel bad for them ... not realizing what they are missing.

Commit Access: It's a Social Problem

2009-02-28T09:47:00.000-08:00

Many proponents of distributed version control systems (DVCSs) say the biggest advantage is that anybody can create a branch and begin working on a project. Whereas, for a centralized system (such as Subversion), the would-be contributor needs to have commit access before they can contribute.

Let's walk through this.

Obviously, this contributor can grab a tarball, make their change, and send a patch file to the project's mailing list. No commit access is required to do that. Why a DVCS, then? Well... the DVCS simplifies the retrieval and application of the patch (by the project's developers, or third-party users of the project). The contributor also gets use a version control system while developing the patch, which I'll just axiomatically state as a Good Thing.

Okay. So if they don't have commit access, then a DVCS is very handy. What would the scenario be if they did have commit access? The contributor could develop their change on "trunk" or on a branch. We've already stated this is a would-be contributor -- not one of the regular developers who already has commit access. It really doesn't make sense for this person to modify trunk directly, so let's just say the work is being done on a branch.

So why would this potential contributor not have commit access? Really? All their work is happening on a branch. It isn't like they're going to mess up the project from over there. They're going to generate some commit emails, sure, but maybe the other developers could then provide pointers, assistance, and feedback earlier than if the contributor had arrived with a patch, as a fait d'accompli. This is source control, people. Anything changed can always be reversed. No permanent harm is possible.

So why do potential contributors not receive commit access to a branch, as soon as they ask for it? For social reasons. It certainly isn't technical. Projects have an us versus them attitude, and they don't get to commit to our repository.

For reference, I'll note that the Apache Software Foundation provides branches to Google Summer of Code students. These students arrive with no credentials, they get a branch, and then work on their code over the summer. When they are done, the work can be merged back to trunk, if it is acceptable. It has worked out very well for all involved.

In the Subversion project, we set up branches for developers to try out their ideas. We say these developers have "limited commit access" rather than "full commit". I'll also note that there are no technical limitations on their commit access. Those developers could commit to trunk if they tried. But social restrictions prevent them from doing so. We've never had a problem with rogue developers, since it is so easy to undo any mistakes or intentional harm, and to remove their access.

In this respect, DVCSs are simply a workaround to social barriers put into place by projects. They do not address the core problem: projects should be inclusive rather than exclusive.

Answer One, Ask One

2009-02-26T18:17:00.001-08:00

whurley is trying out a fun series of question/answer posts on his site. He asks somebody a question, they answer, and get to ask him a question. He asked me whether distributed version control systems (DVCS) are bringing about large changes in the Open Source ecosystem. I asked him about Netbooks.

Check out the questions and answers.

Giving Back

2009-02-18T12:30:00.000-08:00

Every now and then, I run into a meme in the Open Source world about "giving back". That if a person or company uses some Open Source software (OSS), then they are now obligated to give back (in the form of code, patches, money or whatever).

Why is this?

I write OSS because I enjoy coding. I have no need to keep it to myself, so I share it with others. But I don't attach strings to that sharing. That is why I've released it as Open Source. If I wanted strings, or expectations of "giving back", then I should have applied a restrictive license.

Question to other OSS authors: do you expect your users to give back? Why?

Why do we allow this "give back" meme to continue? If we are truly sharing our code, then it should not be with these kinds of expectations.

Westvleteren 12

2009-02-18T00:58:00.000-08:00

I've blogged about this beer before. Even used a picture outside of the brewery for a profile picture. Fabulous stuff, as many reviewers would say, and a couple sites call it the best beer on the planet. I'm certainly a fan.

It is just so hard to get. They don't sell the stuff into the retail chain. You can't simply go to your neighborhood bar and grab a case. This is fine -- the monks are only making as much profit as they need, rather than being sell-outs. But it makes it hard as a consumer.

So... Answer is that the abbey will sell you cases. But you have to call ahead of time. And call. And call. AND CALL. The problem? The line is busy. There are so many people trying to get their beer reserved, that you need to be hovering over your Redial key.

This morning, I finally got through. Talked to the monk and reserved three cases of the Westvleteren 12, to be picked up next Tuesday. It'll be a nice long drive through the Belgian countryside, but it is worth it.

You know... Belgium does have its high points!

Taxes, Again?

2009-01-30T20:15:00.000-08:00

First, there was Geithner with his late taxes. Big story on Capitol Hill. "Oh no! He doesn't pay his taxes properly! He can't be Mr Finance!"

Now we have Daschle and an amended return for taxes due to "value received" in terms of a car and driver. "Oh no! A second failure for Obama!"

You know what? Fuck all that. The root of the problem here is not people trying to stiff the government. It is not people trying to get out of taxes. The real problem? Taxes are too god-damned complicated.

These freakin' lawmakers want to point fingers at these candidates and hang them out to dry for tax problems. It has nothing to do with the actual payments... they're just being political bitches. Find a wound they can rub salt in. Grind it down.

You. Me. Those candidates. Any damned human in the United States is going to make mistakes on their taxes. And you know why? It is those same bitchy congressman poking at each other. They passed the laws. They made Tax Law this complicated.

Taxes are an Industry. You wanna know how much H&R Block makes? All those independent tax accountants and attorneys? Intuit with their TurboTax. And helluva lot more than I can mention here. It is a bajillion dollar industry. And that industry has lobbyists. They want to keep the taxes complicated. Those congressman are not going to make this crap any simpler.

Result? People make mistakes. Even those candidates. They're human.

(and no, I have no opinion on them; I'm just pissed off at the hypocracy of Congress pointing fingers at the candidates about their taxes, when it is Congress that made it impossible for any rational human to properly follow the tax code)

(real answer: flat tax. seriously.)

The History of Python

2009-01-28T06:45:00.000-08:00

Guido has been kind enough to allow me to post on a blog about The History of Python. I've just published my first post, talking about Microsoft shipping Python code back in 1996.

From the Wreckage Known as Perl...

2009-01-13T02:54:00.000-08:00

Many years ago, I was setting up cvsweb to display CVS repositories on my web server. There was something funky going on (don't recall what), and so I went to go and fix the script. No problem, right? It's Open Source, after all!

Yeah. Right.

Hey, I'm no slouch as a programmer, and can actually get around a bit in Perl. But that cvsweb script is an absolute disaster!! (I suspect the FreeBSD peeps have fixed it tremendously; it was core to their version control for many years; but I'm talking back in 1999).

I'll repeat that again: DISASTER

Global variables. Functions with strange side effects. Sloppy organization. Poorly named functions and variables. Few comments.

One of the things that I've learned about Perl over the years is that it takes an advanced Perl programmer to produce maintainable code. Novice or intermediate programmers produce crap. The language is just so fluid and forgiving, that it is easy to create write-only code. I recommend Python to people because you only need to reach an intermediate level to produce reasonable code.

Back to cvsweb. After staring at the depths of hell for a while, I realized there was really one solution to the problem: rewrite it all in Python. I spent a weekend doing just that. About 2500 lines of Perl became 2500 lines of Python, and I published it as "ViewCVS".

That tool has since grown support for Subversion, been renamed to ViewVC, and is being further developed and maintained by some friends.

The Power of Facebook Relationship Status

2009-01-01T09:51:00.000-08:00

Everybody seems to notice and watch your Relationship Status on Facebook. Your updates may come and go, pictures may be posted, or other weird things may fall into the feed on your wall, and it goes right past your friends. But everybody notices changes in your Relationship Status.

One year ago, I changed my status from unstated to "Single", and got an outpouring of concern that I was "now" single. Of course, I had been for a long time, but just didn't throw it "out there" on Facebook. I do have to confess that doing so may have changed my outlook on life. "I'm now telling people explicitly that I'm single" -- that does indicate a change in mindset, really.

This New Year's Day, I got to change my status to "Engaged". What a difference a year makes! If you had asked me where I thought I would be in a year... this certainly would not have been on the list! But it is, and I am incredibly happy about it (obviously, duh). It has been just eight hours since I made that change, and there are already lots of comments from my friends! (mostly of the "congrats" form)

People wake up from a long night of New Year's Eve partying, and check Facebook. It's a crazy world we live in, but I think it is also a great way to "get the word out" and let your friends know that your life has changed.

Thank you everybody for the warm wishes so far, and thanks in advance for the wishes not yet said.

Bestest Xmas Present Ever

2008-12-31T14:26:00.000-08:00

The word "Yes"

Star Wars: The Clone Wars (TV Series)

2008-12-16T18:24:00.000-08:00

I've been watching a bit of the new Star Wars animated series. It's quite fun! Lots of light saber battles, jedi tricks, bad guys, and some great effects and animation.

I came in partway thru the series, so I may need to dig up some old episodes.

Now... while this looks interesting, if you are looking for excellent television, then go watch Dexter. Best show ever. But it just finished its third season, so it is time to find new things :-)

Java doesn't work? Switch to Python

2008-12-07T22:50:00.000-08:00

My friend Anthony just pointed me at this post from Larry Page 12 years ago. Apparently, he never found a solution. The first Google web crawler was written in Python instead.

Seriously funny stuff.

Unique Rectangle (Type 4)

2008-12-04T21:48:00.000-08:00

A couple months ago, I downloaded a sudoku application onto my iPhone. I wanted something to do during the "in between times" -- while sitting on a train, waiting to meet up with somebody, or riding a parking lot shuttle. I started with the free version, but eventually upgraded to the paid version to avoid daily limits on the number of puzzles.

The app is called "Enjoy Sudoku", and it is really good. It has a hint mode that will give you a hint, you can ask for more info, and then one more time. It will basically walk you through what is going on, and explains the solving technique.

One of those solving techniques is the "Unique Rectangle (type 4)" and is quite handy. I've learned about BUGs, X Wings, and Locked Pairs. Lots of crazy stuff. This little application for the iPhone has taught me quite a bit about solving sudoku puzzles, and I'm continuing to get better at recognizing when a technique can be applied.

If you're looking for sudoku for your iPhone, then Enjoy Sudoku is a great choice.

Layers of an Onion

2008-10-09T00:52:00.001-07:00

Earlier, I had mentioned that one of my first coding projects was to work on Subversion. I took on the herculean task of rewriting a library that has existed since Day One (the "working copy" library for those who may know what I'm talking about). Over the past eight years, it has slowly grown and expanded and been tweaked and prodded and poked. At this point, it is a morass of barbed tangles that nobody wants to get near. A rewrite has been requested for several years now, but nobody had time or cajones to jump in. Well... I decided that enough was enough, and Subversion really needed this work done if it was going to continue to improve and evolve.

As I've dived into it, it has been like peeling layers of an onion. I go to fix something, but to do that, I need to fix something else, which leads me on down to a third thing. Making headway is slow, but I have made some progress. At a minimum, I'm getting a lot of the code cleaned up in support of the larger task.

Subversion 1.6 is scheduled to come out in December. Only a portion of my overall work will be in there, with the bulk to appear in 1.7, next summer. It's a big and long project, but this work will help (literally) millions of Subversion users around the planet.

Where am I?

2008-09-16T15:02:00.001-07:00

I'm two weeks into a long trip to the east coast of the US and to Europe. I've been here in Washington DC, and will depart for Bardstown, Kentucky on Thursday.

"Bardstown?!" ... Yeah, I'd never heard of it either, but it hosts an annual Bourbon Festival. Being a huge bourbon fan, I'm off to meet a few friends for the weekend and the festival.

After that, off to Munich for Oktoberfest for a week. Then down to Brussels for about two weeks, with little day trips here and there.

Back to Munich for a few days for a conference (SubConf; I'm giving a keynote), then to Spain for a week at another conference. Finally home after that!

Over seven weeks altogether. Whew! I'll periodically post pictures to Facebook and/or Flickr, will update my Twitter/FB status, and will continue to post here occasionally.

Burning Man

2008-08-24T19:40:00.001-07:00

I'm currently driving to Burning Man while writing this post. This is my first year... Very excited!!

I've heard rumours of Internet connectivity, but even if there is, I'm going to be mostly offline for a week.

Woot!

Visiting Bourbon County

2008-08-21T20:29:00.000-07:00

Many of you may already know of my love for bourbon. Woodford Reserve, in particular. For about a year, I've been talking with friends about a trip to Bourbon County, Kentucky to visit "the homeland". Well, we've started to organize the trip for mid-September. I've set up a Facebook event for the Bourbon County trip. Unfortunately, I know that I've forgotten to invite some bourbon lovers onto the event, so if you're game, then please add yourself to the event! And please forward to other bourbon aficionados that I've forgotten.

I'm really looking forward to the event! I hope you can join us in the fun!

WeGame

2008-08-20T16:49:00.001-07:00

On my recent status updates (to Twitter and Facebook), I mentioned hanging out at the WeGame offices. A friend of mine asked about that, kind of wondering whether I'm working here or contracting or something. Nah... nothing so dramatic.

My friend Jared started WeGame last year, and has a big cozy office here in SoMa with a dozen or so people. I pack up my laptop and come over to get out of the loft. Hang out with real people, a fridge of drinks, access to a printer and fax, network connectivity, and whatnot. It is kind of like the coworking concept that my friends over at Citizen Space have been encouraging and talking about for a long while.

Westvleteran

2008-08-20T01:00:00.001-07:00

In my first post on this blog, I mentioned going to Europe. I spent about four days in various places in Belgium, and several days in Amsterdam. One of the days in Belgium, I got a chance to go and visit the Westvleteran Brewery. This was an awesome experience, given that the Westvleteran 12 is pretty consistently rated as the best beer in the world.

My first experience with the Westvleteran 12 was in February 2006 when I was in Brussels for the FOSDEM conference. I remembered it as "good, but with sort of a sweet/bock style flavor." I'm not a bock beer fan, so on this trip, I was approaching the 12 with a little trepidation. At the cafe there next to the abbey, I ordered a pint. Woah. Way way better than I remembered. No wonder it gets such great ratings.

Getting a hold of Westvleteran beer is difficult. The abbey only sells to individuals, and only one case maximum at a time. And not all varieties on any given week. The stuff is in huge demand, but they won't alter production. The abbey doesn't need more money, so they don't make more beer.

If you get a chance to go, or a chance to try the Westvleteran beers, then go for it! You won't be disappointed.

(and yes, I brought some home with me...)

Hawaii

2008-08-15T14:04:00.001-07:00

People who follow my Twitter feed have seen that I'm currently in Hawaii (spur of the moment trip!). Kauai to be precise. This is the first time on this island. I've been to Oahu and Maui a couple times each, and the Big Island once. Kauai is a pretty island, and lightly populated.

Yesterday, I had a Hawaiian "plate lunch" with Kahlua pig and Laulau. Wow. An incredibly tasty meal! Another local item I learned about is Koa wood. It has a beautiful grain that shows through when polished. I saw some very artistic boxes made of Koa wood at a store here in Lihue. One other "local item" is the Red Dirt t-shirt that I bought. Some very cool products!

This visit is fun, but a bit somber because the reason for the visit is to care for a sick family member (doing okay!). I'll be back to Kauai in October and that will be a "real" vacation.

Projects?

2008-08-02T07:50:00.000-07:00

Thanks everyone for the well-wishes!

The most frequent question i've been asked is which Open Source project(s) I'll be working on. The non-helpful answer is "whatever seems cool" :-) seriously, though... I do have some ideas now, but nothing really ties me to them except if they're interesting.

Big ones right now are the svn working copy library, the serf library, and the new Apache web server. Thinking about some others, but first up is to jump back into svn.

Transitions

2008-07-29T17:27:00.000-07:00

I never really paid attention to blogs until I started working as the Engineering Manager for Blogger at Google, back in April 2004. Oh, certainly read some blogs here and there, but never really felt like I had anything to say in a blog. Once I started working on Blogger, I "got it" but did not want to write one for fear of people mixing my opinions with Google's opinions.

Today, that has changed. I have left Google and thought this post about "why" would be a great start for my own little corner of the Internet.

My time with Google has been awesome. It is an amazing company to work for. After Blogger, I spent several years working with Chris DiBona to shape Google's Open Source policies and strategies. I traveled and spoke about Open Source. For a while, I was involved with Google Gadgets, and Google App Engine. But I was getting distracted (more on that in a bit) and I sorely missed writing code. Last year, I switched things around and started to work with Neal Gafter and the Google Calendar team to add CalDAV support (just launched yesterday!). After that, I worked again with the App Engine team for a while.

But I've really been missing out on writing Open Source code. It has been several years since I have significantly contributed code to any Open Source project, and that has been a growing dissatisfaction for me. I've been involved with Open Source for over fourteen years... it is one of the things that I love to do.

I've also been greatly distracted by a major upheaval in my life. As many of you may know, about two years ago, I moved out of the house and a relationship that was over 18 years long. I've since moved to San Francisco, met many wonderful new friends, and have met someone that brings new joy into my life.

My departure from Google is going to allow me to get back into Open Source development. It is going to allow me to travel. And it is going to allow me to explore where I'm going with my new life. I see a world and a lifetime of opportunity with this move, and am tremendously happy about it.

And no, I'm not going to be working any time soon. Please feel free to contact me about short-term projects, and I'll keep it in mind, but I don't foresee any real interest until at least January.

In closing... I'm here for a few days, then off for a wonderful week in Europe. More trips are planned later this year, and I'll write about those as they come up.

See you around, my friends.