Today I was working with a Ruby on Rails application, configured to use MySQL 5.6.10, that kept failing to perform a column-modifying migration:

class SomeModel < ActiveRecord::Migration
 def up
   change_column(:some_models, :some_value, :integer, default: 0, null: false)
 end

 def down
 end
end

The error message, from the Mysql12 adapter, was "Invalid use of NULL value".

Using the test database I could change the schema manually, from the MySQL prompt. Cut I couldn't do so in the development database. As it usually is, the test database held no records. That should have been my clue.

Eventually I realized that in the development database the "some_models" table contained many records, at least some of which held NULLs for "some_value".

After performing a manual update:

update some_models set some_value = 0 where some_value is null;

I was able to run the migration.

In August, 2010, I wrote,

it seems likely that "hover-free Mobile Safari" will become a historical anomaly.

I don't know whether or not their implementation is any good, but Samsung is bringing hover to multi-touch devices. Fun times:

Samsung Launches The Galaxy S4:

• Air View: Hover with fingers to get a preview of content
• Air Gesture: Wave your hand to change music, take a call, scroll a web page

I've been trying to re-install a stack of MS SQL Server installations, from 2005 to 2012, with no luck. "Uninstall" of MS SQL Server 2008 kept failing because "the following products are installed:"

Microsoft SQL Server 2008 R2 RsFx Driver

Of course the RsFx (Report Streaming?) driver showed up nowhere in the list of installed programs.

This post solved my problem:

SQL 2008 R2 – Uninstall of SQL fails with error about RsFx Driver or Common Files still being installed:

"Option B – Get the product GUID (Identifying number) from WMI using WMIC and uninstall using MSIEXEC /X {GUID} "

I had trouble reading the text describing the actual shell commands to use, because it rendered white-on-white in my browsers (Safari, Google Chrome). Happily it was easy enough to just select the region with the mouse and paste into a plaintext text editor buffer :)

Here's more about the research predicting "higher end" temperature increases.

Climate Predictions: Worst-Case May Be Most Accurate, Study Finds:

"Hovering several thousand feet above Earth's surface, in the troposphere—the part of the atmosphere where clouds can form—dry zones play a primary role in the future climate.

…

The scientists compared the observed relative humidity in the dry zones to 16 different climate models used in the most recent study by the Intergovernmental Panel on Climate Change.

Since we don't have good ways to observe or predict how clouds will form, focus instead on relative humidity, which we can observe. How well do the climate models predict the relative humidity data that has actually been observed?

Fasullo and Trenberth found that the three models that best matched the humidity observations were the same ones that predict the hottest future, with temperatures increasing 8 degrees F before century's end. The least accurate models overpredicted relative humidity and projected lower increases in temperature.

Fasullo used the analogy of an eye: 'The dry zones are like the iris of the climate system. With warming, the iris dilates, decreasing cloud cover and allowing in more heat.' Models that don't provide for that expansion of the dry zone fail to accurately depict observed data, he explained."

The article ends with a small equivocation:

Karen Shell, a climate scientist from Oregon State University who was not involved in the research […]: "It's a promising technique. It's one study, but if this relationship holds up, it implies the climate sensitivity is on the higher end of the range."

Climate science is Nate Silver and U.S. politics is Karl Rove | Grist:

"Empiricism won. It didn’t win because it’s a truer faith or a superior ideology. It won because it works. It is the best way humans have figured out to set aside their prejudices, their perceptual limitations, their cognitive shortcomings, and get a clear view of what’s to come."

Let’s be clear about this.... There are serious scientists who doubt that human civilization can endure at all in the face of 7.2 degrees. And we are headed for 10.8.

…on the biggest, most pressing risk facing the country, those involved in U.S. politics might as well be witch doctors. Or worse, Karl Rove.

It’s not a wise approach, because in the climate race, losses are permanent and irreversible. There will be no recount.

For more than 20 years I've been hearing that our response to climate change needs to take into account the health of the economy. OK. How healthy will our economy be when we can't keep factories cool, or grow enough food to keep its employees from starving?

Shortly after iOS 6 was released I upgraded my iPhone. About the same time I set up iClouds backups of Safari bookmarks. Within days (hours? It's all a blur) I discovered that I could no longer sync Safari bookmarks via iTunes. Worse, I could no longer edit them on my iPhone.

The problem proved to be a corrupt schema in Mobile Safari's bookmarks database.

Here's the quick repair procedure. It does not require you to restore your phone to factory settings. (It should go without saying, but: do this at your own risk!)

1. Buy iBackupBot — it will save a lot of hassle in editing your phone backups
2. Export Library/Safari/Bookmarks.db
3. Using sqlite3 /path/to/exported/Bookmarks.db, run the following commands on your exported bookmarks database to re-create missing tables and indices

   CREATE TABLE bookmark_title_words (id INTEGER PRIMARY KEY, bookmark_id INTEGER NOT NULL CONSTRAINT fk_bookmark_id REFERENCES bookmarks(id) ON DELETE CASCADE, word TEXT, word_index INTEGER);
   CREATE INDEX title_bookmark_id_index ON bookmark_title_words(bookmark_id);
   CREATE INDEX title_word_index ON bookmark_title_words(word);
   

4. Re-import the modified Library/Safari/Bookmarks.db
5. Use iTunes to "Restore from backup…"

Note that the last step should read "Use iBackupBot to restore the bookmarks file to your iPhone." I haven't yet tried doing that. I didn't even know iBackupBot could restore individual files until after my problem had been solved. Ah, ignorance :)

Also note that the exact set of required schema restoration commands may vary from one phone to another. These were the bits which were missing on my phone.

I learned a lot of interesting trivia while bumbling along toward this solution. Writer's block permitting, I'll try to summarize it in subsequent posts. Meanwhile, here are some links which helped me find a solution. Thanks to all for sharing!

• Solving iOS 6 Battery Drain Problems
• The iPhone Backup Browser project at Google Code

This item showed up on today's "4 Short Links" by Nat Torkington, on O'Reilly Radar. I was struck by Nat's chosen excerpt:

What matters isn't the ancestor relations between things but what they can do for you. That, of course, is where interfaces come into Go. But they're part of a bigger picture, the true Go philosophy. If C++ and Java are about type hierarchies and the taxonomy of types, Go is about composition.

Setting Go aside for a moment, that first sentence captures one of Python's most useful traits: duck typing. Or, I don't care who your parents are. Can you do this job?

In any case, Rob Pike's post is thoughtful and well written, and its treatment of "programming in the large" makes me want to go play with Go.

command center: Less is exponentially more:

" Less is exponentially more

[…]

I was asked a few weeks ago, "What was the biggest surprise you encountered rolling out Go?" I knew the answer instantly: Although we expected C++ programmers to see Go as an alternative, instead most Go programmers come from languages like Python and Ruby. Very few come from C++.

[…]

Python and Ruby programmers come to Go because they don't have to surrender much expressiveness, but gain performance and get to play with concurrency.

It has been bothering me that my Wordpress installation on webfaction wasn't using SSL for admin. Mike Pirnat explains how to fix this, concisely and clearly. Thank you!

Mike Pirnat: Configuring Wordpress SSL Login and Admin on Webfaction

For a recent iPhone app I wanted a multi-line editable text field, in which I could display placeholder text so my user would know what to do with the field.

Placeholder text is a common affordance provided by several UI classes such as UITextField. But UITextView in iOS 4 does not support placeholder text.

A web search turned up several implementations. Some of these seemed too complex, and some failed to provide the behavior I wanted:

• If the view is empty, show the placeholder.
• Whenever the view is the first responder, hide the placeholder.

Here's my implementation. It's very basic. For example it doesn't let you customize the color of the placeholder text; and the placeholder is always centered vertically within the bounds of the text view. But it does provide the behaviors listed above.

PlaceholderTextView.h

#import <UIKit/UIKit.h>
@interface PlaceholderTextView : UITextView {
    UILabel *placeholderLabel;
    NSString *placeholderText;
}

@property (nonatomic, retain) NSString *placeholderText;
@end

PlaceholderTextView.m

#import "PlaceholderTextView.h"
@implementation PlaceholderTextView
@synthesize placeholderText;

- (void)hidePlaceholder {
    if ([placeholderLabel superview]) {
        [placeholderLabel removeFromSuperview];
    }
}

- (void)maybeShowPlaceholder {
    if ([placeholderText length] > 0) {
        if (![self hasText] && ![placeholderLabel superview]) {
            CGRect frame = CGRectMake(0, 0, self.bounds.size.width, self.bounds.size.height);
            placeholderLabel.frame = frame;
            placeholderLabel.text = placeholderText;
            [self addSubview:placeholderLabel];
        }
    }
}

- (void)initCommon {
    if (!placeholderLabel) {
        [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(beganEditing:) name:UITextViewTextDidBeginEditingNotification object:nil];
        [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(endedEditing:) name:UITextViewTextDidEndEditingNotification object:nil];
        placeholderLabel = [[UILabel alloc] init];
        placeholderLabel.numberOfLines = 0;
        placeholderLabel.backgroundColor = [UIColor clearColor];
        placeholderLabel.textColor = [UIColor lightGrayColor];
        placeholderLabel.font = self.font;
        placeholderLabel.lineBreakMode = UILineBreakModeWordWrap;
        placeholderLabel.textAlignment = UITextAlignmentCenter;
        [self maybeShowPlaceholder];
    }
}

- (void)awakeFromNib {
    [super awakeFromNib];
    [self initCommon];
}

- (id)initWithFrame:(CGRect)frame {
    self = [super initWithFrame:frame];
    if (self) {
        [self initCommon];
    }
    return self;
}

- (id)initWithCoder:(NSCoder *)aDecoder {
    self = [super initWithCoder:aDecoder];
    if (self) {
        [self initCommon];
    }
    return self;
}

- (id)init {
    self = [super init];
    if (self) {
        [self initCommon];
    }
    return self;
}

- (void)dealloc {
    [[NSNotificationCenter defaultCenter] removeObserver:self];
    [placeholderLabel release];
    [placeholderText release];
    [super dealloc];
}

- (void)setPlaceholderText:(NSString *)newValue {
    [newValue retain];
    NSString *oldValue = placeholderText;
    placeholderText = newValue;
    [oldValue release];
    [self maybeShowPlaceholder];
}

- (void)beganEditing:(NSNotification *)notification {
    [self hidePlaceholder];
}

- (void)endedEditing:(NSNotification *)notification {
    [self maybeShowPlaceholder];
}

@end

http://gordonazmo.wordpress.com/2011/02/09/fixing-vmware-tools-vmhgfs-on-newer-kernels-probably-anything-2-6-36/

Many thanks!

Occasionally I need to create an iPhone app which can either take a picture or select one from the user's library. Testing such an app on the iPhone simulator is a pain; the simulator doesn't include a simulated camera, and it doesn't provide a way to sync photos from iTunes.

I keep forgetting how to load images into the simulator's photo library. Here's a method that works well, courtesy of Stack Overflow:

• Launch the iPhone Simulator.
• In the simulator, launch Safari.
• Drag an image from the Finder into the simulator. It should appear in Safari.
• Click and hold the mouse on the image.
• An action sheet should appear. Choose 'Save Image'.

To confirm that the image got added you can launch Photos in the simulator. The image should appear in your Saved Photos album.

A trailing slash can be hard to see.

This morning I tried to commit to a Mercurial repository which contained sub-repositories. One of those subrepos had been added a few days earlier, along with the corresponding .hgsub entry. .hgsub had been committed explicitly, without incident.

$ vi .hgsub
  # add entry for src/baz/foo
$ hg ci -m "Some comment." .hgsub
$

$ hg ci -m "Blah."
abort: path 'src/baz/foo/' is inside repo 'src/baz/foo'

Say what?

It took a few minutes of quiet contemplation before I finally saw the problem:

abort: path 'src/baz/foo/' is inside repo 'src/baz/foo'

Once the extraneous slash was removed from .hgsub, everything was fine.

From Engadget:

"In particular, we are relaxing all restrictions on the development tools used to create iOS apps, as long as the resulting apps do not download any code. This should give developers the flexibility they want, while preserving the security we need.

"In addition, for the first time we are publishing the App Store Review Guidelines to help developers understand how we review submitted apps. We hope it will make us more transparent and help our developers create even more successful apps for the App Store."

I try to keep this blog focused on software development: design, bug fixes, etc. That's why the following post originally appeared elsewhere.

It appears here, now, because it's loosely connected to software development and because — well, it's interesting (to me). I've been seeing so many references to the idea of proximity-based user interfaces, it seems likely that "hover-free Mobile Safari" will become a historical anomaly.

Technical Note TN2262: Preparing Your Web Content for iPad:

"For example, a mouse pointer can hover over a webpage element and trigger an event; a finger on a Multi-Touch screen cannot."

This is certainly true for multi-touch devices available now, but I wouldn't be surprised to see, someday, touch-sensitive devices which are also proximity-sensitive.

Update 2010/07/08: Others are thinking about implications of hover on multi-touch devices: http://trentwalton.com/2010/07/05/non-hover/ via http://news.ycombinator.com/item?id=1497108

I'm not a Star Trek fan, but a recent Ars Technica article, exploring the similarities between the iPad and ST:TNG's PADD, caught my eye. It provides several good examples of how science fiction guides technology development. This excerpt seems particularly relevant:

"Still, what new frontiers are out there for interacting with computing devices? Michael Okuda believes that removing the touch requirement will bring new advances in gesture-based control. "Once you don't have to physically touch the screen," he told Ars, "I think yet another window is going to open up.""

Update 2011/01/27 AppleInsider reports on Apple's recent hover-related patent work.

rentzsch.tumblr.com: [C4 release]:

"With resistance to Section 3.3.1 so scattershot and meek, it’s become clear that I haven’t made the impact I wanted with C4. It’s also clear my interests and the Apple programming community’s interests are farther apart than I had hoped."

I signed up for an iPhone developer account this spring, just before the change to the developer agreement which included the revised 3.3.1. The change was galling. Never mind Flash; Apple was showing what it thinks of its developer community. "We can further limit you any time we want. Not a big deal. Just rewrite your code."

I bought back into Apple in 2002, drawn by OS X. OS X developers are still free from the strictures imposed on iP* developers. Let's hope it stays that way. If not, then it's back to Linux for me.

Meanwhile, here's hoping for good things from HP/Palm, Notion Ink, and the Google Android ecosystem.

Yesterday I tried to update an Ubuntu 9.10 VM, running VMware Fusion 3.1 beta, to Ubuntu 10.04. Two significant problems arose:

1. At the login screen, the keyboard didn't work
2. the boot process paused because hgfs could not be mounted

Here's a log of the problems and worries encountered, together with their solutions.

Grub Configuration

Late in the Ubuntu upgrade process a warning appeared, saying Grub could not be configured (or words to that effect — I wasn't paying close attention) and asking for confirmation that the installation should continue without any changes to grub's configuration. I confirmed. After all, this was a VM instance with only a single operating system installed.

Can't Mount hgfs

On reboot the console reported that a problem had occurred while mounting /mnt/hgfs. If offered to continue without mounting, and I accepted.

Unresponsive Keyboard

When the Ubuntu login screen appeared, the keyboard was unresponsive. This made it difficult to enter my password.

Thanks to this thread in the VMware Communities forums, I was able to login after enabling the onscreen keyboard. (Click on the accessibility icon — the "little man in the circle" — check the onscreen keyboard toggle, and restart the VM.) Once logged in, the keyboard worked just fine.

VMware Tools Out of Date

VMware detected that VMware Tools were out of date for this VM. (It may have reported that they weren't installed. I can't recall.)

No xorg.conf?

I started the VMware Tools installation process and encountered a warning I hadn't seen before:

Detected X.org version 7.6.6.


The configuration file /etc/X11/xorg.conf can not be found.  Do you want to create a new one? (yes/no) [yes]

Not sure what to do, I accepted the default response.

Keyboard Fixed

After the tools finished installing I rebooted the VM. This time, at the login screen the keyboard worked. A wild guess is that the VMware Tools re-installation fixed this problem by reconfiguring the console.

Fixing hgfs Ownership

/mnt/hgfs had been auto-mounted, but as usual the ownership and permissions were wrong. I patched this as documented in a previous post.

Fixing hgfs automount Failure

Ubuntu continued to complain at startup that it had failed to mount hgfs. It continued to request user permission to skip mounting hgfs. But once the boot completed /mnt/hgfs was mounted with correct permissions.

To prevent Ubuntu complaining of failure to mount hgfs, I added a "nobootwait" option to its entry to tell mountall(8) not to hold up the boot for this filesystem. The entry now reads:

.host:/ /mnt/hgfs vmhgfs rw,ttl=1,uid=my_uid,gid=my_gid,nobootwait 0 0

Now Ubuntu boots without complaint, the keyboard works, and /mnt/hgfs is mounted with correct ownership and permissions.

In VMware Fusion I share a folder with many of my VMs. In a new Ubuntu 9.10 VM, that shared folder is always mounted with the wrong owner and group IDs. Here's a fix.

After booting the Ubuntu VM for the first time, I downloaded and installed the latest vmware-tools distribution, using the VMware "Virtual Machine -> Install VMware Tools" menu item. Then I added an entry to /etc/fstab, specifying the correct owner and group ID for the /mnt/hgfs mount point.

Unfortunately, the owner and group ID specifications were ignored. As it turns out, /etc/init.d/vmware-tools does not examine /etc/fstab when deciding whether or how to mount the shared filesystem.

Here's how I changed /etc/init.d/vmware-tools to address this problem.

Old

# Mount all hgfs filesystems
vmware_mount_vmhgfs() {
  if [ "`is_vmhgfs_mounted`" = "no" ]; then
    vmware_exec_selinux "mount -t vmhgfs .host:/ $vmhgfs_mnt"
  fi
}

is_vmhgfs_in_fstab() {
    if grep -q " $vmhgfs_mnt " /etc/fstab; then
        echo "yes"
    else
        echo "no"
    fi
}
# Mount all hgfs filesystems
vmware_mount_vmhgfs() {
  if [ "`is_vmhgfs_mounted`" = "no" ]; then
    if [ "`is_vmhgfs_in_fstab`" = "yes" ]; then
        vmware_exec_selinux "mount $vmhgfs_mnt"
    else
        vmware_exec_selinux "mount -t vmhgfs .host:/ $vmhgfs_mnt"
    fi
  fi
}

I finally presented something at a Python conference. Many thanks to Vern Ceder for organizing the PyCon 2010 poster session!

Here's a copy of the poster, which describes a way to run QUnit-based JavaScript unit tests using Python's unittest facilities:

PyCon Poster.pdf

The code described by the poster is available from bitbucket.org, under the BSD license.

I recently upgraded my iMac. Migration Assistant moved all of my files to the new machine without issue -- or so it seemed.

I had created Mercurial repositories in a couple of virtualenv environments, to track changes locally.[1] I didn't notice that Mercurial had put each virtual environment's .Python file under revision control.

Shortly after completing the migration I made changes in one of these virtual environments. A quick 'hg status' before committing, and...

$ hg status
abort: data/.Python.i@13b27e856c38: no match found!

WTF?

After much investigation it appears that the following has happened.

1. Mercurial represented the .Python link in its .hg/store/data directory as ._Python.i
   1. The leading underscore appears to be Mercurial's way of noting that the 'P' should be capitalized.
2. I think Migration Assistant uses ditto to copy files.
   1. ditto saw the leading '._' and concluded this was an orphaned resource file.[2] So it didn't copy the file.
3. Mercurial knew that it was supposed to have a .hg/store/data/._Python.i file; when it couldn't find it, it decided the repository was corrupted.

Luckily the problem cropped up before I traded in the old machine, so I was able to copy across the missing files manually.

In my experiments, the problem manifested only when the capitalized dotfile was a symbolic link, not when it was a regular data file.

Well... the above write-up contains several unproven assertions, e.g. about the conditions under which Mercurial will create a '._' filename. I'm not really sure whether this is a bug or merely a caveat regarding an obscure corner condition. For now, the easiest workaround is:

Don't Track virtualenv .Python Files With Mercurial.

[Update 2010/02/22: Someone has already filed this as a Mercurial bug.]

[2] OS X still supports something ike resource forks. In tarballs and other non-OS Extended filesystems, resource forks are represented as dot-files with a leading underscore. (See Norman Walsh's blog for more info.) For example, the resource fork for a file named 'foo.txt' might be '._foo.txt'.

Once in awhile I write Python scripts which produce static HTML pages and then use the webbrowser module to open the results in a browser. I do this just rarely enough that, when the scripts fail on OS X with the following error, I have to waste time diagnosing the problem:

execution error: An error of type -2110 has occurred. (-2110)

Despite the obscure phrasing, the cause of the error is simple. I'm passing a filesystem pathname to webbrowser.open, when I should be passing a file: URL.

This produces the error:
webbrowser.open("/path/to/file.html")

This works:
webbrowser.open("file:///path/to/file.html")