She played a pivotal role in my life, long before I was even conceived. After her mother, my grandmother, had passed away, Barb took it upon herself to put my mother through college. I doubt she ever understood this nuclear engineering thing my mother studied, but she supported it. And eventually my mother met another engineer and here I am today.

It wasn’t unexpected - she had been declining for a while - but I’m surprised at how choked up I am about her passing. Outspoken, disapproving, warm and alive - often in the same sentence.

I remember a message she once left on our answering machine. Her voice sounded so similar to my dead mother’s voice, with the same Pennsylvania accent, that it sent chills up my spine.

Thanks Barb, for all you’ve done and all you were - you’ll be missed - and in the end, what more is there, than to be missed?

I was recently discussing the problem of tracing program execution through code that doesn’t exist.  Sometimes, usually to understand and debug a program, you need to trace through it. If the code doesn’t exist, its hard to follow where it goes. Actually, the code exists, its just not in source control or otherwise easily searchable.  Let me explain.

In Ruby, whose source code is a lot like its executable format, its trivial to write code that writes code - so you, the programmer, don’t have to. The Rails web framework uses this a lot, for creating whatever form of ‘find’ strikes your fancy: find_by_firstname, find_by_firstname_and_gender, find_by_pet_species_and_viciousness, etc. - as long as your database table has the appropriately named columns then it will find what you’re looking for with no work on your part!  Actually, some work is required - you need to know that most any routine starting with ‘find’ is probably auto-generated by Rails and thus will be un-findable in the source tree.

A more complex example is the routing helpers - a shorthand way of saying “put a link here on this web page that will take the user to that web page over there.  Until you realize that most any method ending in ‘path’ is a URL helper then you’ll be confused by code that references user_edit_path or formatted_pet_list_path or formatted_pet_species_list_path.  The last one says that you want to get the path to a specially-formatted list of the various pet species represented by the system (e.g. “http://myPetSite.com/pet_species/list.csv”). Fairly clear once you know what it does, but fairly obtuse until you get to that point.  And it is, trust me on this, shorter, more maintainable and more clear than the alternative (after, of course, you learn to read it).

So this difficulty in finding the source to “formatted_pet_species_list_path” (or something like that) started a discussion that eventually got around to the idea that Ruby and Rails uses code to write code.

“Its the 11th commandment”, said one, ” - thou shalt not write self-modifying code!”

Yes and no. Call me irrationally exuberant or say that I drank some sugary flavored colored water - but I don’t think its as cut-and-dried as it used to be.  This commandment, when Moses brought it down from on high, was written for a compiled language, where the source was very very different from the executing code, and it actually did modify the code being executed.  This is bad bad bad - its hard to understand, it leads to intractable bugs, it’ll make you go blind, melt glaciers and other bad stuff.  No argument there.

But is this what Rails is doing with Ruby? I’d say its different.  Instead of modifying existing code it is merely creating code that did not exist before. Its more akin to a pre-processor that could, just before compiling a program, generate all the permutations of finding database rows from columns X, Y and Z (and any number of others).  Looked at this way, it appears to be similar to a C++ template - generic code that a programmer has written to simplify the writing of code that does similar things in similar ways.  The main difference is that with Ruby there is no pre-processor - the routine gets created at the time it is first used.

The “template” in this case is a routine called method_missing that gets handed the name and arguments of any routine that doesn’t exist.  It looks for a function name matching the form ‘find_by_X‘ where X makes sense for the database table in question (of course, if it doesn’t match the correct format then it just passes the name and arguments onward for some other routine to either make sense of or to burp up an error).  Once the routine is created it is available to be called again, with none of the overhead incurred when it was first created. More importantly, it was created with no additional overhead on the part of the programmer - to write it, test it, debug it or modify it. This, in my opinion, is a huge advantage - but I’m a lazy developer who doesn’t want to write or debug any code that I don’t have to.

Kool-aid anyone?

Biometric authentication transactions to cease at 11:59:59PM March 19, 2008

SAN FRANCISCO - (March 19, 2008) - Solidus Networks, Inc., dba Pay By Touch, regretfully announced today that it will no longer process biometric transactions on behalf of its merchant customers and consumer membership base, as of 11:59:59PM March 19, 2008.

On December 14, 2007, Solidus Networks filed for U.S. bankruptcy protection under Chapter 11. As part of the company’s restructuring, it was determined that the enterprise could no longer support the biometric authentication and payment system as it currently exists, based on lack of funding and current market conditions.

Other non-biometric Solidus Networks business units will continue on their current business paths.

Solidus Networks extends its sincere gratitude to the shoppers, merchants, vendors, investors, partners, and employees who have been supporting the company’s vision since its first biometric payment transaction in 2002.

I like reading code. Its like a novel and I want to read it cover-to-cover. Some, such as Why’s Camping framework, I struggle to comprehend. But most code that I read comes up slightly short. Like a novel with some mis-spellings, awkward phrasing or repeated analogies, I mentally mark it as “could be better”. And sometimes I really do sit down and write something better - maybe just for my own amusement but often for a useful purpose.

I recently had the experience of reading some code that parsed a variable-length binary data structure. This sort of thing comes up often when parsing a file format or communications protocol. Most of the code looks fairly similar because it does similar stuff: ignore one byte, read the next four as the length of the following junk, read two important bytes, ignore two more, read another four-byte length and skip past the following N bytes - ad nauseum.

I’ve written it in C, and it looks something like this (ignoring error conditions like getting to the end of the buffer):

ptr = &data;                  // start at the beginning of our data
ptr++;                        // skip junk we don't care about
UInt32 len = *(UInt32 *) ptr; // get the 4-byte length
len = ntohl(len);             // convert from network byte ordering
ptr += sizeof(UInt32);        // skip past the length we just read
ptr += len;                   // skip past the data we don't care about
UInt16 cost = *(UInt16 *)ptr; // read our important two bytes
cost = ntohs(cost);           // convert to the correct byte ordering

In Ruby, this tends to be shorter due to the handy String.unpack() routine, which takes a concise format string to define how many bytes to read and what to do with them. “a3″ reads 3 bytes as a string, “N” reads 4 bytes in network order, “n” reads 2 bytes in network order, etc. The code above could be rewritten in Ruby like this:

array = data.unpack( "a1N")        # read the junk and the 4 length bytes
len = array[1]                     # only get the length value we care about
data = data[5..-1]                 # throw away the stuff we just read
array =  data.unpack( "a#{len}n" ) # define the length to read on the fly
cost = array[1]                    # get our data in its correct ordering
data = data[(len+2)..-1]           # again, throw away what we just read

This code works fine, but its not much more readable than the C code. A first step would be do define a string.unpack!() routine, where the ‘!’ exclamation clues us in that it modifies the object we’re working with. In this case, the modification is to eat (discard) the data we just read. This shortens the code to:

array = data.unpack!( "a1N")       # read the junk and the 4 length bytes
len = array[1]                     # only get the length value we care about
array =  data.unpack!("a#{len}n")  # define the length to read on the fly
cost = array[1]                    # get our data in its correct ordering

But again, this isn’t much more readable (in my opinion) than the C code. Additionally, it doesn’t help us understand the code much better in the case where our format string is “a3Nna5″ and we need to remember which item in ‘array’ corresponds to the ‘n’ in the string (in this case, it is array[2]). After a test iteration or two, what I finally hit upon was to encapsulate the behavior we want in a separare Unpacker class, that automatically eats the data it reads and stores the results in an internal Hash object, to map the name ‘len’ or ‘cost’ to the data. I also combined the format string and the resulting variable so we can clearly see the relationships. The result looks like this:

u = Unpacker.new(data)
u.u! "a1        => unused
      N         => len"
u.u! "a#{u.len} => unused
      n         => cost"

Now we can clearly see which values are ignored, which are given meaningful names, and how the format codes relate to the meaning of the data. Changing it to reflect a better understanding of the underlying data will be very easy. Note that the only reason its in two statements is to define a value for u.len before we use it - blocks of fixed-length data can be one statement.

The code to implement the Unpacker class is only about 30 lines of Ruby - including the string.unpack!() routine that can be reused separately.

class String
  def unpack! format
     array = self.unpack(format+"a*")
    self.replace array.pop
     return array
   end
end
class Unpacker < Hash
   attr_reader :data
 def initialize string
     @data = string
    super
  end
  # format string is expected to have whitespace between each
  # "unpackCode=>variableName" pairing (which can have whitespace
  # around the "=>").  u! was picked to be short so it would
  # look nice, and to connote a destructive "unpack!" operation.
  def u! format
    format.gsub(/\s*=>\s*/,'=>').strip.split(/\s+/).each do |segment|
    src,dst = segment.split(/=>/)
    self[dst] = @data.unpack!("#{src}")[0]
 end
end
# Hash_with_Attrs - For the simplicity of using either u.len or u['len'],
# makes a hash appear to have members for each hash entry. Many thanks
# to Why_ for collecting this handy routine on his a href= RedHanded blog.
# Note of Caution: 'len' is fine but 'length' would not be since u.length
# would give the number of entries in the hash, not the just-parsed value.
def method_missing(meth,*args)
  meth = meth.id2name
  if meth =~ /=$/
    self[meth[0..-2]] = (args.length<2 ? args[0] : args)
  else
    self[meth]
  end
end
end

Update: An even cleaner and shorter way would be to implement a DSL as a module so the code above could look like this:

a 1,    :unused
N       :len
a :len, :unused
n       :cost

(and yes, this is valid Ruby code)

[Update 1/29/2008: the network is inconsistent.  Upon a boot or un-hibernate it may be completely incapable of finding my router - but then later it is fine. I'll continue trying to track it down... using the Windows OS!] 

# xkcd.rb
# extract all the titles from xkcd comics since they
# tend to be too long to fully show in the browser

# USAGE: ruby -rubygems -rxkcd.rb -e 'Xkcd.new.show_all'

require 'open-uri'
require 'hpricot'

class Xkcd
  DOMAIN = 'http://xkcd.com/'

  def show id = 343  # 343 is the NSA/RSA one
    begin
      @hp = Hpricot.parse( open( "%s/%d/" % [DOMAIN,id.to_i] ) )
      (@hp / :img).each do |el|
        puts "%4d: %s" % [id.to_i, el[:title]] if el[:title]
      end
    rescue
    end
  end

  def show_all
    0.upto(400) do |i|
      show i
    end
  end
end

http://www.catalogchoice.org/

So yes, Virginia, Ruby and Rails do scale.

The context for this discussion is a website (nameless, sorry) that I’m interested in working on. The startup site, yet another type of social network, holds the promise of allowing for some very interesting and powerful interactions - but unnecessarily limits its users as it guards scarce server resources and data security. Furthermore, and I’m going out on a limb here, I suspect that these mis-perceptions are one of the reasons this startup has had difficulty in raising much-needed funds. Some examples:

• Users are automatically logged out after a few idle minutes, with no option of changing the time period before auto-logout (or choosing “Keep me logged in” for single-user computers). This seems a bit draconian given that there is nothing accessible on the site that couldn’t be gathered in other ways - no bank statements, social security number or mother’s maiden name.
• A PDF document containing the public profile data for your social circle can be generated for off-line access, but only by a subset of the social circle and only for a short period of time. I think this is intended both for security and to guard scarce resources (such as server time and bandwidth). The former concern is misguided - anyone receiving the PDF can circumvent security by immediately sending it to bad people - which is unstoppable once you provide off-line access). The scarcity of server time or bandwidth can be overcome by delegating it to someone else such as Amazon’s ECC or S3 services.
• New people can be invited to the social circle, but only by a small initial set of users - and those invitations expire relatively quickly. Its unclear why this decision was made, but I suspect it was due to some perception of scarcity or security. All it appears to do is add yet another unnecessary barrier to entry.

In spite of these issues, and others, I’m still captivated by the underlying ideas that it represents and by what it could become in the future. Hopefully I can rapidly prototype my vision for an improved site and use it as a starting point to land a dream job.