Playing with Prism

This weekend, I decided to goof off with Mozilla Lab’s Prism Project. It is the Mozilla Foundation’s implementation of the Site Specific Browser (SSB) concept. I set up or downloaded applications for Napster, Pandora and Gmail. Out of the box, it has novelty value, but little of practicality uniqueness. In its current form, you fire up Prism, provide a URL, and create one or more shortcuts. when the link is clicked, it will launch in its own private browser session, without toolbars, extensions, or navigation keys by default. This is nothing that Internet Explorer hasn’t been doing for a long time and even Firefox can do most of it (with the unfortunate exception of not starting an independant browser session). You can also set a custom icon on it so that it “Feels” a little more like a desktop application. Prism’s wiki indicates that you can do more with it, like custom styling, but I haven’t seen anyone actually use these capabilities. The bundles I tried (like Gmail) simply create the shortcut and use a custom icon.

A great deal of the issue here is, no doubt, what Joel Spolsky calls the Chicken and the egg problem: to be interesting, Prism needs bundles, but no one is willing to create the bundles for Prism because Prism isn’t interesting yet. If and when it catches on, the idea has potential. Gmail, for example, can run beautifully in the browser, but have nice tie-ins on the desktop, like showing a message count when it is in the system tray, start composing an e-mail when I click an e-mail link and allow drag and drop of files (particularly for attachments).

So far, it has had one very pragmatic benefit. As many Linuxers are aware, Flash and sound are both known to be somewhat problematic. Having downgraded to Ubuntu 8.04, Flash locks up sound when run in Firefox which means that, to run a desktop sound application (like, say, Amarok) I have to kill Firefox to release the lock. This works and, with Firefox’s session management, is almost liveable, but is quite annoying. Using Prism for the Flash sites (which, for me, are few) allows me to kill the Prism “application” and release the lock, leaving my browsing and work untouched. This problem seems to have been sorted out in later versions of Ubuntu–to which I will not upgrade until suspend/resume and hibernate/resume work.However, this benefit really just makes up for two shortcomings elsewhere: the abominable state of Linux sound and the fact that Firefox does not run with separate processes for each tab (ala Chrome).

Mozilla’s is not the only project attempting to work off of the SSB concept. A quick googling turns up many options, tough most are limited to either Windows or Mac OS X. The idea itself seems to be a bridge. From where and to where, it is difficult to say. We could be bridging from desktop applications to web applications. From web applications to RIA applications on the desktop. From web back to the desktop. The motivation probably varies by implementor. For Adobe, it is most probably a bridge from web applications to RIA. Once you are started with Air, it will probably be a lot simpler to get you over to move to Flex than it would be to simply jump directly from web programming to the hybrid world of Flex. Mozilla would rather see you use Google everything than see you use Microsoft products as this keeps the browser as the focus of your world. So, they want to create a bridge that eases the desktop oriented users (which is, for the most part, medium to older users, as the younger ones are used to doing everything over the web). to the web.

So, to sum up, Prism has some potential, though right this minute it isn’t the biggest whoop in the world. However, it can still be kind of nice for odds and ends.

Another MySQL Diversion…

At the fine MySQL performance blog, the post http://www.mysqlperformanceblog.com/2009/05/21/mass-killing-of-mysql-connections/, the author gives a sequence of MySQL queries to generate and run a sequence of queries to kill database connections. Once again, I had made a comparable home-brew script and so, as another diversion, I will provide my version:

  1. #!/bin/bash
  2.  
  3. for process in $(mysqladmin -uroot -ppassword processlist | awk "BEGIN { FS=\"|\" } \$4 ~ /$1/ { print \$2 }")
  4. do
  5. echo "Killing process $process"
  6. mysqladmin -uroot -ppassword kill $process
  7. done

Where, naturally, one would substitute the correct login information. The security information aside, I like this method a bit better as it is less interactive (I simply run the script on a given host and it kills all connections from that host) and it does not create any temporary files to be cleaned up later. The caveat, of course, is that it kills all connections for a host–it does not allow for cherry picking.

syntax-rules in Common Lisp

I was hacking some Scheme code (about which a post will be forthcoming) recently and got to thinking that syntax-rules was nice. Really, really, nice. For those unfamiliar with it, both Scheme and Common Lisp include support for macros, which are, basically, readtime code transformations. These transformations are not unlike those done with #define in C and C++, in that they happen before the code is ever run or compiled, but are much more powerful. Common Lisp macros are functions that accept a form has input and produce another form as output to be evaluated. Scheme macros, on the other hand, are usually defined with syntax-rules, which allows the input to be transformed according to rules. In this sense, the Scheme macro system is similar to Awk or XSLT. From all I can tell, the systems are equal in terms of raw power. However, there are cases when the Scheme version is simpler or more concise. So, I consulted google. It turns out that Dorai Sitaram has implemented Scheme macros in Common Lisp. It is an interesting little library, to say the least. I thought one comment the author made was odd, though: “The ellipsis is represented by `…’ in Scheme. This Common Lisp implementation uses `***’ instead, since `…’ is disallowed by Common Lisp’s identifier conventions.” This is almost true. If you use … directly at the top level, you will indeed get an error. However, the spec does provide for using odd characters in symbols using the | syntax. So, this fails:

(let ((… 1)) …)
  1. but
  2. <pre lang="lisp">(let ((|| 1)) ||)

works. “But wait a minute,” you may say, “|…| isn’t the same as …”. Well, yes, it is. In the eq, eql, and equal senses of the word, … is |…|. Perhaps it would be more precise to say that … is the print name for both appearances of the form…which is what matters.

The nice point for Scheme is that, of course, you can use either macro system with no extra dependencies (assuming, of course, R5+RS compliance). However, I cannot say which system I like better. syntax-rules can be kind of nice, particularly for comparably simple macros that involve symbols as keywords (i.e. something like (do … with foo)), but for anything more complex, the rules get so tangled that you are probably better of doing things the Common Lisp way (which is, I’m sure, why Scheme includes support for low level macros like that).

Excel Dates -> Epoch Dates

There are a lot of posts about converting Unix Epoch dates to Excel (which is a number representing the number of days since 1900-01-01 vs. the number of seconds since 1970-01-01) and a few on converting Excel dates to Epoch.

The quick formula for converting an Excel date to an Epoch date is:

(EXCELDATE – 25568) * 60 * 60 * 24

This is slightly modified from a comment to a post that I can’t seem to find spur of the moment.

Old Friend…

This week, I have been tasked with making some changes to code that was gnarly to write and has gone untouched in some time. I needed to backtrack and look at an older edition, so I fired up good old svn log. Snipped down, I saw these two lines:

r154 | mmcdermott | 2009-05-18 16:04:22 -0500 (Mon, 18 May 2009)

r117 | mmcdermott | 2008-05-21 15:42:22 -0500 (Wed, 21 May 2008)

These were two consecutive checkins. That’s right: this code had gone unaltered for almost exactly one year. Believe me, that is rare where I work. But what was nice about the whole thing was that I was able to look at that code and not call myself an idiot. Now that the dust had cleared, I want to rearrange a few pieces of code (and I’ll probably take this chance to do it) to reduce the number of AJAX callbacks, but the code was readable, clean, and sane.

As developers, we have all had those slap-my-forehead-what-was-I-thinking? moments when reviewing our old code. When I was in college, I spent one spring break (in addition to doing all of my semester projects), learning Scheme (this also being, I might add, my first encounter with any form of Lisp) and using it to write an application to normalize a table schema. I looked back at it later after I had learned more about Scheme and found the SRFIs and realized that I had basically reimplemented SRFI-1 and, since I was just learning the language, reimplemented it badly.

So it is really nice to look at code that was written under the gun (as we will all have to write code, at some time or another) and say that it was a legitimately good job.

All I really want in a web framework…

As of late, there has been a great deal of attention paid to web frameworks and these huge time savings that are supposed to be based on the framework selected. Ruby on Rails is, perhaps, the most famous and the one with the most devoted base. I played with Rails for a bit and the honest truth was that I liked the Ruby part a lot better than the Rails part. Rails tried to do a lot, to “help”–and I didn’t like it. I have written similar dislikes for CakePHP which is, by its own admission, an attempt to bring Rails to PHP. Cake is not the only attempt to give PHP users the bliss of Rails. There is also PHP on Trax and Akelos and heaven knows how many others.

It’s not just the PHP boys hopping at the Rails hype. The BBC created a Perl on Rails. Heck, on cliki I even found a project named Lisp on Lines that endeavors to provide a “Rails-like” environment in Common Lisp.

ASP.NET is a little better. It doesn’t try to help with ORM or a lot else, but I can’t help but feel that it has a tendency to be a little stiff at times. Though it is definitely one of the better frameworks out there (and, yes, it is only a framework; contrary to all-too-popular belief, you can write ASP.NET in any .NET language).

What I am saying is that I, personally, want a minimalist framework. I want to be able to assemble a toolkit based on the project at hand, not what is encouraged by the framework. I don’t want to write a plugin to be able to use a bit of JavaScript effectively. Here is what a web framework should provide:

  • Spackling over request details. I need the contents of the GET and the POST, but I sure as heck don’t want to write a GET and POST parser for every application. Preferably, this would also spackle over the exact source of the request. When writing an application, I don’t care whether the request came over CGI, FastCGI, an IIS filter, or mod_lang. I want to the variables and that’s it.
  • A templating engine. A simple, minimalist templating engine. Nothing too fancy and, if I so desire, I should be able to ignore it with impunity or even substitute another. This should be one of the most loosely coupled pieces of the framework. As time goes on, I also expect a minimum of widgets and gadgets to be provided as we get an increasingly large amount of sites and apps whose UI is AJAX/Flash/Insert-RIA-here driven. I really like Smarty for PHP work (which has a plugin API, but not controls in the ASP.NET sense). ASP.NET’s templating engine is good, but its server-side controls are biased towards a method of development that is either outdated (controls rendered and handled serverside) or opaque (how does that callback work, again?).
  • Pretty URL support. Yes, there is mod_rewrite and friends, but it makes things much cleaner to have only a couple of mod_rewrite rules and handle the rest application side.
  • Session management. HTTP is stateless, but web applications, as we all know, are not. Session management is not difficult so much as it is an annoyance to write with every app.

Data access should not, as a rule, be involved. Particularly, data access in which it is difficult to perform raw queries against the database. Generic, all encompassing data access layers are hard to use because they make a number of assumptions (in the name of simplicity) that simply do not hold true for everything. Worst of all are the schema assumptions that frameworks like Cake make.

Django is the exception to the rule, though. Their DAL is excellent. You write an object model that inherits from some base classes. This is used as a very thin layer and you can still easily (yes, easily is the key word; I know you can write raw queries in Cake and Rails, but they try to hide it as much as possible) drop down into plain old SQL. This way, the developer is not boxed in by such things as the dictionary included in the framework.

One time, I wrote an app in CakePHP and I used a term for an entity, I cannot remember what, that had a plural not recognized by Cake; now Cake requires that all tables have plural names. The plural for this word was not recognized, even though it was the correct English usage. This caused the DAL to go haywire and cost me an hour or two of debugging.

The overwhelming philosophy is to provide basic support for that which needs to be done in the act of serving up a web application and leave everything else to the user. The new wave of frameworks tries to do as much as possible for the user. In this sense, a framework is like any other tool or any other application: when you try to do the user’s work for them and are overzealous about making things “easy”, you get underfoot and make things difficult. RoR has made the “blog in 10 minutes” example almost mandatory for a new framework. Well, it’s great that you can throw together a simple blog in 10 minutes, but who wants to? Therein lies yet another rub: RoR and its kin are tuned towards mass producing almost-clones of certain types of applications. I, for one, do not like writing lots of almost-clones. I’d rather download Drupal or any one of a thousand other open source packages and configure that than write a custom almost-clone. What I want in a web framework is small. So, can I have that framework, yet?

Debugging Ubuntu

As always, it seems, the ubgrade to the latest and greatest Ubuntu means fixing a couple of things. At one point, this meant wireless and power management, but times are improving! Now it’s just power management. I have taken to using ndiswrapper for my wireless cards and find it a lot easier than trying to hack the open source ones to kind of, maybe work. To me, that’s a biggie because my main PC is a laptop. A laptop that I carry, open, use for a while, then close and put away. Rinse, rewash, etc. The point is, that I often work on the go and expect to be able to flip open my laptop and pick up where I left off for a quick few minutes of work or play.

I found two problems on my Compaq Presario F700 laptop on Ubuntu 9.04: suspend and a clock error. In the first, suspend worked beautifully–but only on AC. If I attempted to either suspend or resume while on battery power, the whole thing came crashing down. If, on the other hand, I suspend while on AC, unplugged the computer, ported it around, then plugged it up and resumed, everything was fine. The second issue was similar: only on batter power, I would get sundry kernel messages relating to the clocksource (usually, “clocksource tsc is unstable”).

The latter I fixed by adding the option hpet=disbled to my kernel options in grub. The former, I never did figure out and downgraded to Ubuntu 8.04. Suspend works there with the hints described here: https://wiki.ubuntu.com/NvidiaLaptopBinaryDriverSuspend , minus the Xorg configuration. Moments like this almost make me want a Mac…

Cygwin Gripe

In my last post, I made a passing reference to an annoyance with Cygwin’s setup.exe program. First of all, though, I want to say that, by and large, I love Cygwin. It makes life livable in Windows-land. The ability to run a full Linux-esque command line environment alongside the rest of Windowsland is absolutely wonderful. But I have a couple of gripes (the other one being that man pages load slowly; come on, it’s just some possibly compressed text being filtered through some troff macros and piped to a pager; what’s the deal?). The most important being the setup.exe utility. Cygwin uses one application as both its setup utility and its package manager. On the surface this has some nice symmetry and you only have to develop one app. Win, right? Well, not if you have to use it semi-regularly.

The biggest thing is that you MUST go through the menus selecting your download path, your line encodings, your mirror, and redownload setup.bz2. The repetitiveness is slow. Then you get to the package list and they have this tree structure that you have to muck around in. Unless you go ahead and set view to full and check “Keep” (because, if I want one package, I don’t want to upgrade every package in the system while I am at it). Then I have to scroll through a mile long list (the problem that the tree structure exacerbates) and find the package. Why?! This little ritual is time consuming but, more to the point, it is outrageously annoying. I only actually switched line encodings on a Cygwin install once (a few years ago)–and it was by accident and promptly jacked up X (it turns out, on Cygwin, that changing line encoding throws X fonts completely out of whack; beautiful). Now, of course, I simply leave everything on Unix and use a translation utility when necessary, which isn’t that often as the applications I would run on Windows text files (most especially vim) handle the line encoding difference automatically. But I have to click through the option EVERY TIME! What would be really, really nice is having the setup.exe for those who want it, but providing an apt-get-like interface with all those options stashed, from install, in a conf file. That way, I could type something like:

  1. $ cyg-get install openssh

It turns out, I’m not the only one to ever think this, either. Someone else wrote a cyg-apt in Python. Kudos to the author, but why does he have to do this anyway? And why isn’t it an official part of the project?

ASDF-Install

If you have seen any number of articles on this blog (which is highly doubtful), you realize that I often use this as my place to gripe about my favorite technologies (paradox intended). Today, I am here to gripe about asdf-install. For those not fortunate enough to know what I am talking about, asdf-install is a package manager running atop of asdf (a system definition facility for Common Lisp; think of it as being high order packages with lots of meta data and the ability to load files and dependencies semi-automatically). I have used it before in Linux environments, but today I tried it for the first time with clisp and Cygwin on (where else?) Windows.

IT DOES NOT WORK!!!!!!!!!!!!!!!!!

Okay, now that I have screamed in agony, I shall explain. First, it complained about gpg which was easily fixed with a trip to Cygwin’s setup program (which is another complaint for another time). I kill clisp and try again. Now the cycle goes:

  1. I run (asdf-install:install ‘some-package)
  2. asdf-install complains that the GPG keys do not match. This is, mind you, after I download the key chain off of common-lisp.net and import it. The keys SHOULD match because everything SHOULD be up to date. But no, it fails. Just to be clear, I have randomly tried a wide selection of packages on common-lisp.net. Everything seems fine, but gpg still insists it isn’t. Perhaps I did something wrong. At any rate, this is all experimental. Let’s just allow it and see what happens…
  3. asdf-install proceeds, then spits out this error:
    gzip: stdin: not in gzip format
    /bin/tar: Child returned status 1
    /bin/tar: Exiting with failure status due to previous errors

For everything everywhere. At this point, I would assume that I was either an idiot or just missed something about asdf-install in my quick view of the docs. But I have set this up and used it before. This issue, so far as I can tell, only happens on Cygwin. I then googled asdf-install cygwin and I find a myriad of complaints. Not identical to mine, with all kinds of advice from ranging from just don’t use asdf-install to apply patches and so on and so forth. Needless to say, my end conclusion was that, while I could sit here and try to figure out what’s wrong, it just isn’t worth it. That’s right. I copped out. But I still find the whole process of darcs getting a repository and then adding the location to my .clisprc.lisp file a little wearying. So, I whipped up this little shell script to make life easier:

  1. #!/bin/bash
  2.  
  3. cd ~/lisp
  4. darcs get http://common-lisp.net/project/$1/darcs/$1/
  5. echo "(push #P\"${HOME}/lisp/$1/\" asdf:*central-registry*)" >> ~/.clisprc.lisp

Not as nice as asdf-install would have been. For example, it really only works with common-lisp.net (which is good, as that is the sourceforge of the lisp world: almost everything of importance is hosted there anyway) and does no dependency resolution. Finally, it is a little hackish, relying on common-lisp.net’s typical project URL scheme.

Usability < Operability

A theme it seems that I hear a lot is something to the effect that it doesn’t matter how awesome the code/functionality is if it isn’t “usable”. This view seems, especially, to come from developers and designers who work in commercial, shrink wrapped software and, coming from this point of view, it is probably true. If the user can’t glance through a few tabs and know how to do what needs to be done, then it might as well not be there. They probably won’t talk to anyone or read a manual, unless they feel truly and absolutely desperate.

However, if you work in corporate IT, this particular piece of the world gets turned on its head. Operability takes first place over usability and the reason is simple: if the users don’t have what they want or need, they will come to IT and demand it. If it is already there, it will be demonstrated if it isn’t, it will more than likely get added.

Not of course, that this absolves internal programmers of responsibility. Far from it. I know that I have learned a great deal by probing deeper, especially with the question “what are you trying to accomplish?”. It is just that, in this case, the very proximity of “the user” changes the whole dynamic. More than likely, the will tell you that this means too much work, or that they cannot do X.

So, where do the lines lie? I do not know exactly. The point I am more interested in here is not where they lie, but that at some point, usability really is less important than operability. Indeed, this is probably more apt to be true in highly specialized software of any kind, whether that software is an in-house developed PHP package or some $3 million CAD program for designing microchips where investment in the software itself is a given and the users have to use the software or else they cannot do their jobs.