jqGrid Frustrations

I just got a jqGrid (not my first, I might add) put together that took way too much time. While I am going to go back and add some features now, I was struggling to get the blasted thing to do the easiest task in the world: show some data. That’s it. No paging. No searching. Nothing fancy, just show the data.

  1. $(document).ready(function() {
  2.     $('#grid').jqGrid({
  3.         url: 'my_callback.php',
  4.         datatype: 'json',
  5.         colNames: ['col0', 'col1', 'col2', 'col3', 'col4', 'col5'],
  6.         colModel: [
  7.             {name: 'id', index: 'id', width: 50},
  8.             {name: 'foo', index: 'foo', width: 200},
  9.             {name: 'baz', index: 'baz', width: 100},                                        
  10.             {name: 'quuz', index: 'quuz', width: 150},                                            
  11.             {name: 'bar', index: 'bar', width:150},                                        
  12.             {name: 'schnoodle', index: 'schnoodle', width: 150}
  13.             ],
  14.         mtype: 'GET',
  15.         rowNum: 20,,
  16.         viewrecords: true,                                                                              
  17.         imgpath: '../javascript/themes/basic/images',                                                  
  18.         caption: 'My Caption',
  19.         });
  20. });

Where, of course, a few of those items have been anonymized. As vanilla a set up as you could ask for. I agonized over the JSON data, making sure it matched the default JSON reader (which, I might add, I used last time I used this component). Finally, trial, error, and Google prevailed. This is the code that worked:

  1. $(document).ready(function() {
  2.     $('#grid').jqGrid({
  3.         url: 'my_callback.php',
  4.         datatype: 'json',
  5.         dataType: 'json',
  6.         colNames: ['col0', 'col1', 'col2', 'col3', 'col4', 'col5'],
  7.         colModel: [
  8.             {name: 'id', index: 'id', width: 50},
  9.             {name: 'foo', index: 'foo', width: 200},
  10.             {name: 'baz', index: 'baz', width: 100},                                        
  11.             {name: 'quuz', index: 'quuz', width: 150},                                            
  12.             {name: 'bar', index: 'bar', width:150},                                        
  13.             {name: 'schnoodle', index: 'schnoodle', width: 150}
  14.             ],
  15.         mtype: 'GET',
  16.         rowNum: 20,,
  17.         viewrecords: true,                                                                              
  18.         imgpath: '../javascript/themes/basic/images',                                                  
  19.         caption: 'My Caption',
  20.         });
  21. });

See the difference? That’s right. There is a datatype entry and a dataType entry. Mind you, when I took either of these away, the grid broke in some way. Leave them both, it is fine. A quick grep of the code shows that, predictably, after this angst, parts of the code use datatype and other parts use dataType. JavaScript is case sensitive, so it matters. Now, the version of jqGrid in the code base is not the newest, but at v. 3.2.4 it should have been pretty stable. No doubt an upgrade would fix this (all right, after this I do have some doubt), but I am ticked. How could no one have noticed this after 3 versions? It’s not like I downloaded version 0.01. Or was running an SVN bleeding edge edition. It failed on an exceptionally simple example. jqGrid is a nice component, over all, but far, far too brittle.

PS – the link that put me on the right trail is here: http://stackoverflow.com/questions/259435/jqgrid-with-json-data-renders-table-as-empty.

Setting up Mendelson AS2 HOWTO

Overview

AS2 is a wire protocol for transferring files between two organizations. This guide explains how to get the mendelson open source as2 server up and running. The instructions are slanted towards set up on a Debian box, though set up on any *NIX system or Windows should be very similar.

The goal in this guide will be to set up two independant machines, one for test and one for production, and get them talking to one another.

Finally, this guide was developed and tested with mendelson AS2 version 1.1 and Debian 5.0.

Installation

Get the zip file from the Mendelson AS2 sourceforge page. No installation is needed (though an installer is provided for Microsoft Windows boxen), just unpack the files in some location. What I did, was:

  • create a separate user to run the AS2 software (named, cleverly enough, as2user)
  • Unpack the software in a directory in the as2user's home directory.
  • Run the software in a GNU Screen session
  • For the GUI portion, run a lightweight window manager (IceWM was my choice) and a VNC server.

As always, your mileage may vary. Out of the box, the mendelson as2 server is configured to interact with the mendelson test server, and nothing else. The next step is to set up the keys.

Configuring Keys

This process can be done with Portecle, keyman, or the OpenSSL toolchain. The most user-friendly of these is Portecle, which is also the one that Mendelson recommends. Because Portecle is pretty straightforward (and, if you choose one of the other tools, you almost certainly know what you are doing anyway), we will skip the exact sequence of clicks or commands needed for this.

Mendelson AS2 stores its keys in the certificates.p12 file in the root of the install directory. The password for this store is, incidentally, test. The first thing we need to do is recreate the private keys. What we do is delete the keys (or delete the store and create it afresh with new keys with this name) and create new ones. The names of the keys are Key1 and Key2.

After creating the private keys on both machines, we need to export certificates for each, then exchange them. If mendelson AS2 is running, the certificates and keys can always be reloaded by clicking File → reload key store.

Patching the Scripts

Mendelson AS2 comes with DOS Batch files and Bash scripts to launch the AS2 server on Windows and *nix machines, respectively. Not to be degrading or anything, but the bash scripts do not appear to have been well tested. I had to make the following changes to them:

  • Both the mendelson_as2_start.sh and mendelson_as2_stop.sh files used Windows line endings, instead of UNIX. The dos2unix script (available in the Debian and Ubuntu package managers) fixed this problem.
  • Make both of the aforementioned files executable
  • At the top of the mendelson_as2_start.sh file, there is a line setting the CLASSPATH. I had to modify it to CLASSPATH=as2.jar:jetty/start.jar:jetty/lib/servlet-api-2.5-6.1.1.jar

Once the appropriate changes have been made to the start scripts, just run:

./mendelson_as2_start.sh

from the install directory.

Configuring the Local Station

Before the server can receive messages, it must be configured as a local station. By default, a local station will already be set up. The parameters just need to be adapted to match the actual environment. Pretty much, all you will have to change right off the bat is the MDN (URI), which is set to a mendelson domain. While here, you will also need to select the keys you generated above for the local station under the Security tab.

Configuring Partners

After a minute or two, the GUI will pop up. It is here that the AS2 partners must be set up before files can be exchanged. Take the following steps:

  1. Click the button labeled “Partner” (or go to File → Partner).
  2. Fill out the forms. The rest of this should be fairly obvious, but to go over it:
    • Misc
      • Name
      • AS2 ID
      • email address – a contact
      • comments
    • Keys – if you imported the keys above, the certificate for the trading partner should be available from the drop down. Select it or bad things will happen—I promise.
    • MDN – the URI of the recipient.
  3. Click Ok

Sending Messages

This part is easy. Copy a file to the intended recipient’s directory on the server. By default, mendelson is set to poll for new files every 10s (a little inotify support here would rock). In general, from the mendelson install directory, the location will look like:

/messages/<partner name>/outbox

Conversely, the messages will be received on:

/messages/<local station name>/inbox

Copy and run. The main windows on client and server will show their respective progresses.

Configuring HTTPS

At this point, if everything has gone according to plan, messages can be exchanged in plain HTTP. In many situations, however, we want to exchange messages over HTTPS for added security. To do this, we must:

  • Configure Mendelson AS2 to use HTTPS
  • Generate new keys for the HTTPS store
  • On the sender, import the certificates for the recipient

This may sound a little confusing, given that we discussed generating keys above. It turns out that Jetty (the HTTP server & client that Mendelson AS2 uses) has its own separate, independent keystore for sending over HTTPS. Moreover, the keys are expired, which is probably just as well because it makes us generate fresh ones.

Under the main Mendelson AS2 directory, there is a directory named jetty/etc containing Jetty’s configuration files. Jetty itself uses jetty.xml and an example configuration for SSL is in jetty-ssl.xml. Copy the following code from jetty-ssl.xml into jetty.xml:

<Call name="addConnector">
  <Arg>
    <New>
      <Set name="Port">8443</Set>
      <Set name="maxIdleTime">30000</Set>
      <Set name="keystore"><SystemProperty name="jetty.home" default="." />/etc/keystore</Set>
      <Set name="password">OBF:1vny1zlo1x8e1vnw1vn61x8g1zlu1vn4</Set>
      <Set name="keyPassword">OBF:1u2u1wml1z7s1z7a1wnl1u2g</Set>
      <Set name="truststore"><SystemProperty name="jetty.home" default="." />/etc/keystore</Set>
      <Set name="trustPassword">OBF:1vny1zlo1x8e1vnw1vn61x8g1zlu1vn4</Set>
    </New>
  </Arg>
</Call>

You’ll notice that it references the defunct keystore. Create a new keystore, populated with two keys from the following commands:

keytool -genkey -alias Key1 -keypass changeit -keysize 1024 -keystore my.keystore -keyalg RSA -storepass changeit

keytool -genkey -alias Key2 -keypass changeit -keysize 1024 -keystore my.keystore -keyalg RSA -storepass changeit

where my.keystore is the filename of the new keystore and changeit is the password for the store.

For password, keyPassword, and trustPassword, put the values corresponding to those used in the keytool commands.

From the destination, export a certificate and import it into the keystore that was just created.

Conclusion

Once you have the AS2 server up and running, the process of adding real life partners is fairly similar. The only other parting tip I can offer (thanks to the forums) is that if, at any step, something goes wrong the start up script can be patched to provide a lot more debugging information by changing the last line of mendelson_as2_start.sh to read:

java -Xmx192M -Xms92M -classpath $CLASSPATH -Djavax.net.debug=all de.mendelson.comm.as2.AS2

the addition is -Djavax.net.debug=all. As implied this will dump all sorts of goodies to the terminal.

Appendix – Signed MDNs

As of Mendelson AS2 1.1 – build 29, there is a bug that causes verifcation errors with signed MDNs sent by mendelson AS2 to non-mendelson AS2 servers. The solution is to get the b29 source module from CVS (on Sourceforge), change all occurrences of “\n” in the message strings to “\r\n” in MDNText.java, then navigate to de/mendelson/comm/as2/message and run:

javac -classpath /path/to/as2.jar MDNText.java

this will create a new MDNText.class file. Unzip the jar somewhere that will not trample anything, replace the MDNText.class file, and create a new jar. This updated file should solve the problem. At least, it did for me. Your mileage may vary, of course.

Sources

The “Business Perspective” is a False Canard

Flipping through the C++ FQA, the phrase “from a business perspective” popped up a number of times and it occurred to me how often I have heard that phrase or something like it to refer to the needs of the management as opposed to the needs of a programmer. In fairness, I must also add that the author of that fine document is semi-quoting from the C++ FAQ. As I stared at those words, something jumped out at me: when it comes to tech, there really isn’t any such thing as a “business need” because the geeks and the suits ultimately want the same thing.

What kinds of things do we find in the “business perspective”? Well, how about these:

  • Economy of price – we need to keep costs down in order to increase our margin
  • Economy of time – closely intertwined with economy of price, but still separate in that we want to get our product to market ASAP, even aside from price, to help grab up marketshare
  • Capability – it must do whatever it is that we need it to do

That should pretty much cover it. The friction between the two groups does not come from these basic wants. Developers do not want to work longer on a project than is necessary. By and large, they want to do it and move on.

About the only time I see that there is a real collision, is when developers try to make their own jobs a little more interesting. Even here, we see that this is mostly unconscious. The developer trying to interestify the job usually believes consciously that they are solving some problem that is stalking the whole project. This is hardly unique to the developer side of the equation as we (or, at least I) have seen the business types getting all distracted by shiny little trinkets.

So, then, at the end of the day, the friction seems to come less from the core concerns (which are, more or less, shared by both parties) but how they are perceived. But the phrase “business perspective” is a lead-in to a pack of nonsense.

MySQL Hatred

Further anti-MySQL bile. I hate MySQL, this morning. While doing some nice sysadmin type stuff, I wanted to either lock a database down or (better yet) take it offline completely, while leaving everything else untouched. A quick command in MS SQL Server or a few clicks of the mouse, if you are so inclined. Easy. MySQL does not have this basic, basic admin feature. Hacks from the intrawebs include:

  • FLUSH TABLES WITH READ LOCK
  • Changing user permissions.

What the heck? I have to tinker with user permissions to TAKE A DATABASE OFFLINE? And flush tables locks all right–every table in every database. If you’re running one database, that’s fine. Me, I’ve got closer to 30. Sure, in our case, this is because we have a couple of apps that are badly designed.  But still. What if I had two? Say, a blog and a wiki? Same problem. Take one down, take the other down. Or fiddle with permissions. I’m sorry, this is just wrong.