Announcing SpiralWeb version 0.1

Version 0.1 of SpiralWeb is available for download at http://pypi.python.org/pypi/spiralweb/0.1. To install, make sure that you have Python and pip, then run pip install spiralweb to download and install. The project home page can be found at https://gitorious.org/spiralweb.

About SpiralWeb

SpiralWeb is a Literate Programming system written in Python. Its primary aims are to facilitate the usage of literate programming in real-world applications by using light weight text-based markup systems (rather than TeX or LaTeX) and painlessly integrating into build scripts. It is language agnostic. The default type-setter is Pandoc’s extended markdown, but new backends can be readily added for any other system desired.

For more information on literate programming, please see literateprogramming.com.

Usage

The syntax is minimal:

@doc (Name)? ([option=value,option2=value2...])?

Denotes a document chunk. At the moment, the only option that is used is the out parameter, which specifies a path (either absolutely or relative to the literate file it appears in) for the woven output to be written to.

@code (Name)? ([option=value,option2=value2...])?

Denotes the beginning of a code chunk. At present, the following options are used:

  • out which specifies a path (either absolutely or relative to the literate file it appears in) for the tangled output to be written to.
  • lang which specifies a language that the code is written in. This attribute is not used except in the weaver, which uses it when emitting markup, so that the code can be highlighted properly.

@<Name>

Within a code chunk, this indicates that another chunk will be inserted at this point in the final source code. It is important to note that SpiralWeb is indentation-sensitive, so any preceding spaces or tabs before the reference will be used as the indentation for every line in the chunks output–even if there is also indentation in the chunk.

@@

At any spot in a literate file, this directive results in a simple @ symbol.

In Search of C# Omnicomplete for Vim

By day, I write in C#, mostly on a stock .NET install (version 4, as of this writing; I expect that the principles laid out here will transfer forward as the Vim ecosystem is fairly stable). I often find myself switching back and forth between Visual Studio 2010 (with VsVim) and gvim 7.3. Frankly, I should like to spend more time on the gvim side than I do. While a great deal of time and effort has gone into customizing my vimrc for .NET development, I often find myself switching back to VS in order to get the benefits of Intellisense when working with parts of the very hefty .NET Framework that I do not recall by memory.

Every so often, I do some fishing around for something useful to make my Vim Omnicomplete more productive. In this post, I will layout my newest attempt and analyze the findings. As such, this post may or may not be a tutorial on what you should do. In any event, it will be a science experiment in the plainest sense of the word.

First, the hypothesis. While checking out the Vim documentation on Omnicomplete, we see that the Omnicomplete function for C makes heavy use of an additional tag file, generated from the system headers
[http://vimdoc.sourceforge.net/htmldoc/insert.html#ft-c-omni]
, and that this file is used in conjunction with what Omnicomplete understands about the C programming language to make a good guess as to what the programmer likely intends.

It should be possible then, with minimum fuss, to generate a similar tag file for C#. It may also be necessary to tweak the completion function parameters. We will look at that after we have checked the results of the tag file generation.

It turns out that Microsoft releases the .NET 4 Framework’s sourcecode under a reference-only license
[http://referencesource.microsoft.com/netframework.aspx]
. The initial vector of attack will be to download the reference code, and build a tag file from it (this seems well in keeping with the intent behind the license—if this is not so, I will gladly give up the excercise). The link with the relevant source is the first one (Product Name, “.NET” and version of “8.0” as of this writing). The source was placed under RefSrc in Documents.

After running:

ctags -R -f dotnet4tags *

in the RefSrc\Source\.Net\4.0\DEVDIV_TFS\Dev10\Releases\RTMRel directory, we got our first pass at a tag file. A little googling prompted the change to this
[http://arun.wordpress.com/2009/04/10/c-and-vim/]
:

ctags -R -f dotnet4tags --exclude="bin" --extra=+fq --fields=+ianmzS --c#-kinds=cimnp *

Then, as the documentation says, we added the tag to our list of tag files to search:

set tags+=~/Documents/RefSrc/Source/.Net/4.0/DEVDIV_TFS/Dev10/Releases/RTMRel/dotnet4tags

When this is used in conjunction with tagfile completion (C-X C-]) the results are superior to any previous attempts, particularly in conjunction with the Taglist plugin
[http://www.thegeekstuff.com/2009/04/ctags-taglist-vi-vim-editor-as-sourece-code-browser/]
.

With this alone, we do not get any real contextual searching. For example, if we type something like:

File f = File.O

and then initiate the matching, we get practically any method that begins with an O regardless of whether or not said method is a member of the File class.

If we stop here, we still have a leg up over what we had before. We can navigate to .NET framework methods and fetch their signatures through the Taglist browser—but we would still like to do better.

The only reason the resulting tagfile is not here included, is that it is fairly large—not huge, but much too large to be a simple attachment to this post.

Symbols vs. Keywords in Common Lisp

I was resuming work on my Sheepshead game today (more will be coming in time on this), and it occurred to me: what is the difference between a symbol and a keyword? If you type

(symbolp 'foo)

and

(symbolp :foo)

both return

T

but, if you type

(eq 'foo 'foo)
(eq :foo :foo)

both return

T

yet

(eq :foo 'foo)

returns

NIL

Finally, if you type

(symbol-name 'foo)

and

(symbol-name :foo)

both return

"FOO"

So, what gives? Both are symbols and symbols with the same print name, at that. The difference, is that keywords are all generated in the KEYWORD package, but symbols identified with the QUOTE operator are generated in the current package. So,

* (symbol-package 'foo)                                                                   
#<PACKAGE "COMMON-LISP-USER">

but

* (symbol-package :foo)                                                                   
#<PACKAGE "KEYWORD">

Just a quick little tidbit.

Is the brain a computer?

I was listening to Donald Knuth give an Author’s Talk over at Google and he did it in soft of a Q&A setup. One of the questions was, I think, particularly evocative: “is the brain really a computer and if so, what are the theological implications?” When I heard that, I put the video on hold to get my thoughts down. Here is the answer, as I would have given it (and I’ll listen to Knuth’s answer in a second):

“If you want to think of the human brain as a computer, I suppose it is as good an analogy as any. If it is, it would be a biological computer unlike any in existence. But I do not see this posing any problems, theologically. One of the most important ideas in Christianity is that a human being has both a body and a soul. It is interesting. C.S. Lewis once said that you do not have a soul. You are a soul. You have a body. This seems particularly pertinent, because if you want to say that the human body is equipped with a massively powerful computer, fine. It is still under the control of the soul which truly is the essence of the human being. Finally, Faith is interested in the soul. The fact that your soul is operating a computer does not change its overall status and does not pose any difficulties to the Christian faith–it does not even change anything, any more than determining the composition of the rest of the body has.”

That’s pretty close to my knee jerk reaction. Naturally, I have had a few (five or so) minutes to refine it so it probably would not have come out so well had I said it, but the content would have been more or less the same.

Ant and Irony

The authors of Ant deserve a prize for irony. In the manual’s introduction, it says that “Makefiles are inherently evil as well.” I won’t deny the shortcomings of make, but these guys say that make is evil, then go and build an XML based build system. I mean come on, if there is anything worse than worrying about tabs it is having to manually write XML.

The Old Schooled Office Suite

Lately, I have been working on getting my few office suite needs moved over to something a little more text based. I find GUI apps pretty inefficient for these kinds of tasks. Understand that this is not an open source vs. closed source discussion. OpenOffice.org Writer and Microsoft Word get grouped together, as one is basically an open source cross platform version of the other. This is actually contrasting two different application philosophies. I am, in fact, looking for something that is more usable.

Usability is a commonly trotted out concern when evaluating software. Now, when the word is thrown out, it is loaded with a specific meaning that is actually improper to its use. Usability is conflated with something that has a flat learning curve. So flat, that it is assumed that the hypothetical user could learn the application every time they start it. It is assumed that it usability refers to the flat learning curve as opposed to a tool that requires a little more upfront effort. This is unfortunate, because usability is actually a slider between the two extremes of “easy to learn” and “efficient to use”. Where a “usable” application falls on this continuum is up to the user in question. Commercial software naturally gravitates towards the “easy to learn” end of the spectrum, because it is the end that accommodates the greatest slice of the populace. Even the more hardcore of us can get by with software that still has the training wheels on.

So, the usability I am looking for is one of efficiency not of expediency. I want apps that will allow me to produce what I want fast and roll on. I am willing to accept a steeper learning curve upfront, it it means time savings later on. This is, for example, why I am using Vim for day to day text editing (like programming) rather than jEdit or syn. Sure, it took longer to learn upfront, but I can move through a text file fast.

Some other considerations for me was that I like using tools that I can use from a straight terminal because it means that I can access them over SSH. No need to sync files, just remote in to the machine carrying the files. This may not be a concern for most users out there, but it makes my life a lot more convenient.

As an example, this little spiel started with the search for an application I could use to draw process flows in a declarative manner. It does not take much to be able to pull open Visio or Dia and draw the diagram. When you think about it, that drawing process takes longer than seems necessary. You click the tools in the pallets. You click through the properties. You drag. You drop. You do a lot of work to draw a picture. However, if you can simply declare the relationship, it would be a lot more efficient. That is what I was looking for, but what I’ve set up is a nice little replacement, for everyday use, for OpenOffice.org/Microsoft Office. So, here goes my list.

Graph/chart drawing & diagramming

Graphviz. More typically used for diagrams produced in mathematical articles (writing on graph coloring algorithms, for instance), this little tool will produce directed or non-directed graphs. The syntax is simple and C-like, allowing one to declare nodes, with optional properties, and edges connecting them.

Email

The biggest two CLI email clients are mutt and pine. Mutt is clearly the more configurable of the two, but Alpine (the new, completely OSS version of Pine) is quick and easy. My biggest annoyance with Alpine is its heavy verbosity. There are plenty of “are you sure?” prompts. Ultimately, I found Sup. Sup is self-described as being a sort of text-based version of Gmail. Sup wins hands down with a nice interface and a powerful search engine.

Word processing

Word processors have gotten huge. Nonetheless, most of what they have to offer is seldom used. The Word power users I have heard mention this always do so with a touch of regret. As though we all should spend more time learning to use Word. The fact is, most of those features go unused because people don’t need them. I fit firmly in that category. I wanted a markup system I could use so that my documents would be composed in pure text (allowing me my precious Vim), but easily run through into something more print ready or presentation ready.

I considered TeX or LaTeX briefly. However, two things about them bothered me. First, there was the sheer ugliness of the markup. The large quantity of line noise interjected into my manuscript as I was composing was simply too much. It distracted too much and was too cluttering. Secondly, there was the heavy mathematical and scientific bias. A good portion of the documents I work on are related to my fiction writing. The ability to pretty print mathematical formulae is not helpful. So, the trade off was simply not worth the benefits.

Wiki syntax is a good example of what I wanted. Plain, non-cluttering, and very close to an ASCII version of all formatting. A little bit of research brought me to Markdown, Pandoc (which accepts Markdown), and AsciiDoc. My favorite of these is AsciiDoc. Its native backends are HTML and DocBook. However, DocBook (around which AsciiDoc provides a wrapper utility) can be converted to many formats, including PDF, ODF, and RTF. So, if you use this, you can create something that just about anyone can open. Pandoc actually offers more native backends than AsciiDoc (including DocBook, so the same ups apply), but generates plainer HTML. Sure, I can style it. I can make it default by aliasing pandoc to pandoc -c mystyle.css. Someday, I may even do it, but right now I am lazy. I don’t want to have to write a reasonable CSS file for quick and dirty use. So, I am using AsciiDoc for now because it caters to my sheer laziness.

Spreadsheets

Most uses of spreadsheets are evil. It seems that a lot of the world tries to use Excel like a database system—which it isn’t. However, their utility for quick little things cannot be ignored. Besides, I still have to deal with the spreadsheets I receive. I found two modern day, command line driven spreadsheet systems: sc and Oleo. sc is older, its source having its origin as a public domain application on USENET. Its interface is largely inspired by vi and less. Oleo is GNU’s spreadsheet application and so, understandably, takes its cues from Emacs. The most important command, C-x C-c, is the same. Naturally, when you put it that way, I have no choice but to fall in with sc. It is a nice little piece of work, allowing quick manipulation of data. Neither of these applications will generate pretty pictures for use in a word processor, but we quickly find ourselves rescued by the UNIX philosophy of a tool for every job. sc/Oleo do not need to provide ways to dump out pretty pictures—that is not their job. They crunch data. We use something else to create graphs.

Tasks

Those TODOs that we all have to keep. Microsofties keep them in Outlook. Anti-computerites either keep them on paper or not at all. The best text-based system I have seen is TaskWarrior. It is truly awesome.

Calendering

Another application that is integrated with Outlook and Exchange for most users. The command line applications are, of course, more loosely coupled. The best event scheduling calendar I found was Pal. Pal looked like it was more heavily geared towards the way people think of calendering.

Instant Messaging

IM is not a part of the traditional office suite, but, it is becoming a regular part of most offices. Finch is the clear winner on this front. There are plenty of text based IRC tools, but Finch is built on top of libpurple and supports all of the protocols that Pidgin does.

Music player

Again, I suppose this is not really a part of the traditional office suite, but we all want one. There is a reason Windows ships with Media Player. It is because Microsoft recognized that people wanted one and if it didn’t ship with Windows people would (horror of horrors!) use non-Microsoft software. My favorite GUI music player is Amarok. On the CLI side, Moc and CMus are excellent choices. MOC seems a little more intuitive.

Loose Ends

Unfortunately, the formats version of musical chairs does not usually permit us to keep everything strictly text and HTML. Those of us who are still part of the real world have to accept and send files in popular formats. Fortunately, OpenOffice.org allows exchanges between the backends of many of the formats that would be required. Unoconv, a Python script to interface with the headless version of OpenOffice, makes those conversions easy from the command line.

I am sure that there are people for whom this type of setup would be simply unworkable. Computing ability aside, it emphasises work on content, not work on layout. Obviously, if you are making flyers every day, AsciiDoc is not for you. You could insert images (generating your word art through some ImageMagick kung-fu) and generate a PS file that you lpr off to your printer, but it would probably not be an optimal workflow. However, I do work on content. My programs are content. My stories are content. I am a content person, not a design person, so, for me, this is an excellent layout.

I Hate Info Docs

GNU’s insistence on building their own documentation system instead of sticking to good old UNIX’s man pages is obnoxious. They provide second rate man pages and pretty good info docs. Why? Why not use some sort of generator that spits out the same data as both man pages and info pages? Anyone who wants to navigate from Emacs or use the schmanzy fancy navigation can use them and those of us who prefer man will still get full documentation. Doesn’t seem like so much to ask. So, GNU: knock it off. I hate info and trying to suck me into using it isn’t going to help matters any.

So Much for WORA

The promise of Java is WORA-Write Once, Run Anywhere. I decided to tinker with the Blackberry development one of the options for which is Java. So, I set up Eclipse (which is way harder than it should be; people found Vista’s branding or Linux distros confusing? Eclipse makes them look positively simplistic) and installed the Blackberry JDE plugin. After restarting Eclipse what greets me? A message that the IDE could not find a Windows USB DLL. I’m running this on Linux because I thought Java was platform independent. There are some USB wrappers in Java that hide the OS. What is the point of using Java if we are going to be tied to a single OS?

Moving Time

I am moving Latrunculi’s headquarters over to Mad Computer Scientist. Information is on the Latrunculi page (see the navbar on the right). I will be updating the Latrunculi page and the SourceForge page shortly. After I finish out a few graphics bugs, it will be time to get cracking on the AI.