Summary

This is the second post of two about a light weight way to implement a thesaurus. In Part I, I described how to set up a script that provides access to an on-line thesaurus. In this Part, I describe how to write a set of simple syntax rules to provide colour and highlighting for the output.

Here is a screenshot of the finished syntax rules (using dummy data):

If you don't like these colours, you can change the rules to use whatever colours you prefer.

How Vim does it

Vim does syntax highlighting in two parts:

• A set of rules that define a (group of) word, and
• A colour description for each rule.

Set of rules

An example is

Notice each rule has a name that is a unique identifier: cStatement, cLabel, cTodo and cBlock.

In fact, it can be much more complicated than that. Rules can be contained within rules (for example, nested if-thens), rules can apply only if another rule has triggered (a '(' as part of a condition, but not as part of a comment), rules can apply to entire regions, and so on.

However, for our simple purpose which is to provide slight colour hints to a plain text list, we can ignore all that, to which we say, "Thank goodness." Check out sh.vim or even simple old c.vim to see why.

Colour description

which translates as: for rule name "Comment", for a simple terminal set to bold, for a colour terminal set the foreground to colour number 8, for the GUI version of Vim set the foreground to colour number #7C7C7C.

You can think of the process as going like this: when Vim displays a file, each token or word is checked against the set of rules and assigned a rule name, and is then colourised according to the highlighting for that rule name.

You can see your highlighting description with :hi Comment, which shows the current highlighting for rule "Comment" for the filetype in the current buffer. To see all highlighting, use :hi by itself.

From this you can see that the rules and the highlighting scheme are tightly coupled. Each rule has a name, and that name should be in the colorscheme¹ . Vim controls the tight coupling by providing a set of standard names which, if we were writing rules for a new programming language, we should use. Those names are things like "Comment", "Keyword", "Statement", and so on.

However, we are creating our own set of rules for this specific instance of text, so we can do what we like. Which we shall.

Syntax rules

The data

Typically, you get a set of entries comprising Main Entry, Definition, Synonyms, Notes and Antonyms. Not every entry is present, often Antonyms and Notes are absent.

As well, if the word is common, you often get repeating sets of entries for very similar words, as in the example where the entry for this is also shown. (This isn't real data, by the way; this doesn't appear with which normally.)

Set up some rules

• Entry names distinguished (highlighted differently) from entry contents, i.e. "Main Entry:" should be different from "which".
• The main entry word should be bolded or similar to make it stand out.
• The main word melds too easily into "pronunciation..", which makes it hard to pick out. Diminish everything to the right of the main word.
• Each entry should be distinguished so you can scan them easily. For example, "Main Entry" should stand out, "Definition" less so, "Synonyms" more so.

Here is an example rule set that roughly implements the first, second and fourth items.

" Include the colon as part of the word
setlocal iskeyword+=:

" Rules
" a keyword
syntax keyword  thesSynonyms Synonyms:
" this entry name has a space, so needs a regex 
syntax match  thesMainEntry /Main Entry: */ 
" this entry should include the line to the end
syntax region thesDefinition start=/Definition: /  end=/$/

" Highlighting
" link the highlighting to the defined 
" name "Keyword" in the colorscheme
hi link thesMainEntry Keyword
hi link thesSynonyms  Statement
hi link thesDefinition  Todo
" specify the colours directly
hi  thesMainEntry term=bold 
 \ctermfg=White cterm=bold guifg=6 gui=bold
" or, keep the current colour, but bold it
hi  thesMainEntry term=bold cterm=bold gui=bold

which gives this:

You can see that the keyword "Synonyms:" has matched a rule set, and the highlighting for the rule "Statement" in my colorscheme (likely different to your colorscheme) has been applied.

The regex pattern for "Main Entry: " has matched a rule set and the highlighting for the rule set has been applied.

Similarly for "Definition", the rule set has matched to the end of the line, and the highlighting for the rule "Todo" in my colorscheme has been applied.

The real deal

Rules

" Vi syntax file
" Language: text dump from online thesaurus
" Maintainer: Nick Coleman
" Last Change:  2012 Feb 18
" Remark: for the online thesaurus script by Nick Coleman
  
if exists("b:current_syntax")
    finish
endif

" Setup
" syntax clear    " only useful for testing
syntax case match
setlocal iskeyword+=:

" Entry name rules
syntax match thesMainEntry /Main Entry: */ contained
syntax region  thesDefinition start=/Definition: /  end=/$/
syntax keyword thesNotes Notes: contained
syntax keyword thesSynonyms Synonyms:
syntax region  thesAntonyms start=/Antonyms:/  end=/$/ 

" give the pronunciation region a special name
syntax region thesPronunciation start=/pronunciation \[/ end=/$/ contained

" Entry contents rules
syntax region thesMainWord start=/Main Entry:/  end=/$/ contains=CONTAINED keepend
syntax region thesNotesEntry start=/Notes:/  end=/^ *$/ contains=thesNotes,thesAntonyms keepend 

" Highlighting

hi link thesMainEntry     Keyword
hi  thesMainWord      term=bold cterm=bold gui=bold
hi link thesDefinition      String
hi link thesNotes     Number
hi link thesNotesEntry      Number
hi link thesSynonyms      Statement
hi link thesAntonyms      Todo
hi link thesPronunciation   Comment

let b:current_syntax = "thesaurus"

Some rules have "contains". This allows a rule within a rule, the classic example being Todo appearing within a comment where you want a different colour to make Todo stand out. "keepend" is part of that, it stops both rules at the first end pattern match rather than the final end match. :h usr_44 section 44.5 for more.

Setup

Recall from Part I that the script sets the thesaurus' buffer to filetype thesaurus. Put the above syntax file in $HOME/.vim/syntax/thesaurus.vim and the buffer will pick up the syntax rules automatically.

Windows users can put it in C:\Program Files\Vim\vimfiles\syntax\thesaurus.vim or the equivalent if using Vista or Windows 7. If you don't have administrator privileges, find where Vim thinks $HOME is by (within Vim) trying :echo $HOME or :version and putting it in $HOME\vimfiles\syntax\thesaurus.vim.

Trying it out

You probably want to use your own highlighting. A tip: to easily see the effect of your changes reload the highlighting for the data buffer with :setlocal filetype=thesaurus.

To see the colours that your colorscheme uses for a particular rule use :hi <rule> as in :hi Statement. To see all colours, use hi by itself :hi.

hi Search ctermfg=white ctermbg=darkblue

Summary

This is the first post of two (second here) about a light weight way to implement a thesaurus. It is great for what I need, which is the occasional use of a thesaurus for writing text such as this article. Once it is set up, you can forget about it and just use K whenever you want to look up a word.

A nice bonus or synergy of using an online source is that the website also returns a definition for the word, so it functions as a simple dictionary as well.

The second post (here) will deal with how to use Vim's built in syntax rule sets to provide highlighting and nice colours.

There are two or three simple steps:

1. a vim script that passes the cursor word to an external shell script.
2. a shell script that looks up the word using an online thesaurus, then parses the output to remove unnecessary cruft.
3. an optional syntax file to provide highlighting and colours.

Here is what it looks like with a quick-n-dirty syntax rule set:

Sample output for "quick"

I use Vim for programming and it is great for that. I use it when I need to edit files on remote servers that I have ssh'ed in. I also use Vim for writing text because it is a great text editor. I am writing this in Vim, for example.

Often, one of the things you want when writing text is a thesaurus, which is the topic for today.

Why on-line

You download a thesaurus (the Moby one is common), point Vim to it and you are good to go. However, a thesaurus can consume a lot of memory. The Moby file is 24 MB, although to be fair Vim doesn't use anything like that. If you are using Vim remotely, you may not want to use any extra memory. For example, I have a remote VPS that has only 128 MB of total memory, so every megabyte is critical. This is especially true if you use the thesaurus only rarely; the trade-off is not worth it.

The bigger issue for me is that Vim does not handle a long list of alternative words very well. It is fine with a short list where you use it just like auto-complete, but it seems to get confused with a long list. Unfortunately, Moby often serves up a long list, and you frequently find you have inadvertently changed your perfectly good word to something completely different. You then have to Undo and try again.

In fact, I got so frustrated with the Vim/Moby combination I ended up unsetting the thesaurus feature in Vim after just a couple of months.

The good news is that there is an alternative, and that is to use one of the many on-line thesauruses.

I have set things up so that a script calls a command-line browser to query for a word and dump the output from the online website, then parses it to remove extra cruft such as sidebars, headers and footers, and white space, and puts it into a scratch buffer in Vim itself for me to browse through and perhaps copy a word from.

Vim setup

Copy and paste the following into your .vimrc or wherever you prefer. I have it in a common.vim file in my ftplugin directory, where my various filetype scripts can source it if they want.

fun! ReadThesaurus()
   " Assign current word under cursor to a script variable
   let s:thes_word = expand('<cword>')
   " Open a new window, keep the alternate so this doesn't clobber it. 
   keepalt split thes_
   " Show cursor word in status line
   exe "setlocal statusline=" . s:thes_word
   " Set buffer options for scratch buffer
   setlocal noswapfile nobuflisted nowrap nospell 
     \buftype=nofile bufhidden=hide 
   " Delete existing content
   1,$d
   " Run the thesaurus script
   exe ":0r !/home/nickcoleman/bin/thesaurus " . s:thes_word 
   " Goto first line
   1
   " Set file type to 'thesaurus'
   set filetype=thesaurus
   " Map q to quit without confirm
   nmap <buffer> q :q<CR>
endfun
" Map the K key to the ReadThesaurus function
noremap <buffer> K :call ReadThesaurus()<CR><CR>

The script is fully commented for you to follow what is going on.

Notice I specified the location of the shell script that Vim is calling to be $HOME/bin/thesaurus. Change this to wherever you are going to put the shell script.

I call the scratch buffer "thes_" so that I have a name for it which means I can easily re-use the buffer again. Otherwise Vim would create a new buffer every time and go through buffer numbers like crazy. In the unlikely event you have an actual file called "thes_", change the script to use some other wacky name. I could have used the Vim function tempname() to generate a unique buffer name, but "thes_" is meaningful and easy to remember if you want to unhide the buffer later.

I set the status line to show the word I am looking up. Sometimes the website will return a different word, a near synonym, so the status line reminds me exactly which word I am looking up. An example is looking up the word "the" which returns the definition and synonyms for "histrionical". No, I don't know why.

I include a line to set the filetype to "thesaurus". Its purpose is to allow me to set up anything special for that filetype later on. A possible use would be to create a syntax file to do special highlighting or colouring. That will be the topic for the next post.

The mapping for K has a second <CR>. It eats up the "Press ENTER or type command to continue" prompt. There are other ways to do this, but they can mess up the display or have the side effect of applying globally instead of just this buffer.

By the way, in case it isn't obvious, the reason I split the functionality in to two—part- Vim and part-shell scripts—instead of doing it all in Vim is because Vim's scripting language, always an awkward beast at the best of times, would make it too hard. Best to use the good parts of each and combine them.

Thesaurus script

Originally I wrote a quick-n-dirty one-liner, but I decided the script might be useful outside Vim as well, so I tidied it up to be a useful script that you can call from the command line.

This is the shell script that Vim calls. Put it in the location you specified in the Vim script above. It is straightforward, apart from the sed call, and there are enough comments so you can see what is happening.

#!/bin/sh

# This searchs an online thesaurus, cuts as much cruft out as possible,
# and displays the definitions.

#URL='http://www.merriam-webster.com/thesaurus/'
URL='http://thesaurus.com/browse/'

# Display help, and quit.
function usage {
    cat << EOF_HELP
    Usage: $(basename $0) [-r][-h] word
    Display the parsed results of an online thesaurus search for <word>

    -r  Raw results; no filtering.
    -h  Display this help.

EOF_HELP
    exit 
}

# Check for a parameter.  Get the output from the website
function get_thes {
    [  -z "$1" ] && usage
    $(readlink -e $(which links)) -dump "${URL}$1" 
}

# Check for a -h parameter or absence of parameter.
[ "$1" = "-h" -o -z "$1" ] && usage

# If -r just get the raw output, otherwise pass it through sed.
if [ "$1" = "-r" ] ; then
    shift
    get_thes $1
else
    # Sed is doing many things: 
    # Print out "no results found for " and quit. 
    # Print out only the lines I'm interested in, which are:
    #   No results found for
    #   Main Entry:
    #   Definition:
    #   and all from Synonyms: to Antonyms:
    # Put a new line after Antonyms.
    get_thes $1 | sed -n -e '/No results found for/ {' \
      -e ' s|^[ \t]*\(No results .*\)|\1|p ; q} '\
      -e '{/Main Entry:/p ' \
      -e '/Definition:/p ' \
      -e '/Synonyms:/,/\(Antonyms:\)\|\(^$\)/p} ' \
      -e '/Antonyms:/ a \ '
fi    

I use thesaurus.com because I found it has a wider coverage than merriam-webster.com. If you want to use a different website, you will need to write your own sed script to parse it.

I separated the sed actions into discrete chunks above so you can get a clearer picture of what sed is doing, but they appear as one line in the actual script. That line is below, for you to cut-n-paste.

    get_thes $1 | sed -n -e '/No results found for/ { s|^[ \t]*\(No results .*\)|\1|p ; q} ;{/Main Entry:/p ; /Definition:/p ; /Synonyms:/,/\(Antonyms:\)\|\(^$\)/p} ; /Antonyms:/ a \  '

Install

I prefer that this script is only available in buffers where I am writing text. That way, I can keep the K mapping for unix man pages in buffers where I am writing code.

The way to do that is to put the Vim script in its own file that is called only by text-like filetypes. Scripts that are unique to filetypes are put in ~/.vim/ftplugin or ~/.vim/after/ftplugin. I created files named text.vim, xml.vim and blog.vim that are in the ftplugin directory. (XML is treated as text because most of my XML is text data.) I put the thesaurus script in a file called common.vim and I source common.vim from within all the above ftplugin scripts.

If ftplugin (and perhaps autocmd) are a mystery, see :h ftplugin and :h autocmd.

With that, I am done. To use, move the cursor in Normal mode underneath the word of interest and press K.

Bugs

I noticed once that the script loses permissions to a temporary file that Vim uses internally. It seems to only happen if a remote ssh session in screen is unexpectedly terminated, and not always then. Closing and re-opening Vim (yes, a pain) will fix it.

Additional Info for Windows Users

Summary

1. Install sed and links. It takes only a few seconds each to download and install. The default install is fine.
2. Create a batch file to get and parse the thesaurus data. The batch file will be an abbreviated form of the shell script above, tailored for Windows.
3. Point vim to the location of the batch file.

Install Links & Sed

Create DOS Batch File

@"c:\program files\links\links.exe" -dump http://thesaurus.com/browse/%1 | "c:\program files\gnuwin32\bin\sed.exe" -n -e "/No results found for/ { s|^[ \t]*\(No results .*\)|\1|p ; q} ;{/Main Entry:/p ; /Definition:/p ; /Synonyms:/,/\(Antonyms:\)\|\(^$\)/p} ; /Antonyms:/ a \  "

There are a couple of differences to the unix script. Sed needs double quotes surrounding its commands instead of single quotes. The paths to the executables need double quotes because of the spaces in the path. I put a '@' in front of everything to prevent DOS from echoing back the entire command. Finally, I put %1 instead of $1 for the DOS way to pass in the parameter.

It is probably worth opening a cmd.exe window and testing the batch file. Assuming you called the batch file "thesaurus.bat", run thesaurus.bat loose and you should get a listing back after a few seconds with all the synonyms of "loose".

Point Vim

In Windows, $HOME expands to "C:\Documents and Settings\{user}".¹ However, it doesn't expand in a script if it is contained within quotes. So the vim script should be changed from

to

If you put the batch file in a sub folder, add that folder name in front of '\thesaurus.bat' like this: '\my_folder\thesaurus.bat'.

If it does not work, because you tested the batch file itself beforehand, the problem is almost certainly in how you specified the location of your batch file in the vim script.

]]> http://www.nickcoleman.org/blog/index.cgi?post=vim-thesaurus%21201202170802%21general%2Cblogging%2Cinternet%2Cprogramming%2Csoftware%2Cunix/feed/ http://www.nickcoleman.org/blog/index.cgi?post=tilingwindowmanager%21201005041032%21software http://www.nickcoleman.org/blog/index.cgi?post=tilingwindowmanager%21201005041032%21software#comments Tue, 04 May 2010 10:32:00 -0700 Nick software http://www.nickcoleman.org/blog/index.cgi?post=tilingwindowmanager%21201005041032%21software/ I would hazard a guess that almost all unix-like system users are using a desktop of some sort or another. KDE is very popular, along with Gnome, Fluxbox and so on. Some people like all the bells and whistles such as menus, fancy graphics, feedback via sounds, and beautiful backgrounds. Others prefer a utilitarian environment and don't want anything that gets in the way or slows the system down.

I'm in the latter group, probably because my hardware is invariably a few years old with not the latest and greatest graphics card. It's not such a big deal nowadays, but fancy graphics used to slow things down too much, and now I'm used to the leaner look. I'm a touch-typist too, so I like to keep my hands on the keyboard. It really slows me down and interrupts my chain of thought if I have to lift my hands to go and move the mouse. I'm not a point-and-click person and I invariably use keyboard shortcuts even when I'm using a fancy desktop.

I wonder how many people know that you don't need a desktop at all. Before you throw up your hands in horror at early memories of a single xterm in X11,

Xterm in X11 without a window manager

The environment we're talking about is a tiling window manager. In this environment, there is no desktop. Instead, the screen displays one or more frames and each frame has only one application running in it. A single frame is the same as an application using the full screen, or you may have two frames split either vertically or horizontally on the screen, or three or more. A common setup for programming might be one large frame on the left, and two or three vertical frames on the right with one showing, say, a man page and the other the progress of a compile. Here is an example with 4 frames:

Tiling Window Manager with four frames

Typically you can resize or move a frame around with simple keystrokes. In the example above, you might move frame (or window) 4 up by one frame, so now your display might show the right hand column as window two, window four then window three below.

Why bother with this when your desktop can do the same thing? Two reasons: you do everything without your hands leaving the keyboard; and you have a "fixed" layout that doesn't change no matter what you do with your applications. For example, if you open a new application in a desktop it typically might open in the centre of the screen and take up about two-thirds of available space even if it covers up other applications. That can be annoying. In a tiling window manager, it opens up within the frame and takes up the frame's available space. You know exactly where and how the application will appear.

If you are a touch-typist, you will find it an absolute boon to be able to move across to a spare frame, start up an app, and move back to where you were, all done by using just a few strokes of the keyboard.

If you're a programmer, you will find the layout that you like and tend to use it all the time because it makes you more productive. As a result, almost all tiling window managers allow you to give a name to a particular layout and save it so it can be re-used via a concept called grouping. This ability is very similar to the desktop's concept of a virtual desktop where you might have a shell on Desktop One, a browser and email client on Desktop Two, and so on. In a tiling window manager you might have a group called "shells" with a layout like the image shown above, a group called "browser" with just a single frame with the browser in it, and so on. It is usually just a single keystroke like Alt-1, Alt-2, etc to switch between groups.

Another benefit of a tiling window manager is the speed. It is blistering fast to switch groups. That is to say, it is blistering fast to switch between your display of several xterms and your display with a browser in it. I imagine that is because you have removed a layer of abstraction by not dealing with a desktop, but directly with the window manager.

For me, there are very few drawbacks. In fact, I can't think of any drawbacks. The only thing missing is a status bar like at the bottom of a Desktop window that has a clock, weather, disk space and so on. As we shall see in a coming post, it is very easy to create such a display using tools that work in a similar way to conky and dzen2. Stay tuned for my 700 line shell script that provides any system information you might want and is efficient and easy on resources (the secret is that very little of the 700 lines is actually executing at any one time).

If you want to give a tiling window manager a go, check the Wikipedia article that has more details and a pretty comprehensive list of available window managers. I use Musca because it is very configurable and it lets me pass commands to it from a script, a very useful feature as we shall see in the forthcoming article. ]]> http://www.nickcoleman.org/blog/index.cgi?post=tilingwindowmanager%21201005041032%21software/feed/ http://www.nickcoleman.org/blog/index.cgi?post=phphate%21201002120843%21software%2Cprogramming http://www.nickcoleman.org/blog/index.cgi?post=phphate%21201002120843%21software%2Cprogramming#comments Fri, 12 Feb 2010 08:43:00 -0700 Nick software programming http://www.nickcoleman.org/blog/index.cgi?post=phphate%21201002120843%21software%2Cprogramming/ I've recently had to use PHP to do some work on this site. After using Python for a while and not having used PHP for a few years, I'd forgotten why I hate programming in it.

The documentation is appalling. I've been trying for two days¹ to figure out how to do a very simple (you would think) thing: display the local time. The docs beat around the bush and tell you how to get time difference from GMT, how to format the date, etc, etc. All common vanilla stuff. But how do I simply show the freakin' time?

I've tried echo localtime(). Nope. Ok, how about echo localtime(time());. Nope, it's an array, for god's sake.

How about this helpful suggestion for the PHP developers. echo localtime(), called without parameters, should do just that, using locale timezone defaults, not return an array that presumably I then have format myself.

At least Python has meaningful API calls that you can half-guess yourself.

The situation is not helped because the official documentation can be edited by users. Can you believe it? The official API documentation can be edited by your non-official user. What other language allows that? Answer: none that I use, for very good reasons.

API documentation serves as the official reference. You don't want any johhny-come-lately to be able to add their well-meaning but stupid comments into the mix. God knows API docs can be confusing enough without a user being led up the garden path by some naive self-taught PHP programmer who wouldn't know rigourous methods if they leapt up and bit him on the bum.

So you are trying to find out how to print the time. You start to read the API docs. One thousand lines later, you have read a couple of hundred lines of docs and about five times as many lines again of threaded comments that go off on tangents or become meaningless is-too-no-it-isn't dribble.

Adding to the frustration is that, in their eternal wisdom, the doc site maintainers have decided that syntax colouring is a good thing. Normally it is, I use it all the time in vim. Except that I get to choose the colours in vim. I don't get to choose on their web page where their colours are forced on me. Too bad if I have colour vision deficiencies. Too bad if I have trouble reading orange text on a gray background (yep, that's their default). Too bad if I hate their mono font. I can't change it. Yet, for some reason, I can choose to display the page in Bulgarian. If I want.

That sort of symbolises my problem with the whole language. It's a bizarre mix of strange choices. Personally, I would have designed the web page so that colour theming for the viewer was a bit of a higher priority than seeing the page in Bulgarian. Call me odd, but there you are.

Alright then, yes, I am being a bit facetious. I'm sure that French and German speakers warrant some docs in their language, but English docs that one can't read because of typesetting choices also deserve better.

Hate, hate, hate PHP.

Ok, rant mode off now.

Here is a screen shot of a very simple pie chart. Note the extra features such as a slice that pops out to identify it, a pop up with more information, and sorted columns in the table. These features come with it, you don't need to do anything.

The image shows a pie chart and a table. You can show either or both plus several other types of graphs. I think it looks professional and neat.

The graphs are generated using javascript and ajax, and the browser will go and load the Google javascript (which I guess could cause a small delay, though on my site it is less than a second). Here is the actual page from which the screen shot was taken. You will notice it is quite quick in practical terms. Try clicking on one of the wedges, one of the coloured index icons and a column header.

Here is the code that produced this display:

<html>
  <head>
    <!--Load the AJAX API-->
    <script type="text/javascript" src="http://www.google.com/jsapi"></script>
    <script type="text/javascript">
    
      // Load the Visualization API and the piechart package.
      google.load('visualization', '1', {'packages':['piechart', 'table']});
      
      // Set a callback to run when the Google Visualization API is loaded.
      google.setOnLoadCallback(drawChart);
      
      // Callback that creates and populates a data table, 
      // instantiates the pie chart, passes in the data and
      // draws it.
      function drawChart() {

      // Create our data table.
        var data = new google.visualization.DataTable();
        data.addColumn('string', 'Task');
        data.addColumn('number', 'Hours per Day');
        data.addRows([
          ['Work', 11],
          ['Eat', 2],
          ['Commute', 2],
          ['Watch TV', 2],
          ['Sleep', 7]
        ]);

        // Instantiate and draw our chart, passing in some options.
        var chart = new google.visualization.PieChart(document.getElementById('chart_div'));
        chart.draw(data, {width: 400, height: 240, is3D: true, title: 'My Daily Activities'});
	var table = new google.visualization.Table(document.getElementById('table_div'));
	table.draw(data );
      }
    </script>
  </head>

  <body>
    <!--Div that will hold the pie chart-->
    <div id="chart_div"></div>
    <div id="table_div"></div>
  </body>
</html>

It might look a fair bit more than the few lines of code I mentioned above, but take out the comments and notice that 5 lines are the data itself. Twelve lines of code generate everything. Most of it is template stuff and you provide only a few parameters yourself like table type, column headings and so on. You can see that all of it goes in the <head> section, with just a couple of <div> tags in the body.

Parameters

The chart type is set when you load the vizualization api itself. You specify an array of packages; here I've chosen pie chart and table.

The column names are set when you add a column of data. Then you add the rows of data and there are several ways to do this. The simplest is to place it in an array. Just note that this means your data will be viewable in the source. For simple charts this may not matter, and you can get the data externally in which case it won't show up.

The title, width and height are set when you draw the chart.

Finally, the chart is displayed in a <div> section that you specified when you created the chart.

The point is that it is extremely simple and you don't need to know any javascript at all.

More

The API can draw a range of different chart types such as pie, horizontal bar, vertical columns, lines, scatter plots, and even images like piles of money. You can also get the API to show minimum and maximums, averages, aggregators (similar to SQL aggregators), and get the data from an external source like a Google spreadsheet or a javascript file instead of hard coding it into the source code of the chart's page. If you are a bit more advanced and know python, you can use a python module to talk directly to the API and generate everything within python.

Google's visualization documentation starts here, and the docs for graphs are here. While you are there, check out the Interactive Samples section to see how the different graph types can be used.

Sweet Spot

]]> http://www.nickcoleman.org/blog/index.cgi?post=chartsandgraphsbygoogle%21201002040825%21software%2Cpython%2Cinternet/feed/ http://www.nickcoleman.org/blog/index.cgi?post=kukkaisvoima%21200912201044%21python%2Csoftware%2Cblogging http://www.nickcoleman.org/blog/index.cgi?post=kukkaisvoima%21200912201044%21python%2Csoftware%2Cblogging#comments Sun, 20 Dec 2009 10:44:00 -0700 Nick python software blogging http://www.nickcoleman.org/blog/index.cgi?post=kukkaisvoima%21200912201044%21python%2Csoftware%2Cblogging/ I wanted a simple light-weight personal blogging tool that lets me write in flat files. Like many others, I very much dislike editing text in a browser, especially anything more than a few lines. Browsers seem so overloaded with junk and javascript nowadays that their response time has crept up to several tenths of a second, very frustrating for a touch-typist who needs response times of around a tenth to catch errors ¹. So, this tool needed to work with flat files that I could edit in vim and easily transfer up to the blog host.

As well, it needed to be light-weight. I can't see my blog ever running to more than a few hundred entries per year max. Tools like Wordpress et al are too heavy for me, in the sense that their processing slows things down by comparison.

Python would be good, I prefer it to PHP. I really like it, it is a great language.

Also, I wanted some simple things like categories and comments, and search would be nice too.

Originally, I had tried pyblosxom, but I found the lack of documentation frustrating. I put a few days into it and got the core working and a couple of plugins. Eventually though, I gave up after two days of unsuccessfully trying to get the comments plugin to work. Plus, I couldn't get the logging to work so I couldn't debug the problem. I decided that almost a week's worth of time was enough.

(To be "sort of" fair to pyblosxom, they are in the middle of a version upgrade, which means older plugins are broken and docs are behind the pace. That doesn't solve my immediate need though, so pyblosxom is out for me. And, really, I don't get the whole "I'd rather be programming" thing. I write docs as I go, they form the application's specifications².)

All of which motivated me to look for something else, which led me to Kukkaisvoima, for which I'm grateful. I downloaded and installed Kukkaisvoima on my local machine. It is very useful for my needs: a light-weight simple blogging tool for flat files, written in Python, with comments and categories. The archive and search are a nice bonus.

Kukkaisvoima in a nutshell

The text file's name is used to generate the post's date and categories. For example, the file name for this entry (now changed, so it won't work as a link) is "kukkaisvoima:2009-12-20:python,software,blogging.txt", which kukkaisvoima tokenises into three parts separated by a ':'; the first is (presumably) not used internally, the second is used to generate the post's date, and the third is a comma-separated list of categories for this post. Simple and effective.

Kukkaisvoima automatically shows in the sidebar a list of categories, month archives, and a search box.

Each post can have comments or they can be disabled globally, but not for individual posts. There is very limited checking on comments: required username and email, a simple question-and-answer check for a live person (no captcha or akismet), and html tags are stripped from the comment's text. IP addresses are not logged. There is no database engine, so SQL injection is not a problem.

My Modifications

I modified it a little to suit my needs, which is not a criticism. We all have our own idea of what we need and how important they are.

The main things I wanted to implement were: recording the IP address of people making comments; removing the need for <p> tags to signify a new paragraph when writing a blog entry; and removing the need for a date token in the blog text's filename. Fairly minor stuff, except I really wanted to record the IP address, it was a priority for me.

The IP address turned out to be quite easy once I realised that, like PHP, Python has it in the external environment. I added a hidden attribute to the comment form that recorded os.environ['REMOTE_ADDR'], and added attributes for it to the appropriate classes and functions. In the comments display function, I print it out with the least-significant octet displayed as .xxx, for privacy. (It's still recorded in full, just not displayed.)

The tags thing was easy too. I added a line to the display function that checked for a newline at the start of the line, and replaced the line with "<p>\n". Now I simply put a blank line in the text, and the html renderer adds a <p> tag. (I really don't like writing HTML when I'm in the middle of writing text. It interupts the flow.)

My next change will be to the file name tokenising, to remove the need for the date in the file name. For my needs, I'm happy to use os.stat.CTIME to generate the post's date.

I also made some minor format changes: the date display uses a strtime() format; date appears directly under the headline rather than with the comments; IP address for each comment; and some spacing and colour changes in the CSS.

In the CSS, I did change many px attributes to either % or em. I have a wide-screen laptop and fixed-size px meant the columns were quite narrow compared to the screen's real estate. Again, thanks to the clear relationship between divs in the html and the css, it was very easy to find which classes and ids needed to be changed.

All in all, it has been a great learning experience. I have brushed up on my Python and my CSS understanding. The author deserves kudos for making the codebase clear and logical with easily understood relationships.

Oh, it's Finnish for "flower power" according to google translate.