Briefly, % is the operator that allows you to do C style sprintf operations. If you don't know them, they let you substitute values into a string, like this: print "%s is the time" % "now" which give the result "now is the time". The '%s' is an indicator to insert a string here, and the string to insert is in the second part of the expression, the 'now'. There are many other types of formatted insertions: int, float, scientific and so on as well as string.

If you do any sort of web programming, you use the print statement a lot. After all, the purpose of your program is to generate HTML for the web server to serve up to your users, and you make the HTML by outputting strings using print.

If your coding is anything like mine, you start off by using lots and lots of print statements, just to get the ball rolling and getting some sort of output. Often the prints are interspersed with a bunch of calculation statements , creating a bit of a jumble. Later, I go back and refactor the block to bring all the calcs to one place and all the prints to one place, then try and consolidate them to reduce the number of statements. (Although the server does do some caching to reduce the IO between server and app, it still makes sense to give your input to the server in as few statements as possible.) In particular, I try to move print statements around so I can consolidate them into big blocks of """ ... """. This has two benefits: it reduces the number of print statements, and, perhaps more importantly, I can more easily see the structure of the HTML since I can format the tags so they appear in logical blocks rather than spread over one long line.

By the way, while doing this I discovered that print """ ... """ can use the % operator too. I didn't know that and assumed for some reason that you could use it only for "normal" print statements. That was quite a discovery and made generating the HTML much easier, both to create and debug. So now I can write print """ ... %s """ % "now" and I get the correctly formatted substitution in my output.

However, using substitution in large blocks of HTML becomes cumbersome. Say you want to insert a new value in the middle of a block of HTML. You now have to count the parameters and hope you are inserting in the correct place: print """ <h1> some word %s inserted</h1> <p>another word %s inserted</p> <p>another word %s inserted</p> <p>another word %s inserted</p> <p>another word %s inserted</p> <p>another word %s inserted</p> <p>final word %s inserted</p> """ % ('now', 'is', 'the', 'time', 'all', 'good', 'men')Tricky. Especially when you come to maintain it six months later.

I've intentionally made a mistake in the above example (the values are missing the word 'for' in the start of the well known phrase "now is the time for all good men to come to the aid of their party"). How would you fix it? First you need to work through the %s's to figure how where to insert the extra %s, then you would need to count through the substitution values to figure out where to add the new value of 'for'. Very easy to make a mistake, and a pain to actually do.

Aside: it's probably not really an issue when you're dealing with multiple print statements. Each print probably has a substitution list of only a few values. But once you have consolidated print statements, as in this use case of writing HTML, each print may have a dozen or more substitutions. That's when it gets tricky.

There is a better way to do string formatting and that is to use a dictionary. Many python people don't use this, if they are anything like me, because the doc page does a pretty poor job of explaining it, at least in terms of why you would want to. The docs show the syntax, but not an example of its benefits. We will come to the benefits in a second.

This is how you use a dictionary instead of a tuple of values. Create a dictionary (called phrase in this example) with all the values and keys to represent them. It's not much extra work; the values fall out when you are doing the calculations to generate them and it is a simple matter to put them in a dictionary at that time. Then, in the print statement, use the keys to insert the values from the dictionary. Here's the dictionary: phrase = { 'word1': 'now', 'word2': 'is', 'word3': 'the', 'word4': 'time', 'word5': 'for', 'word6': 'all', 'word7': 'good', 'word8': 'men', 'word9': 'to', 'word10': 'come', 'word11': 'to', 'word12': 'the', 'word13': 'aid', ...} Not a great choice of key names, but I don't want to get bogged down in details. I want to make the example clear, so I chose the key to reflect the word's position within the phrase. (In actuality, you would choose key names that reflect the purpose of the variable, such 'name' or 'email'.)

Here's the print statement: print """ <h1>some word %(word1)s inserted </h1> <p> another word %(word2)s inserted</p> <p>another word %(word3)s inserted</p> <p>another word %(word4)s inserted</p> <p>another word %(word5)s inserted</p> <p>another word %(word6)s inserted</p> <p>final word %(word7)s inserted</p> """ % phrase Notice the change: the value's key is placed between parentheses that are between the '%' and the indicator 's'.

Some points: the key is specified between parentheses placed in between the substitution operator '%' and the substitution type 's' (or d, f, g, etc); the key is not stringified, but typed as it appears in the dictionary without the enclosing ''; and finally you need only to specify the dictionary name at the end of the print statement.

You can see this is very easy to maintain. If I want to insert a value, I don't need to count any values to make sure I'm specifying the correct one in the correct place, I just specify it by key name. It's much harder to get wrong.

An added benefit is that there is (unintuitively) less typing since you specify only key names, not like dictionary[key].

Finally, and this is the real kicker, the number of keys in the dictionary doesn't have to be the same as the number of substitutions. This is great; it is such a benefit that even by itself it means that a dictionary is much better than using parameter substitution. With "normal" substitution, the tuple has to have the same number of items as there are substitutions (unless you can slice the tuple), and the values have to be in the order that you want to use them (rather than unordered). You have to have a one-to-one match between each %s and 'value'. You can't have too few values (obviously), but you also can't have too many, which is annoying sometimes when working with tuples and lists. With a dictionary, you can pre-compute any number of values and just pick off the ones you need.

So what? Say you are generating a <form> which is re-using values from a previous POST, and that the form is quite complex with about 30 different elements ranging from textboxes, drop-down lists to radio buttons. If you use "normal" %s substitution, the print statements become very complex and it becomes hard to maintain and change. (I had real life experience of this when making my ephemeris.) Instead, store all the values in a dictionary and recall them by using their key name, and you will find maintenance of the form becomes a snap.

I hope this gives you a real life example of the benefits of using a dictionary in string formatting. I find it both better and easier to use for anything involving intermixed calculations and print.

]]> http://www.nickcoleman.org/blog/index.cgi?post=pythonprintform%21201009230930%21programming%2Cpython/feed/ http://www.nickcoleman.org/blog/index.cgi?post=python%21201007071205%21programming%2Cpython http://www.nickcoleman.org/blog/index.cgi?post=python%21201007071205%21programming%2Cpython#comments Wed, 07 Jul 2010 12:05:00 -0700 Nick programming python http://www.nickcoleman.org/blog/index.cgi?post=python%21201007071205%21programming%2Cpython/ I've been spending the last couple of months teaching myself Python properly. I already had some basic knowledge of it, enough to do, say, some tasks equivalent to Bash shell scripting, but I knew I didn't know the language properly. I thought I would jot down some of the references and texts I found useful.

If you know any programming language at all, you will find Python easy. Its syntax is natural and clear. If you have formal programming skills, and by that I mean that you can program in another language fairly proficiently, you will find that Python has an elegance that delights; it is a beautiful language. On top of that, it has object-oriented features, data-oriented features, and some functional programming features. As well, it has the rapid development facility of any scripted language, and yet it is fast.

On to texts. I read a great deal and already have several hundred books, so I now intentionally limit my book-spending budget. I found that this wasn't a hinderance here, though. There are some excellent texts online that are free. Please bear in mind that I was not looking for beginner books since I am a programmer; I was looking for books that assume some programming experience. I found three particularly useful:

• Dive Into Python by Mark Pilgrim.
• Building Skills In Python by Steven Lott.
• Thinking In Python by Bruce Eckel of Thinking in C++ and Thinking in Java fame.

These three complement each other by covering slightly different ground.

Dive Into Python covers Python in several chapters, going from simple features to advanced features. It is brief, but comprehensive. I think of it as a primer. The latter chapters become more specific to certain tasks such as xml processing, using the http module, and so on. There is a lot of detail without much verbiage, which I quite like. Skills are built by following examples that build up on previous ones in each chapter.

Building Skills in Python is a very comprehensive text. This one is more like a text book that you would use in a university or college course on Python. It seems to cover the full gamut of Python, as far as I can tell. There is a number of exercises at the end of each chapter, useful for testing your skills. The exercises intentionally are not particularly easy and are more like a mini-project than a one-or-two sentence answer. I suspect the author doesn't expect you to do each one.

The text can be searched and the source can be viewed. This is now my reference book.

Thinking In Python is different to the other two, and from Eckel's Thinking in C++ Vol 1. It is more like Eckel's Thinking in C++ Vol 2 in that is more about design patterns and less about teaching the language. I'm yet to finish this one so I'll leave it at that for now and come back later.

A newcomer will benefit from both of the first two books. I would start with Dive Into Python first then, when the content becomes a little too specific for your needs, which will happen after several chapters, switch to Building Skills In Python. That should give you a good grounding.

]]> http://www.nickcoleman.org/blog/index.cgi?post=python%21201007071205%21programming%2Cpython/feed/ http://www.nickcoleman.org/blog/index.cgi?post=chartsandgraphsbygoogle%21201002040825%21software%2Cpython%2Cinternet http://www.nickcoleman.org/blog/index.cgi?post=chartsandgraphsbygoogle%21201002040825%21software%2Cpython%2Cinternet#comments Thu, 04 Feb 2010 08:25:00 -0700 Nick software python internet http://www.nickcoleman.org/blog/index.cgi?post=chartsandgraphsbygoogle%21201002040825%21software%2Cpython%2Cinternet/ Have you sometimes wanted to make a quick graph or a pie chart to display on your website? Decided it was too hard for the amount of effort and given up? There is a very simple solution that is remarkably easy. You can use Google's Visualization API to create the graphs for you with just a few lines of code.

Here is a screen shot of a very simple pie chart. Note the extra features such as a slice that pops out to identify it, a pop up with more information, and sorted columns in the table. These features come with it, you don't need to do anything.

The image shows a pie chart and a table. You can show either or both plus several other types of graphs. I think it looks professional and neat.

The graphs are generated using javascript and ajax, and the browser will go and load the Google javascript (which I guess could cause a small delay, though on my site it is less than a second). Here is the actual page from which the screen shot was taken. You will notice it is quite quick in practical terms. Try clicking on one of the wedges, one of the coloured index icons and a column header.

Here is the code that produced this display:

<html>
  <head>
    <!--Load the AJAX API-->
    <script type="text/javascript" src="http://www.google.com/jsapi"></script>
    <script type="text/javascript">
    
      // Load the Visualization API and the piechart package.
      google.load('visualization', '1', {'packages':['piechart', 'table']});
      
      // Set a callback to run when the Google Visualization API is loaded.
      google.setOnLoadCallback(drawChart);
      
      // Callback that creates and populates a data table, 
      // instantiates the pie chart, passes in the data and
      // draws it.
      function drawChart() {

      // Create our data table.
        var data = new google.visualization.DataTable();
        data.addColumn('string', 'Task');
        data.addColumn('number', 'Hours per Day');
        data.addRows([
          ['Work', 11],
          ['Eat', 2],
          ['Commute', 2],
          ['Watch TV', 2],
          ['Sleep', 7]
        ]);

        // Instantiate and draw our chart, passing in some options.
        var chart = new google.visualization.PieChart(document.getElementById('chart_div'));
        chart.draw(data, {width: 400, height: 240, is3D: true, title: 'My Daily Activities'});
	var table = new google.visualization.Table(document.getElementById('table_div'));
	table.draw(data );
      }
    </script>
  </head>

  <body>
    <!--Div that will hold the pie chart-->
    <div id="chart_div"></div>
    <div id="table_div"></div>
  </body>
</html>

It might look a fair bit more than the few lines of code I mentioned above, but take out the comments and notice that 5 lines are the data itself. Twelve lines of code generate everything. Most of it is template stuff and you provide only a few parameters yourself like table type, column headings and so on. You can see that all of it goes in the <head> section, with just a couple of <div> tags in the body.

Parameters

The chart type is set when you load the vizualization api itself. You specify an array of packages; here I've chosen pie chart and table.

The column names are set when you add a column of data. Then you add the rows of data and there are several ways to do this. The simplest is to place it in an array. Just note that this means your data will be viewable in the source. For simple charts this may not matter, and you can get the data externally in which case it won't show up.

The title, width and height are set when you draw the chart.

Finally, the chart is displayed in a <div> section that you specified when you created the chart.

The point is that it is extremely simple and you don't need to know any javascript at all.

More

The API can draw a range of different chart types such as pie, horizontal bar, vertical columns, lines, scatter plots, and even images like piles of money. You can also get the API to show minimum and maximums, averages, aggregators (similar to SQL aggregators), and get the data from an external source like a Google spreadsheet or a javascript file instead of hard coding it into the source code of the chart's page. If you are a bit more advanced and know python, you can use a python module to talk directly to the API and generate everything within python.

Google's visualization documentation starts here, and the docs for graphs are here. While you are there, check out the Interactive Samples section to see how the different graph types can be used.

Sweet Spot

]]> http://www.nickcoleman.org/blog/index.cgi?post=chartsandgraphsbygoogle%21201002040825%21software%2Cpython%2Cinternet/feed/ http://www.nickcoleman.org/blog/index.cgi?post=kukkaisvoima%21200912201044%21python%2Csoftware%2Cblogging http://www.nickcoleman.org/blog/index.cgi?post=kukkaisvoima%21200912201044%21python%2Csoftware%2Cblogging#comments Sun, 20 Dec 2009 10:44:00 -0700 Nick python software blogging http://www.nickcoleman.org/blog/index.cgi?post=kukkaisvoima%21200912201044%21python%2Csoftware%2Cblogging/ I wanted a simple light-weight personal blogging tool that lets me write in flat files. Like many others, I very much dislike editing text in a browser, especially anything more than a few lines. Browsers seem so overloaded with junk and javascript nowadays that their response time has crept up to several tenths of a second, very frustrating for a touch-typist who needs response times of around a tenth to catch errors ¹. So, this tool needed to work with flat files that I could edit in vim and easily transfer up to the blog host.

As well, it needed to be light-weight. I can't see my blog ever running to more than a few hundred entries per year max. Tools like Wordpress et al are too heavy for me, in the sense that their processing slows things down by comparison.

Python would be good, I prefer it to PHP. I really like it, it is a great language.

Also, I wanted some simple things like categories and comments, and search would be nice too.

Originally, I had tried pyblosxom, but I found the lack of documentation frustrating. I put a few days into it and got the core working and a couple of plugins. Eventually though, I gave up after two days of unsuccessfully trying to get the comments plugin to work. Plus, I couldn't get the logging to work so I couldn't debug the problem. I decided that almost a week's worth of time was enough.

(To be "sort of" fair to pyblosxom, they are in the middle of a version upgrade, which means older plugins are broken and docs are behind the pace. That doesn't solve my immediate need though, so pyblosxom is out for me. And, really, I don't get the whole "I'd rather be programming" thing. I write docs as I go, they form the application's specifications².)

All of which motivated me to look for something else, which led me to Kukkaisvoima, for which I'm grateful. I downloaded and installed Kukkaisvoima on my local machine. It is very useful for my needs: a light-weight simple blogging tool for flat files, written in Python, with comments and categories. The archive and search are a nice bonus.

Kukkaisvoima in a nutshell

The text file's name is used to generate the post's date and categories. For example, the file name for this entry (now changed, so it won't work as a link) is "kukkaisvoima:2009-12-20:python,software,blogging.txt", which kukkaisvoima tokenises into three parts separated by a ':'; the first is (presumably) not used internally, the second is used to generate the post's date, and the third is a comma-separated list of categories for this post. Simple and effective.

Kukkaisvoima automatically shows in the sidebar a list of categories, month archives, and a search box.

Each post can have comments or they can be disabled globally, but not for individual posts. There is very limited checking on comments: required username and email, a simple question-and-answer check for a live person (no captcha or akismet), and html tags are stripped from the comment's text. IP addresses are not logged. There is no database engine, so SQL injection is not a problem.

My Modifications

I modified it a little to suit my needs, which is not a criticism. We all have our own idea of what we need and how important they are.

The main things I wanted to implement were: recording the IP address of people making comments; removing the need for <p> tags to signify a new paragraph when writing a blog entry; and removing the need for a date token in the blog text's filename. Fairly minor stuff, except I really wanted to record the IP address, it was a priority for me.

The IP address turned out to be quite easy once I realised that, like PHP, Python has it in the external environment. I added a hidden attribute to the comment form that recorded os.environ['REMOTE_ADDR'], and added attributes for it to the appropriate classes and functions. In the comments display function, I print it out with the least-significant octet displayed as .xxx, for privacy. (It's still recorded in full, just not displayed.)

The tags thing was easy too. I added a line to the display function that checked for a newline at the start of the line, and replaced the line with "<p>\n". Now I simply put a blank line in the text, and the html renderer adds a <p> tag. (I really don't like writing HTML when I'm in the middle of writing text. It interupts the flow.)

My next change will be to the file name tokenising, to remove the need for the date in the file name. For my needs, I'm happy to use os.stat.CTIME to generate the post's date.

I also made some minor format changes: the date display uses a strtime() format; date appears directly under the headline rather than with the comments; IP address for each comment; and some spacing and colour changes in the CSS.

In the CSS, I did change many px attributes to either % or em. I have a wide-screen laptop and fixed-size px meant the columns were quite narrow compared to the screen's real estate. Again, thanks to the clear relationship between divs in the html and the css, it was very easy to find which classes and ids needed to be changed.

All in all, it has been a great learning experience. I have brushed up on my Python and my CSS understanding. The author deserves kudos for making the codebase clear and logical with easily understood relationships.

Oh, it's Finnish for "flower power" according to google translate.