Refreshing iptables

The last time I played around with IPTables was about 6 - 7 years back. I have been working with a Linux box again from the last few months and I am absolutely thrilled about it. Nevertheless, I have also been meaning to set up a simple IPTables firewall on my machine.

Since I had forgotten all my IPTables concepts, I decided to hunt the Internet for some good articles. I found some really nice resources to refresh my memory as well as learn new things.

Here is a nice 3 part video series from Linux Journal.

1. iptables part 1
2. iptables part 2
3. iptables part 3

The entire series takes less than a half hour and is a good refresher or introduction to IPTables.

Here is a nice picture which explains how a packet is routed through various chains in the ip tables. If you are looking for a quick refresher, this might help you out.

If you want more details, here is a tutorial, and yet another tutorial, and Netfilter's excellent documentation on IPTables.

So now there is no excuse for not setting up a little firewall on your Linux box.

Inheritance vs. composition depending on how much is same and how much differs

I am reading the excellent Django book right now. In the 4th chapter on Django templates, there is an example of includes and inheritance in Django templates.

Without going into details about Django templates, the include is very similar to composition where we can include the text of another template for evaluation. Inheritance in Django templates works in a way similar to object inheritance. Django templates can specify certain blocks which can be redefined in subtemplates. The subtemplates use the rest of the parent template as is.

Now we have all learned that inheritance is used when we have a is-a relationship between classes, and composition is used when we have a contains-a relationship. This is absolutely right, but while reading about Django templates, I just realized another pattern in these relationships. This is really simple and perhaps many of you may have already have had this insight...

We use inheritance when we want to allow reuse of the bulk of one object in other objects such that it can change a few things as per it's requirements. So in a way we have a template which we want to allow reuse of in the greater part and modification of in the smaller part. On the other hand we use composition when we want to reuse another object in this object. So we have a template in which we want to plug some holes in functionality.

Does this make sense?

Slashy strings in Groovy

Groovy has done a great job of enhancing Java Strings. It offers a lot of features like String interpolation with GStrings, triple quoted Strings, multi-line Strings, and slashy Strings.

In this post I will talk about slashy Strings in Groovy. But before doing that let us see how we represent a regular expression in Java. Let's say I have a list of fully qualified file names and I want to match all files in my 'c:\tmp' directory.

I would create a String to represent my regex in Java like this:
String exp = "C:\\\\tmp\\\\.*"

The four '\' are needed because '\' is a meta character in regular expressions, so we need to represent a '\' as a '\\'. Because a '\' is used for escaping special characters in Java, we need to represent '\\' as '\\\\'. Wow doesn't this look cumbersome. Well, this is not just one case. Regular expressions make use of the '\' character for special classes. Everytime we want to use a '\w' or a '\d' or something like that we will have to use '\\w' and '\\d' instead.

Slashy Strings in Groovy give us a way around this by allowing us to represent regular expressions just like they would be represented without having to escape the '\' in Java.

Using them we can write

def file = /C:\\tmp\\.*/

Notice that slashy Strings have to be surrounded by forward slashes, and are mostly used when Strings need to represent regular expressions.

Testing Groovy domain classes

If you are trying to test Grails domain class constraints by putting your unit test cases in the 'test/unit' directory, then your tests will fail because the domain objects will not have the 'valdate' method. This can be resolved in two ways:

1. Place the test cases inside test/integration (which will slow things down)
2. Use the method 'mockForConstraintsTests(Trail)' to create mock method in your domain class and continue writing your test cases in 'test/unit'

What follows is some example code around this finding.

I am working on a Groovy on Grails project for a website to help programmers keep up and refresh their skills.

I started with some domain classes and then moved on to write some unit tests. When we create a Grails project using

grails create-app

,
it creates several directories, one of which is a directory called 'test' for holding unit tests. This directory contains two directories, 'unit', and 'integration' for unit and integration tests respectively.

Now begins an interesting journey with writing unit tests in Groovy. I wanted to write tests for my domain classes. A simple domain class like the one I have below can contain only properties and constraints for those properties. This is what Trail.groovy looks like.

class Trail {
 String shortName
 String name
 String description

 def hasMany = [learningObjects:LearningObject]

 static constraints = {
   shortName(maxSize:6, blank:false)
   name(maxSize:75, blank:false)
   description(maxSize:2048, blank:false)
 }
}

I wanted to start writing unit tests to test the constraints. When Grails creates a domain class, it injects several methods in it at runtime , one of which is a method called 'validate'. This method is called before the domain object is saved and it will return a false if the domain object has violated any constraints. So, I created a simple unit test to test the constraints of Trail.groovy

This is what my initial test case looked like.

import grails.test.*

class TrailTests extends GrailsUnitTestCase {

  def trail

  protected void setUp() {
    super.setUp()
    trail = new Trail(shortName:'sname',
                            name:'Java 101',
                            description:'This is a basic Java course')
  }

  protected void tearDown() {
    super.tearDown()
  }

  void testShortNameConstraints() {
    assertTrue(trail.validate())
    trail.shortName = 'thisisaverylongname'
    assertFalse(trail.validate())
    trail.shortName = ''
    assertFalse(trail.validate())
  }
 
  //... further methods not shown
}

When I ran my tests using

grails test-app

(See how well Grails integrated testing), I got a bunch of errors that told me that the 'validate' method was not found. If you enjoy looking at stack traces... feast your eyes:

Testcase: testShortNameConstraints took 0.159 sec
 Caused an ERROR
No signature of method: Trail.validate() is applicable for argument types: () values: []
groovy.lang.MissingMethodException: No signature of method: Trail.validate() is applicable for argument types: () values: []
 at TrailTests.testShortNameConstraints(TrailTests.groovy:20)
 at _GrailsTest_groovy$_run_closure4.doCall(_GrailsTest_groovy:203)
 at _GrailsTest_groovy$_run_closure4.call(_GrailsTest_groovy)
 at _GrailsTest_groovy$_run_closure2.doCall(_GrailsTest_groovy:147)
 at _GrailsTest_groovy$_run_closure1_closure19.doCall(_GrailsTest_groovy:113)
 at _GrailsTest_groovy$_run_closure1.doCall(_GrailsTest_groovy:96)
 at TestApp$_run_closure1.doCall(TestApp.groovy:66)
 at gant.Gant$_dispatch_closure4.doCall(Gant.groovy:324)
 at gant.Gant$_dispatch_closure6.doCall(Gant.groovy:334)
 at gant.Gant$_dispatch_closure6.doCall(Gant.groovy)
 at gant.Gant.withBuildListeners(Gant.groovy:344)
 at gant.Gant.this$2$withBuildListeners(Gant.groovy)
 at gant.Gant$this$2$withBuildListeners.callCurrent(Unknown Source)
 at gant.Gant.dispatch(Gant.groovy:334)
 at gant.Gant.this$2$dispatch(Gant.groovy)
 at gant.Gant.invokeMethod(Gant.groovy)
 at gant.Gant.processTargets(Gant.groovy:495)
 at gant.Gant.processTargets(Gant.groovy:480)

After some Googling I realized that the 'validate' method is injected into domain objects by the Grails framework, and therefore it does not exist when we instantiate the domain object from test code. I later found that if a test is placed inside 'test/integration' then the test cases are created as the objects would have been created by the Grails framework. So the domain objects would have the 'validate' method. But I also read in another place (I cannot remember where), that if possible it is k better to mock these special methods instead of putting tests in the integration directory, since these tests take longer to instantiate and can slow down the overral running time. And I found in yet another place that there exists a method called 'mockForConstraintsTests(Trail)' which will inject the 'validate' method along with some other methods. So, I added this mock method to my setup after which I was able to run my tests successfully.

  protected void setUp() {
    super.setUp()
    mockForConstraintsTests(Trail)
    trail = new Trail(shortName:'sname',
                            name:'Java 101',
                            description:'This is a basic Java course')
  }

The rest of the class is the same.

So to summarize, if you are trying to test constraints in Grails domain classes, you have two options:

1. Place the test cases inside test/integration (which will slow things down)
2. Use the method 'mockForConstraintsTests(Trail)' to create mock method in your domain class and continue writing your test cases in 'test/unit'

I will post more such episodes of how Google helped me resolve issues I faced while developing :-)

Groovy on Grails

Grails is a web application framework build on top of well known languages, frameworks and libraries. It uses the Groovy programming language which gives it power because of it's dynamic nature. Under the hoods, Grails uses Spring framework(for MVC), Hibernate (for OR mapping), Quartz (for scheduling), Log4J (for logging), JUnit for unit testing, and Canoo Webtest for functional testing.

We already have a Gazillion frameworks, do we need Grails ?

Typically web development in Java is a very long and arduous process. When a web project is started, many configuration files have to be set up after which the project development begins. Even then we have to create domain objects, basic controller functionality, the model, stylesheets, views, and other aspects of business logic. Grails makes things easier by allowing us to do all of these things rapidly and in fewer lines of code. So how does Grails allow us to do all this with fewer lines of code? Grails uses convention over configuration.

If you have developed a JEE app with Struts, you probably remember configuring struts-config.xml to map ActionBeans, Actions, Action classes, Global forwards, and so on. A typical development workflow with Struts is as follows:

1. Create struts-config
2. Create ActionBean classes and write their validators
3. Create Action classes and delegate persistence and other things to model classes
4. Create a Hibernate configuration file
5. Create views with JSP's using Taglibs
6. Write unit tests (these two are not really the last two... remember TDD)
7. Write functional tests

Whew.... isn't that a lot of work to do? In Java I have always felt like there is just a lot of coding and configuration that needs to be done to make even the smallest web application.

Grails takes away a lot of this pain. This means we do not have to specify and code the smallest of things if we follow certain conventions. We however have the ability to break the conventions if we want to (in case of integrating with legacy databases, etc) by adding configuration.

So, for example lets assume we want to make a simple application to manage a music collection. We start with creating a Grails app with a grails command, after which we create a domain class. Thats it. Grails will scaffold the application for us, meaning it will create the entire infrastructure to give basic CRUD functionality with our domain object (this includes the controller, and views for adding, deleting, updating, and modifying data). So within minutes we have something we can use. Off course in most situations we will not want something as simple as this. We will have to extend the application in various ways. But I suspect, even after extending the application we will still have saved some time by using Grails, and there is another benefit. Because Grails gives us something really quickly, we can use the Agile approach where we quick feedback on what we are trying to accomplish, and every feedback loop actually gives us something that is usable and .

Because Grails is written in Groovy which is a JVM language, we can take advantage of all the Java libraries that we have become familiar with and also gain benefits from the JVM's optimizations.

A few other features of Grails which I have noticed and like are:

1. Ability to upgrade to a newer version of Grails with a single command
2. Automated creating of test classes (thus prodding you to write unit tests)
3. Ability to use different databases for testing, development and deployment
4. Command for creating domain objects, and controllers
   
5. Inbuilt UI templating
6. Host of third party plugins that extend the Grails framework
7. Extremely simple mechanism to create Tag libraries

These are some of the features that I have read about, but I am sure there are many more.

If you build web applications in Java, Grails is definitely be something you should look into.

More about Grails coming up in future posts.

Internet and new media for teaching and learning

Today, Freeman and I did a video talk for a faculty development program at SCIT on how teachers can use the Internet and New Media for teaching and learning.

Internet And New Media For Teaching

View more presentations from parag.

Some useful links for those who are new to this medium.

Google Groups is a good and simple way to start a mailing list.

Google Reader is a good blog reader.

For hosting a blog, I recommend either Blogger or Wordpress to start with. If you want more control, you can download the Wordpress software and install it in your own server.

ITunes is a good client software for listening to podcasts. But ITunes works only on Windows and Mac. For those running Linux, the Miro Player is a good option for video podcasts, and Amarok or JPodder for audio podcasts.

ITunes university has several podcasts hosted by universities worldwide.

If you want to create a podcast, you can use Audacity for recording them and Odeo for free hosting.

Those interested in Twitter can head straight to Twitter.com and here is an article describing how Twitter was used by a professor in the classroom.

To host your bookmarks on the net, you can use Delicious.

Free wiki hosting is available at PBWiki. If you want to install your own wiki, then download Mediawiki, or Twiki.

Moodle is the course management system that we spoke about.

Some educators who have inspired me by sharing their knowledge are: Stephen Downes, Konrad Glogowski, and Leigh Blackall

CompSci videos at Stanford

I came across some very interesting videos published by Stanford. These videos are recordings of some computer science courses offered at Stanford. I still have not had a chance to see all of them, but I am sure they will be very informative.

• Programming methodology series at Stanford (lectures 23/24 may be useful for developers who already know programming) - video
  
• Programming abstractions series at Stanford (looks like an interesting lectures. They focus on data structures, implications of using certain data structures, recursion, algorithm analysis like the Big O notation) - video
• Programming paradigms series at Stanford (Talks about imperative, OO, & functional programming. Also discusses many good low level things like pointers, big/little Endian, representation of data as bytes, and also touches on algorithms, assembly code, etc) - video
• Machine learning series at Stanford - video

I hope you find these useful. I will share more links as I discover them.

Some more impressions of Python

List comprehensions: List comprehensions are pretty neat. Before studying Python I did not know what they were. This is a wonderful way to perform some operation on an entire list. Let me explain with an example. Let's say we have a list of numbers and want another list containing the double of all the numbers greater than 5 from the first list. In Java we would have to do something like this:

private static List getDoubleOfMembers(List numbers) {
List result = new ArrayList();
for(int num : numbers) {
  if(num > 5) {
    result.add(num*2);
  }
}
return result;
}

However, in Python we can perform this operation using list comprehension like this:

[n*2 for n in numbers n > 5]

Isn't this nice?

Lambda expressions: Python supports Lambda expressions but it does not support closures like Groovy. I wish Python supported closures.

Unchecked exceptions: The blogosphere has had more than it's share of discussions on checked vs. unchecked exceptions. I will not spam your feed reader with any more on this topic. Suffice to say, I like them. In case you want to read more about this debate, this Developer works article is a good starting point.

Pickle: Python has a module called pickle for object serialization and deserialization. Using pickle we can save an object to a file and then reconstruct it as well. We can also use pickle to transmit objects over Internet sockets. There is C implementation of pickle called cPickle which is many times faster. Python does not seem to use anything to signify that an object can be pickled (like the Serializable interface of Java), however, it does have a notion of picklable and non picklable objects. I am still not sure if Python has the equivalent of SerialVerUID for forward compatibility of serialized objects.

repr: Python has a function called repr() which converts an object into a canonical (stringified... if that is a word :-) ) form.
Call repr(obj) to get the string form of an obejct. The return value can be controlled by implementing the __repr__() function in the class. Typically the following should be true:

eval(repr(obj)) == obj

I like this statement. Joshua Bloch wrote in his book Effective Java, Item 9, that if a class's toString() method is documented, then it should return a string such that we should be able to recreate a similar object from that string using the class's String constructer. The repr() function fulfills that criteria.

My first impressions of Python for the second time

I had worked a bit in Python many years back. Since then I have forgotten almost everything I learned back then. I think the phrase "Out of sight out of mind" applies perfectly to my mind.

Since the last few days, I have started relearning Python, and this time I am recording my impressions of Python after having come to it from a Java background.

Indentation: Python uses indentation to specify blocks of code, instead of curly braces. I like this, because we anyways indent code to increase readability, so why not achieve two tasks together. Code looks much cleaner without the curly braces. However there may be a little downside. Everyone in the team will have to set up their IDE's in the same way. Things might fall apart if some people use tabs and others use spaces for indentation.

Access modifiers: Python does not have public, private, and protected keywords. Everything is public. However, private members can be specified with a leading single underscore. If we use double leading underscores then Python's runtime will do name mangling and make it difficult to directly access those members. Note that I use the word difficult, and not impossible. Personally I like not having access modifiers. Using convention to specify a member as private is fine in my books, because I have never thought of access modifiers as a security feature. It is rather a feature that prevents well meaning programmers from making mistakes. If there is a simple convention like leading underscores signifying a private member, then well meaning programmers will not access it directly. We do not need to treat developers as children and put fences all around the place.

String interpolation: In a previous blog post I wrote about how Python supports string templating, but not interpolation. This is one feature I wish Python supported.

String and Unicode: Python strings are not Unicode by default. For Unicode strings there is a separate Unicode type. On the other hand in Java all strings are always Unicode. Update: Some friends just informed me that Python 3000 has Unicode strings by default.

Multiple inheritance: Python supports multiple inheritance. Again I am not sure I like this.

Numbers: I tried this code in Python

max64BitInt = 2**64 - 1
bigNum = max64BitInt + 126
print 'This is a big number ', bigNum

and it prints the answer correctly. So Python does not have the 64 bit limit for integers the way Java does. In Groovy, numbers can be BigInteger types and can be arbitrarily large. I do not know if this is true in Python. However what we have is definitely better than Java's support for working with large numbers.

Magic method: Python has magic methods. These methods begin and end with double underscores, like __str__(). Magic methods in Python have special meaning, in that they exist to be called back when a certain procedure is invoked. For instance the __len__() method of object o is invoked when we call the len(o) function. It returns the logical length of the object. This is a nice feature because it allows the system to have certain standard functions like len(), repr(), print(), etc and allow developers to have magic methods in their classes if they want their classes to respond to these standard functions. This is how Python also supports operator overloading, as well as list iteration and indexing.

Procedural and OO programming styles: Even though everything in Python is an object, Python supports both procedural and object oriented programming styles. I believe it is also possible to write functional code in Python. But I do not know enough about it. That is a topic for a future post. I like support for procedural programming, because it makes it easy to create small useful scripts. One of the reasons I could never write those in Java was because I would have to create a class and a main method and also put the script through the compile phase. But then again, Java was never meant to be a scripting language. Python also has globally available functions like len(), print(), repr() etc. For example len(o) takes an object o, (maybe a list or a string), and returns it's length. In Java we would have invoked o.length(). I guess there are pros and cons to both approaches. Java's style is more object oriented, but it does not guarantee uniformity. We use the length attribute to get an array's length and the size() method to get the length of a List. In Python if an object o has a logical length, we can get it by invoking len(o).

I have some more impressions on Python, which I will publish in the next post.

String interpolations in different languages

I am learning Groovy and refreshing my Python skills as well. While working on some examples, I realized that the operation of concatenating string to produce a larger string is a very frequently used operation. Very often parts of the string we are creating are already held in variables. A crude way of creating the larger string, is to simply concatenate everything as shown in this simple toString() method of the Person object.

public class Person {
  //showing only relevant code
  public String toString() {
    return "Name: " + name + " salary: " + salary + " address: " + address;
  }
}

However, this is not very readable. A better way is to use either string interpolation or templating. Java gives us a couple of ways to do templating.

We can either write the toString() method like this to become much more readable:

public String toString() {
  return String.format("name: %s salary: %d address: %s", name, salary, address);
}

Or we can use the MessageFormat class:

return MessageFormat.format("name: {0} salary: {1} address: {2}", name, salary, address);

I personally think the last option is more readable. Moving to Python, it also gives us ways to do templating:

return "name: %s salary: %d address: %s" % (name, salary, address));

There is yet another way:

 print "name: %(name)s salary: %(salary)d address: %(address)s" % locals()

Notice that both, Java and Python, support templating but not interpolation. Groovy on the other hand supports interpolation which is far more elegant and concise.

public class Person {
  //showing only relevant parts of the code
  public String toString() {
    return """name: ${name} salary: ${salary} address: ${address}"""
  }
}