This week I was told that my way of going through each elements in an ArrayList was not the most efficient. I should be using an Iterator.

No reason to doubt an experienced programmer so I changed my code.

I never trusted anyone when I did troubleshooting before. I always need to see the evidence for myself. So I decided to do this code to check the validity of the data I was given:

/**
 * Testing the performance of different ways to go through a list of items
 */
package com.cinq.test;

import java.util.ArrayList;
import java.util.Iterator;
import java.util.NoSuchElementException;

/**
 * @author mc
 * @version 0.1.0
 *
 * last updated on 2011-11-27
 *  initial release of these tests
 */
public class Performance {

    /**
     * @param args
     */
    public static void main(String[] args) {

        long dstart = 0;
        long dend = 0;

        // Allocate an ArrayList
        ArrayList<String> data1 = new ArrayList<String>();
        ArrayList<String> data2 = new ArrayList<String>();
        for ( int i = 0; i < 10000; i++ ) {
            data1.add("abc" + String.valueOf(i));
            data2.add("xyz" + String.valueOf(i));
        }

        // Start going through all the elements with a get index method
        dstart = System.currentTimeMillis();
        for ( int j = 0; j < data1.size(); j++ ) {
            data1.get(j);
            for ( int k = 0 ; k < data2.size(); k++) {
                data2.get(k);
            }
        }
        dend = System.currentTimeMillis();

        System.out.println("Get Index for " + data1.size() + " by " + data2.size() + " elements took " + (dend - dstart) + " ms.");

        dstart = System.currentTimeMillis();
        for ( String s1 : data1 ) {
            for ( String s2 : data2 ) {
                // nothing
            }
        }
        dend = System.currentTimeMillis();

        System.out.println("For each for " + data1.size() + " by " + data2.size() + " elements took " + (dend - dstart) + " ms.");

        dstart = System.currentTimeMillis();
        Iterator<String> it1 = data1.iterator();
        try {
            while ( it1.next() != null ) {
                try {
                    Iterator<String> it2 = data2.iterator();
                    while ( it2.next() != null ) {
                        //nothing
                    }
                } catch ( NoSuchElementException e0 ) {
                    //ignore
                }
            }
        } catch ( NoSuchElementException e1 ) {
            // ignore
        }
        dend = System.currentTimeMillis();

        System.out.println("Iterator for " + data1.size() + " by " + data2.size() + " elements took " + (dend - dstart) + " ms.");
    }

}

The output I get from this is:

Get Index for 10000 by 10000 elements took 12 ms.
For each for 10000 by 10000 elements took 271 ms.
Iterator for 10000 by 10000 elements took 166 ms.

This makes no sense. I should see the performance of the Iterator way of doing things to be faster than my get(index) way. I will have to ask my co-worker tomorrow why I am getting this bad performance. Conclusion to follow in the next few days.

Sometimes you have this basic question that you  are too shy to ask around because you should know. Reality is that I always have a few of those and eventually you have to look around for what others are doing if you want to get better.

I just completed some code and I was wondering if using the return code for errors was a good idea or if I should have used exceptions everywhere.

After reading a few articles I think that I did the right thing.

One key sentence that I like is: exceptions are for exceptional conditions. So you should see an exception if the failure is not expected and abnormal to the working of your application. A good point.

Others are saying that you should crash loudly. They are saying to use exceptions as much as possible so that it gets your attention and you fix the issue sooner than later.

The argument that return code makes the application more stable seems to be a bit simplistic because you might be hiding issues that will be more difficult to identify later.

There is room for both approach and I think that experience tells you what to do. Always using one or the other does not seem to make sense. Balance.

Return code makes sense right now because this Java code needs to integrate with some Bash code. If the solution was purely Java I think that I would need to change this strategy.

I may start using more checked exceptions since it gives me a better sense of balance and allows to have more information about the failure.

Continuous improvements.

Wow.

This new type of debugger is something on its own.

It allows you to record your application executing and then replaying it has you need to figure out where the errors are. You simply need to watch their video to get the “wow” factor right away.

It simplifies the code you have to do and gives you a better insight in your application.

I am quite curious to know how it would work on JBoss servers running multiple apps.

http://www.chrononsystems.com/

I have a lot to learn about the pattern and how to be effective at doing continuous delivery of a web app. This presentation is giving a few hints on what you need to be successful:

1. Monitoring
2. Testing
3. Remediation

The monitoring portion is more than just availability. You need to be able to detect any failure quite rapidly otherwise you won’t know to which release to rollback or what to remediate. Monitoring your logs is probably the next thing I need to implement and see how good I can get this. Jez Humble talks about monitoring revenue and it is a great idea for real businesses, it just does not apply to my web app. You can monitor the number of transactions or logins instead of revenue but I will keep that as my next step. Logs first.

Testing, I have a clue on what needs to get done but the cascading of the different type of testing was not something I had in mind until I listened to this presentation. Unit testing is the first step to what needs to get done and I knew that but the next few steps are a bit more clear now.

The remediation pattern that were shown are good ones. Failure does not mean a full code rollback. You can deploy features and turn them on later. I especially liked his Facebook example where the code was there and user loging in were making calls to the back-end without knowing it. This allowed the team to see how the code/app would behave under a more realistic load. That is clever.

Now I have to write down how this is going to change my architecture for this web app. It certainly will improve the plan.

If I wanted the least amount of trouble I would probably use OpenShift on a Fedora laptop or a RHEL6 server.

Since I always give myself a challenge, I decided to use MacOSX to work with OpenShift. Not that it is a bad idea but it gives additional challenges I did not expect. There is always something positive about these challenges. When the basic things don’t work you have to learn about them and how everything is setup. It is always the best setup to learn. Certainly a quick success on that aspect.

Installing the rhc tools to create a domain and app on openshift requires that you have XCode installed. Only $5 to get the app from the Mac App Store but I did not know that it was installing an installer. Once I learned that I needed to install from the installation I was golden to get something done. So I had a relatively empty application from the WSGI template setup on my MacBook Air.

It is great to code on the laptop but sometimes I prefer to use the iMac with the bigger screen and more comfortable setup. I tried to simply do a git pull to get the repo but it would always fail on me. After getting some help from the OpenShift Express Forums I learned that I needed to copy the files from .ssh that were created with the rhc tools. The config and libra_* files were copied and my git pull worked as it should.

Basic things but it is great when it works and that you have learned where things are.

Now I have this great idea that I should try to put Spring Python in OpenShift Express. Looking for more learning, I guess.

I have read and seen a few presentations where the team works in a continuous deployment mentality. I find it very fascinating because I am in situation where it would be of great benefit to me to have this environment.

This is the latest presentation I have look at:
http://www.infoq.com/presentations/Continuous-Deployment-50-Times-a-Day

First reason for this CD environement: no QA team. I am responsible for product/service and I am alone in taking care of it. I have no one to check if what I am doing is good and not breaking any existing features. To achieve the level of comfort needed for continuous deployment you need to have testable code and that is my achille’s heal on this project. There was a bit of code written for a few tests but I added afew features and modified many things without modifying the test code or adding any. I need to change my mentality to get a lot more tests in there so that I can feel at ease when modifying code or adding features.

Next reason is the speed to deploy to your customer the features and change they need. When it takes week to deliver a change people become impatient and they lose respect or trust for your abilities to help them. With the current change management process we have, it takes at least a week to deliver the smallest change. This is a setup for failure and no one can feel comfortable in it. If I implement CD correctly then deploying new code should be this regular activity that no one cares to track. It is simply normal operations.

Since I love numbers, I picked up on the statistical analysis that they do to confirm that the new deployment is good. They look at the number of errors but also the number of request on each page to see if there is a significant differences. This is interesting and is very viable when you have a good amount of traffic. In my case the traffic is small enough that the smallest anomaly will create a significant statistical difference. I will have to think and read more about how to address this one so it gives me good KPIs.

The funny fact that excited me is the automation. It is funny to my project because my main goal with my project is to automate what we do to help us be more efficient. So CD is about automatind the deployment on my automation tool. I love irony.

I am in the middle of adding features to a web application that uses the Spring Framework and since I have not had any training I am learning as we go. As much as I know better this is where I am anyway.

I have been fighting for a few days with a java.lang.NullPointer issue and it was driving me crazy. I knew that it was some small details that was escaping me but I just could not put my finger on it.

I got a breakthrough yesterday evening by re-reading a form post about the fact that you can’t use the new operator on objects that are wired by the Spring Framework. I was also missing the getter and setter method for that entity and then everything started to work. Small victory that certainly pleased me because it was starting to get to my self esteem.

I was a bit mad at myself for not understanding the tip that was given in the forum on the first read. I was looking for something else and the truth was starring me so obviously. The madness will certainly make sure I remember to pay attention to every recommendation experienced people make when answering other beginners like me.

Understanding how the framework does its things is starting to sink in and I am starting to see the full potential this framework has to quickly develop web application.

If I can just spend some time to read on it and understand all the libraries and a few more founding concepts I could stop wasting my time fighting with it.

The trivial issues are the one that drive you nuts because you waste too much time for something that should have taken 2 minutes at most.

Today I was fighting with a spring application that uses hibernate to talk to an embedded derby database.

So simple.

Because I am getting a java null pointer exception when I am trying to save an entity I decided to empty the embedded derby database directory so I could start fresh and make sure that this was not the issue. Again simple.

I started to have all sorts of issues in my spring application logs telling me that it could not create the database (I have the property for derby set to create=true).

All I needed to do was to the delete the directory itself and not just the content.

An hour wasted searching on Google and no one was as dumb as I and posted anything about this issue.

This article is a must read:

http://timothyfitz.wordpress.com/2009/02/10/continuous-deployment-at-imvu-doing-the-impossible-fifty-times-a-day/

The discipline and the level of automation to be able to continuously deploy new code is inspiring.

It may sound impossible to do right now but I think that once you start taking the first step on this path you will not want to go back. This is a lot more fascinating than the current type of testing I see done on my code. It is also more fascinating to think that you can deploy new code easily all the time than having to go through all the paperwork that I am seeing all the time.

FASCINATING!

Learning Python is a lot more fun than I anticipated.

I think that the fun comes from a no expectation mentality and that everything I do with python is currently simple and helps greatly to automate some boring tasks.

I am not sure that I can complain about the fact that from one version to an other the changes are sometimes drastic and being able to understand and find the differences is sometimes more work that I would like. This issues comes from the fact that RHEL5 has Python 2.4 and that my other platforms have Python 2.7.

Today I was trying to understand how to pass a datetime to a xmlrpc server and some functionality was introduced in 2.5 (allow_datetime). It took me 20-30 minutes to understand why it was not working on RHEL5.

As any good learning I am making many errors in small scripts so I hopefully avoid when the real apps come around.