Tech That!

The rantings of a mad computer scientist

Ice Cream Sandwich: Death by paper cuts

I’m a big fan of Android. I have written a few apps, have two phones running it, and I happily recommend it to friends. There is one problem though; the Android experience is riddled with paper cuts. Seemingly minor annoyances that all together make for a frustrating user experience. Google attempted to mitigate some of these in Ice Cream Sandwich, but they somehow managed to introduce a slew of new ones.

In June 2010, Mozilla started the Paper Cuts project. This aimed to fix a lot of the minor quirks that annoyed people working with Firefox all day. I believe it’s about time Google does the same with Android. We don’t need another full overhaul, we just want a more polished experience. So, without further ado, my Android paper cuts:

  • There is no way of hiding applications from the application launcher without uninstalling or disabling them. I don’t use Places, but I still want Google Maps…
  • And why can’t the search bar on top of the homescreen be hidden? Personally, I have never used it. Not a single time. Why should it occupy an entire row on my homescreen?
  • Enabling and disabling auto rotate requires a swipe, three taps and multiple presses of the back button to accomplish. Get real. At least automatically enable rotate when watching video… There are so many ways this could be done: Long click back button, in the power menu, in the notification drop-down. Please give us a shortcut.
  • The whole point of having soft buttons is that they are context aware. This does not just mean hide in fullscreen and rotate. Why is the back button visible when there is no back action? Why is the home button there if I’m already on the homescreen? Couldn’t these provide shortcuts for more useful features in these instances?
  • Why oh why does long click on the soft buttons not have any effect? They are always present, why not utilize them? Toggle auto-rotate for instance… Long press the recent applications button could switch to the latest open application to make switching between two applications very easy.
  • In general, getting to anything takes too many taps. Try setting an alarm. Open app drawer, click clock, click set alarm, click add alarm. Isn’t this common enough that four clicks is a bit too much?
  • The menu button. Or the action overflow. Call it what you’d like. Make it consistent. Why does it appear in three different places? Actually, I don’t want to know why. I just want it to be consistent.
  • Android Market is too slow. It should not take over a second to load, even with a very slow internet connection. Cache the previous content and show that immediately, then update it when fresh information comes in. Usually, we go to the market for a specific action, like opening My Apps. It shouldn’t be necessary to wait for, or even load, the frontpage… We need to see a quick reaction to our inputs.
  • And can someone please fix the millisecond delays that are present for almost every single action? It’s just enough to be noticed, which is not a good thing. This is especially noticeable when using the soft buttons, since they are so often used. I don’t care what is causing it. I’ve read all the blog posts on applications doing too much work on the GUI thread or the fact that user interaction events are prioritized as high as on iOS. I don’t care. It has to be fixed.
    • I have a suspicion that the reason many of these actions feel slow is the little animations that have been placed everywhere in ICS. Closing an app with the back button should not take close to a second – even with animations.

These minor things, and many more like them, taint the Android experience and make it inferior to what it could, and should, be.

Please Google, sit down with a group of your users and have them play around with an Android phone. Note down what bothers them even the slightest, and fix what you can. This will do more for users than any redesign. Make Android feel polished.

Advertisements

Linux homedir dotfiles

Most first-time Linux users come from a world where every program has a settings dialog in which to configure the program; indeed some have never opened those dialogs at all. In Linux, however, many applications still use configuration files located in your home directory. These are often prefixed with a period (.bashrc, .xinit, etc), and referred to as dotfiles for this reason. The reason behind the dot-prefix is that in Linux, files with filenames starting with a period are hidden in most listings by default. Since you don’t want to see all your config files every time you try to find something in your home directory, it makes sense to make them hidden. (If you don’t believe me, open a command prompt and run “ls -la” to see all the dotfiles as well)

So, let’s have a the dotfiles you will most likely want to change. Note that these are primarily for command-line applications, but since you’re using Linux, you will more than likely come in contact with the command line sooner or later.

.bashrc

This file contains all the configuration options for your shell (the application that handles everything that happens in your terminal window).

Below is my current .bashrc file:

# Check for an interactive session
[ -z "$PS1" ] && return

# Color aliases
alias ls='ls --color=auto'
alias grep='grep --color=auto'

# Common aliases
alias more='less'

# Convenience aliases
alias svim='sudo vim'
alias srm='sudo rm'

# Prompt
PS1='[\u@\h \W]\$ '

# Tab completion for sudo
# Disabled because it also autocompletes filenames...
#complete -cf sudo

# Sets the editor to use
export EDITOR='vim'

# Add local bins to path
export PATH="$PATH:/usr/local/bin"

# Other aliases
# make
alias ,='make'
# git
alias .='git pull'
alias ..='git push'
# file handlers
alias o='xdg-open'
# update commands
alias yu='yaourt -Syu --aur'
alias pu='sudo powerpill -Syu'
alias yi='yaourt -S'
# online tools
alias t='tweet'

Note that you should change this to suit your preferences. If you don’t use git, then remove the git aliases or change them to something else you find useful!

.xinitrc

The .xinitrc file is a small, but important, file which controls what happens when you log in and start your X session (The “startx” command). If you’re running a graphical login manager such as GDM, KDM or SLiM, then these will execute startx for you, thus still invoking .xinitrc.
Since I’m running XFCE, my .xinitrc contains “exec startxfce4”.
The “ssh-agent” part runs the entire window manager under a single SSH agent daemon, allowing me to only enter the password for my private keys once in a single session.

exec ssh-agent startxfce4

.Xresources

The .Xresources file configures the visual aspects of all X applications (most notably xterm and xscreensaver).
Note that a lot of these settings will not change anything if you are using another terminal editor such as gnome-terminal or Konsole.
In my .Xresources file below (which is heavily inspired by the Arch Linux Wiki page on the subject), I define a “better” color theme, terminal settings such as font and how my xscreensaver should look. Feel free to experiment these and come up with the perfect settings for your tastes!

One important thing to note is that I am setting xterm to use the Inconsolata font – a very good monospaced terminal font. It is probably available for your Linux distro through the packaging system (Look for ttf-inconsolata)

! terminal colors ------------------------------------------------------------

! tangoesque scheme
*background: #111111
*foreground: #babdb6
! Black (not tango) + DarkGrey
*color0:  #000000
*color8:  #555753
! DarkRed + Red
*color1:  #ff6565
*color9:  #ff8d8d
! DarkGreen + Green
*color2:  #93d44f
*color10: #c8e7a8
! DarkYellow + Yellow
*color3:  #eab93d
*color11: #ffc123
! DarkBlue + Blue
*color4:  #204a87
*color12: #3465a4
! DarkMangenta + Mangenta
*color5:  #ce5c00
*color13: #f57900
!DarkCyan + Cyan (both not tango)
*color6:  #89b6e2
*color14: #46a4ff
! LightGrey + White
*color7:  #cccccc
*color15: #ffffff

! xterm ----------------------------------------------------------------------

xterm*geometry:           100x25
xterm*faceName:           inconsolata:pixelsize=15
!xterm*font:             -*-dina-medium-r-*-*-16-*-*-*-*-*-*-*
xterm*dynamicColors:      true
xterm*utf8:               1
xterm*eightBitInput:      true
xterm*saveLines:          512
xterm*scrollTtyKeypress:  true
xterm*scrollTtyOutput:    false
xterm*jumpScroll:         true
xterm*multiScroll:        true
xterm*toolBar:            false

! xscreensaver ---------------------------------------------------------------
!font settings
xscreensaver.Dialog.headingFont:        -*-dina-bold-r-*-*-12-*-*-*-*-*-*-*
xscreensaver.Dialog.bodyFont:           -*-dina-medium-r-*-*-12-*-*-*-*-*-*-*
xscreensaver.Dialog.labelFont:          -*-dina-medium-r-*-*-12-*-*-*-*-*-*-*
xscreensaver.Dialog.unameFont:          -*-dina-medium-r-*-*-12-*-*-*-*-*-*-*
xscreensaver.Dialog.buttonFont:         -*-dina-bold-r-*-*-12-*-*-*-*-*-*-*
xscreensaver.Dialog.dateFont:           -*-dina-medium-r-*-*-12-*-*-*-*-*-*-*
xscreensaver.passwd.passwdFont:         -*-dina-bold-r-*-*-12-*-*-*-*-*-*-*
!general dialog box (affects main hostname, username, password text)
xscreensaver.Dialog.foreground:         #ffffff
xscreensaver.Dialog.background:         #111111
xscreensaver.Dialog.topShadowColor:     #111111
xscreensaver.Dialog.bottomShadowColor:  #111111
xscreensaver.Dialog.Button.foreground:  #666666
xscreensaver.Dialog.Button.background:  #ffffff
!username/password input box and date text colour
xscreensaver.Dialog.text.foreground:    #666666
xscreensaver.Dialog.text.background:    #ffffff
xscreensaver.Dialog.internalBorderWidth:24
xscreensaver.Dialog.borderWidth:        20
xscreensaver.Dialog.shadowThickness:    2
!timeout bar (background is actually determined by Dialog.text.background)
xscreensaver.passwd.thermometer.foreground:  #ff0000
xscreensaver.passwd.thermometer.background:  #000000
xscreensaver.passwd.thermometer.width:       8
!datestamp format--see the strftime(3) manual page for details
xscreensaver.dateFormat:    %I:%M%P %a %b %d, %Y

.vimrc

If you want to do any kind of advanced text editing (programming, etc), you’ll want a proper editor. And vim is definitely a suitable option when configured properly.
My .vimrc below is largely inspired by Steve Losh’s “Coming Home to Vim“, though it does contain some of my own preferences as well.
With regards to Vim, there are a couple of plugins I recommend you download. The script below only contains a couple of them: ZenCoding and CloseTag, but have a read of the above article and find your own combination.

 " Get indentation
set autoindent

set guioptions-=T " Remove toolbar
set vb t_vb= " No more beeps
set backspace=2 " Backspace over newlines

" GUI fixes
set ruler " Where am I?
set ttyfast
set laststatus=2

" Relative line numbers
set relativenumber

" Permanent undo
set undofile

" Autocomplete
set wildmenu
set wildmode=list:longest

" Fix tabs
set shiftwidth=4
set softtabstop=4
set tabstop=4
set expandtab

" Get syntax
syntax on

" Style vim
colors clouds

" Keymaps
let mapleader = ","
nnoremap <tab> %
vnoremap <tab> %
inoremap <F1> <ESC>
vnoremap <F1> <ESC>
vnoremap <F1> <ESC>
inoremap jj <ESC>

" Good-bye arrow keys... =(
nnoremap <up> <nop>
nnoremap <down> <nop>
nnoremap <left> <nop>
nnoremap <right> <nop>
inoremap <up> <nop>
inoremap <down> <nop>
inoremap <left> <nop>
inoremap <right> <nop>
nnoremap j gj
nnoremap k gk

" Proper search
nnoremap / /\v
vnoremap / /\v
set ignorecase
set smartcase
set gdefault

" Script plugins
au Filetype html,xml,xsl,php source ~/.vim/scripts/closetag.vim
au Filetype html,xml,css,php source ~/.vim/scripts/zencoding.vim

" Filetype indenting
filetype indent on

" Load indentation rules and plugins
" according to the detected filetype.
if has("autocmd")
  filetype plugin indent on
endif

" Settings
set nocompatible
set modelines=0
set encoding=utf-8
set scrolloff=3
set showmode
set showcmd
set hidden

set showcmd       " Show (partial) command in status line.
set mouse=a       " Enable mouse usage (all modes) in terminals

Developing applications for Android – gotchas and quirks

Over the past couple of days, I have developed my first two applications for the Android Market; Tuples and PencilWise. The first is a simple tool for storing key-value pairs, or tuples, of information you need to remember such as your bank account numbers, your PGP fingerprints, etc.. The second is a mobile version of the puzzle page by the same name.

As this was my first attempt at an Android application, I encountered quite a lot of first-time gotchas and quirks with the Android development platform, and I figured I’d share them here so others can resolve them quicker than I did. I’ll also give some hints for how to accomplish common task,s such as strike-through text, that might not be as obvious as they should be.

All of this is based on Android 2.1, but I assume most of them apply to any 2.0+ API.

Quirks and Gotchas

View reuse

Since Android mostly runs on devices that lack in the area of processing power, several optimizations have been implemented in the core of the OS. Unfortunately, these can have some strange side-effects.

One of these optimizations is that Android reuses views (the common base class for all elements drawn on screen) once the given element is no longer in view on the screen. This means that any pointers you might have to a view, will actually point to whatever Android has reused the view as, not the element you originally thought it was.

An example of this annoyance can be observed in the following code:

public class PencilWise extends ListActivity {
    View activeElement;
    // ...
    @Override
    public void onCreate ( Bundle savedInstanceState ) {
        // ...
        this.getListView( ).setOnItemClickListener ( new OnItemClickListener ( ) {
            public void onItemClick ( AdapterView<?> parent, View view, int position, long id ) {
                MyActivity.this.activeElement = view;
                MyActivity.this.showDialog ( DIALOG_ANSWER );
            }
        } );
    }
}

The showDialog method will display the answer dialog, which needs to know what question the user has opened. The problem is that by the time the dialog opens, the view passed to onItemClick might have been reused, and so activeElement would no longer point to the element the user clicked to open the dialog in the first place!

To achieve the desired effect, you need to store the position or the ID instead, and use something like: “this.getListAdapter ( ).getItem ( this.activeElement )”, which will return the cursor used to fetch the element behind the selected item.

View reuse and formatting

A bi-product of the view reuse of Android is that formatting options on the view are actually kept when a View is reused. So, if you for instance set that a TextView should be painted with a strike-through (see further down), ever time Android reuses that view, the new view will also be striked even though you never explicitly made it so..

For instance, say you used a ListAdapter for a ListView as such:
the getView method here is used to dynamically manipulate elements drawn in the List whenever they are accessed from the adapter. This can be used to, as shown in the example below, only strike through elements that hold true for a given condition

this.setListAdapter ( new ArrayAdapter<String> ( this, android.R.layout.simple_list_item_1, new String[] { "Hello", "World" } ) {
    @Override
    public View getView ( int position, View convertView, ViewGroup parent ) {
        TextView v = (TextView) super.getView ( position, convertView, parent );
        if ( MyActivity.this.answered.getBoolean ( v.getText ( ).toString ( ), false ) )
            v.setPaintFlags ( v.getPaintFlags ( ) | Paint.STRIKE_THRU_TEXT_FLAG );
        return v;
    }
} );

This code would seemingly be working, but once you start getting lots of elements, and the user starts to scroll, you will see that apparently random elements get striked. This is because of view reuse.
The solution is to undo the paint flag if the condition is not true, like this:

this.setListAdapter ( new ArrayAdapter<String> ( this, android.R.layout.simple_list_item_1, new String[] { "Hello", "World" } ) {
    @Override
    public View getView ( int position, View convertView, ViewGroup parent ) {
        TextView v = (TextView) super.getView ( position, convertView, parent );
        if ( MyActivity.this.answered.getBoolean ( v.getText ( ).toString ( ), false ) )
            v.setPaintFlags ( v.getPaintFlags ( ) | Paint.STRIKE_THRU_TEXT_FLAG );
        else
            v.setPaintFlags ( v.getPaintFlags ( ) & ~Paint.STRIKE_THRU_TEXT_FLAG );
        return v;
    }
} );

Now, each view will be drawn correctly.

Dialog reuse

Android also reuses dialog boxes, which means that if you, for instance, have an input field in the dialog that the user types something in, closes the dialog, and opens another dialog with the same ID, the text will remain in the field. This can be very annoying if the value in the input field should depend on another action the user has performed (such as what element was clicked to open the dialog).

The obvious fix would be to reset the content of the text area whenever a dialog is created, which you would think would be done in the method “onCreateDialog” using something like this:

@Override
protected Dialog onCreateDialog ( int id ) {
    // ...
    ( (EditText) dialog.findViewById ( R.id.text1 ) ).setText ( "" );
    // ...
}

Turns out, however, that this won’t work, since onCreateDialog is only run once per dialog ID. The proper way is to also override “onPrepareDialog”, and do any dynamic changes to the dialog there. So, onCreateDialog is for setup/format, and onPrepareDialog is for manipulating content.

Dynamic dialog titles/messages

After reading the above gotcha, you might also be tempted to put the setTitle and setMessage calls on the AlertDialog in onPrepareDialog since they might change between invocations of the dialog. The problem is that Android will not display the title or the message if they have not been set in the dialog returned from onCreateDialog. So, to use setTitle in onPrepareDialog, you have to make sure that you run setTitle in onCreateDialog with some placeholder text as well, otherwise your title/message calls in onPrepareDialog will not have any effect!

Dealing with SQLite

SQLite is not as easy as it could be in Android, but the SQLiteOpenHelper class does help quite a bit.

In order to utilize SQLite in your application, you need a class representing each database you want to use that extends the class SQLiteOpenHelper. Each such class needs to define three methods:

  1. The constructor
  2. onUpgrade
  3. onCreate

The constructor will theoretically only contain line:

super ( context, DATABASE_NAME, null, DATABASE_VERSION );

where DATABASE_NAME and DATABASE_VERSION are values defined by you to represent the current database. DATABASE_NAME can be any string, and DATABASE_VERSION should be an int that you will increment if you do any structural changes to your database.

This brings us to onUpgrade. This method is called if the database version number is different between the last time the app was run and this time (i.e. the user has updated your app in the market and the new version has a different DB schema). Here, you should provide update paths from every version of your database schema to the next such as adding/dropping tables and columns.

Finally, we have onCreate, which is called the first time the database is accessed. Here, you also usually only need a single statement per table you are creating: “db.execSQL ( TABLE_CREATE );” where db is the SQLiteDatabase object the method is passed and TABLE_CREATE is the SQL statement used to create the given table.

When accessing your database later, you need to call the method getReadableDatabase() or getWritableDatabase() on your helper class instance object to actually execute query(), delete(), insert() and update() statements.

SQLite and CursorAdapters – Where the fun starts

There is one little quirk that is not very well advertised in the SDK documentation for Android, and that is the fact that all Cursor objects used as source for list adapters MUST have an ID column. And not just any ID column, but one with the name set in BaseColumns._ID and a type of “INTEGER PRIMARY KEY AUTOINCREMENT”. So, if you intend to use any table in SimpleCursorAdapter or CursorAdapter, make sure your CREATE_TABLE statement includes the column

BaseColumns._ID + " INTEGER PRIMARY KEY AUTOINCREMENT"

Handling screen orientation changes

Android has another quirk that caused me quite a bit of headache when a bug was reported a couple of days ago:

If a dialog box was open, and the user changed the orientation of the screen, the dialog box would no longer contain the data from before the screen flip.

Turns out, whenever the screen orientation is changed, Android actually relaunches the current activity, thereby resetting all instance variables of that class! Thus, if I render the contents of the dialog based on what element in a list a user clicked (which is quite common), and the id of the element the user clicked is stored in this.activeElement; when the user changes orientation, that variable is reset to 0, and the code in onPrepareDialog does not know what element was originally pressed any more. This leads to it not changing the content, and only the structure from onCreateDialog is actually used (i.e. the title is only set to ‘__’).

After digging a bit around, I came upon this blog post dealing with the problem.

The solution is to use what Android refers to as an Application. This is kind of like a global registry for all of your application that will remain unchanged as long as your program runs. Thus, we can store any instance variables there, and access instance variables like this:

First, we create a new file holding the application class:

public class MyApplication extends Application {
    public int activeElement = -1;
}

Next, we define in our application manifest (AndroidManifest.xml) what our application class is

<application [...] android:name="MyApplication">

And now, we can access variables from our activities using:

( (MyApplication) this.getApplication ( ) ).activeElement;

It might be good to define a getter and setter in your activity for ease of use:

private int getActiveElement ( ) {
    return ( (MyApplication) this.getApplication ( ) ).activeElement;
}

private void setActiveElement ( int element ) {
    ( (MyApplication) this.getApplication ( ) ).activeElement = element;
}

How-tos

Strike-through text

Given a TextView view v, use:

v.setPaintFlags ( v.getPaintFlags ( ) | Paint.STRIKE_THRU_TEXT_FLAG );

To undo the strike-through, we use some bitwise magic:

v.setPaintFlags ( v.getPaintFlags ( ) & ~Paint.STRIKE_THRU_TEXT_FLAG );

Redraw all elements in a ListView

If you use an ArrayAdapter a, you can call

a.notifyDataSetChanged();

If you use an ArrayAdapter in a ListActivity:

( (ArrayAdapter) this.getListAdapter ( ) ).notifyDataSetChanged ( );

If you have a ListView ‘lv’ that uses an adapter that implements CursorAdapter, use:

((CursorAdapter) lv.getListAdapter()).getCursor().requery();

More flexible string handling through resources

Say you want to output the string “Hint: ” followed by some dynamically fetched hint. Your first thought might be to hard code ‘”Hint: ” + this.getHint()’, but it should be apparent that this is not the right way to do it. Android provides the strings.xml file for exactly this purpose. So, we define a string by the name of “hint_text” in strings.xml with the content “Hint: “. How do we access this string from our code?

Well, from any activity you can do this by simply calling “this.getString” which takes the ID of the string resource. In our case this would be

this.getString ( R.string.hint_text )

With our code looking like this (to put this in a toast):

Toast.makeText ( this, this.getString ( R.string.hint ) + this.getHint ( ), Toast.LENGTH_SHORT ).show ( );

You then happily compile and run the code, just to see that your text gets printed out as “Hint:Some hint” without the space you put in. The reason for this is that XML trims spaces inside elements, and as such, your space is lost. There are two wrong ways of fixing this, and one right one. The wrong ones are:

  1. Use the XML entity for a space:
  2. Hard code the space in your string concatenation in Java

If you want to do it properly, you define your string like this: ‘<string name=”hint_text”>Hint: %s</string>’. For those of you familiar with the printf family of functions, this should look familiar. What we are doing here is to tell Android that our string will contain dynamic content (here a string, represented by %s). We can then give the data that should be inserted into the text as additional parameters to the getString method like this:

Toast.makeText ( this, this.getString ( R.string.hint, this.getHint ( ) ), Toast.LENGTH_SHORT ).show ( );

RSS feed for AlertBox

For those of you concerned with web usability, there are few better sources than Jakob Nielsen’s AlertBox.

Although his site is quite infrequently updated, it contains very informative posts. Unfortunately and ironically, the site does not have a RSS feed.

After an especially boring afternoon, I decided to see if I couldn’t do something about that, so with the use of this PHP Feed Generator class and some HTML parsing, I came up with the following script to generate an RSS feed from the AlertBox website:

Just download the FeedGenerator FeedWriter class from the given link, put the following code and FeedWriter.php in the same directory, upload it to a webserver with PHP support, and voilla, you have an AlertBox RSS feed.

Enjoy!

<?php

date_default_timezone_set("UTC");
class FetchAlertbox {
    private $contents = array();
    private function fetch() {
        $page = file_get_contents("http://www.useit.com/alertbox/index.html");
        $startList = stripos ( $page, '<ul>' );
        $page = substr ( $page, $startList, stripos ( $page, '</ul>', $startList ) - $startList + strlen ( '</ul>' ) );
        $page = str_ireplace ( '<p>', '', $page );
        $page = preg_replace ( "/<li>(.*)/i", '<li>$1</li>', $page );
        $page = preg_replace ( "@</?strong>@i", "", $page );
        $p = @DOMDocument::loadHTML($page);

        foreach ( $p->getElementsByTagName("li") as $article ) {
            $url = '';
            foreach ( $article->getElementsByTagName("a") as $link ) {
                if ( preg_match ( "@\w+.html@i", $link->getAttribute('href') ) ) {
                    $url = 'http://www.useit.com/alertbox/' . $link->getAttribute('href');
                    break;
                }
            }

            if ( $url === '' ) {
                continue;
            }

            $contents = str_replace ( "\n", " ", $article->textContent );
            $contents = preg_replace ( "/\s{2,}/", " ", $contents );
            $date = 0;
            $possibleStartDate = strrpos ( $contents, '(' );
            if ( $possibleStartDate !== false ) {
                $timestring = substr ( $contents, $possibleStartDate + 1, strpos ( $contents, ')', $possibleStartDate ) - $possibleStartDate - 1 );
                $parsedDate = strtotime ( $timestring );
                if ( $parsedDate !== false ) {
                    $date = $parsedDate;
                    $contents = str_replace ( "($timestring)", "", $contents );
                }
            }

            $this->contents[] = array ( 'url' => $url, 'title' => trim ( $contents ), 'date' => $date );
        }

        return $this->contents;
    }

    public function fetchWithSummaries($type = "RSS2", $limit = 10, $start = 0) {
        if ( empty ( $this->contents ) ) { $this->fetch(); }
        include ( "FeedWriter.php" );

        $feedType = RSS2;
        $feedDate = DATE_RSS;
        switch ( $type ) {
            case 'RSS1':
                $feedType = RSS1;
                break;
            case 'ATOM':
                $feedType = ATOM;
                $feedDate = DATE_ATOM;
                break;
        }

        $feed = new FeedWriter($feedType);
        $feed->setTitle("Alertbox");
        $feed->setLink("http://www.useit.com/alertbox/");
        $feed->setDescription("Current Issues in Web Usability - Bi-weekly column by Dr. Jakob Nielsen, principal, Nielsen Norman Group");

        if ( $type === 'ATOM' ) {
            $feed->setChannelElement('updated', date($feedDate, $this->contents[0]["date"]));
        } else {
            $feed->setChannelElement('pubDate', date($feedDate, $this->contents[0]["date"]));
        }

        $feed->setChannelElement('language', 'en-us');
        $feed->setChannelElement('author', array('name' => 'Dr. Jakob Nielsen'));

        if ( $type === "RSS1" ) {
            $feed->setChannelAboute("http://www.useit.com/alertbox/");
        }

        for ( $i = $start; $i < $start + $limit && $i < count ( $this->contents ); $i++ ) {
            $p = @DOMDocument::loadHTML ( file_get_contents ( $this->contents[$i]["url"] ) );
            $blockquotes = $p->getElementsByTagName("blockquote");
            if ( $blockquotes->length > 0 ) {
                $this->contents[$i]["summary"] = trim ( str_replace ( "Summary:", "", $blockquotes->item(0)->textContent ) );
            } else {
                $contents = $p->saveHTML();
                $endHeadline = stripos ( $contents, "</h1>" );
                $this->contents[$i]["summary"] = trim ( strip_tags ( substr ( $contents, $endHeadline + strlen ( "</h1>" ), stripos ( $contents, "<p>" ) - ( $endHeadline + strlen ( "</h1>" ) ) ) ) );
            }

            $item = $feed->createNewItem();
            $item->setTitle($this->contents[$i]["title"]);
            $item->setLink($this->contents[$i]["url"]);
            $item->setDate($this->contents[$i]["date"]);
            $item->setDescription($this->contents[$i]["summary"]);
            $feed->addItem($item);

//            echo date ( "F jS, Y", $this->contents[$i]["date"] ) . ": " . $this->contents[$i]["title"] . " -> " . $this->contents[$i]["url"] . "\n";
//            echo "\t" . $this->contents[$i]["summary"] . "\n\n";
        }

        $feed->genarateFeed();
    }
}

$s = new FetchAlertbox();
$s->fetchWithSummaries(isset($_GET['type']) ? strtoupper ( $_GET['type'] ) : 'RSS2', isset($_GET['limit']) && ctype_digit($_GET['limit']) ? $_GET['limit'] : 10);

WPA2-Enterprise wireless with netcfg

At Bond University the IT department has now finally taken the step from an open wireless network with a login proxy to a proper WPA2 Enterprise setup.

Apart from giving much higher security since data is not transmitted unencrypted through the air, this also makes it a lot easier to automatically connect to the internet. With the previous setup, a post_up netcfg hook was needed that used some specially crafted cURL code to post the login form on the proxy. Ugly as hell, and not very reliable or secure either.

The only issue is that WPA2-Enterprise has a lot of different configuration options, so it took a while to figure out the exact setup to use.

First of all I had a look at the configuration options for Windows XP here: http://www.bond.edu.au/student-resources/student-support/computing-support/for-students/wireless-access/index.htm. The advantage of looking at the XP setup guide is that it doesn’t have as much fancy auto-detection as 7 or Mac OSX, so they give some more low-level details that we can base our setup on.

In this setup I decided to use netcfg simply because it provides a flexible utility for managing multiple wireless configurations and switching between them in a simple fashion. Furthermore, it abstracts away some of the uglier command line options of wpa_supplicant into configuration options.

All netcfg configuration files live in /etc/network.d. Each file represents one network, and contains at the very least for a wireless connection an interface specification, the SSID of the network and an IP configuration line. Have a look in the /etc/network.d/examples directory for, well, examples..

For WPA2-Enterprise however, we need quite a few more parameters:

CONNECTION='wireless'
INTERFACE=wlan0
SECURITY='wpa-configsection'
ESSID='BondStudents'
IP='dhcp'
CONFIGSECTION='
  ssid="BondStudents"
  key_mgmt=WPA-EAP
  eap=PEAP
  group=CCMP
  pairwise=CCMP
  identity="<student ID goes here>"
  password="<password goes here>"
  priority=1
  ca_path="/etc/ssl/certs"
  phase2="auth=MSCHAPV2"
'

Quite a mouthful.. In essence, this means we are using WPA enterprise with CCMP (an AES-based encryption algorithm) and MSCHAPV2 authentication with PEAP (TLS) encapsulation. As if that helped..

The most interesting line here is: ca_path=”/etc/ssl/certs”.. Why would we need to import certfiicates? Well, turns out Bond uses a SSL certificate that is not exactly mainstream called “UTN-USERFirst-Hardware”. This certificate is usually included in the certificate directory of most distros, but is not very secure as it is not a trusted signing authority. If you want to know more about this, then Google it.

Well, that should be it. You should be able to run “sudo netcfg BondStudents” if you’ve called the configuration file “BondStudents” and you’ll be online!

Using WordPress as a website backend

How often have you found yourself creating a great new website design for a friend, family or a client, just to realize after all your HTML and CSS is done that the thing will need an administration panel? Way too often, the administration comes as an afterthought, and thus it ends up being incomplete, hard to use or in some cases absent. Two months down the track you end up being contacted again to do some “site updates”, and these just keep coming.

Making a good administration panel takes a bit of work, and it takes planning to ensure that the client can update the parts of the site as he/she wishes, and more importantly, that your interface gets all the information it needs. If your design has a thumbnail for every post, your administration panel has to provide the opportunity to upload an image and attach it to a post.

For years, I have been making these admin pages myself, until I read a post on WordPress custom themes. This post was on how to write a theme for WordPress, but it hinted at the possibility of having a standalone site that used WordPress as its backend, thus providing a full-fledged, well designed and familiar administration interface to any site. At first I was skeptical to the idea because I thought WordPress was way to inflexible to allow for the variety of pages I make. I was wrong. You see, if you really thing about it, all you need to make most websites are two things: Posts in categories and dynamic text/HTML blocks.

To give you an idea on how flexible WordPress is, and how it can be used, consider the following website which I am currently working on. The site is for a sound production company, and will contain the following:

  • Short posts giving updates on what the studio is working on at the moment. These have to have a short text to be shown on the front page, a feature image and a rich-text full article.
  • An image feed showing images of the sound studio
  • A page with the various projects the studio has completed. Each project has an image, a description and one or more music tracks. Each track has two variants, a streaming-quality MP3 and a high-quality WAV.
  • An about page with two separate columns, one on the sound designer, and one on the company
  • A list of previous clients with a short description and a link to the client’s website

At first, this seems outside the scope of WordPress which only deals with Posts and Pages, but let’s have a closer look.

The posts are clearly just regular WordPress posts. Let us put them in a category called “Frontpage”.

The image feed is just a stream of images. We here have two choices, use WordPress’ media library, and attach all images to be shown in the feed to a static page, or use a flickr stream. I opted for the latter, but WordPress could have handled this fine!

Each music project can be a blog post (you heard me right) in the category “Projects”. We can then attach images and songs to the post using the WordPress media library. The two versions of each music file can be given the same name, and we can use the MIME type to distinguish between them.

The two text blocks 0n the about page can be two WordPress Pages (let’s call them ‘designer’ and ‘studio’)

Finally, the list of clients can be implemented using a set of links in a link category.

So, how would we go about and convert our static HTML into a fully dynamic site with a complete administration interface? Let’s dig into some code.

Getting access to WordPress from your code

The first step to a well-integrated site is to include the following code at the top of any page that needs to access WordPress functions. Naturally this does not need to be included in included/required files.

define('WP_USE_THEMES', false);
require('./wp-blog-header.php');

This gives you access to a whole set of WordPress functions (http://codex.wordpress.org/Function_Reference) that will aid us in integrating your site with WordPress. Unfortunately, the WordPress API is not very well structured, and naming conventions are a bit all over the place, but we’ll make do.

Getting content from WordPress

From the WordPress function list, there are a couple of terms you need to become accustomed to in order to start using the API. First of all, “The Loop”.

Now, we will not actually be using the loop to display any of our pages for two reasons: “The Loop” is a magic thing that mysteriously figures out what posts/pages to display, and is associated with some magic methods such as get_the_author which magically contain data about the “current” post, whatever that is. Second, it provides very little flexibility for selecting only certain posts/pages.

I will not go into the details of “The Loop” here, I will just say that it is a while loop that most WordPress templates use to print blog posts, pages, etc. to abstract away the backend query. There are several methods in the WordPress API that depend on being used in The Loop, and these usually contain “the” in the function name. Avoid these!

Next, in WordPress, pages are posts. Special types of posts, but posts nonetheless. This means that if you fetch a page, the various fields available will be the exact same as those for post, and they will be named post_title, post_content, etc…

When printing the content of posts (that is, any field that has a rich text field as input), WordPress depends on you running the data through another magical function: wpautop. This function automatically adds p tags where it thinks is appropriate to mimic the appearance in TinyMCE in the admin panel. Always put post_content through this function, otherwise your output is going to look very weird indeed.

Finally, the WordPress API usually returns objects or lists of objects. This is very convenient for most uses, but it also means that you have to take care in those cases where it doesn’t. One such method that you will probably be using is wp_get_attachment_image_src; this function actually returns a numerically indexed array.

Most function that return objects contain all the fields outlined in the appropriate table in this database diagram: http://codex.wordpress.org/Database_Description. Note that almost all models will contain an ID field which comes in very handy. Most other columns are prefixed by the name of the table, and this prefix is also used in the object attributes.

Getting posts

When getting blog posts, the main function to think about is get_posts. This function has a plethora of configuration options, but usually, you will only need the numposts option, and maybe offset. In the case of the website I was developing, I wanted just the posts in a given category

foreach ( get_posts ( 'numberposts=7&category=4' ) as $post ) {
    echo $post -> post_title;
}

This is a very simple example which only prints the title of each post, but you get the drift.

If you want a full version of a post given its ID, you would use the quite similar get_post (http://codex.wordpress.org/Function_Reference/get_post) function. This function takes a post ID, and returns an object representing that post. This object tells you nothing about the author or any attached images, so these will have to be fetched separately as such:

$post = get_post ( $_GET['p'] ); // This is probably quite insecure. Sanitize your input!
$author = get_userdata ( $post -> post_author ); // Here we should do some error checking on $post first
$images = get_children ( array(
    'post_type' => 'attachment',
    'post_parent' => $post -> ID,
    'post_mime_type' => 'image'
) );

There is a bit of voodoo going on here, so let’s take it step by step.

The first line should be pretty straightforward, we simply get the appropriate post object by its ID (which we take from the query string).

The next line is also quite simple, we fetch the user data of the user with the ID matching that of the author of the post.

Now we get to the strange bit; get_children. You see, WordPress treats almost everything as posts. Even attachments, no matter the type, are considered posts, and are part of the page/post hierarchy. Thus, to get the attachments of a post or page, we are actually getting all the children of the given object of the type ‘attachment’. I have also added a filter on ‘post_mime_type’ to ensure we only get images. Notice how WordPress sometimes uses strings as arguments, and other times uses arrays? Turns out you can usually get away with both approaches… Someone should really write a wrapper class to sort out that mess, but until then, we’ll have to deal with it. The good part though is that the process for getting the images for a page is exactly the same, just replace get_post with get_page!

The most interesting part though is showing an image you’ve fetched. WordPress “conveniently” provides user-customizable thumbnails for all uploaded images. Unfortunately, these tend to be cropped in weird ways, and are very unpredictable and unlikely to look nice. When printing an image, you have a choice between several formats, amongst others ‘thumbnail’ (the default) and ‘full’. The only one that gives you an uncropped image is ‘full’, but this will give you the image in its original resolution. True, the users can edit and scale the images in the admin panel, but how many end-users can you expect to do that? Unfortunately there is no way around that at the moment AFAIK, but one happy day…

Anyway, until then, you have a choice of two functions for printing your images: wp_get_attachment_image and wp_get_attachment_image_src. They both take the same arguments, but the difference is that the first one prints a full ‘img’ HTML tag with the alt, title, width and height attributes already set, whereas the second one just returns the image url, the widht and the height as a numerically indexed array that you can decide what to do with. They both take the ID of the image as a first parameter, and the size you want as the second. Here, you can either give a predetermined size such as ‘thumbnail’, get the full image with ‘full’ or get a cropped thumbnail that fits inside a certain box by passing an array of two values, width and height as such: array ( 64, 64 ). If you want to get the image description and title yourself, those are stored in the object you used to get the ID for wp_get_attachment_image_*, i.e. in $images[$i].

Other attachments (mp3s for instance)

When it comes to getting other post attachments, this is actually quite trivial once you know how to fetch images. Instead of using ‘post_mime_type’ => ‘image’, you simply use another MIME type. On the site I am developing, I will use the MIME type for mp3 which is ‘audio/mpeg’ as far as WordPress can tell (you can see this in the WordPress admin panel -> Media). I would therefore substitute  ‘post_mime_type’ => ‘image’ with ‘post_mime_type’ => ‘audio/mpeg’. Simple as that!

To get the direct URL for a non-image attachment, you can use the wp_get_attachment_url function. As for getting the high quality version of a file, this is just a matter of selecting an attachment with the same title (i.e. the name of the file without the extension), but a different MIME type.

Dealing with pages/editable content boxes

Now, for the boxes on the about page which the administrators of the page should be allowed to edit. This is as easy as just creating two new pages in the WordPress admin and noting down the name you use. Back in your code, you can then use the following snippet to print the content of the box/page:

$page = get_page_by_title ( 'About box left' );
echo wpautop ( $page -> post_content );

By now this should look familiar. We are simply fetching the page by its title, and then passing the pages content through wpautop, and echoing the result.

Lists of links with descriptions

Our final challenge for this site will be to fetch the list of clients. We’ve already determined that we are going to use the WordPress Links library because this provides exactly the fields we need, a title, a URL and a short description. However, if you start looking through the WordPress API for anything related to links, you will come up empty handed. The reason for this is that in their wisdom, WordPress decided to call links “bookmarks” in their API for the sake of clarity. The function we are looking for here is called get_bookmarks, and again we may specify lots of parameters. In our case, however, we are only concerned with one of them; category. Since we may want to add other links later that should not show up in the clients list, we create a link category from the WordPress admin and note down the category ID. In my setup it was 3, and so my code to get the links/bookmarks becomes:

foreach ( get_bookmarks ( array ( 'category' => 3 ) ) as $link ) {
    echo '<a href="' . $link -> link_url . '">' . $link -> link_name . '</a>';
    echo '<blockquote><p>' . $link -> link_description . '</p></blockquote>';
}

Of course, this is a simplified version of the end result, but it should give you enough of an idea to get you on your way.

Final thoughts

As you have now seen, this entire page can now be administered fully through WordPress with its quite good admin panel, and the user won’t even think twice about WordPress really being a blogging tool. In fact, neither should you, because as you can see, it is more than flexible enough to be used for quite complex websites. Your users will be happy with a comfortable admin interface, and you won’t have to touch a single piece of admin code!

Random lag spikes in Windows 7

Update: It turns out the issue is even more general than I thought at first. It is in fact not just the HTC mobile sync program that causes these freezes; it is in fact any mobile syncing application. After launching the Sony Ericsson PC Suite today, I noticed that the issue returned immediately. I killed the process, and it ended again. Seems as though Windows 7 has issues dealing with mobile syncing software if the device is not connected..?

For the past couple of weeks, my graphics card had been acting up. Or so it seemed. Every 5-6 seconds, the screen would freeze for a brief moment – just long enough to notice. It was only the screen though, because music playing in the background would keep going, and any other application would proceed as though nothing had happened. This lead me to believe that my nVidia 8600 GT was to blame.

After looking around the web, I found that several people were complaining about random lag spikes with the newest nVidia drivers, and thus my suspicion was reinforced. I tried uninstalling the drivers, and installing 6 different, previous versions of the driver pack, but to no avail.

A couple of days ago I’d had enough – it is really annoying when the mouse stops moving every five seconds – and so I decided to order a brand new graphics card. After all, if the graphics card is acting up, what is more likely to fix the issue than a new graphics card?

Well, it arrived in the mail yesterday, and to my great surprise, the issue persisted. And this was an ATI card – no nVidia drivers were even installed on the system. The mystery remained.

In order to find the cause of these lags, I fired up the Task Manager and kept an eye on the CPU usage graph. It turned out that one of my CPU cores spiked at about 10% every time the screen froze. I switched to the process view, but there were no processes suddenly jumping to the top on every freeze… Odd.. There was a process called fsynsrvstarter – with the suspicious description “TODO: <description>” quite close to the top. It didn’t use any CPU, it just sat there.. And so I figured I’d Google it just in case.

At this point, I came across this forum post explaining that this process was spawned every 4 seconds by HTC Sync. This seemed as though it aligned too perfectly with my spikes, so I decided to uninstall HTC Sync and see what happened. And what do you know? The spikes were gone..

So, I now own two graphics cards (albeit one somewhat outdated) which both work well, but are brought to their knees by such a complex piece of software as a mobile phone syncing program (an inactive one at that).

I hope this post will solve your problem before you decide to buy a new card like I did.

Inline website administration

Almost all modern websites require some sort of administration, and this usually involves creating a separate administration page where articles can be added and users managed. Lately, I’ve been making quite a few new websites that will be released in the upcoming year, and all of these have been quite simple sites with a single user and where the administration consists mainly of adding simple news updates and updating page text. For these sites, a full blown administration panel is not necessary, and is also quite inconvenient as the user will have to go back and forth to see the results. So, what are the alternatives?

(Live examples are not available at the moment, but might come later)

AJAX driven, on-page administration

Here, the user (the person administering the website) is allowed to edit content on the same page as the content through a rich text area in a popup, and the text is then changed afterwards to reflect the users edits.

The simplest, and in my experience most flexible way of doing this is through named fields. Each block of text on the site gets its own unique name, and is linked to a plain text file on the server. In my small site setups, I usually use a structure like this:

/
 pages/
  about.inc.php
  projects.inc.php
  bio.inc.php
 api.php
 page.php
 index.php

The .inc.php files can either be plain text or contain PHP code. The most important thing is that they have a unique name. The files the usually look something like this (simplified for clarity – remember security and error checking!)

<?php

// page.php
function printBlock($name) {
    if ( !file_exists ( 'pages/' . $name . '.inc.php' ) ) return;
        echo '<div id="' . $name . '" class="editable">';
        require 'pages/' . $name . '.inc.php';
        echo '</div>';
    }
}

// api.php
require 'page.php';
$action = $_GET['a'];
$block = $_GET['e'];
switch ( $action ) {
    case 'get':
        printBlock ( $block );
        break;
    case 'post':
        file_put_contents ( 'pages/' . $name . '.inc.php', $_POST['content'] );
        break;
}

// index.php
require 'page.php';
?>
<!-- HTML structure -->
<!-- Then, whenever you're printing a block or page that should be editable, call printBlock -->
<?php printBlock ( 'about' ); ?>

Next, you will have to make some sort of JavaScript hook to make all editable areas editable. I like to use a combination of CKEditor, a simplified version of lightbox and jQuery so the end result looks something like this when a user double clicks on a box with the editable class:

Upon saving, jQuery sends a AJAX request to the api.php file with the updated contents, and also changes the contents of the block on the page using the .html() method on the element with the same ID as the block name.

In-line administration

On some sites, popup boxes simply won’t cut it. In fact, they might even become a bit cumbersome when working with news articles and such where you might want a live preview of the article as you’re typing it. Earlier, one had to have a rich text editor with a “Preview” button, but now we have a much better tool available: contentEditable. This awesome attribute allows you to tell the browser to allow the user to change the contents of an element on your page at will. Consider these screenshots that illustrate adding a new news post on a page utilizing this attribute for administration:

Before adding the article

Adding a title

Editing the post body

After saving the new post

As you can see, this is a very simple way of creating and editing posts – and immediately seeing how it would look on the page. The major drawback is that you cannot easily accept rich inline content such as images and video, or even simple text formatting. On the other hand, such features often clutter the articles anyway. On this site, I have overcome this by allowing file attachments that are placed beneath the article based on their type (images are shown in a gallery strip, videos are embedded, etc.) Text formatting is achieved through a markdown-like syntax handled by JavaScript. There is no rich text logic in the backend.

Using contentEditable is quite simple. All you have to do is use JavaScript’s setAttribute/removeAttribute functions on any element you want to be editable. Set the attribute to true when you want it turned on, and remove it when you want it off. Apart from this, everything is quite straight-forward and very similar to the previous method of popup administration. JavaScript sends the new content to the backend, which saves it and returns the HTML rendering of the content as it would be displayed when loading the front page regularly. JavaScript then swaps the editable post area with the HTML from the server and disables editing on it.

Rounding up

Both these techniques provide quite intuitive and easy-to-access administration equivalents to classical admin-panel interfaces. They are not especially complex to build either, though they provide the user with a more comfortable and usable way to manage their sites. If you have any questions regarding these techniques, don’t hesitate to use the comment field below or e-mail me at jon <you know what goes here> thesquareplanet <and you know this one as well> com.

Developing for the modern web

Web development today is a constantly struggle between three major stakeholders: the customer, the designer and the developer. The customer tries to push through his or her (often distorted and silly) mental image of the website, the designer wants to be original, creative and fancy creating lots of intricate designs with fancy visual effects, and the developer who attempts desperately to explain to both the customer and the designer why what they’re doing is a bad idea (heavy background images, crammed pages, no whitespace, confusing visual effects…). The developers aren’t all good either though – They tend to put in as many fancy tricks and solutions in the final product as they can, often resulting in exotic bugs in various browsers and usually ungraceful downgrading™. In all of this, one stakeholder is often wholly forgotten, even though it is probably the most important one; the users.

Users often don’t know the first thing about how the web works. They don’t care whether the site is optimized for Firefox, Internet Explorer, Chrome or Safari (in fact, they probably don’t even know what a browser is…) The users want a site that is visually appealing, but not distracting – informative, but not cluttered – clear, but not over-simplified – and most importantly, one that is responsive. When a user does something, they should begin to see something happening within .1 seconds (http://www.useit.com/alertbox/timeframes.html) to feel as though they aren’t being slowed down by the site itself. Furthermore, the total loading time for whatever action the user initiates should be less than a second for the user not to fall out of his or her “flow”. Way to many websites violate these simple rules, causing the site to feel unresponsive to the user, and the users are likely to jump to the next site on their list.

In this post, I hope to show you how to make your website faster – mainly through optimizing the initial page load. In order to do this, there are three steps that need to be taken: Combine, Compress and Communicate. Repeat after me: Combine, Compress and Communicate.

Combine

Many developers seem to think (albeit erroneously) that many small files are better than few large ones. This might seem intuitive since a smaller file downloads faster than a large one, and you would think they could all be gotten out of the way quicker. The truth is quite different. Due to limitations of the HTTP protocol, the browser has to initiate a new request to the server for every single file, causing quite a bit of overhead when having to download several files. Also, modern browsers limit the amount of simultaneous downloads to 6, meaning downloading all of your small files will go even slower. Add to this the sequential nature of JavaScript, and the fact that the browser stops loading the page once it hits a JavaScript piece (external or not), and doesn’t continue loading until the JavaScript file is finished downloading and has been interpreted.

Therefore, you should work to combine as many of your files as possible. Don’t jump to put all your scripts and styles inline, however (you will understand why in Communicate). Instead, you should attempt to combine all your CSS files into one, all your JavaScript into another, and all your images into a third. Ideally, you should need no more than three external files on your site. So, how do you go about doing this?

CSS and JavaScript

Combining CSS and JS files shouldn’t itself be a problem.. Open up a text editor, copy-paste all of your CSS or JS into that file, save it and upload. You should probably still keep the separated files for readability though. Of course, modern web applications are usually a bit more complicated. For instance, you might have a stylesheet that is only included on sites with ads on them or a JavaScript file that is only needed on your frontpage. In these cases, your should look into using a combinator. One of the best sites describing the techniques of combining is this one. The mod_concat plugin for Apache2 provides several advantages over traditional scripting approaches especially with regards to communication (as will be discussed later)

Images

All your images should be done as sprites. Ideally, you should even be able to put every single image on your site into a single png image. Do this, and you will substantially reduce the loading time of your site. For an introduction to CSS sprites, have a look here.

Compress

All your CSS and JS files should be compressed to reduced overall download size. Again, it is usually a good idea to keep the original, uncompressed versions of the files, and re-compress the files whenever you change them. For CSS, I recommend the YUI compiler (http://www.refresh-sf.com/yui/). It does JavaScript as well, but Google’s recently released Closure Compiler seems to be even more effective at compressing it. You can find it at http://closure-compiler.appspot.com/home. With the Closure Compiler, you can also select the advanced compiler which will decrease the total file size even more, but will mess up your files’ external API. This means that any functions you define inside your files won’t be available from the outside by the same name. The internal workings of the file will be preserved though.

Apart from minimizing the files, you should also compress them using something like GZip which is natively supported by several browsers. To see how to do this automatically with Apache2, have a look at http://www.cyberciti.biz/tips/speed-up-apache-20-web-access-or-downloads-with-mod_deflate.html.

Communicate

OK, so all of your files are combined and compressed, and you’ve never seen the CSS and JavaScript download so quickly. How can it possibly go any faster? Quite simple – by preventing the browser from having to download the files at all. Modern browsers include a lot of caching technology to prevent them from downloading unnecessary data from the server. The problem is that many web servers do not communicate properly the states of the files, and the browsers can thus not determine if a file has changed or not; and therefore they download the file just to make sure. So, what should you do?

First of all, you need to tell your web server to send out as much data as possible about your file. This especially applies to dynamic files such as those created by PHP. Have a look here for a more thorough discussion of this topic.

Second, files that are GZipped by Apache don’t always get an expiration date, causing the browser to re-download the file on every page load. To overcome this problem, have a look at the first answer on this page

Final thoughts

In the course of this post, I hope I have given you an overview of what can be done to speed up the loading time of web pages, and enough pointers to keep you going in your quest for the best speed your website can achieve. This is an ever-expanding topic, and new techniques are always appearing, so you should attempt as best you can to keep up to speed (pun intended) on the newest advances in the field.

Happy speeding!