Shaving response times and adding progress with spawn

One of the reasons that lds.org is such a hassle is that their server responds so slowly.

I want this application to be user-friendly. The problem is, how do I make an abstracted layer faster than the original?

I don't.

But I do fake it as best I can and I'll make up for it later with HIJAX. The idea is that I'll use spawn to fork blocks of code into the background (like the generator I created in the last post) so that the rendered page returns sooner and I can do a little ajax polling in the background on the client-side to update the client page as needed. In the meantime, I'll allow the user to take care of things which aren't on the 'critical path', like grouping and formatting options.

Installing spawn in rails

sudo apt-get git-core
cd ~/Code/TheWardMenu
./script/plugin install git://github.com/tra/spawn.git
#app/controller/directory.rb  
 ...  
 spawn do
   while record = @photo_directory_generator.next
     member = Member.new(record)
     member.ward = @ward
     member.save     end
 end

Using links2 as a quick test for transfer speeds from lds.org here's a way that things could go

User initiates request

• 4s - TheWardMenu.com responds
• 5s - User enters credentials
• 1s - twm receives request
• 3s - Lds.org has responded
• 1s - Update Profile page parsed
• 0s - spawned the block of code that gets the directory
• 1s - response sent

The user has spent the last 6 seconds choosing pdf options rather than dying of boredom

• 4s - the text-only directory is now in the database
• 0s - the photo downloading process is spawned

The user can now review the text version of the directory for mistakes and see some real-time sorting as well as see the % complete on the picture downloads via a polled feed which gets only pictures that are more recent than the last polling.

• 2m - the photos are downloading

The user finishes adjusting the settings and hits 'Download PDF' and is given the approximate wait time needed to download the remaining photos (which are already being downloaded into a hidden div so that the browser caches them).

The user leaves the site happy or makes adjustments to a fully viewable picture directory and downloads the pdf again (and it only takes a few seconds this time around)