Digital Carpenter - Doculicious

Doculicious.com Launch FAQ

nospam@example.com (Chris Carpenter) — Mon, 12 Jan 2009 10:15:26 +0000

Decreasing AJAX web app loading times

nospam@example.com (Chris Carpenter) — Wed, 24 Dec 2008 09:39:05 +0000

When developing Doculicious, one of my main concerns was how fast the heavy JavaScript parts would load and run in the browser. Doculicious uses a fairly customised version of the Dojo Toolkit, and the raw files that were being loaded for some pages added up to over 550KB - made up of about 50 individual files, all being loaded by Dojo with its packaging system. This was fine for development and great for debugging, but I knew I had to do something before this went into production. In this post I'll talk about some of the things we use in Doculicious to make our JavaScript smaller and load faster, but still easy to develop on. Hopefully it should be general enough to help anyone who has a site using JavaScript, but we use Dojo so is a bit focused on its packaging system which includes multiple files at run-time. If your app doesn't use Dojo but has multiple JavaScript files, you'll probably still get some benefit out of this.

There are 2 main ways to speed up a JavaScript application - Make the JavaScript load faster into the browser, and make it run quicker. There's a great article at IBM developerWorks called AJAX performance analysis that goes into both of these. The first section talks about the tools to use in Web application development - Firebug and YSlow, which are both required tools on any development box I use. The next goes into reducing the network transfer time, and this is where I'll focus on what we did with doculicious.

Reducing File Size and HTTP requests

The Dojo Toolkit uses a packaging system similar to Java, and it's a great way to separate out all the files and keep them organised and easy to develop. We've created our own package for doculicious and one of our JavaScript heavy pages may require up to 50 individual files to be included. Because of current browser connection limits, this is way too much, and it's much better to limit the number of JavaScript files to one or two instead of loading many.

What I wanted to do for doculicious was to create individual files for each JS heavy page, with each file including just the JavaScript code it needed. This individual file could then be used with the dojo.js file so that all the required JavaScript for a page would be in 2 files. Because Dojo automatically includes JS files depending on which widgets and functions are used, I needed to define which ones were used on which page so that I could join them all together. Firebug is great for this, and using the Net tab quickly lets you see all the included JS files:

My initial goal was to create a range of "base" files that could be concatenated together in an Ant script to create a JavaScript file for any doculicious page. Using Firebug to find the common files across each of our pages, I made a list of all the Dojo specific files that were needed, and also of our custom doculicious files. At this point I realised that each of our pages used very similar Dojo functions and widgets, so I made the decision to manually join these together into one file that could be included across all the files I needed. This is probably not the best thing to do if you are using the latest version of Dojo or another packaging system, as when changes are made it will be harder to integrate them, but our base Dojo is version 0.4.3 and I do not plan on upgrading it. We've only found a couple of bugs over the past 18 months, so I'm comfortable that this base is stable.

Concatenating your JavaScript into a single file is great for production code, but you still need to develop on it. So when doing this we need to keep our individual files available for development, while having a simple process to make the production code. I use Eclipse with the MyEclipse plugin as my IDE. The JavaScript project is structured like below, with the js folder being included in the web project so that it is deployed along with it:

JavaScript Project
   - buildscripts
      - build
         - functions
         - widgets
      - lib
      - build.xml
   - js
      - digitalcarpenter
         - system
            - Command.js
            - DocumentState.js
            - State.js
            - ...
         - widgets
            - Canvas.js
            - Elements.js
            - ...
      - dojo
         - dojo_functions_widgets_minimal.js
         - dojo.js

Underneath the dojo folder is the dojo_functions_widgets_minimal.js file which contains all the dojo functions and widgets that our pages need. The digitalcarpenter folder is our base package, and contains all our individual JavaScript files, including widgets and system classes. Under the buildscripts folder is our build file and a build folder to hold temp files. The lib folder holds some tools we will use later. The ant script is pretty basic, here's what it does:

At the beginning we define the paths that are used and the files we are going to create.
We then build the directory structure for the build
Then in copy we move all the files we are going to use into the build/functions and build/widgets. Taking care to not move stuff that isn't needed, such as testing files, images and css.
Next we define a regular expression replace to remove all lines with "dojo.require" on them. This line is the way Dojo knows which files to include, and it's not needed in our final file because all the code is together. In fact things break if these are left in. Depending on which JS library you use, you may not need to do something similar when cancatenating your files together.
The next area is where we actually create our specific file. First we copy the base file to our working directory, then we concat all the individual files together, appending them to this file. Order may be important. Some of the widgets in our code include other widgets, so they need to be added before-hand.
The last step is calling ./lib/custom_rhino.jar. This is the Dojo ShrinkSafe application which removes comments and shrinks down the code. I highly recommend that you use this. ShrinkSafe has worked great for all of my JS, and I've never had errors. Because it parses the JavaScript it does a great job. It will even tell me if I've left some horrible bug in the code.
At the end is the create and clean methods you use to run the whole thing.

<?xml version="1.0" encoding="ISO-8859-1"?>
<project name="DigitalCarpenter_JavaScript" default="create"
	basedir=".">
	<description>
		Creates the javascript files needed for Doculicious.
	</description>
	<!-- set global properties for this build -->
	<property name="build" location="build" />
	<property name="widgets" location="${build}/widgets" />
	<property name="functions" location="${build}/functions" />
	<property name="lib" location="lib" />
	<property name="dojo_src" location="${basedir}/../js/dojo" />
	<property name="dc_src" location="${basedir}/../js/digitalcarpenter" />
	
	<property name="dojo_base" value="dojo_base" />
	<property name="dojo_minimal" value="dojo_functions_widgets_minimal" />
	<property name="base_workshop_dojo_file" value="workshop_dojo" />
	<property name="document_workshop_js" value="document_workshop" />

	<target name="init">
		<!-- Create the build directory structure used to create the JS files -->
		<mkdir dir="${build}" />
		<mkdir dir="${widgets}" />
		<mkdir dir="${functions}" />
	</target>
	
	<target name="copy" depends="init">
		<copy file="${dojo_src}/${base_workshop_dojo_file}.js" todir="${build}" />
		<copy file="${dojo_src}/${dojo_minimal}.js" todir="${build}" />
		<copy todir="${functions}">
			<fileset dir="${dc_src}/system" />
		</copy>
		<copy todir="${widgets}" includeEmptyDirs="false">
			<fileset dir="${dc_src}/widget">
				<exclude name="_package_.js" />
				<exclude name="ButtonField.js" />
				<exclude name="InlineEditor.js" />
				<exclude name="RichEditor.js" />
				<exclude name="TestWidget.js" />
				<exclude name="Popup.js" />
				<exclude name="templates/*" />
				<exclude name="templates/buttons/*" />
				<exclude name="templates/images/*" />
			</fileset>
		</copy>
	</target>
	
	<target name="remove_dojorequires" depends="copy">
		<replaceregexp match='dojo.require' replace="//dojo.require" byline="true">
			<fileset dir="${widgets}"/>
		</replaceregexp>
		
		<replaceregexp match='dojo.require' replace="//dojo.require" byline="true">
			<fileset dir="${functions}"/>
		</replaceregexp>
	</target>

	<target name="document_workshop" depends="remove_dojorequires">
		<copy file="${build}/${base_workshop_dojo_file}.js" 
                               tofile="${build}/${document_workshop_js}.js" />
		<!-- NOTE: The order of the below files is important -
		            later code expects the previous code to be available -->
		<concat destfile="${build}/${document_workshop_js}.js" append="true">
	    	<filelist id="dw_functions" dir="${functions}">
			    <file name="Utils.js"/>
			    <file name="SHA1.js"/>
			    <file name="Constants.js"/>
			    <file name="Command.js"/>
			    <file name="DocState.js"/>
			</filelist>
			<filelist id="dw_widgets" dir="${widgets}">
				<!-- file name="Resizer.js"/ -->
			    <file name="ImagePicker.js"/>
			    <file name="FloatingImagePicker.js"/>
			    <file name="TextField.js"/>
			    <file name="Element.js"/>
			    <file name="Canvas.js"/>
			    <file name="ColorPalette.js"/>
			    <file name="FontSizePicker.js"/>
			    <file name="FontPicker.js"/>
			    <file name="RichTextToolbar.js"/>
			    <file name="Dialog.js"/>
			</filelist>
	  	</concat>
	  	
	  	<antcall target="-rhino-compress">
			<param name="srcFile" value="${build}/${document_workshop_js}" />
		</antcall>
		<!-- Copy the completed file to the js/dojo directory - 
                           will overwrite the one already there -->
		<copy overwrite="true" 
                               file="${build}/${document_workshop_js}.js" 
                               tofile="${dojo_src}/${document_workshop_js}.js" />
	</target>
	
	<target name="-rhino-compress" unless="nostrip">
		<copy overwrite="true" file="${srcFile}.js" tofile="${srcFile}.uncompressed.js" />
		<java jar="./lib/custom_rhino.jar" 
                              failonerror="true" 
                              fork="true" 
                              logerror="true" 
                              output="${srcFile}.js">
			<arg value="-strict" />
			<arg value="-opt"/>
			<arg value="-1" />
			<arg value="-c" />
			<arg value="${srcFile}.uncompressed.js" />
		</java>
	</target>
	
	<target name="create" depends="clean, init, copy, remove_dojorequires, document_workshop" 
                          description="creates the javascript files">
		<echo message="All done! :)" />
	</target>

	<target name="clean" description="clean up">
		<!-- Clean up the files used -->
		<delete dir="${build}" />
		<delete dir="${widgets}" />
		<delete dir="${functions}" />
	</target>
</project>

Cool, we've now got our Ant file ready to create the individual files. The example above creates only one file. Our production one creates a couple of different files. Because this allows you to pick and choose individual files you can customise the generated JS very specifically. Doculicious has two main "applications". One is the front end web form and the other is the template/style creator. They use similar functions and widgets, but doing it this way I can save about 30kb on the uncompressed files by only including the stuff each page needs.

Now when I'm developing on my code I can just comment out the single file call at the top of the HTML page and uncomment the dojo.require lines to use the individual files, allowing us to program on nicely formatted files that have comments and line breaks. When everything's tested and ready for prod we just re-comment the dojo.requires, uncomment the individual line, run ant create and increment the jsVersion variable so that clients get the new version.

<!-- Comment out the following line to test the individual js files
<script type="text/javascript" src="http://www.digitalcarpenter.com.au/js/dojo/document_workshop.js?v${jsVersion }"></script>		 
-->
<script type="text/javascript">
	<!-- Uncomment these to test the individual js files-->
	dojo.require("digitalcarpenter.widget.Element");
	dojo.require("digitalcarpenter.widget.Canvas");
	dojo.require("digitalcarpenter.system.DocState");

Now our production code is all nicely together in one file. We still have commented and formatted code to develop on, and running it all through ShrinkSafe has also lowered the file size. Our uncompressed file has gone from 370KB to 262KB, but most importantly we've gone from about 50 individual file calls to 2. The file size still seems large, but with mod_deflate on Apache2 it comes down to 68KB and with aggressive caching, clients should only need to download it when we change it.

Blogged with the Flock Browser

Tags: AJAX, Web development, web 2.0, JavaScript,

A fun doculicious example

nospam@example.com (Chris Carpenter) — Wed, 17 Dec 2008 09:46:26 +0000

With the launch of Doculicious.com I thought I'd take the opportunity to post a couple of examples of how it looks and works on an external site.

The original launch article has an example of a business/government form, but I also want to show a more graphical and fun one. The following is an Award Certificate that can be used for many types of occasions. The border, flower decoration and title images are all changeable, just click them to choose a different image.

This is one of the free awards certificate templates on doculicious.com, so with even just a free account you can copy this and change bits around and put it on your own site or blog. You could create more border and text images to have it work for whatever type of award you like.

<div style="margin-bottom: 20px;">Your browser does not support frames, so you can't complete this form here. However, you can <a href="http://www.doculicious.com/img/screenshots/2b9d83ae0a087d5e.pdf">click here to download a PDF file of this form</a></div>
powered by doculicious

Launch of www.doculicious.com!

nospam@example.com (Chris Carpenter) — Tue, 16 Dec 2008 16:04:42 +0000

I'm happy to finally announce the launch of our new web application, www.doculicious.com. It's been under development for the past couple of months, and is a great enhancement on the underlying technology that we've used to build our previous applications.

Doculicious is similar to our previous sites in that it allows people to create documents in their browser, the difference is that we've opened it up so that anyone can create templates and embed them on their own websites.

Because of it's support of PDF, Doculicious is perfectly placed to offer solutions similar to what a PDF file solves, and together with it's web based delivery, it can act like a web form too. We've chosen to focus on this idea of merging a web form and a PDF file, and allow the editing of a document directly in the browser using easy and simple fields, just like a web form. It's an alternative to offering a PDF download that offers many other benefits.

A Doculicious template provides a downloadable PDF, but only after the user has filled in the fields - so the PDF is complete and ready to print, fax, mail or whatever else may be required. After the form is filled out we store the details that were entered and make it accessible to the form owner. This means it can be processed straight away, the data will have less mistakes, there's no retyping or handwriting to read and it's exportable as CSV or as the original PDF. The form owner can go in and edit or add details themselves. There's also integrated notifications, redirection after submissions, editable confirmation pages and you can make unique styles for each template, allowing you to make them match any design. Templates can also be made with a tracking code so when the PDF is printed and sent back, it's easy to match up with the data in the system.

We're really happy with the current system, but we have many features we want to add, and plan on working with our existing and new clients to keep making Doculicious better.

Doculicious has a free account that gives almost all the functionality listed above (styles and redirection are the only premium features), so if you have a need for a PDF application form, or a form where you'd like the data in PDF format, or even just a standard web form, give it a try or at least check out the example page and see just a few ways it can be used www.doculicious.com

Make sure you keep reading to see the example template ...

Continue reading "Launch of www.doculicious.com!"

Doculicious.com Launch Imminent

nospam@example.com (Chris Carpenter) — Tue, 09 Dec 2008 07:51:00 +0000

We plan on launching our new site, doculicious.com, in the next few days. With this launch we will be decommissioning the AustralianBusinessForms.com website, as all the ASIC forms will now be within Doculicious. This is really exciting, and adds a whole new range of features for users, as they can now use front-end web forms to have their clients complete some parts of their forms, while still being able to complete the rest of the form themselves.

Currently we are clearing out the doculicious test database and forum so it's all shiny and new on launch. Also, after the last Open Coffee meet-up in Sydney I had someone mention a feature that I had planned on implementing, but had put off. On the train ride back I couldn't stop thinking about how I could do it. Luckily, the widgely website had a similar functionality developed, but was not moved into production. When I got back to the office and looked at the code I realised it would do everything required, and more. So my launch plans were put off a little as I put this new change in and had it tested. I was actually happy to do this, as for a while I'd had concerns about launching without this feature .. so hearing that it was something people would want gave me the impetuous to get it in.