refactoring the codebase into logically-related modules
In the former article of the series
- We rendered the user interface of our widget inside an anchor element in publisher’s website;
- We asynchronously loaded widget styles;
- We implemented a very naive authentication mechanism.
- We also defined a job queue _wdq (similar to google analytics’ _gaq) to be able to push async commands to our widget;
- And we defined a _wdAsyncInit callback (similar to Facebook’s fbAsyncInit) to be able to notify the publisher when our widget is ready and responsive.
In the following articles, we will improve these mechanisms.
However, in this tutorial our subject matter is a little different:This tutorial is not directly related to JavaScript widget development specifics;
and it’s more about code reorganization.
Logically Splitting Widget API File
If you remember from the former article we used to have a single monolithic api file (api.v.0.1.js) to manage the entire widget initialization flow, event registration, and other API behavior. In a real production application, it’s highly possible that our api code will grow over a few hunderds of thousand lines of code. We will have hard time improving, developing and debugging our codebase.
My experience shows that chaos is the inevitable fate of any software project that lives for several years, and gets developed by different teams of developers along the way. Although we cannot avoid it, at least we can minimize the consequences of it, by organizing chaos in an orderly manner as much as we can.
Refactoring to the Rescue
One way to manage our codebase, is to keep our files in logically-coherent modules; and to segregate those modules further into tiers (we’ll come to that later).
By splitting our code into logically-coherent modules;
- We will have less merge conflicts when working with teams,
- And it will be easier to find the exact point of failure, when a functionality breaks.
A clever refactoring is not a waste of time. Per contra, it will gain you a lot of time, and help you easily find and fix bugs in the long run.
We’ve seen a refactoring case study in the BPC architecture article.
In this tutorial we will follow a similar approach for layering our widget codebase.
BPC Architecture in Review
Here’s an overview of the BPC architecture for a recap:
Four Tiers of Separation
In the figure above, the dashed arrows stand for DOM Event callbacks, and asynchronous server-side callbacks (AJAX/JSONP etc).
The moral of the story is:
- All callbacks go to the delegation tier.
- Delegation tier cannot call the communication tier or the presentation tier directly, it should call the behavior tier first.
- Presentation tier is the “sink“, it cannot call anyone else.
- The only exception is the persistence tier: everyone can read from it, but only the behavior tier can write to it.
Similar to the model above, in this article we will organize our codebase into well-defined behavior, communication, delegation, and persistence tiers; where each tier will contain one or more modules.
Improve BPC Architecture Further
And to improve the structure further, we will decouple our modules as much as we can:
Instead of having modules call public methods of one another, we will be raising custom events, for inter-module communication.
This idiom is widely known as the Publish-Subscribe Pattern.
Using the pubsub pattern, we will decouple our modules further:
Any given module will not have direct knowledge of any other module talking to it.
This loose coupling;
- Will give us the flexibility to easily move code pieces around;
- And, since the modules only know about themselves and no one else, it will be easier to unit test those modules individually in their sandbox.
Application Directory Structure
Enough introductory talk; now let’s see how we apply all this theory to our widget API.
To begin with, here’s the directory struture of our application:
Widget API Directory Tree
In the lib folder we have helper libraries (such as o2.js) and widget modules (i.e. the /lib/wd/ folder).
The widget modules raise events for inter-module communication. The modules also directly use helper libraries for utility functions (such as JSONP calls, and DOM event delegation).
We will come to the implementation of these very soon.
As seen in the image above we have 5 tiers (represented by folders): behavior, comunication, communication, delegation, persistence and presentation.
Each tier, further down, contains modules.
And each module is nothing but a static object under window._wd.protecteds namespace.
For example the Init module under the behavior tier, resolves to window._wd.protecteds.Init
The “protecteds” namespace is an informal indicator that these modules are meant to be used by the internal functionality of our widget.
Protected modules are not meant to be accessed publicly by the publisher.
Establishing Inter Module Communication
In order to send messages to other modules, each module either publishes events:
// protecteds/behavior/init.js
/*
* Load initial widget state data from the server.
*/
function loadState(config) {
log('Init.loadState(');
log(config);
log(')');
// Behavior -> Communication
p.pub('SEND_GET_PARAMS', [config]);
}
…or subscribes to events:
// protecteds/communication/proxy.js
/**
* @function {static} Proxy.subscribe
*
* Subscribes to relevant events.
*/
me.subscribe = function() {
log('Proxy.subscribe()');
...
// Get widget parameters from the server.
sub('SEND_GET_PARAMS', function(params) {
log('event<SEND_GET_PARAMS');
log(params);
log('>');
get(
concat(url.API_ROOT, path.PARAMS),
params,
callback.sendGetParams_complete
);
});
...
};
One caveat though:
Implementing a pubsub pattern is a bit like using a hand grenade.
In the hands of thoughtful experts, you can expect to have a sky scraper demolished safely and in a reasonable time frame.
However letting amateurs mess with the pubsub pattern is akin to giving a box of hand grenades to a gang of monkeys:
You’ll end up with a very noisy and messy scene when you least expect it.
And every once in a while, even experts act like monkeys
.
.Split Main API file into #regions
This is a habit I grew back in my .net/C# development years.
In Visual Studio, you can separate logically distinct parts of the code with #region pragmas. Those #regions will be rendered distinctly to indicate that they are logicially-related groups of code.
Here’s the new api.v.0.1.js split into regions
(I’m replacing parts of the code with … to save some screen real estate):
...
(function(window, document, isDebugMode) {
'use strict';
/* #region version */
/*
* Should match beacon version timestamp.
*/
var versionTimestamp = '20120720135547909116';
/* #endregion */
/* #region widget state */
/*
* Resources to be loaded asynchronously.
*/
var scriptQueue = [];
/*
* This will be set after resource initialization.
*/
var o2 = null;
/* #endregion */
/* #region module common constants */
/*
* Query Formation
*/
var kAnd = '&';
var kEmpty = '';
var kEquals = '=';
var kQuery = '?';
/*
* Regular Expression
*/
var kCompleteRegExp = /loaded|complete/;
/*
* Tags
*/
var kHead = 'head';
var kScript = 'script';
/*
* Mime Types
*/
var kScriptType = 'text/javascript';
/*
* Globals
*/
var kO2Alias = '_wd_o2';
var kWidgetAlias = '_wd';
/*
* Common Widget Keys
*/
var kReadyState = 'readyState';
/* #endregion */
/* #region exported configuration */
/*
* Parameter Names
*/
var param = {
GUID : 'guid',
RANDOM : 'r',
VERSION : 'v',
USERNAME : 'u',
PASSWORD : 'p',
ACTION : 'action',
PAYLOAD : 'payload'
};
/*
* Element IDs
*/
var elm = {
LOGIN_BUTTON : 'wd_btnLogin'
};
/*
* Widget Ready States
*/
var readyState = {
LOADED : 1,
LOADING_DEPENDENCIES : 2,
LOADED_DEPENDENCIES : 3,
BEGIN_PROCESS_QUEUE : 4,
BEGIN_RENDER : 5,
COMPLETE : 6
};
/*
* URL
*/
var url = {
API_ROOT : 'http://api.widget.www/',
O2_ROOT : 'http://api.widget.www/lib/o2.js/',
WIDGET_LIB_ROOT : 'http://api.widget.www/lib/wd/v.0.1/',
LIB_ROOT : 'http://api.widget.www/lib/'
};
/*
* Path
*/
var path = {
BEACON : 'api/v.0.1/beacon',
CSS : 'css/v.0.1/widget.css',
LOGIN : 'api/v.0.1/login',
PARAMS : 'api/v.0.1/params'
};
/*
* Custom Events (used for inter-module messaging)
*/
var event = {
BEGIN_RENDER : 'wd-begin-render',
CSS_LOADED : 'wd-css-loaded',
DELEGATE_EVENTS : 'wd-delegate-events',
FIRE_ASYNC_INIT : 'wd-fire-async-init',
INSERT_BEACON : 'wd-insert-beacon',
LOAD_STATE : 'wd-load-state',
OVERRIDE_QUEUE : 'wd-override-queue',
PROCESS_QUEUE : 'wd-process-queue',
RENDER_DOM : 'wd-render-dom',
RENDER_LOGGED_IN : 'wd-render-logged-in',
RENDER_WIDGET : 'wd-render-widget',
SEND_GET_PARAMS : 'wd-send-get-params',
SEND_LOAD_CSS : 'wd-send-load-css',
SEND_USER_LOGIN : 'wd-send-user-login',
USER_LOGGED_IN : 'wd-user-logged-in',
USER_LOGIN : 'wd-user-login'
};
/* #endregion */
/* #region helper methods (exported) */
/*
* Does nothing, and that's the point.
*/
function noop() {}
/*
* Logs to console for debug mode.
* Does nothing in release mode.
*/
var log = function(stuff) {
...
};
/*
* Sets the internal ready state.
*/
function setReadyState(state) {
window[kWidgetAlias][kReadyState] = readyState[state];
}
/* #endregion */
/* #region sanitization */
// Publisher has forgotten to provide initialization data.
if (!window[kWidgetAlias]) {
log('Widget namespace cannot be found; exiting.');
return;
}
// To avoid re-defining everything if the bootloader is included in
// more than one place in the publisher's website.
if (window[kWidgetAlias][kReadyState]) {
log('Widget has already been loaded; exiting.');
return;
}
/* #endregion */
/* #region protecteds namespace (export root) */
/*
* The "protected" methods are shared across modules, but they
* are not intended for public use.
*/
window[kWidgetAlias].protecteds = {};
/* #endregion */
/* #region widget initialization */
/*
* Asynchronously inserts a script element to the head
* of the document.
*/
function insertScript(root, src) {
...
}
/*
* Revalidates cache for this bootloader script, if there's a newer
* version available. The changes will take effect only AFTER the user
* refreshes the page.
*/
function checkForUpdates() {
...
}
/*
* Exports protected methods for intra-module use.
*/
function exportProtecteds() {
...
}
/*
* Loads the next resource after the former one
* has loaded successfully.
*/
function loadNext(root, loader, callback) {
...
}
/*
* Loads the given script.
* <strong>callback</strong> is the function to be executed after
* there's no resource left to be loeded next.
*/
var loadScript = function(root, src, callback) {
...
};
/*
* Loads an array of scripts one after another.
*/
function loadScripts(root, ar, callback) {
...
}
/*
* First loads necessary o2.js components in noConflict mode.
* Then loads protected modules.
*/
function loadDependencies(callback) {
log('o->loadDependencies(');
log(callback);
log(')');
setReadyState('LOADING_DEPENDENCIES');
loadScripts(url.LIB_ROOT, [
'o2.js/o2.meta.js',
'o2.js/o2.core.js',
'o2.js/o2.string.core.js',
'o2.js/o2.jsonp.core.js',
'o2.js/o2.dom.constants.js',
'o2.js/o2.dom.core.js',
'o2.js/o2.dom.load.js',
'o2.js/o2.event.constants.js',
'o2.js/o2.validation.core.js',
'o2.js/o2.event.core.js',
'o2.js/o2.event.custom.js',
'o2.js/o2.method.core.js',
'o2.js/o2.collection.core.js',
'wd/v.0.1/protecteds/behavior/init.js',
'wd/v.0.1/protecteds/behavior/queue.js',
'wd/v.0.1/protecteds/behavior/widget.js',
'wd/v.0.1/protecteds/communication/proxy.js',
'wd/v.0.1/protecteds/delegation/callback.js',
'wd/v.0.1/protecteds/delegation/event.js',
'wd/v.0.1/protecteds/persistence/config.js',
'wd/v.0.1/protecteds/presentation/dom.js',
'wd/v.0.1/protecteds/presentation/rendering.js'
], callback);
}
/* #endregion */
/* #region widget initialization flow */
// At the end of the initialization flow, readyState will be finally
// set to COMPLETE. When the readyState is COMPLETE, it means that
// the widget UI has been rendered, the events have been bound,
// widget job queue has been processed, and the widget is completely
// ready and responsive.
setReadyState('LOADED');
checkForUpdates(versionTimestamp);
loadDependencies(initialize);
/* #endregion */
}(this, this.document, true));
Adding Modules as Dependencies
One thing we add to our widget’s initialization flow is the URL‘s of our protected modules.
We first load them as dependencies, and let each module subscribe to custom API events once they are loaded.
We’ll come to the module subcription soon.
Here’s the set of files we load as dependencies:
/*
* First loads necessary o2.js components in noConflict mode.
* Then loads protected modules.
*/
function loadDependencies(callback) {
log('o->loadDependencies(');
log(callback);
log(')');
setReadyState('LOADING_DEPENDENCIES');
loadScripts(url.LIB_ROOT, [
'o2.js/o2.meta.js',
'o2.js/o2.core.js',
'o2.js/o2.string.core.js',
'o2.js/o2.jsonp.core.js',
'o2.js/o2.dom.constants.js',
'o2.js/o2.dom.core.js',
'o2.js/o2.dom.load.js',
'o2.js/o2.event.constants.js',
'o2.js/o2.validation.core.js',
'o2.js/o2.event.core.js',
'o2.js/o2.event.custom.js',
'o2.js/o2.method.core.js',
'o2.js/o2.collection.core.js',
'wd/v.0.1/protecteds/behavior/init.js',
'wd/v.0.1/protecteds/behavior/queue.js',
'wd/v.0.1/protecteds/behavior/widget.js',
'wd/v.0.1/protecteds/communication/proxy.js',
'wd/v.0.1/protecteds/delegation/callback.js',
'wd/v.0.1/protecteds/delegation/event.js',
'wd/v.0.1/protecteds/persistence/config.js',
'wd/v.0.1/protecteds/presentation/dom.js',
'wd/v.0.1/protecteds/presentation/rendering.js'
], callback);
}
Widget Initialization
We also added a couple of extra lines to the widget initialization code:
/*
* Initialize after loading prerequisites.
*/
function initialize() {
...
exportProtecteds();
var config = wp.Config.get();
config[param.GUID] = o2.String.generateGuid();
subscribe();
wp.pub('LOAD_STATE', [config]);
}
Let’s expand the above code, line by line.
We first export a set of methods that will be commonly used in all modules:
/*
* Exports protected methods for intra-module use.
*/
function exportProtecteds() {
log('o->exportProtecteds()');
var wp = window[kWidgetAlias].protecteds;
wp.sub = function(name, callback) {
var nom = wp.event[name];
if (!nom) {
log(['wp.sub: No such event for "', name, '"'].join(kEmpty));
return;
}
o2.Event.subscribe(nom, callback);
};
wp.pub = function(name, payload) {
var nom = wp.event[name];
if (!nom) {
log(['wp.pub: No such event for "', name, '"'].join(kEmpty));
return;
}
o2.Event.publish(nom, payload);
};
wp.event = event;
wp.log = log;
wp.noop = noop;
wp.o2 = o2;
wp.path = path;
wp.setReadyState = setReadyState;
wp.url = url;
wp.param = param;
wp.elm = elm;
}
wp is an alias for window._wd.protecteds namespace. We bind a bunch of helper functions and configuration objects to that namespace to access them from other modules.
Then we call subscribe methods of related modules (which makes the modules subscribe to related event, as the name implies):
/*
* Trigger modules to subscribe to events.
*/
function subscribe() {
var wp = window[kWidgetAlias].protecteds;
wp.Init.subscribe();
wp.Queue.subscribe();
wp.Event.subscribe();
wp.Widget.subscribe();
wp.Proxy.subscribe();
wp.Rendering.subscribe();
}
o2.js PubSub Internals
The internals of the pubsub mechanism is managed by o2.event.custom.js module, which is an extension to the o2.Event class.
The o2.event.custom module exports three methods: o2.Event.publish, o2.Event.subscribe, and o2.Event.unsubscribe as follows:
(function(framework) {
'use strict';
var _ = framework.protecteds;
var attr = _.getAttr;
var create = attr(_, 'create');
var def = attr(_, 'define');
var require = attr(_, 'require');
var exports = {};
var kModuleName = 'Event';
var me = create(kModuleName);
/*
* Aliases
*/
var isArray = require('Validation', 'isArray');
var cache = {};
exports.publish = def(me, 'publish', function(name, argv) {
if (!name) {
return;
}
var delegates = cache[name];
var args = argv || [];
if (!isArray(args)) {
args = [args];
}
var i = 0;
var len = 0;
if (!delegates) {
return;
}
for (i = 0, len = delegates.length; i < len; i++) {
try {
delegates[i].apply(null, args);
} catch (ignore) {}
}
});
exports.subscribe = def(me, 'subscribe', function(name, callback) {
if (!name) {
return;
}
if (!cache[name]) {
cache[name] = [];
}
cache[name].push(callback);
var handle = {
name : name,
callback : callback
};
return handle;
});
exports.unsubscribe = def(me, 'unsubscribe', function(handle) {
var name = handle.name;
var callback = handle.callback;
var delegates = cache[name];
var i = 0;
var len = 0;
var delegate = null;
if (!delegates) {
return;
}
for (i = 0, len = delegates.length; i < len; i++) {
delegate = delegates[i];
if (delegate === callback) {
delegates.splice(i, 1);
len = delegates.length;
i--;
}
}
});
}(this.o2));
Where window._wd.pub and window._wd.sub methods wrap around o2.Event.publish, and o2.Event.subscribe methods respectively.
Read the Source Luke
You can view the source code from this github history snapshot.
Conclusion
That concludes our quick tour on widget refactoring 
.
In this article;
- We divided our widget into logical modules, and grouped those modules into different tiers;
- We also implemented a custom publisher subscriber messaging system for inter-module communication;
- As a corollary to the above; almost all of the modules never exposed public methods, and instead modules sent and received messages using the pubsub mechanism above, rather than calling methods on each other.
- This pubsub mechanism allowed “loose coupling” between modules; which, in turn, will make it possible to unit test each module independently.
Next Up?
In the following article, we will convert our application with something more meaningful, use CORS for cross domain POST requests, detect if CORS is supported, and discuss alternatives if it’s not. We will also discuss minification, compression and deployment strategies; and ways to secure our widget and prevent it from certain XSS and CSRF attacks.
In short we will try wrap things up in the next article .
Until then, feel free to share your comments and suggestions.
