Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

February 27 2014

09:22
Tags: Mobile

February 26 2014

14:30

February 24 2014

12:38
Tags: Mobile

February 19 2014

11:41
Tags: Mobile

February 11 2014

12:13

Four Ways To Build A Mobile Application, Part 3: PhoneGap


  

This is the third installment in a series covering four ways to develop a mobile application. In previous articles, we examined how to build a native iOS and native Android tip calculator. In this article, we’ll create a multi-platform solution using PhoneGap.

Adobe’s PhoneGap platform enables a developer to create an app that runs on a variety of mobile devices. The developer accomplishes this largely by writing the user interface portion of their application with Web technologies such as HTML, CSS and JavaScript. PhoneGap’s development tools then bundle the HTML, CSS and JavaScript files into platform-specific deployment packages. PhoneGap supports a wide variety of platforms:

  • iOS
  • Android
  • Windows 8
  • Windows Phone 7 and 8
  • BlackBerry 5.x+
  • WebOS
  • Symbian
  • Tizen

For this article, we’ll focus on getting our sample FasTip application running on iOS and Android:

ios_android-550

As with the previous articles in this series, all of the code for our application may be obtained from a GitHub repository.

Applications built with PhoneGap use the mobile platform’s Web view to render content. As such, the content will appear nearly identical on each platform, much as any Web page would. While you can style the controls differently on each platform, take care in doing this. This issue is covered in detail in the section below on multi-platform considerations.

Plugins: Closing The Gap On Native Features

PhoneGap essentially wraps a Web view of your HTML, CSS and JavaScript in a native application. This is required because the Web view in an application does not inherently support many device features, such as access to the file system or the camera. PhoneGap has a bridging mechanism that allows JavaScript running in the Web view to invoke native code contained in the application. PhoneGap comes complete with plugins to support device capabilities such as the following:

  • accelerometer,
  • camera,
  • contacts,
  • file system,
  • media playback and recording,
  • network availability.

A full list of capabilities for each platform is available on PhoneGap’s website. If these capabilities aren’t enough, PhoneGap may be extended with plugins that enable the developer to access more device features, including these:

  • barcode scanning,
  • Bluetooth,
  • push notifications,
  • text to speech,
  • calendars,
  • Facebook Connect.

In previous versions of PhoneGap, a GitHub repository contained a set of prebuilt plugins. With the arrival of PhoneGap 3, a new plugin architecture has resulted in the old repository being deprecated. A registry has been created for all plugins compatible with PhoneGap 3.

Some command-line tools are also provided to make it easy to add the plugins to the repository for your project. We’ll see these later on in this article. PhoneGap also publishes documentation and examples on how to write your own plugins. Of course, development of plugins assumes familiarity with the native platform on which the plugin is to be supported.

An HTML, CSS And JavaScript Foundation For Mobile Development

The majority of PhoneGap’s capabilities lie in non-visual components — things that access the file system, network availability, geolocation, etc. PhoneGap does not provide much assistance with building the user interface itself. For this, you must rely on the HTML and CSS foundation that you’ve built yourself or on a framework. Applications written for mobile browsers must respect the limitations of the given mobile platform (processing speed, screen size, network speed, touch events, etc.).

Unless you have been working with HTML and CSS for a long time and are well aware of these issues, developing an effective mobile application without some sort of framework can be daunting.

Fortunately, some mobile frameworks have arisen to help with this. Here are just some of the offerings in this area:

These frameworks vary from CSS-oriented libraries (like Topcoat) to complete MVC-based libraries with sets of mobile UI controls (like Sencha Touch). Discussing the differences between these frameworks in detail is beyond the scope of this article and would require an entire series of articles. Visit the websites above and try some of the demonstrations on several mobile devices.

One distinction to be made is that some frameworks support a wider variety of devices and device versions than others. Some mobile frameworks are built on a particular MVC platform. For example, Ionic is built on the AngularJS framework. This might make a particular library more attractive to developers who are already familiar with the respective MVC framework.

Frameworks such as Sencha Touch abstract the DOM from the developer through the use of APIs, freeing the developer from having to worry about details of browser vendor implementations. Sencha also provides graphic development tools, such as Sencha Architect, to aid the development process. Sencha provides commercial support and training, too, which are worth investigating.

I’m often asked to recommend a framework for mobile development. But the right tool depends largely on your application’s functionality, the mobile platforms you need to support, your demand for commercial support and your experience with Web and mobile development in general. I’m encouraged by some emerging frameworks, particularly Ionic, but as of the time of writing, Ionic is still in the alpha stage. If you are looking for something with a longer track record, then Sencha Touch or, for some cases, jQuery Mobile might be appropriate.

To be expedient and because of the wealth of tutorials and documentation available, I’ve written our FasTip application in jQuery Mobile, which provides a wide range of support for mobile devices. It’s a good choice for applications that do not require significant customization to the user interface. The simplicity of our application makes it well suited to jQuery Mobile.

PhoneGap Or Cordova: What’s In A Name?

In the process of learning about PhoneGap, you’ll encounter the name Cordova. In fact, the command-line tool that we’ll use in just a moment to create our PhoneGap project is named Cordova. Simply put, Cordova is the open-source project that is the basis of PhoneGap. Think of it as the engine that drives PhoneGap.

In most cases, the names may be used interchangeably. With “PhoneGap” already being trademarked, a new name was needed when the project was open-sourced — hence, Cordova. “PhoneGap” continues to be used by Adobe for commercial products. Most people still refer to
the project as a whole as PhoneGap.

phonegap-cordova-550

Installing PhoneGap

With the introduction of version 3, installing PhoneGap’s tools and setting up projects have been greatly simplified, thanks to the new command-line interface (or CLI). The command-line tools are based on Node.js; thus, you must install it before installing PhoneGap. Once Node.js is in place, simply run one command for the node package manager to install PhoneGap’s tools:


npm install -g cordova

Native SDKs: The Rest of the Equation

The command-line tools provide only the PhoneGap portion of the tools that are necessary for development. You will still need the mobile SDKs for the platforms that you wish to support. For this article, we’re mostly concerned with iOS and Android. For iOS, you’ll need to install the same developer toolset that we used when building native Objective-C applications.

Refer to the earlier article on native iOS development for instructions on installing those tools. This will give PhoneGap the tools necessary to compile and build your app into something that can be deployed to an iOS device. Likewise, to build applications for Android, you’ll need the Android SDK.

The previous article on native Android development provides some assistance in locating these tools and installing them. In addition, you may wish to consult PhoneGap’s documentation for details on installing for Android and iOS.

FasTip In PhoneGap

Given that FasTip is such a simple application, we don’t have to leverage much of PhoneGap’s functionality to access device features. Most of our time will be focused on the Web components needed to build the user interface. Unlike with native platforms, there is no vendor-supplied integrated development environment (IDE) for PhoneGap that is similar to Xcode or Eclipse.

Developers may use whichever IDE they please. I use JetBrains’ excellent WebStorm IDE for Web development because it suits PhoneGap development nicely. We’ve developed much of the FasTip application in WebStorm and tested it on the desktop version of Google’s Chrome browser, before packaging it for mobile devices in PhoneGap.

webstorm-550

The figure above shows the FasTip application loaded in WebStorm. Note that relatively few files make up this project:

file-structure

The user interface is shaped by index.html. This one file contains all of the HTML used by this application. The HTML starts out with this head section:


<head>
  <meta name="viewport"
    content="initial-scale=1, minimum-scale=1, maximum-scale=1">
  <meta charset="utf-8">
  <title>FasTip - Tip Calculator</title>
  <link rel="stylesheet" href="css/themes/default/jquery.mobile-1.4.0.css">
  <link rel="stylesheet" href="css/app.css">
  <script src="js/jquery.js"></script>
  <script src="js/jquery.mobile-1.4.0.js"></script>
</head>

The viewport meta tag scales the width of the page to match the width of the device’s browser. If this is not present, then the mobile browser wouldn’t know that the page is optimized for mobile use and would zoom out to display the page as though it were on a desktop browser.

Next, we load two style sheets. The vast majority of the CSS is contained in the jquery.mobile.css file, with a few minor application-specific CSS overrides in the app.css file. The head section then loads the scripts needed to support jQuery and jQuery Mobile.

Navigating Screens, jQuery Mobile-Style

Unlike our native apps, in which we had separate view controllers or activities for each screen, here we’ll place the HTML markup for both screens in one file. jQuery Mobile supports the concept of a page being divided into sections that make it appear to the user as though they are navigating between screens. In fact, the user would never leave this single Web page.


<div data-role="header">
  <h1>FasTip</h1>
  <a href="#settingsPage" id='settingsButton' class="ui-btn-right&quot
    data-role="button" data-icon="gear">Settings</a>
</div>
<div data-role="content">
  <form>
  <div data-role="fieldcontain">
  <label for="billAmount">Bill Amount:</label>

Above is the code in the HTML file for the header of the main input form for the tip calculator. Note that many of the HTML elements have data- attributes. Pay particular attention to the data-role attribute, which indicates to jQuery Mobile the purpose of the various elements on the page. The data-role="header" attribute defines the upper part of the page, which looks something like the UINavigationController in iOS apps. The data-role="content" attribute is the main content of our page and displays the main form of the tip calculator.

jqm-data-roles-500-opt

For our native iOS and Android apps, we had to write some code to enable the user to navigate between pages. In PhoneGap, a simple HTML anchor tag takes the user to the settings page:


<a href="#settingsPage" id='settingsButton' class="ui-btn-right"
  data-role="button" data-icon="gear">Settings</a>

Further on in the HTML, we see this:


<div data-role="page" id="settingsPage" data-add-back-btn="true" >
  <div data-role="header">
   <a href="#mainPage" class="ui-btn ui-icon-delete ui-btn-icon-left"
    data-rel="back">Cancel</a>
   <h1>FasTip - Settings</h1>
  </div>
  <div data-role="content">

This div element defines the boundary of the settings page in our application. The page has a div in it for the header and the content, just like the main page. Note how the first anchor tag in the header div defines the “back” and “cancel” button that enables the user to return to the first screen. The data-rel="back" attribute signifies that the link should close the current page and navigate to the main page, just like hitting the “back” button in a Web browser. In fact, pressing the dedicated “back” button on an Android device would perform the same function.

jQuery Mobile takes care of performing the transitions between these virtual “pages” in our application. Because much of the user interface and interaction between the controls is defined declaratively in our HTML, we have considerably less code to write to get the application running. All of our code is confined to a single app.js file that is just 41 lines long. The meat of our application starts with this snippet:


$( document ).on( "ready", function(){
  $('#calcTip').on('click', calcTip);
  $('#saveSettings').on('click', saveSettings);

  var tipPercentSetting = localStorage.getItem('tipPercentage');

  if (tipPercentSetting) {
   tipPercent = parseFloat(tipPercentSetting);
  }

  $('#tipPercentage').val(tipPercent);
});

When the document is ready, event handlers are attached to the buttons in order to calculate tips and save settings. In this example, we’re using HTML5’s localStorage object to persist the setting for a tip percentage. PhoneGap supports localStorage and will save the content in the proper directory of the native platform to ensure that the data is backed up with the device’s normal backup mechanism. When the user choses to save their settings, the following code is run:


var saveSettings = function() {
  try {
   var tipPct = parseFloat( $('#tipPercentage').val() );
   localStorage.setItem('tipPercentage', tipPct);
   tipPercent = tipPct;
   window.history.back();

  } catch (ex) {
   alert('Tip percentage must be a decimal value');
  }
};

Note the use of localStorage.setItem to write the tip percentage under the key of tipPercentage. Once the user has saved their tip percentage, they return to the previous screen. Because jQuery Mobile integrates with the browser’s History API when transitioning between pages, we can leverage the History API here and simply jump back to the previous page via window.history.back();. On Android where the device supports a “back” button, this will have the same effect as hitting the browser’s “back” button and moving back one screen.

For more complex apps, jQuery Mobile also enables you to dynamically load new pages, rather than place all of the markup in one file. This is the preferred approach if your application has several screens, because maintaining all of the markup in one file would become difficult and the mobile browser would have to parse that large document before rendering anything. Forcing the browser to process a large document would slow down the startup time of your application.

Calculating the value of the tip is done with the following code:


var calcTip = function() {

  var billAmt = Number( $('#billAmount').val() );
  var tipAmt = billAmt * tipPercent/100 ;
  var totalAmt = billAmt + tipAmt;
  $('#tipAmount').text('$' + tipAmt.toFixed(2));
  $('#totalAmount').text('$' + totalAmt.toFixed(2));

};

This is standard jQuery code for reading the bill amount, calculating the tip percentage and returning the results to the display fields on the page. When the user taps on the calcTip button, the calcTip function is invoked via an event handler that is attached when the DOM is loaded and ready:


$( document ).on( "ready", function(){
  $('#calcTip').on('click', calcTip);
  …
});

The interaction between the DOM and the HTML and CSS should be familiar to anyone who writes Web pages. Of course, other frameworks, such as Backbone.js and AngularJS, may be employed to bind data, render content from templates and so on. This is an important aspect of PhoneGap: It does not dictate that particular libraries or CSS be used. Use whatever tools you want to get the job done.

Getting Our Project Into PhoneGap

PhoneGap provides a command-line tool, named Cordova, to set up a new project, build an application, run the application and install plugins. Follows the steps below to set up a project in PhoneGap.

Step 1

To create our application, navigate to an empty directory and use the create command:


$ cordova create fastip org.traeg.fastip FastTip

This command will create a subdirectory, named fastip, to hold our application files and to establish a new application named FasTip, with an org.traeg.fastip namespace.

Step 2

Switch to the fastip directory that PhoneGap has just created from the command above.

Step 3

We’ll need to tell PhoneGap which platforms we want this app to run on. We’re supporting iOS and Android, so run the following two commands to support them:


$ cordova platform add ios
$ cordova platform add android

Step 4

We can test our project to see whether PhoneGap has been properly initialized:


$ cordova build ios
$ cordova emulate ios

This will compile the application for iOS and then run it in the iOS simulator.

cordova-splash-500

To run the application on Android, you would use the following commands:


$ cordova build android
$ cordova emulate android

Note the directory structure that PhoneGap has established:


platforms/ios
platforms/android

Note also that the sample code is copied by PhoneGap into the respective iOS and Android subdirectories. When you run the build operation, PhoneGap updates the respective build directories and places the updated files in a project suitable for each platform. If you examine the iOS directory, you’ll see that an .xcodeproj file has been created.

In fact, this project may be opened in Xcode, like any other iOS project. Similarly, the Android project may be readily imported into Eclipse. Do not change the contents of your application’s files in the platform directories, or else they will be overwritten the next time the build process is run.

Step 5

We can place the various files that comprise our application in the www folder directly in the project’s root. The next time the cordova build command is executed, the application’s content will be copied to the respective platform directories, and the application will be run with that new content.

file-structure

Note that our Web application’s entire structure resides in this www directory. When you create a new PhoneGap project, it is preconfigured to run whatever is in the index.html file in the www directory. In addition to the Web files that we’ve added to the project, one more file is important to a PhoneGap application, config.xml. This file controls various options used in the build process, including the display of a startup splash screen, viewport scaling, supported device orientations, etc. The settings for this file will vary according to the platform, so review the settings for each.

Also, note that PhoneGap has a merges directory. In this directory you can place platform-specific files, which will be copied to the respective build directories. In this way, you can customize HTML, CSS and JavaScript files to each platform.

Considerations For Supporting Multiple Platforms

We’ve seen that, by using PhoneGap and a framework like jQuery Mobile, building a cross-device application is easy. One caveat with jQuery Mobile is that both iOS and Android users would see the same UX; that is, the UX would not be tailored to the respective platform, much like a Web page looks the same whether viewed on OS X or Windows.

Attempting to mimic the native platform controls in HTML and CSS is generally a bad idea. While you could approximate the look of the native platform, mimicking the feel of the platform is exceedingly difficult. The controls might not respond to taps and other gestures in exactly the same way. The result is usually a UX that just feels wrong. Successful PhoneGap projects generally eschew this approach in favor of a platform-independent UX.

If your project requires an experience that closely matches that of the native platform, PhoneGap might not be the right choice.

HTML5 capabilities vary widely across devices. While iOS and Android browsers are built on WebKit, some features are implemented only in the latest versions of the platforms. One useful website for checking browser version capabilities is Mobile HTML5. The browsers on some old platforms, particularly Android 2.x, have various rendering bugs, which could also hamper your development.

Frameworks such as jQuery Mobile and Sencha Touch take these compatibility issues into account. If you choose to roll your own solution, perhaps to gain greater control over the UX, then you will need to plan additional time for testing, particularly for old device versions, on which HTML5 support is less consistent.

jQuery Mobile makes it easy to get started and provides a good feature set for simple applications. Much of the functionality on the screens of our sample application is provided by jQuery Mobile itself, leaving us with less code to write. However, the speed of development with jQuery Mobile comes at the expense of customization. While jQuery Mobile does offer a “theme roller” tool to customize appearance, much of that is “chrome” related to colors and visual attributes like rounded corners.

Assess whether your application fits jQuery Mobile’s paradigm. The platform does offer some alternate layouts for things like forms, but the options are still somewhat limited. If your application requires a truly custom look or layout, then you might find jQuery Mobile too confining. Consider one of the frameworks suggested earlier. Or adopt a “best of breed” approach, mixing a CSS-oriented framework (such as Twitter Bootstrap, Zurb Foundation or Ratchet) with an MVC framework (such as AngularJS or Backbone.js).

Some of the large mobile solutions, such as Sencha Touch and Ionic, marry a powerful MVC platform to a set of prebuilt controls focused on mobile development. These tools are attractive and worthy of consideration. But bear in mind the different range of devices that each of these tools supports. At the moment, Sencha Touch supports more devices than Ionic, but jQuery Mobile supports even more. This should also factor in your selection process.

Using A Native Plugin

If you try our sample application on iOS 7, you’ll run into an issue with the device’s status bar that has affected many PhoneGap applications. That is, the status bar now overlaps the top of the application:

ios_status-bar-overlap

You could give the body tag a 20-pixel top margin to make the page clear the status bar, but that would cause problems on iOS 6 and Android. A cleaner solution is to use a native plugin that fixes this specific issue. To add the plugin to our sample project, we’ll use this command:


cordova plugin add org.apache.cordova.statusbar

This pulls the necessary code from the plugin repository into our application. Now we’re ready to invoke the plugin in our code:


$( document ).on( "deviceready", function(){
  StatusBar.overlaysWebView( false );
  StatusBar.backgroundColorByName( "gray" );
});

This snippet of code waits for the deviceready event from PhoneGap, which tells our application that the PhoneGap environment has been initialized and is ready to receive commands from the JavaScript. The StatusBar object has been added to the window namespace via the PhoneGap plugin system so that we can access it via JavaScript.

The following screenshot shows the status bar overlay being turned off, as well as the background color of the status bar being changed to better fit our application:

ios_status_bar_good

Building In The Cloud

Generally speaking, to build an application for a particular platform, you must install the SDK for that platform on your machine. This could be a problem if, for instance, you’re on a Mac and want to target Windows tablets, whose SDK requires you to be on a Windows machine. Adobe offers a service named PhoneGap Build to help in this situation.

PhoneGap Build enables you to upload a ZIP file containing the HTML, CSS and JavaScript of your application. Additionally, PhoneGap 3 enables you to submit an application to PhoneGap Build right from the command line; from there, PhoneGap Build takes over and produces a deployment bundle for the desired platform.

build-500

Once the build process is complete, you may either download the deployment bundle and manually install it to your device or take a picture of a QR code on PhoneGap Build’s website and download the deployment package directly to your mobile device. What’s more, PhoneGap Build supports a feature named Hydration, which enables the application to download updates to the HTML, CSS and JavaScript files whenever it launches.

This means that testers will always be running the latest version of your application, without having to go through the traditional updating process. Note that, at least for now, Hydration is meant for development and testing — not for final production code that will be submitted to the app store.

Development in the Cloud

You can do more than simply compile and package your application in the cloud. Cloud-based Icenium provides a browser-based IDE that lets you edit code in a browser and immediately turn those edits into a deployment package. Using an application in the iOS App Store named Icenium Ion, you can dynamically load updates to your application as you change the code in the Web-based IDE.

In addition, Icenium bundles a license to Telerik’s KendoUI library. I’ve used this to make changes to an iOS PhoneGap application while on a Windows-based laptop. Icenium also offers a Windows desktop IDE named Graphite, which expands on the capabilities of the browser-based tool.

Another alternative to PhoneGap Build is AppGyver. It is not an IDE like Icenium, but it does offer a UX library, named Steroids, that facilities navigation between Web views using some native controls. Note that for page content itself, you must still use HTML and CSS.

The main downside to cloud-based development services is that they have limited support for PhoneGap plugins. However, new plugins are being supported all the time. Determine your application’s needs to see whether a particular cloud-based tool supports the plugins you require.

Debugging Your Application

In most cases, when attempting to debug an application, you’ll be mostly concerned with debugging the JavaScript, examining the console log and perhaps doing some interactive inspection and manipulation of the DOM. Unfortunately, there is no integrated way to debug across all platforms. However, there are several useful techniques, covered below.

Chrome DevTools

I generally find it easiest to work with Chrome DevTools by treating my pages as standard Web pages loaded in a desktop browser (although this doesn’t help with the portions of code that rely on PhoneGap plugins). Chrome DevTools provides an emulation panel that simulates the aspect ratios, touch events and more of various mobile device screens.

Apache Ripple

Apache’s Ripple is a NodeJS project that serves PhoneGap content in a desktop browser. Ripple enables you to experiment with different screen sizes and orientations. It also allows you to simulate geolocation coordinates and network connectivity events, as well as allows JavaScript code to invoke many of PhoneGap’s core plugins while running in a desktop browser.

Ray Camden has more information on using Ripple on his blog. Do not confuse this newer NodeJS-based version of Ripple with the outdated Chrome browser extension. The Chrome extension is no longer supported and is not compatible with PhoneGap 3.

Using Desktop Safari to Debug Web Views in iOS Apps

Starting with Safari 6, Apple has included support for using the Web inspector in desktop Safari to debug Web views running in the simulator. With your app running in the simulator, simply go to the “Develop” menu in Safari and look for the “iPhone Simulator” option. This will connect the Web inspector to the running instance of your PhoneGap application and will allow to you manipulate the DOM, set breakpoints in JavaScript, etc.

Using Chrome DevTools to Debug Web Views in Android Apps

Starting with version 4.4, Android’s Web view is now based on Google Chrome. With this change comes the ability to connect to the Web view from DevTools and to debug interactively, just like you can with iOS. If you don’t have a 4.4 device, fear not: It works with the 4.4 emulator as well. Note that you’ll need to add a special plugin to your PhoneGap project to enable Web view debugging on Android.

Web Inspector Remote

Web Inspector Remote (WEINRE) provides a partial set of debugging features, including DOM manipulation and access to the console. However, it does not support JavaScript debugging. Add WEINRE to your application by including a script tag in your application’s HTML and running a NodeJS server that this script connects to. The main advantage of WEINRE is that it works with earlier versions of WebKit on mobile devices and is one way of debugging cross-platform.

The PhoneGap wiki also has a page on debugging, with links to other tools.

Learning More

Much of the work you’ll do with PhoneGap requires a strong understanding of HTML, CSS and JavaScript. Consult resources such as Nicholas Zakas’ excellent book Professional JavaScript for Web Developers to build a foundation for development with PhoneGap. Of course, other resources exist for jQuery Mobile, AngularJS, Backbone.js and other frameworks. Google and Udacity have recently teamed up to offer an online course on mobile Web development, whose content applies to PhoneGap applications as well.

As for PhoneGap itself, the project has some introductory guides for each platform. And Adobe publishes a free monthly digital magazine, Appliness, covering Web and PhoneGap development. The PhoneGap wiki covers issues such as debugging and security. Finally, some great blogs by Adobe’s technical evangelists cover PhoneGap and Web development:

  • Andrew Trice provides a lot of sample code, along with some great performance tips.
  • Christophe Conraets has a number of articles on ways to structure an application using MVC frameworks. He also covers how to integrate with server resources.
  • Raymond Camden offers a variety of articles on PhoneGap plugins and PhoneGap Build.
  • Holy Schinsky has a series of articles on how to use and write PhoneGap plugins, as well as on how to incorporate push notifications.

If you’d like to further explore the marriage of native code and Web code to produce mobile applications, my Smashing Magazine article on “Mixing HTML5 and Native Code” might be of interest.

In the next and final installment in this series, we’ll explore another way to develop an application for multiple platforms, with Appcelerator Titanium.

(al, ea)


© Peter Traeg for Smashing Magazine, 2014.

Tags: Mobile

February 06 2014

13:10

Applying XSL Transformations To Responsive Web Design


  

To most Web developers, it sounds controversial until you hear the punchline: Last summer, the developers in charge of Google’s Chrome browser floated a proposal that went virtually unnoticed by the technology press, which was to remove support for an established W3C standard that every other browser vendor still supports. The standard in question? Extensible Stylesheet Language Transformations, otherwise known as XSLT.

In reaction to this news, most Web developers would likely shrug and say “So what?” That’s unfortunate.

Transformations are a simple yet powerful technique for separating content and presentation in Web applications. Yet, outside of enterprise and data-processing circles, transformations have failed to gain popularity through XSLT. As a result, Web developers are liable to think that transformations “don’t apply to me,” even though they work with HTML, a structured format ripe for transformation. Thankfully, new transformation frameworks are on the horizon, including work by the inventor of Sass, that hold the promise of a revival.

In this article, we will reintroduce transformations and explore their applications to mobile and responsive design. We will not only teach the old dog new tricks, but show that transformations are relevant to everyone who deals with HTML.

Separating Content And Presentation

A transformation is simply a system that transforms its input into an output according to a set of predefined rules.

The data flow of a transform
The data flow of a transformation. (Large view)

The key thing about transformations isn’t the action they perform but the capability they enable. Transformations create an abstraction that decouples content and functionality from presentation. This separation is a design goal of many frameworks, but transformations facilitate it in powerful and unique ways.

The transformation data flow recast as a separation of concerns
The transformation data flow recast as a separation of concerns. (Large view)

A powerful example of this decoupling occurs in Enlive, a transformation and templating system written in Clojure. Most templating engines use a customized markup language to mix HTML with programming constructs such as loops and variables. Take PHP, for example:

<ul>
  <?php foreach ($task_list as $task) { ?> 
    <li> <?php echo $task ?> </li>
  <?php } ?> 
</ul>

By contrast, Enlive templates use the same plain old HTML that you would get from your designer or PSD slicer. For example, a simple hello-world.html Enlive template might look like this:


<html>
	<body>
		<h1 id="output">Lorem Ipsum</h1>
	</body>
</html> 

Instead of embedding logic in the markup, Clojure code that is associated with the HTML transforms it into output:


  (deftemplate helloworld 
    "hello-world.html" 
    [] 
    [:h1#output] 
    (content "Hello World!"))

Completely understanding the code above is not important, but you’ll probably recognize the h1#output argument as a CSS selector. In this example, a template named helloworld is being defined, in which the h1#output element’s content is replaced by “Hello World!” When executed, this template would output the following:


<html>
	<body>
		<h1 id="output">Hello World</h1>
	</body>
</html>

For more on Enlive, I recommend David Nolen’s excellent tutorial. But the key point is that content and presentation have been decoupled. Website changes, A/B tests and even a redesign could be as easy as getting new HTML from your designer and dropping it in. In addition, the designer doesn’t need to know anything about the templating language and may use HTML classes and IDs, concepts that they already know.

While you don’t need to build a website in this way, the situation is analogous to building a Web page without a style sheet. True, you could design a page purely with inline styles (that is, using only the style attribute), and novice developers often code this way out of expediency. But experienced developers know that a style sheet improves workflow and productivity at scale.

Similarly, by separating content and presentation, you will unlock more productivity and agility for your team. In effect, transformations truly separate the front end from the back end and create a new workspace for the visual design team to operate independently of the rest of the system. In an industry where even simple things like the color of a button can affect conversion rates, enabling your visual design team to iterate more quickly and continually could deliver tremendous value to the bottom line.

Responsive Retrofitting

Transformations are not just useful in a templating system. This decoupling of content and presentation can also be applied to an existing website, enabling developers to apply a new presentation regardless of how the original website was built. This combination of separating content and presentation and its applicability to existing websites is what makes transformations so powerful and useful to a broader audience than currently use them. I’ll illustrate this by responsively retrofitting an existing website using a transformation technology that’s in every browser today (at least for now), XSLT.

XSLT was introduced in the late 1990s as a language for transforming XML and XHTML documents. During the ascendency of XML, XSLT was seen as a solution for separating content and presentation in Web applications built on XML data formats. The W3C recommended XSLT as a standard, and almost every major browser incorporated support for some form of the language.

Now that Google’s Chrome and Blink team has proposed dropping support for XSLT, some might be concerned about using it long term. However, at the time of writing, XSLT is supported in all major browsers, including Chrome and the latest versions of the iPhone and Android browsers. In addition, XSLT may be used both server- and client-side. Server-side implementation works regardless of browser support, and open-source and commercial XSLT engines are available. Finally, JavaScript implementations of XSLT, such as Saxon-CE, could also fill the gap client-side if Google does indeed decide to drop support for XSLT.

Responsive retrofitting is the process of grafting responsive Web design techniques onto an existing website that was not built to be responsive. Although a lot of resources and tutorials on building a responsive website from scratch are out there, retrofitting has curiously received far less attention, despite its enormous value. A lot more old websites exist than new ones, and significant capital and effort have been invested in them.

A long responsive rebuild might not meet the budget or time-to-market requirements for many of these websites. Transformations provide an effective way to make a website responsive without the expense of rebuilding it.

The natural first step would be to retrofit the website purely in CSS. Ben Callahan’s bookmarklet for example, inserts a custom CSS file to make a given website more responsive. However, adding CSS gets you only so far. Eventually, the layout, nesting and order of elements in the original HTML will restrict your design options. John Shirrel describes this as an inherent flaw of CSS:

“You have discovered the weakness of using CSS… CSS does not allow you to transform your document. Elements must remain in the order they appear… There is no real decision-making power in CSS.”

Fundamentally, this process breaks down because CSS and HTML do not purely separate presentation and content. Whenever you’ve found yourself needing to wrap an element in an extra div or span to make the design work, you’ve encountered that breakdown. This is where transformations come in, restoring the abstraction (surprisingly) by enabling us to manipulate the document.

To demonstrate the technique, I’ve created an sample retrofit of Hacker News’ home page, by making the top navigation responsive, using ZURB’s Foundation framework. Naturally, additional design changes beyond these would be required, but it should serve to illustrate the process. Why Hacker News? First, using code from a real website helps to demonstrate the power of the technique. Secondly, the HTML on this website is Goldilocks-sized: not so small that it’s a toy, but not so big as to make our example complicated.

Hacker News' HTML, with Foundation's responsive top bar added by transformations
Hacker News’ HTML, with Foundation’s responsive top bar added by transformations.

The most relevant file in this retrofit is the XSL style sheet transformer.xsl, which you can download directly. In the paragraphs below, we’ll highlight some key points about the code in transformer.xsl. While a full explanation of XSLT is beyond the scope of this article, a quick crash course should make these explanations more understandable:

  • Syntax
    The XSLT programming language is written in a form of XML. For example, an if statement in XSLT would be <xsl:if></xsl:if> element.
  • Parameters
    Parameters to statements like <xsl:if> are specified in the attributes and child nodes of the element.
  • Matching elements
    A common parameter is the match attribute, which selects a collection of nodes. For example, the template specified by the <xsl:template match="body/center"> rule would be applied when the XSLT engine encounters the <center> element that is the child of the <body>.
  • Embedding HTML
    Finally, you can embed bare HTML in a style sheet.

The first thing we need to do is tell the browser to use the rules in transformer.xsl. To do this, we change the document’s Content-Type to text/xml (alternatively, in some browsers, you can simply change the file’s extension from .html to .xml) and add an <?xml-stylesheet> tag like the following to the top of the document:


<?xml-stylesheet type="text/xsl" href="transformer.xsl"?>

When the browser encounters this tag, it will transform the document according the XSLT rules specified in the style sheet before rendering it. This will execute the transformation in the browser (i.e. client-side). But transformations can also be performed server-side. Doing transformations server-side could result in better performance, but we’re doing transformations client-side to make our example accessible to anyone with a browser. For those interested in using this technique client-side, optimization techniques and tools are available for writing performant XSLT.

Another issue is how XSLT handles unspecified elements. By default, XSLT strips out any elements not specified in the transformation rules. This is a design decision that makes sense for XML documents, which typically represent data. However, when transforming a fully formed Web page, you will usually want to change only a relatively small part of the document, and specifying every single element we need to be preserved would be laborious. Thankfully, we can add a construct called the identity transform to the transformer.xsl style sheet:


<!-- XSLT identity transform allows most of the input to pass through to the output. -->
<xsl:template match="@* | node()">
  <xsl:copy>
    <xsl:apply-templates select="@* | node()"/>
  </xsl:copy>
</xsl:template>

The code above might appear cryptic, but in essence it tells the XSLT engine to copy (i.e. <xsl:copy>) every element node and attribute in the input to the output. Adding this to our style sheet will effectively change the default behavior from stripping out elements to passing them through.

The next step is to add Foundation to the document. For example, the following code would add Foundation’s CSS (foundation.css) and its dependencies (normalize.css and Modernizr) to the top of the <head>:


<!-- Install and initialize Foundation -->
<!-- Note the match attribute targeting the document's head element -->
<xsl:template match="head">
  <link rel="stylesheet" href="normalize.css" />
  <link rel="stylesheet" href="foundation.css" />    
  <script src="custom.modernizr.js"></script>
  <xsl:apply-templates select="@* | *"/>
</xsl:template>

Likewise, to add Foundation to the bottom of the <body> element, you would write this:


<xsl:template match="body">
  <xsl:apply-templates select="@* | *"/>
  <script src="zepto.js"></script>
  <script src="foundation.min.js"></script>
  <script>$(document).foundation();</script>
</xsl:template>

The chief difference between the two code samples above are the match attribute values (head versus body, indicating the element being targeted) and the placement of the <xsl:apply-templates> tag (which controls whether our new content will appear before or after the existing content).

Finally, to add the responsive header, we inline new HTML that matches the structure of Foundation’s top bar, and we use the for-each and copy-of commands to populate it with the relevant links from the existing header.


<!-- Add our Foundation top bar -->
   <xsl:template match="body/center">
     <nav class="top-bar">
	
	<!-- Boilerplate removed for clarity -->

        <section class="top-bar-section">
          <ul class="right">

            <!-- Pull the menu links from the old header into our new Foundation header -->
            <xsl:for-each select="//span[@class='pagetop']/a">
              <li><xsl:copy-of select="." /></li>
            </xsl:for-each>

          </ul>
        </section>
     </nav>
     <xsl:apply-templates select="@* | *"/>
   </xsl:template>

There are some caveats to this approach the reader should be aware of. First, in this retrofitting example the transformations were performed by the browser but executing the transformations server-side has a couple of advantages. Server-side transformation reduces the burden on mobile devices, which have less processing, power, and memory capabilities than the server.

The server is also the appropriate place to segment your content to avoid sending unnecessary data over the network and to improve performance. Lastly, you can update the server transformation engine and keep it consistent, instead of dealing with potentially different quirks and levels of XSLT support among browsers. (For example, while XSLT 1.0 is supported in most browsers, XSLT 2.0 is not supported in any, although Saxon-CE is one attempt to add it via JavaScript.)

Second, XSLT’s roots in functional programming make it inaccessible to the average Web developer. It isn’t simply a matter of learning a new syntax. The recursive processing model of XSLT requires a new way of thinking that is unfamiliar to developers of imperative languages, especially developers from a design background who do not have formal training in computer science.

Finally, a larger challenge is that this technique works only for Web pages that are in XHTML (a flavor of HTML that is XML-compliant), because XSLT can transform only XML, not HTML. According to W3Techs, 55% of websites are in XHTML. While this is a majority, it still leaves out a large number of websites. In fact, for this retrofitting example, I worked around this limitation by running Hacker News’ HTML code through an HTML to XHTML converter.

In the next section, we’ll explore how the Tritium transformation language was built to address these issues.

Responsive Delivery

In the example above, we’ve used transformations in the browser to create a responsive experience for an existing website, but conceptually the two approaches overlap. Because responsive Web design is itself about changing presentation across multiple screen sizes, transformations can help in that process as well. Instead of simply pairing different CSS styles to the same fixed HTML as in typical responsive design, we can leverage transformations to change the HTML across devices.

As we explored earlier, the ability to manipulate the HTML (which is missing from CSS alone) not only creates flexibility but actually improves the separation between presentation and content. As a result, maintainability becomes easier because the content is more semantic and less tied to layout. In essence, think of this as moving the breakpoints in responsive design to the transformation layer.

At Moovweb, we’ve leveraged these insights about transformations to implement a technique called responsive delivery, which draws inspiration from responsive Web design, RESS and adaptive design. With responsive delivery, transformations are used to adapt an existing website to different touch points, such as smartphones and tablets.

TThe data flow in this responsive delivery configuration transforms desktop HTML for mobile and tablet end points.
The data flow in this responsive delivery configuration transforms desktop HTML for mobile and tablet end points. (Large view)

Because the transformed website inherits directly from the desktop, responsive delivery provides the unified content and functionality across touch points that you get with responsive Web design, while the majority of the processing is done server-side. And because transformations can be used to manipulate the HTML, we get the benefits that we saw earlier: independent and precise control over the look and feel for each touch point, thanks to an abstraction that separates content from layout.

In some sense, we can think of responsive delivery as part of a larger trend of weaving adaptive design techniques into responsive design and moving more processing to the server, as in RESS.

Also worth noting is that responsive delivery can power desktop websites. In the ideal scenario, the back end would produce semantic but unstyled HTML (affectionately called “wireframe HTML”), and the responsive delivery layer would transform the output for all user touch points.

The data flow in this responsive delivery configuration uses wireframe HTML as the source data for transformations that output to mobile, tablet, app and desktop end points.
The data flow in this responsive delivery configuration uses wireframe HTML as the source data for transformations that output to mobile, tablet, app and desktop end points. (Large view)

Especially in large organizations, this makes for a powerful decoupling of teams. Back-end engineers can focus purely on functionality (i.e. producing the “wireframe HTML”), and front-end designers can focus on the experience (i.e. transforming the wireframe HTML into a styled Web page). Because the channel between both teams is simply HTML, both teams understand it, and passing through implementation details that are inherited by the front end (such as styles and JavaScript) becomes easy.

Consider a simple autocomplete widget. With a responsive delivery approach, the back-end engineer would write the server code to support an autocomplete API and embed the corresponding JavaScript that powers the widget in the wireframe HTML. The designer could then style the widget using CSS and transformations and, if necessary, add JavaScript beyond the stub provided by the back-end engineer. The key point is that HTML is used as a flexible carrier of both content and functionality, as opposed to XML or JSON, which represent only data.

Shown here are the wireframe HTML and the transformed mobile and desktop output used to power Remix Moovweb.
Shown here are the wireframe HTML and the transformed mobile and desktop output used to power Remix Moovweb. (Large view)

One way of applying this approach is to use a software-as-a-service platform to process transformations in the cloud. This is the service that we built at Moovweb to power mobile websites and apps for Macy’s and 1-800-Flowers, among others, using our own open-source transformation language, called Tritium. Tritium was designed by Hampton Catlin, inventor of the CSS preprocessor Sass. He used his experience to create a transformation language that’s accessible to Web designers and developers. Some features that we felt would be important to the language are the following:

  • Familiar syntax
    The syntax should be similar to CSS and jQuery so that it’s more familiar and readable than XSLT’s XML syntax.
  • Imperative style
    We wanted to use an imperative programming style, instead of the function and recursive processing model of XSLT.
  • Input transparency
    The input is passed directly to the output, so there is no need for a construct like the identity transformation (as is the case with XSLT). Stated differently, the identity transformation is an empty document.
  • HTML-compatible
    Tritium was designed to process regular HTML, so it can be used on any website, not just XHTML websites (as with XSLT).

For a simple “Hello world” example, the following Tritium script will select all of the HTML table elements with an ID of foo and change their width attributes to 100%:


# Select all HTML nodes that are table elements with ID of foo.

# The $$() function takes a regular CSS selector
$$("table#foo") {
        # change the width attributes to "100%""
        attribute("width", "100%")
}

The Tritium language has its own official website, and Moovweb has scheduled to open-source the language in 2014. Developers interested in contributing are encouraged to check out the public GitHub repository for the core Tritium parser library.

To compare an example, the analogous Tritium code for adding Foundation to an HTML document, as we did earlier with XSLT, would be this:


  # Install Foundation in the document
  $$("head") {
    inject_top('<link rel="stylesheet" href="normalize.css" />
                <link rel="stylesheet" href="foundation.css" />
                <script src="custom.modernizr.js"></script>')
  }
  $$("body") {
    inject_bottom('<script src="zepto.js"></script>
                   <script src="foundation.min.js"></script>
                   <script>$(document).foundation();</script>')
  }   

Recall that, in the case of XSLT, the relative placement of new content in the document relies on the judicious placement of the <xsl:apply-templates> tag. However, in the Tritium code just above, the straightforwardly named functions inject_top and inject_bottom are used to insert the boilerplate HTML that installs Foundation.

In practice, the Moovweb SDK offers a Foundation scaffold that installs the Foundation boilerplate for you. The scaffold also offers convenience functions to use other components in the Foundation framework. For example, we would use the following functions to create the responsive top bar for Hacker News:

foundation.tbar() {
  foundation.tbar_title("Hacker News", "", "menu-icon")
  foundation.tbar_section("right") {
    move_here("//span[@class='pagetop']//a"){
      wrap("li")
    }
  }
}

As in the XSLT example, the XPath selector "//span[@class='pagetop']//a" is used to select the links for the menu, but the foundation.tbar*() convenience functions make it easier and spare us from having to know the details of the top bar’s HTML format.

As a final example, a full project implementing the responsive retrofit of Hacker News with Tritium is available on GitHub for you to run and play with using the Moovweb SDK. Unlike the XSLT version, which uses HTML downloaded from Hacker News (due to the need to run it through an XHTML converter), you can run this project with the live version of the website.

Transform Your Thinking

Closures once sat obscurely in functional languages until languages such as JavaScript and Ruby brought them into the mainstream. Likewise, transformations have been buried in frameworks that are alien to the average developer, and the popularity of the transformation approach has been married to the fate of XSLT.

This is unfortunate because transformations are more about a new way of thinking than about any particular technology. They enable a powerful separation of content and logic from presentation, and the usefulness of this separation is important in a number of ways. As we move into the post-PC era, transformations provide one part of the answer to serving websites across a wide array of form factors.

Furthermore, the separation enabled by transformations enhances productivity and accelerates the iteration cycle for visual design teams. Meanwhile, for developers who rearrange DOM objects via JavaScript or jQuery, transformations are a new lens on their current workflow, and they open up new doors to optimizing tasks, such as server-side transformation.

In an industry that has no shortage of new ideas, sometimes the most useful thing is to connect new concepts with old ones to make them more digestible. That is what this article has tried to do with transformations. Hopefully, we’ve demonstrated the power of thinking in transformations, showing its relevance to anyone who works with HTML.

(al, ea)


© Ishan Anand for Smashing Magazine, 2014.

Tags: Mobile

February 03 2014

13:20

One Solution To Responsive Images


  

Responsive images have been, and are, one of the hardest problems in responsive Web design right now. Until browser vendors give us a native solution, we have to think on the fly and come up with our own solutions. “Retina” images are especially a challenge because if you have sized your layout with ems or percentages (as you should!), then you cannot be sure of the exact pixel dimensions of each image being displayed.

In this article, we’ll look at one solution to the problem that we implemented on our portfolio website at Etch, where you can see an early working version in the wild.

Requirements

We used a content-first approach on Etch. We knew we wanted to use a lot of images to quickly convey the atmosphere of the company. These would be accompanied by small snippets, or “soundbites,” of text.

The next decision was on image sizes and aspect ratios. To get maximum control over the design, we knew we needed maximum control over the images. We decided to use Instagram as the base for our imagery for the following reasons:

  • The aspect ratio is fixed.
  • Most employees here already use it.
  • Those lovely filters.

Instagram allows for a maximum image size of 600 pixels, so we now had our first set of content constraints to work with: images with a 1:1 aspect ratio, and a maximum image size of 600 × 600. Having constraints on the content side made the design process easier because they limited our options, thus forcing decisions.

When the content was completed, we began looking at the design. Again, to keep maximum control, we decided on an adaptive design style with fixed column sizes. We used grid block elements that match our maximum image size. Each grid block would either be 600 × 600 or 300 × 300, which also conveniently fit our rough plan of a minimum width of 320 pixels for the viewport on the website.

During the rest of the design process, we noticed that we needed two other image sizes: thumbnails at 100 × 100, and hero images that stretch the full width of the content (300, 600, 900, 1200, 1600, 1800). All images would also need to be “Retina” ready — or, to put it another way, compatible with displays with high pixel densities. This gave us the final set of requirements for a responsive images solution for the website:

  • Potential image widths (in pixels) of 100, 300, 600, 900, 1200, 1600, 1800
  • Retina ready
  • Must be crisp with minimal resizing (some people notice a drop in quality with even downsized images)

Having to resize that many images manually, even using a Photoshop script, seemed like too much work. Anything like that should be automated, so that you can focus on fun and interesting coding instead. Automation also removes the chance for human error, like forgetting to do it. The ideal solution would be for us to add an image file once and forget about it.

Common Solutions

Before going over our solution, let’s look at some common solutions currently being used. To keep up with currently popular methods and the work that the Web community is doing to find a solution to responsive images, head over to the W3C Responsive Images Community Group.

Picture Element

First up, the picture element. While this doesn’t currently have native support and browser vendors are still deciding on picture versus srcset versus whatever else is up for discussion, we can use it with a polyfill.


<picture alt="description">
  <source src="small.jpg">
  <source src="medium.jpg" media="(min-width: 40em)">
  <source src="large.jpg" media="(min-width: 80em)">
</picture>

The picture element is great if you want to serve images with a different shape, focal point or other feature beyond just resizing. However, you’ll have to presize all of the different images to be ready to go straight in the HTML. This solution also couples HTML with media queries, and we know that coupling CSS to HTML is bad for maintenance. This solution also doesn’t cover high-definition displays

For this project, the picture element required too much configuration and manual creation and storage of the different image sizes and their file paths.

srcset

Another popular solution, srcset, has recently been made available natively in some WebKit-based browsers. At the time of creating our plugin, this wasn’t available, and it looks like we’ll be waiting a while longer until cross-browser compatibility is good enough to use it without a JavaScript fallback. At the time of writing, srcset is usable only in the Chrome and Safari nightly builds.


<img src="fallback.jpg" srcset="small.jpg 640w 1x, small-hd.jpg 640w 2x, large.jpg 1x, large-hd.jpg 2x" alt="…">

The snippet above shows srcset in use. Again, we see what essentially amounts to media queries embedded in HTML, which really bugs me. We’d also need to create different image sizes before runtime, which means either setting up a script or manually doing it, a tiresome job.

Server-Side Sniffing

If you’d rather not use JavaScript to decide which image to serve, you could try sniffing out the user agent server-side and automatically send an appropriately sized image. As a blanket rule, we almost always say don’t rely on server-side sniffing. It’s very unreliable, and many browsers contain inaccurate UA strings. On top of that, the sheer number of new devices and screen sizes coming out every month will lead you to maintenance hell.

Other Solutions in the Wild

We chose to make our own plugin because including layout code in the HTML seemed undesirable and having to create different image sizes beforehand was not enticing.

If you’d like to explore other common solutions to decide which is best for your project, several great articles and examples are available on the Web, including one on this very website.

  • Choosing a Responsive Image Solution,” Sherri Alexander, Smashing Magazine
    Alexander looks at the high-level requirements for responsive images, and then dissects the variety of solutions currently available in the wild.
  • Which Responsive Image Solution Should You Use,” Chris Coyier, CSS-Tricks
    Coyer takes us through imaging requirements while suggesting appropriate solutions.
  • Adaptive Images
    A solution very similar to Etch’s in its implementation. It uses a PHP script to size and serve the appropriate images. Unfortunately, this wasn’t available when we were coding the website.
  • Picturefill
    This is a JavaScript replacement for markup in the style of the picture element.
  • Responsive Images Using Cookies,” Keith Clark
    Clark uses a cookie to store the screen’s size, and then images are requested via a PHP script. Again, it’s similar to our solution but wasn’t available at the time.

Onto our solution.

Our Solution

With both picture and srcset HTML syntaxes seeming like too much effort in the wrong places, we looked for a simpler solution. We wanted to be able to add a single image path and let the CSS, JavaScript and PHP deal with serving the correct image — instead of the HTML, which should simply have the correct information in place.

At the time of developing the website, no obvious solution matched our requirements. Most centered on emulating picture or srcset, which we had already determined weren’t right for our needs.

The Etch website is very image-heavy, which would make manually resizing each image a lengthy process and prone to human error. Even running an automated Photoshop script was deemed to require too much maintenance.

Our solution was to find the display width of the image with JavaScript at page-loading time, and then pass the src and width to a PHP script, which would resize and cache the images on the fly before inserting them back into the DOM.

We’ll look at an abstracted example of the code, written in HTML, JavaScript, PHP and LESS. You can find a working demo on my website. If you’d like to grab the files for the demo, they can be found on GitHub.

Markup

The markup for the demo can be found in the index.html file on GitHub.

We wrap the highest-resolution version of an image in noscript tags, for browsers with JavaScript turned off. The reason is that, if we think of performance as a feature and JavaScript as an enhancement, then non-JavaScript users would still receive the content, just not an optimized experience of that content. These noscript elements are then wrapped in a div element, with the image’s src and alt properties as data attributes. This provides the information that the JavaScript needs to send to the server.


<div data-src="img/screen.JPG" data-alt="crispy" class="img-wrap js-crispy">
    <noscript><img src="img/screen.JPG" alt="Crispy"></noscript>
</div>

The background of the image wrapper is set as a loading GIF, to show that the images are still loading and not just broken.

An alternative (which we used in one of our side projects, PhoSho) is to use the lowest-resolution size of the image that you will be displaying (if known), instead of the loading GIF. This takes slightly more bandwidth because more than one image is being loaded, but it has an appearance similar to that of progressive JPEGs as the page is loading. As always, see what your requirements dictate.

Dinner 2013 - Showcase by gavmcksnow
(Large preview)

JavaScript

The JavaScript communicates for us between the HTML and the server. It fetches an array of images from the DOM, with their corresponding widths, and retrieves the appropriate cached image file from the server.

Our original plugin sent one request to the server per image, but this caused a lot of extra requests. By bundling our images together as an array, we cut down the requests and kept the server happy.

You can find the JavaScript plugin in /js/resize.js in the GitHub repository.

First, we set an array of breakpoints in the plugin that are the same as the breakpoints in the CSS where the image sizes change. We used em values for the breakpoints because they are based on the display font size. This is a good practice because visually impaired users might change their display’s default font size. This also makes it easier to match our CSS breakpoints with the JavaScript ones. If you prefer, the plugin works just fine with pixel-based breakpoints.


breakpoints: [
    "32em"
    "48em"
    "62em"
    "76em"
]

As we pass each of these breakpoints, we need to check the images to make sure they are the correct size. At page-loading time, we first set the current breakpoint being displayed to the user using the JavaScript matchMedia function. If you need to support old browsers (Internet Explorer 7, 8 and 9), you might require the matchMedia polyfill by Paul Irish.


getCurrentBreakpoint: function() {
      var bp, breakpoint, _fn, _i, _len, _ref,
        _this = this;

      bp = this.breakpoints[0];
      
      _ref = this.breakpoints;
      
      _fn = function(breakpoint) {
        // Check if the breakpoint passes
        if (window.matchMedia && window.matchMedia("all and (min-width: " + breakpoint + ")").matches) {
          return bp = breakpoint;
        }
      };
      
      for (_i = 0, _len = _ref.length; _i < _len; _i++) {
        breakpoint = _ref[_i];
        _fn(breakpoint);
      }
      
      return bp;
    }

After setting the current breakpoint, we gather the images to be resized from the DOM by looping through them and adding them to the plugin’s images array.


gather: function() {
      var el, els, _i, _len;

      els = $(this.els);
      
      this.images = [];
      
      for (_i = 0, _len = els.length; _i < _len; _i++) {
        el = els[_i];
        this.add(el);
      }
      
      this.grabFromServer();
    }

The PHP script on the server needs the image’s src and current width in order to resize it correctly, so we created some serialized POST data to send to the server. We use jQuery’s param method to quickly convert the image into a usable query string.


buildQuery: function() {
      var image = { image: this.images }
      return $.param(image);
    }

The images are then sent via an AJAX request to the server to be resized. Note the single request, to minimize server load.


grabFromServer: function() {
      var data,
        _this = this;

      data = this.buildQuery();
      
      $.get("resize.php", data, function(data) {
          var image, _i, _len;
          for (_i = 0, _len = data.length; _i < _len; _i++) {
            image = data[_i];
            _this.loadImage(image);
          }
        }
      );
    }

Once we have retrieved the images from the server, we can add them to the DOM or replace the image already in place if it has changed. If it’s the same image, then nothing happens and the image won’t need to be redownloaded because it’s already in the browser’s cache.


loadImage: function(image) {
      var el, img,
        _this = this;

      el = $("[data-src='" + image.og_src + "']");
      
      img = $("");
      
      img.attr("src", image.src).attr("alt", el.attr("data-alt"));
      
      if (el.children("img").length) {
        el.children("img").attr("src", image.src);
      } else {
        img.load(function() {
          el.append(img);
          el.addClass('img-loaded');
        });
      }
    }

PHP

With the JavaScript simply requesting an array of images at different sizes, the PHP is where the bulk of the action happens.

We use two scripts. One is a resize class (found in /php/lib/resize-class.php in the demo), which creates cached versions of the image at the sizes we need. The other script sits in the Web root, calculates the most appropriate size to display, and acts as an interface between the JavaScript and the resizer.

Starting with the sizing and interface script, we first set an array of pixel sizes of the images that we expect to display, as well as the path to the cached images folder. The image sizes are in pixels because the server doesn’t know anything about the user’s current text-zoom level, only what the physical image sizes being served are.


$sizes = array(
    '100',
    '300',
    '600',
    '1200',
    '1500',
);

$cache = 'img/cache/';

Next, we create a small function that returns the image size closest to the current display size.


function closest($search, $arr) {
    $closest = null;
    foreach($arr as $item) {
        // distance from image width -> current closest entry is greater than distance from  
        if ($closest == null || abs($search - $closest) > abs($item - $search)) {
            $closest = $item;
        }
    }
    $closest = ($closest == null) ? $closest = $search : $closest;
    return $closest;
}

Finally, we can loop through the image paths posted to the script and pass them to the resize class to get the path to the cached image file (and create that file, if necessary).


$crispy = new resize($image,$width,$cache);
$newSrc = $crispy->resizeImage();

We return the original image path in order to find the image again in the DOM and the path to the correctly sized cached image file. All of the image paths are sent back as an array so that we can loop through them and add them to the HTML.


$images[] =  array('og_src' => $src, 'src' => '/'.$newSrc);

In the resize class, we initially need to gather some information about the image for the resizing process. We use Exif to determine the type of image because the file could possibly have an incorrect extension or no extension at all.


function __construct($fileName, $width, $cache) {

    $this->src = $fileName;
    $this->newWidth = $width;
    $this->cache = $cache;
    $this->path = $this->setPath($width);

    $this->imageType = exif_imagetype($fileName);

    switch($this->imageType)
    {
        case IMAGETYPE_JPEG:
            $this->path .= '.jpg';
            break;

        case IMAGETYPE_GIF:
            $this->path .= '.gif';
            break;

        case IMAGETYPE_PNG:
            $this->path .= '.png';
            break;

        default:
            // *** Not recognized
            break;
    }
}

The $this->path property above, containing the cached image path, is set using a combination of the display width, a hash of the file’s last modified time and src, and the original file name.

Upon calling the resizeImage method, we check to see whether the path set in $this->path already exists and, if so, we just return the cached file path.

If the file does not exist, then we open the image with GD to be resized.

Once it’s ready for use, we calculate the width-to-height ratio of the original image and use that to give us the height of the cached image after having been resized to the required width.


if ($this->image) {
    $this->width  = imagesx($this->image);
    $this->height = imagesy($this->image);
}

$ratio = $this->height/$this->width;
$newHeight = $this->newWidth*$ratio;

Then, with GD, we resize the original image to the new dimensions and return the path of the cached image file to the interface script.


$this->imageResized = imagecreatetruecolor($this->newWidth, $newHeight);
imagecopyresampled($this->imageResized, $this->image, 0, 0, 0, 0, $this->newWidth, $newHeight, $this->width, $this->height);

$this->saveImage($this->newWidth);

return $this->path;

What Have We Achieved?

This plugin enables us to have one single batch of images for the website. We don’t have to think about how to resize images because the process is automated. This makes maintenance and updates much easier, and it removes a layer of thinking that is better devoted to more important tasks. Plug it in once and forget about it.

TL;DR? Let’s summarize the functionality once more for good measure.

In our markup, we provide an image wrapper that contains a <noscript> fallback. This wrapper has a data attribute of our original high-resolution image for a reference. We use JavaScript to send an AJAX request to a PHP file on the server, asking for the correctly sized version of this image. The PHP file either resizes the image and delivers the path of the correctly sized image or just returns the path if the image has already been created. Once the AJAX request has been completed, we append the new image to the DOM, or we just update the src if one has already been added. If the user resizes their browser, then we check again to see whether a better image size should be used.

Pros And Cons

All responsive image solutions have their pros and cons, and you should investigate several before choosing one for your project. Ours happens to work for our very specific set of requirements, and it wouldn’t be our default solution. As far as we can tell, there is no default solution at the moment, so we’d recommend trying out as many as possible.

How does this solution weigh up?

Pros

  • Fast initial page download due to lower image weight
  • Easy to use once set up
  • Low maintenance
  • Fast once cached files have been created
  • Serves image at correct pixel size (within tolerance)
  • Serves new image when display size changes (within tolerance)

Cons

  • Unable to choose image focus area
  • Requires PHP and JavaScript for full functionality
  • Can’t cover all possible image sizes if fluid images are used
  • Might not be compatible with some content management systems
  • Resizing all images with one request means that, with an empty cache, you have to wait for all to be resized, rather than just one image
  • The PHP script is tied to breakpoints, so it can’t be dropped in without tweaking

Responsive image solutions have come a long way in recent months, and if we had to do this again, we’d probably look at something like the adaptive images solution because it removes even more non-semantic HTML from the page by modifying .htaccess.

Wrapping Up

Until we have a native solution for responsive images, there will be no “right” way. Always investigate several options before settling on one for your project. The example here works well for websites with a few common display sizes for images across breakpoints, but it is by no means the definitive solution. Until then, why not have a go at creating your own solution, or play around with this one on GitHub?

(al, il)

SmashingMag front page image credits: PhoSho's front page showcase.


© Gavyn McKenzie for Smashing Magazine, 2014.

Tags: Mobile

January 16 2014

10:55

How To Use Data And Research To Build A Better Mobile Web


  

The value you bring to any project as a designer or user experience professional isn’t only your ability to execute a series of tasks. Good clients will recognize and appreciate the guidance, recommendations and opinions that come with your experience. It’s up to you to be forthcoming and to position yourself as a valuable thinker from the start.

It’s also up to you to base your recommendations on more than subjective opinion. This is especially true in the rapidly evolving world of mobile, where clients typically require as much education as they do execution.

Project-specific research provides the dual benefit of enhancing your general knowledge while improving your ability to make (and sell) decisions as the project progresses. In my experience, gaining that critical buy-in is much easier when recommendations are based on research conducted for the project.

In this post, I’ll refer to specific mobile research projects I’ve conducted and build a case for audience surveys, participatory discovery sessions and review of analytics data.

Practicality Vs. Personality

A paper from Carleton University (PDF) suggests that consumers simply have jobs to do and seek to employ the best product or service to do it. Applying this thinking to how users interact with mobile websites places more importance on “What do I need to find/do” than on anything related to their demographic. Mobile Web users have jobs to do that aren’t driven by age, gender, education, income, etc. This “practicality” is key to how I approach research.

finding-the-right-job-opt
“The jobs that customers are trying to get done cannot be deciphered from purchased databases, but rather from watching, participating, writing and thinking.” (Source of quote and image: Finding the Right Job For Your Product)

Fitting Research And Analysis Into Your Process

Mobile research is a considerable value-add that should be baked into your services to the client from the start. It enables you to make better decisions and to inform your client of what their audience expects from a mobile experience.

I recently finished guiding Loyalist College through a research phase that included a mobile component. The project is currently in production and is slated to launch early this year.

In this case, I faced unique challenges when selling the value of the proposed research to my client. Loyalist had already conducted research, but the research was more focused on uncovering general best practices and on clarifying the psychographics and demographics of its customers.

I suggested adding a more practical, goal-oriented research component (which was ultimately accepted) as a way to help us build a better product. Most of your clients probably understand well who their customers are but lack a solid grasp of what those customers want from a mobile experience.

Audience Research

The best way to determine what users expect to find and do in the mobile channel is to ask.

There are a few ways to approach this.

Participatory Discovery Sessions

The Loyalist College project included two sessions in which members of the target audience were asked, quite simply, what they expected to find and do, and how these tasks might best be performed on a mobile device. I also asked how many of the participants use a smartphone as their primary computing device. The data collection method in a session like this is as follows:

  1. Ask participants to contribute approximately 10 things they expect to find and/or do when using their mobile device in the context of the products or services provided by your client. Each item should be written on a sticky note.
  2. Collect the sticky notes and arrange them visibly, like on a wall or table.
  3. Ask participants to review all suggestions and place a star or mark on the three responses that they think are most important.
  4. Discuss the starred responses with the group to seek additional information.

Surveys

I’m working with 3M Canada on a consumer-facing mobile project that consists of multiple phases of research. In partnership with a research firm, we’re using surveys to learn more about how a very specific, targeted set of customers use their smartphones and what the probability is that they’d be interested in using the application we intend to build. We’ll use the results of this survey to define the features that will ultimately make it easier for users to do their job.

Building and distributing surveys are easy using a tool such as Fluid Surveys or Survey Monkey. If the budget is available, my advice is to seek the assistance of a research firm that can help you accurately craft the survey and find the most relevant set of participants. For a project with a tighter budget, you could certainly create, distribute and analyze a survey using a tool such as Fluid Surveys or Survey Monkey.

In my experience, once you’ve defined the purpose of the research initiative, the following are important to consider:

  • Keep the questions focused and simple. Every question should tie back to the goal of the research and be focused on the expectations and habits of mobile users as they relate to the project. Keeping the questions simple lowers or eliminates any barriers to respond.
  • Avoid open-ended questions. Whenever possible, pose questions that require respondents to answer yes or no, via multiple choice, or by rating on a scale. Data of this nature is much easier to analyze and allows you to build recommendations based on objective responses, rather than have to attempt to further interpret subjective feedback.
  • Use incentives. A survey that I deployed for a project for the Golf Association of Ontario received over 5,000 responses. Our client was kind enough to offer an incentive of entering participants into a draw.
  • Define your audience well. Soliciting responses from a general audience can be difficult, and the results are not always accurate. My suggestion is to work with your client to identify an audience (staff, customers, etc.) and to send the survey directly via an email list or other targeted communication channel.

Analytics Review

Slicing and reviewing analytics data from an existing Web presence can be an effective way to determine what mobile users are doing with the content and features available. For the Loyalist College project, I was interested in a simple report of the most popular content accessed from mobile devices.

Keep in mind, especially when using Google Analytics, that mobile phones and tablets are lumped together in the default “Mobile” group. Because mobile and tablet use cases are typically very different, I like to segment phones explicitly as part of the analysis. Here’s how:

  1. Create a new custom advanced segment by selecting the arrow on the “Audience Overview” landing page.
    GAmobile_two
  2. Name the new segment (something like “Phones”).
    GAmobile_three
  3. In the “Technology” section, choose to include mobile traffic and exclude tablet traffic.
    GAmobile_one

You can now analyze the data and create reports for phones only.

I recently used this approach in a large project for the Middlesex-London Health Unit. Part of our plan was to consider mobile as the cornerstone of a large Web redesign project and to treat the mobile experience as a priority from the start, rather than an afterthought. With access to the client’s analytics account, I was able to slice the data and reveal the following:

  • On average, the website received 250 mobile visits per day.
  • The most popular content for mobile users was a health-related game, information on immunization clinics and career opportunities.

Armed with information such as this, I was in a much better position not only to recommend how the content should be prioritized for mobile users but also to demonstrate the general importance of including mobile as a foundational element of the website redesign project.

I took a similar approach with the Loyalist project. In addition to identifying the number of mobile visitors and the content priorities of those users, I was able to communicate to my client that visits from mobile devices had doubled from 2012 to 2013.

Identifying Themes

Researching the expectations of mobile users has little value until the research is put to use. I like to include a step between research and execution in which I summarize the findings via a series of “themes.” Themes are a familiar concept that simplify the process of grouping findings together, while encompassing a number of tactics to be implemented in the project.

The research that I conducted for the Loyalist project revealed the importance of a centralized events calendar, especially on mobile. Using this data, I created a theme called “Events Matter” and outlined the results of the research and what the new mobile experience would do to capitalize on this theme. It’s important to be specific here and let your client know exactly what you think should be done to capitalize on the results of the research.

The themes can be used to create planning materials, like a creative brief, flowcharts or a scope of work document. Recently, I’ve found myself forgoing these components in favor of building visual concepts directly from the themes and spending more time in rapid iteration, as opposed to drawn-out planning.

Closing Thoughts

Mobile is causing the way we interact with the Web to change at an unforgiving pace, a shift that requires guidance from well-informed professionals like you. Your responsibility is not only to produce great work but to educate and guide your clients through something that most of them can’t keep up with.

Selling research as part of your process will be more effective if you can show clients specific examples of how you’ve used research in the past. My advice is to bake research into your next project as a value-add for the client. You can keep it simple by taking the following steps:

  1. Conduct an analytics review, and use the data to support at least one key decision in the project.
  2. Create a brief survey, and ask the client to provide access to it from their existing website. Again, use the results of the survey to support one key decision.

Continually build your research practice from project to project by going deeper with your analytics reviews, adding more survey questions, integrating discovery sessions and building complete themes that guide the project in a meaningful way.

Research is a fantastic, accessible way to increase your perceived value while helping to shape the future of the Web and ensuring that the software and websites we build keep pace with users’ ever-advancing expectations. Businesses value proof and, in my experience, appreciate a conversation that starts with explaining a body of proof before presenting a set of specific recommendations.

(al, ea, il)


© Jonathan Kochis for Smashing Magazine, 2014.

Tags: Mobile

January 10 2014

13:32

Four Ways To Build A Mobile Application, Part 2: Native Android


  

This article is the second in a series of four articles covering four ways to develop mobile applications. The last article covered how to accomplish this using native iOS development tools. In this article, we’ll look at how to build the same sort of application using native Android tools.

We’ve been building a simple tip calculator. As with the iOS application, this one contains two screens: a main view and a settings view. The settings view persists the default tip percentage to local storage using Android’s SDK support. (The source code for each app is available on GitHub.)

Here’s the UI we’re following for this project:

android-example

Native Android Development: The Tools

For the Android platform, the Java language is used, along with the Eclipse integrated development environment (IDE). Google provides the Android Developer Tools (ADT) plugin for the standard Eclipse IDE to support things like graphical layout definition and debugging. As of the time of writing, Google has also released a new IDE, called Android Studio, in early-access preview. This product is based on JetBrains’ IntelliJ IDE and will eventually replace Eclipse as the official development tool for Android applications. Because Android Studio is still in prerelease form, we’ll use Eclipse to develop this application. Eclipse is very stable, and many Android training resources using Eclipse abound.

The Android development environment is supported on Windows, Mac and Linux. There is no charge for the development tools, and applications may be freely deployed to devices. To submit apps to the Google Play marketplace, developers must sign up for publishing and pay a one-time fee of $25. Submitting apps to Google Play is notably less involved than for iOS. When your application is submitted to Google, it is scanned for malware and common threats. Generally, the application becomes available to the general public within a few hours.

An overview of the toolset can be found on the Android Developers website. Successful Android development entails use of the following tools:

  • Eclipse IDE
  • Android SDK
  • Android ADT
  • Android system images for the Android emulator
  • an Android device (technically not required but highly recommended)

Grab an Installation Bundle to Ease Your Pain

In the past, you had to download, install and configure the tools individually. This took a fair amount of time and often led to confusion for new Android developers. To make the process easier, Google now offers “bundles,” which simplify the installation process. Bundles are offered for each operating system and are available on the page where you download the developer SDK.

Installing Eclipse, the SDK, the emulator and so on is as easy as unpacking a ZIP file into a directory. If you have an existing installation of Eclipse and would prefer to use that instead, there are instructions for adding Android support to it.

Loading An Existing Application Into Eclipse

Once the Android development tools have been installed, you may wish to import an existing project, such as the source code for our sample app. In Eclipse, go to File → Import in the menu, which will display the following dialog:

android-import-step1

Once you’ve chosen the option to import an existing Android project, click the “Next” button, whereupon you will be able to specify the directory where the code that you wish to work with in Eclipse is located.

android-import-step2

Once you’ve selected a directory via the “Browse” button, Eclipse will automatically find any Android project in that directory and show it in the list of projects to import. Simply click the “Finish” button, and the project will appear in your list of projects along the left side of the IDE.

Steps To Building An Application

There are several steps to building an Android application. We’ll cover each in detail later in this article. The steps include the following:

  1. Define the user interface. The UI for an application is generally defined as a series of layout files. These are XML-based files that describe the controls on a screen and the relationship of their layouts relative to one another.
  2. Add image assets, language translations and other resources. Android refers to non-code assets of a project as resources. These are placed in the project in a directory structure defined by the Android SDK. At runtime, Android dynamically loads content from this directory structure. We’ll see later on how different assets and layouts can be loaded to support the wide variety of Android device configurations available in the market today.
  3. Write Java code to respond to various events that occur from the controls on a given screen and from changes in the lifecycle of an application. Java code is also responsible for loading the layout and menu files associated with each screen. And it’s used to control the flow from one screen to the next.
  4. Export the completed Android application as a file that can be uploaded to Google Play or shared with others directly.

The Eclipse ADT For Android Development

The Eclipse IDE provides the standard source-code editing tools, along with a source-level debugger that allows applications to be debugged on both the simulator and a physical device. In contrast to the storyboards used with iOS, a layout editor is used to define screen layouts:

ADT layout editor
(Large view)

The layout editor works on a single screen at a time and is not able to define segues between screens, as in the iOS storyboard editor. However, the Android layout editor does have a very flexible layout system that supports the wide range of screen sizes and resolutions of Android devices.

Unlike the storyboard editor in iOS, you can edit the layouts in either visual mode, as shown above, or an XML-based editor. Simply use the tabs at the bottom of the layout editor to switch between the two views. As the layouts become more complex, being able to edit the layout files directly comes in handy. A snippet of layout XML looks like this:


<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
                xmlns:tools="http://schemas.android.com/tools"
                android:layout_width="match_parent"
                android:layout_height="match_parent"
                tools:context=".MainActivity" >

    <EditText
            android:id="@+id/billAmtEditText"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_alignParentTop="true"
            android:layout_centerHorizontal="true"
            android:layout_marginTop="20dp"
            android:ems="10"
            android:gravity="right|center_vertical"
            android:hint="@string/billAmount"
            android:inputType="number|numberSigned|numberDecimal" >
        <requestFocus />
    </EditText>

    <Button
            android:id="@+id/calcTipButton"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_below="@+id/billAmtEditText"
            android:layout_centerHorizontal="true"
            android:layout_marginTop="19dp"
            android:text="@string/calculateTip" />

    <TextView
            android:id="@+id/TextView01"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_alignLeft="@+id/billAmtEditText"
            android:layout_below="@ID/calcTipButton"
            android:layout_marginTop="18dp"
            android:text="@string/tipPercentage"
            android:textAppearance="?android:attr/textAppearanceMedium"/>
</RelativeLayout>

Android Resources: Support For Various Device Capabilities

The Android SDK was designed from the outset to support a wide variety of device capabilities. Much has been written about the “fragmentation” of devices on the Android platform. However, from the beginning, the Android SDK was designed to support this variety of device attributes, including screen size, pixel density and Android API versions.

Screen layouts, image assets (called drawables), styles and other configuration files are all incorporated in a series of subdirectories underneath a master “resource” directory. The Android SDK documentation illustrates how multiple versions of the same file can be placed in uniquely named directories within this resource structure, so that the proper one is loaded depending on the capabilities and orientation of the device at runtime.

Consider this directory structure:

android-file-structure

The structure above shows uniquely named directories for drawable assets, with suffixes of -hdpi, -ldpi, -mdpi and so on. This permits the developer to supply different image content to correspond with the DPI resolution of a given screen. Similar capabilities extend to things like the layout files, which may supply unique content according to screen size, landscape or portrait orientation, etc. In the example above, we see unique folders for values-v11 and values-v14. This allows for two different styles.xml files to be used for version 11 (Android 3.x) and version 14 (Android 4.x) of the Android operating system.

Note, also, the regular values directory. Android versions prior to version 11 would obtain their styles.xml from this directory. This fallback mechanism is in place for all resources. The SDK will attempt to find the resource in the particular directory and then fall back to a more “generic” resource if it’s not available. All of this occurs without the developer having to write any special code. Simply drop the resources into the proper directories, and the Android runtime will take care of the rest. Each of your resources may be accessed from both the XML and Java files in your application.

The resource system is quite powerful and supports many types of resources. In addition to the drawable, layout, menu and styles for your application, it can also be used to hold application constants for arrays or dimensional values in your application. In this manner, you can load different constant values for various device configurations simply by placing the files in the proper directory structure. You can also use this system to support localization. This is accomplished through the strings.xml file. Here’s an example of the strings.xml file associated with our application:


<?xml version="1.0" encoding="utf-8"?>
<resources>
    <string name="app_name">FasTip</string>
    <string name="hello_world">Hello World!</string>
    <string name="menu_settings">Settings</string>
    <string name="billAmount">Bill Amount</string>
    <string name="calculateTip">Calculate Tip</string>
    <string name="tipAmount">Tip Amount</string>
    <string name="totalAmount">Total Amount</string>
    <string name="title_activity_settings">Settings</string>
    <string name="saveSettings">Save Settings</string>
    <string name="tipPercentage">Tip Percentage</string>
    <string name="percentSymbol">%</string>
</resources>

The base string file is placed in the res/values directory of the application. If you wanted to offer a Spanish version of the same file, you would place it in the res/values-es directory.

Resource IDs

In the process of defining the layouts, you should associate an ID value with each control (or UI widget) that you wish to reference from code. This can be done by specifying an ID value in the properties inspector in the layout editor:

android-eclipse-id-property-500
(Large view)

Layout Managers

Android layout files may use a number of layout managers, or ViewGroups, to arrange the controls on the screen. This is Android’s approach to laying out views, much like the constraints system we saw in iOS. Here are some of the more common ViewGroups used in Android apps:

  • LinearLayout
    This is used to lay out a series of controls in either horizontal or vertical orientation.
  • RelativeLayout
    This is used to position controls relative to one another or to the bounds of their parents’ layout. This is a flexible layout system and is often used as an alternative to nesting linear layouts.
  • ListView
    This is a view group that presents a series of vertically scrolling items, much like a UITableView for those familiar with iOS. In Android, you write a ListAdapter to provide a view for each row of data in a data source.
  • GridView
    This is similar to a list view, but it provides items in a two-dimensional, scrollable grid. Just like the list view, it also uses an adapter to provide view contents for each cell in the grid.

In our sample application, two layout files have been created: activity_main.xml for the main screen and activity_settings.xml for the settings screen. So far, we’ve just defined the appearance of things. Unlike the iOS storyboard editor, Android has no “assistant” tool to directly link the controls in the visual layout editor to the code. We’ll need to write some code to connect these components together and build the application.

Android: Activities And Intents

In iOS, we loaded the logic specific to a given screen into a ViewController. In Android, these separate screens are treated as separate “activities.” Just like in an iOS UIViewController, there is a defined life cycle for an activity that governs when the activity starts, pauses, resumes and stops.

activity-lifecycle

The diagram above comes straight from the Android Developers documentation and shows the lifecycle of an activity. It’s up to the developer to place code in the various methods of the activity to respond to the various states of the lifecycle.


@Override
protected void onCreate(Bundle savedInstanceState) {

    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    tipPctTextView = (TextView)this.findViewById(R.id.tipPctTextView);
    tipAmountTextView = (TextView)this.findViewById(R.id.tipAmtTextView);
    totalAmountTextView = (TextView)this.findViewById(R.id.totalAmtTextView);
    calcTipAmountButton = (Button)this.findViewById(R.id.calcTipButton);
    billAmountTextView = (EditText)this.findViewById(R.id.billAmtEditText);

    calcTipAmountButton.setOnClickListener(new OnClickListener() {
        @Override
        public void onClick(View v) {
            calculateTip();
        }
    });
}

The code above is executed early on in the activity’s lifecycle. In this case, we are loading the layout file that was defined earlier, via this statement:


setContentView(R.layout.activity_main);

Upon successfully loading the layout, we obtain references to the various controls that make up the layout. Notice that we are referencing the layouts and the IDs via the R. notation. These are the resource IDs that we defined earlier in the layout manager. Additionally, we attach a click listener to define what code should run when the user taps on the “Calculate Tip” button.

In the iOS example, we used NSUserDefaults to save and restore the user’s preference for a tip percentage. In Android, the equivalent is sharedPreferences. This can be seen in the code snippet below, where we restore the persisted value for tipPercentage in the local variable tipPctString. Notice that the first time the user runs this app, a default value of 15.0 is used.


private void loadTipPercentage() {
    SharedPreferences preferences =
                this.getSharedPreferences("AppPreferences", MODE_PRIVATE);
    String tipPctString = preferences.getString("tipPercentage", "15.0");
    tipPctTextView.setText(String.format("%s%%", tipPctString));
    tipPercentage = Double.parseDouble(tipPctString) / 100;
}

Defining a Menu for Our Activity

Android applications do not use the NavigationController approach of iOS applications. In Android 4.x devices, the choices that appear in the upper-right of the screen are part of the Android “action bar.”

android-action-bar
The action bar is the dark header in this screenshot. Note the settings button in the upper-right, which is defined by the menu.

We populate the action bar by defining a series of menu options in an XML file, just as we did with the screen layout. The menu is placed in a menu directory within the resources directory. Here are the contents of the menu file used on the main screen of our application:


<menu xmlns:android="http://schemas.android.com/apk/res/android" >
    <item
        android:id="@+id/menu_settings"
        android:orderInCategory="100"
        android:showAsAction="ifRoom"
        android:title="@string/menu_settings"
        android:icon="@android:drawable/ic_menu_preferences"/>
</menu>

Note that, in this case, we have just one menu option to access the settings screen. If more options were placed in the menu than could fit on the screen, then the remaining items would flow into a drop-down menu, accessible via the three vertical dots often seen in Android menus. To use the menu in our activity, we must load it at the correct time. This is done in the onCreateOptionsMenu method:


@Override
public boolean onCreateOptionsMenu(Menu menu) {
    // Inflate the menu; this adds items to the action bar if it is present.
    getMenuInflater().inflate(R.menu.activity_main, menu);
    return true;
}

// Respond to menu selections
@Override
public boolean onOptionsItemSelected(MenuItem item) {
    // Handle item selection
    switch (item.getItemId()) {
        case R.id.menu_settings:
            this.startSettings();
            return true;
        default:
            return super.onOptionsItemSelected(item);
    }
}

Notice how the onOptionsItemSelected method is used to determine what code is run when a given menu option is chosen.

Starting Another Activity

As we’ve seen from the code above, when the user chooses the settings icon from the action bar, the startSettings() method is invoked. This is the code that launches the second activity to display the settings screen:


private void startSettings() {
    Intent settingsIntent = new Intent();
    settingsIntent.setClass(this, SettingsActivity.class);
    startActivity(settingsIntent);
}

Android launches activities via an intent object. In this simple example, we are creating a very specific intent to launch the SettingsActivity class, which contains the code to load and drive the settings screen. Most Android applications use this technique to move the user from one screen to the next in an application.

Using Intents for App Integration

Intents are considerably more flexible than this. It’s possible to use intents to launch a variety of activities. A common intent is to send an item to be shared via social networking apps. If you’ve used an Android device before, then you’ll know that doing something like this typically brings up a menu of applications that support a sharing intent. This list often contains applications such as Facebook and Twitter.

android-share-list-338

This particular sharing intent is known as ACTION_SEND. All of the applications that support sharing are listening for a common intent. When an Android application is installed on a device, its manifest file describes the intents it can process. In this way, several applications may be installed, each of which supports the same intent.

If only one application supports the intent, then that activity is immediately launched. Whereas if multiple applications are installed for the same intent, then Android would show a menu of options, from which the user would make a selection before the intent is sent to the chosen activity.

The intent system in Android is a unique feature that can be used to create workflows across several applications. When building for the Android platform, you should keep this capability in mind because it offers a way to integrate your application with others.

The Settings Screen

Much of what occurs here has already been covered in the section on the main activity. This activity has similar code in the onCreate method to load the appropriate layout file and obtain references to the controls. When the “Save Settings” button is pressed, the following code is run to save the tip percentage value to storage:


private void saveSettings() {
    if (validateSettings()) {
        SharedPreferences preferences =
            this.getSharedPreferences("AppPreferences", MODE_PRIVATE);
        SharedPreferences.Editor prefEditor = preferences.edit();
        prefEditor.putString("tipPercentage",
            tipPercentageEditText.getText().toString());
        prefEditor.commit();
        this.finish();
    }
}

Notice that the last line in the saveSettings method calls the finish() method. This causes the current activity to stop running and returns the user to the prior activity, which in our case is the main screen. Contrast this with iOS’ use of the NavigationController and the way we popped the current view controller off the stack of view controllers.

Running Your Application

Android provides two basic ways to test an application: in the Android emulator or on a regular device such as a phone or tablet. Unlike with iOS, you don’t have to pay any fees to deploy your app to your own device or to share it with other Android users. The only fee that is required is if you wish to feature your app in Google Play.

The emulator enables you to emulate a variety of Android versions, as well as different screen aspect ratios and resolutions. While testing your application on a real device is always a good idea, the emulator helps you to test on device configurations that are not readily available.

android-emulator

To use the emulator, you must create an Android Virtual Device (AVD). Think of this as a virtual phone or tablet. Creating multiple AVDs, each with a unique configuration, is possible to test your app on different screen sizes and memory capacities. Android Developers provides details on how to use the AVD manager to set up these devices.

android-avd-manager

Use Hardware Virtualization to Speed Up That Slow Emulator

The Android emulator has long been maligned for its poor performance on many machines. This is because, in the past, the emulator has been running actual code compiled for the ARM processor on a phone, rather than for the x86 CPU that’s likely on your development machine.

Recently, an x86 hardware-virtualized version has been made available. This version starts up and runs considerably faster than the traditional emulator. The hardware-virtualized version will generally run on a Windows or Mac OS X machine with an Intel Core i3 or later processor that supports hardware virtualization. AMD processors are currently supported only on Linux.

Creating an AVD that uses hardware virtualization requires that you set up Intel’s HAXM support first. The developer documentation provides details on the process to follow for your operating system. Make sure that you are running the latest HAXM support; if you’re on Mac OS X Mavericks, a recently released hotfix resolves issues there as well.

Testing on a Device

To test the application on your own device, you’ll need to enable USB debugging on the device. Google provides detailed instructions on setting up your device for debugging.

android-usb-debug

For Android 4.0 devices, go to Settings → Developer Options in the menu and you’ll see a checkbox to enable this. If you’re on Windows, you might need to install some USB drivers for your device; these can usually be downloaded from the manufacturer’s website. For Mac OS X, you usually will not need any drivers and can just plug in the device via a USB cable.

The Eclipse IDE provides a “targets” option that enables you to specify which device, or AVD, should be used to run your application. This can be found in Run → Run Configurations and Run → Debug Configurations in the menu. The “Targets” tab allows you to select which AVD should be used by default or to set a prompt each time to select a run or debug destination for the application:

android-run-target-500

If you select “Always prompt to pick device,” then the IDE will present this dialog each time you choose to run or debug the application:

android-device-chooser-500

Every time you run or debug the application, an APK file is built and written to the /bin directory of your project’s root directory. This is the application distribution package used for Android devices. It’s analogous to the IPA file for iOS applications. When you select a deployment target via the process above, the Android development tools will automatically install the APK on the target device or emulator and run it.

Sharing Your Application With Others

Submitting an application to Google Play is beyond the scope of this article. The documentation provides an overview of the process. You can also use the export wizard in Eclipse to export an APK file. The resulting APK file can then be shared with others outside of Google Play via open distribution, and it is very useful when sharing copies of your application for testing prior to general release.

Android: Learning Resources

As with iOS, some excellent resources exist for getting started with Android. I recommend the following books and websites:

  • The Busy Coder’s Guide to Android Development, Mark L. Murphy
    This book is comprehensive, and the writing style is easy to follow. The author even offers a free version of the book based on an earlier version (Android 3.x) of the SDK.
  • Beginning Android 4 Application Development, Wei-Meng Lee
    For those looking for a shorter book, this does a good job of covering the basics, without as much detail or scope as Murphy’s book above.
  • Vogella
    This series of brief tutorials shows how to perform common operations in Android applications.
  • Android Weekly
    This weekly newsletter will help you stay current on new blog posts and libraries of interest to the Android development community.

Summary

That wraps up our coverage of developing a native app for Android. We’ve seen some unique capabilities of the platform, such as the resource system, layouts, and the way that activities and intents work together. In the next article, we’ll take the tip calculator cross-platform, looking at what’s involved in developing our app for both iOS and Android using PhoneGap.

(al, ea, il)


© Peter Traeg for Smashing Magazine, 2014.

Tags: Mobile

December 09 2013

13:50

How To Simplify Mobile App Data With Google Analytics And Tag Manager


  

Information about customers has never been available on the scale it is today. Businesses are learning new ways to leverage data to improve themselves on a daily basis. They’re realizing that data collection and data analysis have a measurable return on investment, and decision-makers are asking to see them.

As a developer, business owner or marketer, you need to know how to gather data and how to do it efficiently and in a scalable way. Furthermore, you need to understand what that data means and how to present it.

Two free tools to know about are Google Analytics and Google Tag Manager. When used together, they put the power in the hands of the marketers and data geeks, while freeing developers from the burden of constant marketing-driven updates.

Main Analytics Screenshot
(View large version)

Google Analytics is a widely known and well-respected system for gathering user data. Almost all developers have come across it, and all marketers have at least heard of it.

Main Tag Manager Screenshot
(View large version)

Google Tag Manager is a relatively new system in which all of the different Google code snippets for a website or mobile application can be organized and controlled through a drag-and-drop interface in a Web browser. In the time it normally takes to add Analytics tracking code to a mobile application, developers can add Tag Manager code instead, and then the Tag Manager’s “container” may be updated with any code snippets and tracking services that Google offers individually.

For this article, we’ll talk about adding Google Analytics to a mobile application, using the very future-proof Google Tag Manager to implement it.

Before We Begin

Google Analytics and Google Tag Manager are two separate services, each complex. Understanding the difference between the two and being clear on how they work together is important.

What Are the Tags We’ll Be Managing?

We can track a lot of things in applications right now. Beyond Analytics, we have AdWords conversions and AdWords remarketing, as well as the configurations and customizations of each service.

Traditionally, each service and customization has required new code to be implemented, but Tag Manager allows you to control all of that through an interface in a Web browser. This just makes sense for everyone’s workflow. Google hopes it will soon be the industry standard.

In short, tags are the different kinds of tracking that you might want to add, remove, try out or tweak, and they’re all controlled through the Tag Manager container.

Google Analytics Is A Tag

Google Tag Manager is what you would add to your application, but you’d still need to create a Google Analytics account to track and organize all of that information about the app and its users.

You’ll need to add your Analytics account as a tag within Tag Manager. Then, in the future, you (or anyone else) can add or tweak any additional tags or settings that you want.

Wait, Why Is This Important?

Imagine that you’re the head of marketing and your company is launching a new product or service. Your app has been updated and is awaiting approval from the app store, and marketing materials have been queued and are waiting to get pushed out to your customers. On launch day, your boss asks how the new feature is performing. You pull up the analytics and find that something is wrong with the data. It could be a little tracking nuance that you didn’t account for, data that doesn’t paint a complete picture or even a development bug. With a traditional analytics setup, you’re at the mercy of your development team and all of the various app stores that need to approve changes to code. With Tag Manager, you can make that fix instantly and get your reporting squared up in minutes instead of days.

Likewise, if your boss unexpectedly asks which search ads are driving the most downloads, you can add the AdWords tag through Tag Manager and start seeing results right away. To put it simply, Tag Manager enables marketers to be agile.

Let’s get started.

Set Up Analytics

If you’re totally new to Analytics, then there’s a lot to learn and read. We won’t cover that here, but I do want to be thorough and complete, and a few tips will save you headaches down the road. Google has more information on getting started.

When setting up Analytics, be aware of general best practices for tracking application data through the service.

One concept that is good to understand initially is how to work with client accounts and multiple users. An analytics account has an owner, but it’s shared with various users. So, create the Analytics property through your client’s Google account (if you have a client), and add yourself and anyone on your team as users.

Setting up your app property correctly for future growth is also important. Google shares some best practices.

Now, if you haven’t done so already:

  1. Create a Google account.
  2. Go to Google Analytics, and click or tap “Access Analytics.”
  3. Create a new account. Accounts may hold many properties, and users of an account will have access to all properties in that account.
  4. Create a property in the account (make sure it’s a “mobile app” property, not a “website” property).
  5. Go to the “Admin” section and, in the “Property” column, click “Property Settings.”
  6. Find and record that tracking ID (you’ll need it for Tag Manager).

Set Up Tag Manager

Separate from the Analytics account, we need to create a Tag Manager account. Here are the steps:

  1. Go to Google Tag Manager, and sign in with your Google account.
  2. Click or tap “New Account,” and name it something related to your app.
  3. Create a container. You’ll need one container per app, and you would add a new one only if you made a significant update that changes the tracking data.

Now that both services are set up, it’s time to connect them and start tracking. First, make sure your application is set up with all of the initial Tag Manager stuff. Google offers an SDK and instructions for both iOS and Android that include all of the Tag Manager code and files that you’ll need, with additional Analytics stuff built in. Just follow the instructions.

Get Everything Talking to Each Other

The next step is to add a tag for Analytics in the Tag Manager interface. This is actually several steps.

Create the Tag

First, navigate to the Tag Manager interface. We’ll create the tag for Analytics — that is, we’ll add Analytics to our Tag Manager container, which will interact with the app through that initial code that you implemented with the Tag Manager SDK. When you drill down into your container in the Tag Manger interface in a Web browser, you’ll start off with a message telling you that you have no tags and to “click here” to create one. Either click there or click the tag icon near the top right in the row of icons.

Enter a name, like “Analytics App View” (which is to say, basic analytics tracking), select “Universal Analytics” from the drop-down menu, and enter the tracking ID from the Analytics configuration a minute ago. Set the “Track Type” drop-down menu to “App View.”

You can save now, and your tag will be created. But we’re not done yet. So far, we have a tag, but it will never fire unless we tell it when to fire, using a rule.


(View large version)

Create a Rule

Tag Manager for apps comes with one rule built in. It’s called “Always,” which basically means “when true equals true” — in other words, always. We’ll set the Analytics App “View” tag to fire “always.”

If you’re not already looking at your newly created Analytics tag, click the tag name to edit it. A little ways down, you’ll see “Firing Rules.” Click to add one, and choose Google’s prebuilt “Always” rule. Now your tag is ready to fire.


(View large version)

Publish

We’re almost there. The last step is to publish to your container — that is, to let the application know that you’ve made changes to the tags that Tag Manager is managing.

First, click or tap that “Create a Version” button in the top-right corner. This will “lock in” your current configuration and archive it.

Tag Manager Create Version
(View large version)

Now that your version has been created, you’ll see that the “Create a Version” button has become a “Publish” button, and a “Preview” button has appeared to the left of it. Google’s preview is pretty advanced, so try it out. It’s pretty self-explanatory.

If you like what you see in the preview, hit “Publish.” If everything has gone off without a hitch, then you’re now tracking Analytics data from your application.

Tag Manager Publish
(View large version)

Now For The Data!

Now that you have Tag Manager in place and you’ve added Analytics to your Tag Manager container, you’re finally ready to start analyzing all of that data that you’ll be collecting. Below is a quick explanation of the main information that Google Analytics offers.

We’ll start with the default “App Overview” screen.

Analytics App Overview
(View large version)

First, access your Analytics account and click through to your property. Once you get to your dashboard, you’ll see a few key metrics, organized for you by Google. This is your “App Overview.” Default report shortcuts include “New Users,“ “Active Users,” “Country / Territory,” “Top Device Models,” “Engagements,” “Screens,” “Goals” and “In-App Revenue.”

Initially, the most interesting data will be about the users. Analytics shows you how many users have installed and come back to your app, which screens of your app they see and which devices they use. If this is the first time you’re using Analytics, then you’re finally seeing exactly how people use your app! Amazing!

New Users

The “New Users” section shows not only new users, but active users and user sessions. Users are broken down by operating system (you can track the same app across multiple operating systems in one Analytics property), app version and geography.

If you’re just getting used to viewing Analytics data, then this particular section will be one of the most useful. It makes it easy to tell what’s going on with your app’s usage at a glance.

Active Users

Active users are similar to new users, but the metrics here dive a little deeper and there are more of them. You’ll see some high-level information about screens and more about sessions.

I spend a lot of time in this section, especially early on. Here it’s easy to tell what’s going on with your app. Before you get into more advanced tracking through conversion funnels and specific events, you should get a broad picture of what users do with your app. The concepts and terminology are simple, but the implications and insights are huge.

Similar to a page view in Analytics for websites, a screen view can generate an immense amount of information about users and how they use your application. A “screen” represents a literal screen or view in your app, and Google knows how to track screens automatically, without additional configuration on the developer’s part (beyond implementing the original Tag Manager and Analytics code).

The reason I mention that Google knows how to track screens automatically is that Analytics goes far beyond tracking common data with a predetermined interface. But that’s another topic. For now, we’ll stick to the built-in tracking.

A session is similar to a visit in website analytics, and it represents the duration of the user’s session across one or many screens. By analyzing sessions, you will know when and where users are getting confused, at which points you should add sales messaging or ads, and what your most popular features are.

Below the screen and session data, on the left side of the main content area, lies a submenu that’s easy to miss. I missed it at first myself. (Google Analytics’ interface is always improving, and understanding the product is a challenging undertaking and so will take a little getting used to.) The submenu starts off with “Language” selected, but check out the other options in the list for some more high-level data about users.

At this point, you’re probably already making some interesting connections and assumptions from your data. I’ll keep going through the other default reports, and then I’ll touch on the exciting next steps you can take to more advanced tracking.

Country / Territory

“Country / Territory” gives geographical information via a heat map that shows where your most active users are concentrated. With the drop-down menu in the upper-left corner, which defaults to “Sessions,” you can display any other metric on the map by geographic density.

I’ve noticed that in the business world, the “suits” seem to love this sort of graphical analysis. In an instant, it “tells a story.” Screenshots of this kind of graphic-based data go a long way in internal and external presentations.

Top Device Models

This one’s pretty self-explanatory, but clicking through gives you insight into more than just device models. It touches on screen resolution, whether the user’s device is a touchscreen, and device branding. It also includes a donut graph for at-a-glance analysis.

Depending on the type of application, your users and their habits, and your own goals for your application, “Top Device Models” could be extremely relevant or completely useless.

It will be relevant if you want to know which devices to focus development on. Knowing whether you’re getting more sales from tablets or from phones is important.

It’s much less useful if your app is only on the iPhone. Sure, you’d see which iPhones your users have, but that might not matter all that much to your return on investment.

Engagements

“Engagements” shows you more information about screens and sessions, but with some valuable additional metrics, such as crash reports and events. Crash reports are self-explanatory and invaluable if your app crashes. Events are likely the next thing you’ll learn about once you understand all of Analytics’ basics.

The things you can do with events is where Analytics starts to get really fun.

Screens

“Screens” gives you detailed metrics for each screen. Of course, a lot of the information we’ve been looking at relates to screens anyway, but that’s because screens are the basis for most basic tracking. It makes sense that screen views permeate Analytics’ entire interface. In the screens report, you’ll see things such as which screens are the most viewed, how long users spend on each screen and which screens cause users to exit.

If you want to really tailor the user experience, then the details in this view are important. You can and should control every aspect of your application, and with this information, you can start to visualize how people flow through your application and where to optimize and guide that process.

Goals

You can do so much with Analytics, and goal reporting is just the beginning. Because goals are very specific to your company and your mobile application, a broad explanation wouldn’t do this section justice. At a very basic level, goals involve setting up event listeners and then combining the tracking of those events into “funnels.” A funnel normally ends with a conversion, such as a purchase or registration, and by tracking events that lead to conversions, you can get a great idea of whether and where users are getting confused or discouraged. This requires additional configuration, but you’ll get some powerful insight by tracking non-screen-change actions as events. By chaining them together into conversion funnels, the sky is the limit on the insight you can gain.

In-App Revenue

“In-App Revenue,” like goals, gets a bit specific to each mobile app. Some apps, such as brand-building apps, don’t generate any type of revenue, and so this section wouldn’t be relevant. If your app has any type of sales or e-commerce, though, then you can track and analyze many more metrics, some of which will truly improve your bottom line.

Time Frames

Once you understand the meaning of all of these numbers and graphs, you’ll probably want to know the period of time covered by a given set of data. In the top-right corner, the date-range selector defaults to the last month, not including today.

Select any date range you want. Choose from presets such as “Today” and “Last Week,” or select a range manually. Once you click “Apply,” the time period will carry through to all of the reports until you change it or close the window.

What’s Next?

Those are the basics, but keep exploring. The next things to learn about are events, shortcuts and dashboards.

Events

Events and goal-tracking are the next logical step after screen-tracking, because events enable you to track more specific actions. Get started on the help page for Google Analytics events.

Shortcuts

Shortcuts are saved snapshots of reports that you can customize or create from scratch. After you get used to the reports that Google provides, you’ll find yourself changing the same things every time you need to see data that is specific to your business. This means that it’s time to create a shortcut. Just press the “Shortcut” button in the top toolbar of the report that you want to save, and then you’ll be able to access that report at any time in the future.

Dashboards

Dashboards, then, are collections of reports and/or shortcuts. Similar to the “App Overview” screen, here you can create your own custom dashboard layouts and save them. Better yet, you can share these custom dashboards with members of your team. Create a board for your CEO with big-picture metrics, and create a separate view for developers that focuses on crashes and errors. Share them with the people who need the information. They’ll thank you for the data and the interface’s ease of use.

Moving Into The Future

As with any complex undertaking, understanding the basics of Analytics and starting with a smart configuration can make learning faster and easier. In this article, we wanted to give you insight into Google Analytics for apps but, more importantly, to present an efficient and versatile configuration and workflow, so that once you understand the main concepts, you’ll be set up to grow.

Google Tag Manager and similar products are the way of the future. Because Tag Manager enable us to add code to our application only once, anyone with account access can change tags and tracking information on the fly, without modifying the original code. This means that changes in a data collection system can be made more quickly and more easily. It also means that A/B testing happens faster and with less pushback, and that last-minute campaigns can be tracked at will.

Before Tag Manager, developers had to change the tracking code for every update to Analytics, and then app stores had to approve those changes. Thus, changes were infrequent, and sometimes mistakes wouldn’t get corrected for a week. That all changed with Tag Manager for apps.

Take advantage of it.

(al, ea)


© Andrew Thomas for Smashing Magazine, 2013.

Tags: Mobile

December 04 2013

14:29

Speed Up Your Mobile Website With Varnish


  

Imagine that you have just written a post on your blog, tweeted about it and watched it get retweeted by some popular Twitter users, sending hundreds of people to your blog at once. Your excitement at seeing so many visitors talk about your post turns to dismay as they start to tweet that your website is down — a database connection error is shown.

Or perhaps you have been working hard to generate interest in your startup. One day, out of the blue, a celebrity tweets about how much they love your product. The person’s followers all seem to click at once, and many of them find that the domain isn’t responding, or when they try to sign up for the trial, the page times out. Despite your apologies on Twitter, many of the visitors move on with their day, and you lose much of the momentum of that initial tweet.

These scenarios are fairly common, and I have noticed in my own work that when content becomes popular via social networks, the proportion of mobile devices that access that content is higher than usual, because many people use their mobile devices, rather than desktop applications, to access Twitter and other social networks. Many of these mobile users access the Web via slow data connections and crowded public Wi-Fi. So, anything you can do to ensure that your website loads quickly will benefit those users.

In this article, I’ll show you Varnish Web application accelerator, a free and simple thing that makes a world of difference when a lot of people land on your website all at once.

Introducing The Magic

For the majority of websites, even those whose content is updated daily, a large number of visitors are served exactly the same content. Images, CSS and JavaScript, which we expect not to change very much — but also content stored in a database using a blogging platform or content management system (CMS) — are often served to visitors in exactly the same way every time.

Visitors coming to a blog from Twitter would likely not all be served exactly the same content — including not only images, JavaScript and CSS, but also content that is created with PHP and with queries to the database before being served as a page to the browser. Each request for that blog’s post would require not only the Web server that serves the file (for example, Apache), but also PHP scripts, a connection to the database, and queries run against database tables.

The number of database connections that can be made and the number of Apache processes that can run are always limited. The greater the number of visitors, the less memory available and the slower each request becomes. Ultimately, users will start to see database connection errors, or the website will just seem to hang, with pages not loading as the server struggles to keep up with demand.

This is where an HTTP cache like Varnish comes in. Instead of requests from browsers directly hitting your Web server, making the server create and serve the pages requested, requests would first hit the cache. If the requested page is in the cache, then it is served directly from memory, never touching Apache or the database. If the page is not in the cache, then the request is handed over to Apache as usual, whereupon Apache will create and serve the page, which is then stored in the cache, ready for the next request.

Serving a page from memory is a lot faster than serving it from disk via Apache. In addition, the page never needs to touch PHP or the database, leaving those processes free to handle traffic that does require a database connection or some processing. For example, in our second scenario of a startup being mentioned by a celebrity, the majority of people clicking through would check out only a few pages of the website — all of those pages could be in the cache and served from memory. The few who go on to sign up would find that the registration form works well, because the server-side code and database connection are not bogged down by people pouring in from Twitter.

How Does It Work?

The diagram below shows how a blog post might be served when all requests go to the Apache Web server. This example shows five browsers all requesting the same page, which uses PHP and MySQL.

When all requests go to the Apache Web server.

Every HTTP request is served by Apache — images, CSS, JavaScript and HTML files. If a file is PHP, then it is parsed by PHP. And if content is required from the database, then a database connection is made, SQL queries are run, and the page is assembled from the returned data before being served to the browser via Apache.

If we place Varnish in front of Apache, we would instead see the following:

If we place Varnish in front of Apache.

If the page and assets requested are already cached, then Varnish serves them from memory — Apache, PHP and MySQL would never be touched. If a browser requests something that is not cached, then Varnish hands it over to Apache so that it can do the job detailed above. The key point is that Apache needs to do that job only once, because the result is then stored in memory, and when a second request is made, Varnish can serve it.

The tool has other benefits. In Varnish terminology, when you configure Apache as your Web server, you are configuring a “back end.” Varnish allows you to configure multiple back ends. So, you might want to run two Web servers — for example, using Apache for PHP pages while serving static assets (such as CSS files) from nginx. You can set this up in Varnish, which will pass the request through to the correct server. In this tutorial, we will look at the simplest use case.

I’m Sold! How Do I Get Started?

Varnish is really easy to install and configure. You will need root, or sudo, access to your server to install things on it. Therefore, your website needs to be hosted on a virtual private server (VPS) or the like. You can get a VPS very inexpensively these days, and Varnish is a big reason to choose a VPS over shared hosting.

Some CMS’ have plugins that work with Varnish or that integrate it in the control panel — usually to make clearing the cache easier. But you can put Varnish in any CMS or any static website, without any particular integration with other systems.

I’ll walk you through installing Varnish, assuming that you already run Apache as a Web server on your system. I run Debian Linux, but packages for other distributions are available. (The paths to files on the system will vary with the Linux distribution.)

Before starting, check that Apache is serving your website as expected. If the server is brand new or you are trying out Varnish on a local virtual machine, make sure to configure a virtual host and that you can view a test page on the server using a browser.

Install Varnish

Installation instructions for various platforms are in Varnish’s documentation. I am using Debian Wheezy; so, as root, I followed the instructions for Debian. Once Varnish is installed, you will see the following line in the terminal, telling you that it has started successfully.


[ ok ] Starting HTTP accelerator: varnishd.

By default, Apache listens for requests on port 80. This is where incoming HTTP requests go, because we want Varnish to essentially sit in front of Apache. We need to configure Varnish to listen on port 80 and change Apache to a different port — usually 8080. We then tell Varnish where Apache is.

Reconfigure Apache

To change the port that Apache listens on, open the file /etc/apache2/ports.conf as root, and find the following lines:


NameVirtualHost *:80
Listen 80

Change these lines to this:


NameVirtualHost *:8080
Listen 8080

If you see the following lines, just change 80 to 8080 in the same way.


NameVirtualHost 127.0.0.1:80
Listen 80

Save this file and open your default virtual host file, which should be in /etc/apache2/sites-available. In this file, find the following line:


<VirtualHost *:80>

Change it to this:


<VirtualHost *:8080>

You will also need to make this change to any other virtual hosts you have set up.

Configure Varnish

Open the file /etc/default/varnish, and scroll down to the uncommented section that starts with DAEMON_OPTS. Edit this so that it looks like the following block, which will make Varnish listen on port 80.


DAEMON_OPTS="-a :80 \
-T localhost:1234 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,256m"

Open the file /etc/varnish/default.vcl, and check that the default back end is set to port 8080, because this is where Apache will be now.


backend default {
.host = "127.0.0.1";
.port = "8080";
}

Restart Apache and Varnish as root with the following commands:


service apache2 restart
service varnish restart

Check that your test website is still available. If it is, then you’ll probably be wondering how to test that it is being served from Varnish. There are a few ways to do this. The simplest is to use cURL. In the command line, type the following:


curl http://yoursite.com --head

The response should be something like Via: 1.1 varnish.

You can also look at the statistics generated by Varnish. In the command line, type varnishstat, and watch the hit rate increase as you refresh your page in the browser. Varnish refers to something it can serve as a “hit” and something it passes to Apache or another back end as a “miss.”

Another useful tool is varnish-top. Type varnishtop -i txurl in the command line, and refresh your page in the browser. This tool shows you which files are being served by Varnish.

Purging The Cache

Now that pages are being cached, if you change an HTML or CSS file, you won’t see the changes immediately. This trips me up all of the time. I know that a cache is in front of Apache, yet every so often I still have that baffled moment of “Where are my changes?!” Type varnishadm "ban.url ." in the command line to clear the entire cache.

You can also control Varnish over HTTP. Plugins are available, such as Varnish HTTP Purge for WordPress, that you can configure to purge the cache directly from the administration area.

Some Simple Customizations

You’ll probably want to know a few things about how Varnish works by default in order to tweak it. Configuring it as described above should cause most basic assets and pages to be served from the cache, once those assets have been cached in memory.

Varnish will only cache things that are safe to do so, and it might not cache some common things that you think it would. A good example is cookies.

In its default configuration, Varnish will not cache content if a cookie is set. So, if your website serves different content to logged-in users, such as personalized content, you wouldn’t want to serve everyone content that is meant for one user. However, you’d probably want to ignore some cookies, such as for analytics. If the website does not serve any personalized content, then the only cookies you would probably care about are those set for your admin area — it would be inconvenient if Varnish cached the admin area and you couldn’t see changes.

Let’s edit /etc/varnish/default.vcl. Assuming your admin area is at /admin, you would add the following:


sub vcl_recv {
   if ( !( req.url ~ ^/admin/) ) {
     unset req.http.Cookie;
   }
 }

Some cookies might be important — for example, logged-in users should get uncached content. So, you don’t want to eliminate all cookies. A trip to the land of regular expressions is required to identify the cookies we’ll need. Many recipes for doing this can be found with a quick search online. For analytics cookies, you could add the following.


sub vcl_recv {
  // Remove has_js and Google Analytics __* cookies.
  set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_[_a-z]+|has_js)=[^;]*", "");
  // Remove a ";" prefix, if present.
  set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
}

Varnish has a section in its documentation on “Cookies.”

In most cases, configuring Varnish as described above and removing analytics cookies will dramatically speed up your website. Once Varnish is up and running and you are familiar with the logs, you can start to tweak the configuration and get more performance from the cache.

Next Steps

To learn more, go through Varnish’s documentation. You should understand enough of Varnish’s basics by now to try some of the examples. The section on “Achieving a High Hit Rate” is well worth a read for the simple tips on tweaking your configuration.

Speed Up Your Mobile Website With Varnish
Keep calm and try Varnish to optimize mobile websites. (Image source)

(al, ea, il)


© Rachel Andrew for Smashing Magazine, 2013.

December 02 2013

06:32

Lessons Learned From An App Graveyard


  

Guesstimates by analysts put the number of mobile app downloads this year at somewhere between 56 and 82 billion, with the average user downloading somewhere between 26 and 41 apps, with a smaller subset of those apps being used on a regular basis. Other numbers indicate that 95% of downloaded apps are abandoned within a month and 26% of apps are only used once.

Depending on the user, these abandoned apps are deleted or ignored, never to be opened again. I choose to leave these ignored apps on my phone and tablet, and at last count, I had over 375 apps on my iPhone. I turned off badges for my App Store app because it was a constant reminder that over 250 apps are waiting to be updated, which is intentional. A majority of those outdated apps lie in what I refer to as my app graveyard — buried beyond the first two screens of apps.

Lessons from an app graveyard
Illustration by Nancy C Xu.

My app graveyard is the final resting place for apps that I have downloaded, tried or used briefly but have since left neglected. I leave them on my phone and tablet as a constant reminder of what killed these apps. The following are lessons from my app graveyard that I keep in mind when designing apps, and they might help you, too.

Lesson 1: Validate The Need For An App

Don’t stretch Apple’s catchphrase “There’s an app for that” too far — not everything has to exist as an app. A recent study (PDF, 2 MB) by Compuware found that smartphone users prefer mobile apps to mobile websites, but other research shows organizational strategy shifting away from native mobile apps towards Web experiences. If your content and functionality can be better served to users through a responsive website or Web app, then you have no real need for a native app. While native apps can easily use a device’s capabilities, a few features, such as GPS, can be used by websites, too.

Couple that with offline storage and many websites can do whatever their native equivalents currently do. Visitors to Kansas City can easily do everything they need to from the VisitKC mobile website, instead of downloading and installing the dedicated app, which is now in my app graveyard.

Visit KC - site vs app
VisitKC.com (left) can do everything the VisitKC app (right) has to offer. (Large views: left, right)

How do you decide whether you need an app? The decision should be driven by business goals, user needs and the user experience. The short version is, if you plan to primarily offer content and basic functionality that users will access infrequently from different platforms and devices, then those users might be best served with a responsive website. Also, consider the time, effort and cost involved in creating a platform-specific native app, as well as the ongoing maintenance.

Additional Reading

Lesson 2: Make Sure The App Works As Expected

This one might sound like common sense, but you would be surprised by how many apps do not work as expected or end up crashing (see the image below), often after an update — and that’s not just from personal experience. One- and two-star reviews in the App Store often complain about just that. Look at reviews of Apple’s own Find My Friends or the beautifully designed alarm app Rise — when apps do not do the one thing they are supposed to do, users will stop using it or will use a competing app instead.

Staples’ app had problems authenticating users and would repeatedly lock users out of their accounts, even though the same credentials worked on the website. And the flight-tracking functionality in United’s app frequently does not work, although it works just fine on United’s mobile website, as shown below.

United's mobile app not as reliable as their mobile site
(Large views: left, right)

One way to avoid this is by thoroughly testing your app when releasing a new version or after an OS release. Test on actual devices, with bonus points for testing in mobile contexts (for example, testing an alarm app during deep sleep). Another ongoing proactive method is to use analytics tools to monitor where your app fails users or even crashes.

Problems faced with mobile apps
Source: “Mobile Apps: What Consumers Really Need and Want,” Compuware (PDF, 2 MB)

Lesson 3: Don’t Drain The User’s Device

Apps should judiciously use a device’s resources, including memory, bandwidth and power. Apps that do not minimize their use of a device’s location capabilities or that do not disable location updates as soon as possible are common offenders. Earlier this year, the Taxi Magic app alone drained about 20% of my battery during a 40-minute ride to the airport, leaving me hunting for juice on my way home. Staples’ location-based alerts appear to be active and display the location-indicator arrow even after the app is closed, and even after the app says it will affect battery life (below).

Staples app location based alert issue
Before and after a Staples app update with location-based alerts. (Large views: left, right)

Apple’s “iOS App Programming Guide” offers some factors to consider when tuning an app’s performance. Android has corresponding best practices.

Lesson 4: Follow Design Standards And Guidelines

The mobile experience is built from individual elements that require careful consideration. For instance, apps with a unique interaction design should onboard new users with a quick introduction or a short tutorial on using the app upon first launching, with options for the user to skip and view it later. Yet many unintuitive apps do not do this or do not even offer help, forcing users to give up.

Spellcheck is not an option either, as Mitt Romney’s mobile developers quickly discovered last year when they misspelled “America” in one of their photo-sharing banners as “Amercia”. Though the app was updated within a day, I did not update, but rather buried it in my app graveyard as a reminder that the little details matter.

With Mitt app spelled Amercia.
“Bart Simpson is with Mitt.” (Image: Joao Correia)

When designing for different platforms, follow the design elements and conventions of the respective platforms, be it iOS, Android or Windows.

Lesson 5: Earn Your Users’ Trust By Addressing Privacy And Security Concerns

Ask only for what you need — don’t collect or access information that is not required for the app to function. Many apps need access to such device information as contacts, calendar and photos to be useful, but an app should be transparent about why it needs that data and how it will be used. Apps that consume or store sensitive information (such as financial data) should demonstrate their trustworthiness up front or risk being abandoned.

One app in my graveyard is Card Mate Pro, which stores credit-card information but does not impart a feeling of security or confidence. Another common reason that apps are abandoned is that they ask users to register (or use social sign-in) in order to gain functionality that does not depend on having a profile. The app for Moe’s Southwest Grill (a fast-food chain) required registration before users could use any functionality in the app, including locating restaurants and browsing the menu.

Moe's app required users to login to use the app
Previous versions of Moe’s app required users to register or log in to use it.

Additional Reading

Lesson 6: Give Users Control And Ownership Of Their Data

Apps that create, capture or store user data should offer a way for users to save, back up or archive that information. A couple of apps that capture and organize my kids’ artwork now lie in my app graveyard because there’s no way for me to save or back up those pictures to iCloud or Dropbox. Every app, free or paid, is at the mercy of its developer and can be discontinued at any time or removed from the App Store, as recently happened with the to-do app Orchestra and Apple’s own Cards. Enabling users to easily save their own data gives them confidence that they won’t lose it forever if the app were to be discontinued or automatically upgraded.

Dead App infographic by Stardust
A part of
StarDust’s mobile reputation infographic shows just how many apps are dead. (Large view)

Lesson 7: Don’t Bait And Switch

Descriptions in the App Store (even for free apps) should clearly indicate what is included in the purchase price and what is available via in-app purchases, which will help users decide whether to buy and download that app. Not making this clear up front is wrong, as is forcing users to rate or share an app in order to enable features that they’ve already paid for.

Step Out Alarm Clock is one such app in my graveyard. While it forces you to wake up, it also forces you to rate the app to unlock features that are listed as part of the paid app. While we’re on the topic of ratings and reviews, when prompting users to rate an app, wait until they’ve had an opportunity to use it, and give them the option to rate it later or never rate it, and honor their selection.

Step Out Alarm App does a bait and switch
Step Out Alarm Clock forces users to rate the app to unlock features listed as part of the paid app.

Lesson 8: Load The App Quickly, Without Fanfare Or Annoyance

The “iOS Human Interface Guidelines.” recommend enabling your users to begin using your app immediately:

“It’s often said that people spend no more than a minute or two evaluating a new app. When you make the most of this brief period by presenting useful content immediately, you pique the interest of new users and give all users a superior experience.”

Apple goes on to recommend avoiding a splash screen or other “launch” experience, and instead using a simple launch image. While resource-intensive apps such as games need a few seconds to load, games are among the biggest offenders of the recommendation against splash screens. Moe’s Southwest Grill launches with its loud signature “Welcome to Moe’s” yell, without any option to mute it — which I discovered in a client’s quiet office. The developers have since listened to feedback, and an update last December allows users to mute the greeting.

Lesson 9: Marketing And Incentives Only Go So Far

Apps need marketing — and sometimes incentives — to get noticed and downloaded. However, that alone does not guarantee continued usage. The reservation system that the developers of Mailbox followed when they launched earlier this year, to stagger the app’s rollout, added to the hype for the app. Like many others, I haven’t used Mailbox since I downloaded it to try it out.

Mailbox's reservation system added to the hype

An incentive will get users to download and try an app, but engagement will not last long. The preceding lessons should help you create a good first impression, but users will often need to be nudged to continue using the app. This could take the form of reminders to use the app, as 1 Second Everyday does, or subtle email messages, if users have given you their email address and permission to contact them.

Lesson 10: What The App Store Gives, Apple Can Take Away

With each iOS release, third-party apps become redundant as their primary functionality in integrated into the new OS. iOS 7 suddenly made flashlight apps and bubble-level apps redundant. Still, some third-party apps have survived and thrived by offering additional unique functionality or a better experience. Examples include Instapaper and Pocket (formerly Read It Later), both of which were supposed to be killed by Safari’s Reading List in iOS 5 but are still alive and actively used today, two iOS releases later.

What’s A Developer To Do?

The lessons above are just a few of the reasons why an app may be downloaded by many but actively used by few. The Compuware study mentioned earlier found that 79% of consumers would retry a mobile app only once or twice if it failed to work the first time, and only 16% would give it more than two attempts. This gives app developers a small window of opportunity to make a good impression. So, put your best foot forward, and make sure your app is ready for prime time before launching. As shown below, abandonment is perhaps the least damaging response to a poor app experience.

User reaction to poor mobile app experiences
Mobile Apps: What Consumers Really Need and Want,” Compuware (PDF, 2 MB). (Large view)

If you already have an app struggling in the App Store, make sure to monitor social-media feedback, support emails, reviews on third-party websites, and App Store reviews and ratings to glean the reasons for the abandonment of your app. Services such as App Annie and App Figures will consolidate your reviews, which are a great source to learn about problems as well as suggestions for features. Prioritize and address these problems in your next release, and improve the app’s quality after launch to reduce the chance of your app joining users’ graveyard.

What other lessons have you learned from your own app graveyard? We would love for you to share them in the comments below.

(al, ea)


© Lyndon Cerejo for Smashing Magazine, 2013.

Tags: Mobile

November 22 2013

13:53

Four Ways To Build A Mobile Application, Part 1: Native iOS


  

The mobile application development landscape is filled with many ways to build a mobile app. Among the most popular are:

  • native iOS,
  • native Android,
  • PhoneGap,
  • Appcelerator Titanium.

This article marks the start of a series of four articles covering the technologies above. The series will provide an overview of how to build a simple mobile application using each of these four approaches. Because few developers have had the opportunity to develop for mobile using a variety of tools, this series is intended to broaden your scope.

we’ll start with some background and then dig into iOS.

I’ve built the same simple application with each technology to demonstrate the basic concepts of development and the differences between the platforms and development tools. The purpose of this series is not to convert you to a particular technology, but rather to provide some insight into how applications are created with these various tools, highlighting some of the common terms and concepts in each environment.

FasTip is a simple application to calculate tips. Because this is a simple example, it uses the standard UI controls of each platform:

fastip-app-screens-500

The screenshots above show the application running as native iOS, PhoneGap and native Android applications. Appcelerator Titanium uses native controls, so it looks the same as the native iOS and Android applications. Our application has two screens: a main screen where the tips are calculated, and a settings screen that enables the user to set a tip percentage. To keep things simple and straightforward, we’ll use the default styles of each environment.

The source code for each app is available on GitHub.

Native iOS Development

Most applications in Apple’s App Store are written in the Objective-C programming language, and developers typically use Xcode to develop their applications.

xcode-screen-500

Obtaining the Tools

To build an iOS app, you must use Mac OS X; other operating systems are not supported. The development tools that you’ll need, iOS 7 SDK and Xcode 5, are free of charge, and you can run the app that you build in the iOS simulator, which is part of the iOS SDK. To run your app on a real device and make it available in Apple’s App Store, you must pay $99 per year.

Creating a New Project

Once you have installed Xcode, you’ll want to create a new project. Choose “Create a new Xcode project” from the welcome screen or via File → New Project in the menu.

xcode-new-project-500

For a simple application such as this one, “Single View” is appropriate. Upon clicking “Next,” you will be presented with a dialog to enter some basic information about your application:

new-project-options-500

The value that you enter in the “Class Prefix” option tells Xcode to attach that unique prefix to every class that you generate with Xcode. Because Objective-C does not support “namespacing,” as found in Java, attaching a unique prefix to your classes will avoid naming conflicts. The “Devices” setting lets you restrict your application to run only on an iPhone or an iPad; the “universal” option will enable the application to run on both.

Navigation Controllers and View Controllers

The screen functionality of iOS applications is grouped into what are known as view controllers. Our application will have two view controllers: one for the main screen and one for the settings screen. A view controller contains the logic needed to interact with the controls on a screen. It also interacts with another component called the navigation controller, which in turn provides the mechanism for moving between view controllers. A navigation controller provides the navigation bar, which appears at the top of each screen. The view controllers are pushed onto a stack of views that are managed by the navigation controller as the user moves from screen to screen.

Storyboards: Building the User Experience Visually

Starting with iOS 5, Xcode has had storyboards, which enable developers to quickly lay out a series of view controllers and define the content for each. Here’s our sample application in a storyboard:

storyboard-overview-500

The container on the left represents the navigation controller, which enables the user to move from screen to screen. The two objects on the right represent the two screens, or view controllers, that make up our app. The arrow leading from the main screen to the settings screen is referred to as a segue, and it indicates the transition from screen to screen. A new segue is created by selecting the button in the originating view and then, while the Control key is pressed, dragging the mouse to the destination view controller. Apple’s documentation provides more detail about this process.

storyboard-properties-500

In the example above, we can see that a text field has been selected, and the property panel is used to adjust the various attributes of the controls. When this application was created, the “universal” app option was selected, enabling the app to run on both an iPhone and iPad. As a result, two versions of the storyboard file have been created. When the app is running on an iPhone or iPod Touch, the _iPhone version of the file will be used, and the _iPad version will be used for iPads. This allows a different layout to be used for the iPad’s larger display. The view controller will automatically load the appropriate layout. Keep in mind that if your storyboards expose different sets of controls for the iPad and the iPhone, then you must account for this in the code for your view controller.

In addition to directly positioning items at particular coordinates on the screen, you can also use the Auto Layout system that was introduced in iOS 6. This enables you to define constraints in the relationships between controls in the view. The storyboard editor enables you to create and edit these constraints.

storyboard-constraints-500

The constraints can also be manipulated programmatically. The Auto Layout mechanism is quite sophisticated and a bit daunting to use at first. Apple has an extensive guide on Auto Layout in its documentation.

Associating Storyboards With Your Code

To access the storyboard objects from the code, you must define the relationships between them. Connecting items from the storyboard to your code via Xcode is not obvious if you’re used to other development environments. Before you can do this, you must first create a view controller to hold these associations. This can be done with the following steps:

  1. Choose File → New File.
  2. In the dialog that appears, choose “Objective-C class”:
  3. In the next dialog, give your class a name and ensure that it inherits from UIViewController:
    file-new-options-500
  4. Upon clicking “Next,” you’ll be asked to confirm where in the project the file should be saved. For a simple project, picking the main directory of the app is fine.
  5. Upon clicking “Next,” you’ll see that a new set of files has been created for your view controller. Now, associate that newly created view controller with the view controller in your storyboard.
  6. With the storyboard open, click on the view controller. In the “Identity Inspector” panel, pick the “Class” that this view controller is to be associated with:
    class-picker
  7. Once this process is completed, the code for your view controller will be properly referenced by the storyboard entry.

To reference the controls that you’ve dragged onto a storyboard from your Objective-C code, you’ll need to define these relationships. The storyboard editor has an “assistant editor” view to help with this. Basically, it’s a split-pane view that shows both the storyboard and your code. In this example, we’ll reference a button that’s already been placed on the storyboard:

  1. First, ensure that you’ve completed the steps above to associate the view controller class with the corresponding view controller in the storyboard.
  2. Choose the assistant editor by clicking the icon that looks like this:
    assistant-editor-icon
  3. A split-pane view will open, with the storyboard on the left and your view controller class on the right.
  4. Select the button in your storyboard and, while holding down the Control key, drag from the button to the interface area of your code.
    create-outlet-500
  5. The resulting dialog will enable you to create an “outlet” for the button in your code. Simply give the button a name, and click the “Connect” button in the dialog. You may now reference the button in the view controller from your code.
  6. Let’s hook up a method to be invoked when a person taps on the button. Select the button again, and use the same Control-and-drag maneuver to drop a reference into the interface section of your view controller.
  7. This time, in the dialog box that appears, we’ll associate an “action,” rather than an outlet. Choose “Action” from the “Connection” drop-down menu, and enter a name like this:
    create-action-500
    For the “Event,” use the default of “Touch Up Inside,” and press the “Connect” button.
  8. Note that your class now has an interface with two entries in it:

@interface FTSettingsViewController ()
@property (weak, nonatomic) IBOutlet UIButton *myButton;
- (IBAction)tappedMyButton:(id)sender;
@end

The IBOutlet item is used to identify anything that you’re referencing from the storyboard, and the IBAction is used to identify actions that come from the storyboard. Notice also that Xcode has an empty method where you can place the code to be run when the user taps on the control:


- (IBAction)tappedMyButton:(id)sender {
}

The process above does take some getting used to and could certainly be made more intuitive. After some practice, it will get less awkward. You might find it useful to bookmark the section of the Xcode documentation that describes how to “Connect User Interface Objects to Your Code.”

As we’ll see later, you can also add objects to the view and manipulate their properties programmatically. In fact, applications of even moderate complexity typically perform a lot of manipulation in code. For complex apps, some developers eschew the storyboard and use the code-based alternative almost entirely.

Getting Into the Code

For even the most basic of applications to function, some code must be written. So far in the storyboard, we’ve laid out our user interface and the interactions between the view controllers. But no code has been written to perform the calculations, to persist the settings of the tip percentage and so on. That is all done by you, the developer, in Objective-C.

When an application is running, its overall lifecycle is handled by something called an “application delegate.” Various methods in this delegate are called when key events in the application’s lifecycle occur. These events could be any of the following:

  • the application is started,
  • the application is moved to the background,
  • the application is brought to the foreground,
  • the application is about to be terminated,
  • a push notification arrives.

The events above are handled in a file called AppDelegate. For our sample application, the default handling of these events is just fine; we don’t need to take any special action. The documentation has an overview of the application’s lifecycle and of responding to changes in an app’s state.

The next area of attention is the view controller. Just as with the application delegate, the view controller has its own lifecycle. The view controller’s lifecycle includes methods that are invoked when the following happens:

  • the view controller has been loaded;
  • the view controller is about to appear or has appeared on the screen;
  • the view controller is about to disappear or has disappeared from the screen;
  • the bounds of the view have changed (for example, because the device has been rotated) and the view will be laid out again.

The main code for our application is in the FTViewController.m file. Here is the first bit of code that initializes our screen:


- (void)viewWillAppear:(BOOL)animated
{
    // Restore any default tip percentage if available
    NSUserDefaults *defaults = [NSUserDefaults standardUserDefaults];
    float tipPercentage = [defaults floatForKey:@"tipPercentage"];
    if (tipPercentage > 0) {
        _tipPercentage = tipPercentage;
    } else {
        _tipPercentage = 15.0;
    }
    self.tipAmountLabel.text = [NSString stringWithFormat:@"%0.2f%%", _tipPercentage];
}

In this application, we want to use whatever tip percentage value was stored in the past. To do this, we can use NSUserDefaults, which is a persistent data store to hold settings and preferences for an application. Keep in mind that these values are not encrypted in any way, so this is not the best place to store sensitive data, such as passwords. A KeyChain API is provided in the iOS SDK to store such data. In the code above, we’re attempting to retrieve the tipPercentage setting. If that’s not found, we’ll just default to 15%.

When the user taps the “Calculate Tip” button, the following code is run:


- (IBAction)didTapCalculate:(id)sender {
    float checkAmount, tipAmount, totalAmount;

    if (self.checkAmountTextField.text.length > 0) {

        checkAmount = [self.checkAmountTextField.text floatValue];
        tipAmount = checkAmount * (_tipPercentage / 100);
        totalAmount = checkAmount + tipAmount;

        self.tipAmountLabel.text = [NSString stringWithFormat:@"$%0.2f", tipAmount];
        self.totalAmountLabel.text = [NSString stringWithFormat:@"$%0.2f", totalAmount];

    }

    [self.checkAmountTextField resignFirstResponder];

}

We’re simply reading the value that the user has inputted in the “Amount” field and then calculating the tip’s value. Note how the stringWithFormat method is used to display the tipAmount value as a currency value.

When the user taps the “Settings” button in the navigation controller, the segue that we established in the storyboard will push the settings’ view controller onto the stack. A separate view controller file, FTSettingsViewController, will now handle the interactions on this screen. Pressing the “Done” button on this screen will run the following code:


- (IBAction)didTapDone:(id)sender {

    float tipPercentage;
    tipPercentage = [self.tipPercentageTextField.text floatValue];

    if (tipPercentage > 0) {

        NSUserDefaults *defaults = [NSUserDefaults standardUserDefaults];
        [defaults setFloat:tipPercentage forKey:@"tipPercentage"];
        [defaults synchronize];

        [[self navigationController] popViewControllerAnimated:YES];

    } else {

        [[[UIAlertView alloc] initWithTitle:@"Invalid input"
                                    message:@"Percentage must be a decimal value"
                                   delegate:nil
                          cancelButtonTitle:@"ok"
                          otherButtonTitles:nil] show];

    }

}

Here we’re retrieving the value from the text field and making sure that the inputted value is greater than 0. If it is, then we use NSUserDefaults to persist the setting. Calling the synchronize method is what will actually save the values to storage. After we’ve saved the value, we use the popViewControllerAnimated method on the navigation controller to remove the settings view and return to the prior screen. Note that if the user does not fill in the percentage correctly, then they will be shown the standard iOS UIAlertView dialog and will remain on the settings screen.

In the section above on view controllers and storyboards, I mentioned that the controls in a view can be manipulated programmatically. While that was not necessary for our application, the following is a snippet of code that creates a button and adds it to a particular location on the screen:


CGRect buttonRect = CGRectMake(100, 75, 150, 80);
UIButton *myButton = [UIButton buttonWithType:UIButtonTypeRoundedRect];
myButton.frame = buttonRect;
[myButton setTitle:@"Click me!" forState:UIControlStateNormal];
[self.view addSubview:myButton];

Generally speaking, all of the controls that you place in a view extend from an ancestor class named UIView. As such, buttons, labels, text-input fields and so on are all UIViews. One instance of a UIView is in the view controller. This can be referenced in your view controller’s code as self.view. The iOS SDK positions items in a view based on a frame, also referred to as CGRect, which is a structure that contains the x and y coordinates of the item, as well as the width and height of the object. Note in the code above that the button is instantiated and assigned a frame (location and size) and then added to the view controller’s view.

Running and Debugging an iOS Application

When Xcode and the iOS SDK are installed, so is the iOS simulator, which simulates an iOS device directly on your machine. Xcode has a drop-down menu that allows you to select different device configurations. Pressing the “Run” button in the upper-left corner will build the app and then run it in the chosen simulator.

device-chooser-500

Using the menu above, you can switch between iPhones and iPads of different sizes, as well as between Retina and non-Retina versions of each device.

Debugging is done simply by clicking in the left margin of the code editor, where the line numbers appear. When the execution of your app reaches the breakpoint, the app will stop and the variable values in effect at that moment in time will appear below the code editor:

breakpoint-variables-500-opt

Some things, such as push notifications, cannot readily be tested in the simulator. For these things, you will need to test on a device, which requires you to register as an Apple developer for $99 a year. Once you have joined, you can plug in your device with a USB cable. Xcode will prompt you for your credentials and will offer to “provision” the device for you. Once the device is recognized, it will be shown in the same menu that allows you to switch between device simulators.

In Xcode, by going to Window → Organizer in the menu, you can display a tool that enables you to manage all of the devices visible in Xcode and to examine crash logs and more. The Organizer window also lets you take and export screenshots of your application.

Summary

Thus far, we’ve seen the basics of developing a simple native iOS application. Most applications are more complex than this, but these are the basic building blocks:

  • Xcode
    The development environment
  • Storyboards
    For laying out and configuring the user interface
  • View controllers
    Provide the basic logic for interacting with each of the views defined in the storyboards
  • Navigation controllers
    Enable the user to navigate between the different views

Learning Resources

To get started with iOS development, you might want to consult these useful resources:

This concludes the first segment of our tour of app development. I hope it has given you some insight into the basic concepts behind native app development on iOS. The next article in this series will cover how to build the same application using native Android development tools.

(al, ea)


© Peter Traeg for Smashing Magazine, 2013.

Tags: Mobile

November 15 2013

11:54

SEO For Responsive Websites


  

When Google announced its preference for user-friendly responsive websites in June 2012, I immediately saw an influx of posts that equated responsive design with search engine optimization. This is unfortunate because, while responsive websites can be SEO-friendly, some responsive websites are not.

I’ve detailed some of the common errors that give responsive websites problems in search results in an article on Search Engine Land earlier this year, so it’s nice to be able to do a more in-depth SEO audit of a responsive website here on Smashing Magazine.

I know not everyone’s definition of SEO is the same, so for those of you who don’t know my work, it should be emphasized that fixing all of these issues with SEO will improve the user experience in general; and like most credible SEOs these days, I don’t believe in manipulating search engine algorithms for short term ranking benefits.

The website I’d like to audit today is the US version of Disney Junior. I’ve chosen this website for three reasons: it’s not run by a client or a partner; it exhibits a lot of the SEO issues of many responsive websites; and my two and four year olds are huge fans of the brand and often use my smartphone or our family iPad to visit it. Hopefully, Disney can use this free advice to make its website better for my kids and kids like them who search.

This audit of Disney’s beautiful but often frustrating website shows that mobile SEO doesn’t end once you’ve made a website responsive, and it gives Disney a framework to make its website more usable and findable on search engines.

Is The Website Indexed?

Disney Junior doesn’t seem to have any real issues with indexing; many of its pages have been indexed by Google. See for yourself with a simple site:domain.com search, or verify it in Google’s Webmaster Tools if you use it.

Disney junior website google index
Google has indexed approximately 1,630 pages of Disney Junior. However, the descriptions indicate that we might have problems with content parity.

Obviously a website that isn’t in a search engine’s index will be invisible when people search for it. This is true for responsive websites as well as for websites that use dynamic serving or dedicated mobile URLs. That being said, this tends to be more of a problem with mobile URLs because of the common practice of intentionally nofollow’ing mobile websites in the robots.txt file in order to prevent the mobile pages from competing with the “canonical” pages for link equity.

This practice is misguided because bidirectional annotations (or switchboard tags) can harness that link equity and bring mobile URLs into search results, but that’s neither here nor there when it comes to responsive websites.

While Disney Junior is indexed, some responsive websites, such as Idis are not so lucky. Idis is responsive and innovative, yet only one page of the website has been indexed by Google. Because the website is dynamic and because _escaped_fragment_ hasn’t been used, the URL does not change when a user clicks on different website elements, thus depriving search engines of deep links to include in their indices. If anyone searches for text on any of these pages, they won’t find this interactive, award-winning website.

This could happen to any website that doesn’t have static URLs, of course, but mobile SEO isn’t done once the developer has decided whether to make their website responsive or use dynamic serving or use dedicated mobile URLs.

Is The Website Crawlable?

In order for any website, responsive or not, to be indexed, Google must be able to crawl the website — that is, to follow a link to every unique piece of content and then store that new URL.

To check this, run any website crawler, such as Xenu or Screaming Frog. I prefer Rob Hammond’s SEO Crawler for mobile SEO audits because it allows you to set the smartphone Googlebot as your preferred crawler. The number of URLs is limited but enough to give you a pretty good idea of any crawlability issues. If you own the website, verifying it with Google and with Bing is imperative. Both search engines have tools for developers that specify the crawl errors they encounter, and Google even lets you tell it to ignore certain parameters that might be causing problems. If you don’t own the website or can’t verify it, you can still identify most problems by crawling the website as described above.

When I crawled Disney Junior, it quickly became apparent that the content is hosted on several URLs: DisneyJunior.com, DisneyJunior.Disney.com, WatchDisneyJunior.Go.com and Disney.Go.com/DisneyJunior. This could hinder a search engine from assigning page authority, because a search engine’s spider has a limited crawl budget for every website, based on that website’s PageRank, so if you’re splitting your PageRank between four URLs and three domains, then you’re possibly presenting to Google a website architecture that is less than optimal. More on this later in the section about duplicate content.

Disney’s URLs themselves don’t seem to have any major issues with crawlability because they are mainly static URLs, which search engines have an easier time with. However, the site map could certainly be improved upon. Search engines jointly announced Sitemaps a few years ago as the convention for finding content on websites to index. DisneyJunior.Disney.com does have a site map, but there are a few problems with it, the biggest being that it’s a video site map that contains more than just videos.

disney junior video site map

Site maps are a way for site owners to communicate directly with search engines, so making the information as accurate as possible and not confusing search engines are imperative. Google has site maps for the many different kinds of content included on Disney Junior, so presenting separate maps for images, videos, HTML documents and so on would be best.

Crawlability is not a mobile-specific problem, but getting it wrong can harm a website intended for mobile users just as easily as it can harm a traditional website, and it needs to be done right regardless.

Recommendation

  • Disney should consider hosting all content on a single subdomain to ensure that search engines can properly identify page authority and to ensure that all relevant URLs are crawled efficiently.
  • Disney should also consider breaking down its single video site map, which contains all types of content, into individual site maps for HTML content, image content, video content, etc.

Is The Website Readable Without Images, Flash Or JavaScript Enabled?

A search engine cannot factor what it can’t see into the overall relevance of a page. And while Google can do some amazing things with optical character recognition in Google Glass, Google Drive and Google Goggles, it still only reads text for Google Search. As it says in its “Webmaster Guidelines”:

  • Create a useful, information-rich website, and write pages that clearly and accurately describe your content.
  • Think about the words users would type to find your pages, and make sure that your website actually includes those words within it.
  • Try to use text instead of images to display important names, content, or links. The Google crawler doesn’t recognize text contained in images. If you must use images for textual content, consider using the “ALT” attribute to include a few words of descriptive text.

Is Disney’s responsive website text-based and information-rich, and does it include terms that people would actually use when searching? Not really. If we look at the website through a simple text browser, like SEO-Browser.com, to get a better sense of how a search engine sees the website, then we get a much different picture than accessing it in a browser:

disney junior text comparison

Unlike websites that embed keywords in images or in Flash, search engines aren’t blocked from seeing relevant content here — there’s just not a lot of relevant content to see. The website isn’t readable in this case because there are no words to read.

If we look at a deeper-level page, we’d see some graphic text that search engines would have a hard time accurately processing.

2013-09-23_15-47-07

The text beginning with “Watch Sam Sandwich…” is accessible, but the words in the logo “The Bite-Sized Adventures of Sam Sandwich” are not because they are embedded as a graphic.

Recommendation

  • At the very least, Disney should make embedded text accessible as alt text.
  • When the text is substantial, the developer should consider making it accessible in a Web font, rather than as an embedded graphic.
  • Furthermore, the volume of relevant keywords, such as “games for kids” and other non-brand words, should be increased. This could be done by slightly expanding the text block on each page or, ideally, by designing the page to include some scannable text, keeping progressive enhancement in mind.

Is The Website Easy To Link To And Share?

Many websites, both responsive and not, have URLs that don’t seem to be intended for human consumption. This is detrimental to SEO, of course, mostly because the URLs aren’t memorable and are difficult to share. Given how important search engines view sharing and linking when ranking pages, the more we can do to facilitate linking and sharing, the better off we’ll be in search results.

The print and video URLs for Disney Junior fall into this category, adding random characters to otherwise memorable paths (http://disneyjunior.com/print/stethoscope-4e4e43e2e8368d71cf2086da). For the most part, though, the URLs include keywords and are easy for users and search engines to understand.

The website could go one step further and include social bookmarklets to enable users to share content on social networks. Disney Junior has an active presence on Facebook, Twitter and YouTube, so its creators must understand the value of social media. But they might not understand that leveraging social media is becoming important to organic discovery and search. And given that 78% of Facebook’s user base is mobile and that mobile users spend more time on social networks than PC users, then helping mobile searchers share content whenever they’d like makes sense.

This isn’t a tutorial on how to make social buttons work on a responsive website, but Facebook’s mobile “Like” button works on responsive websites. However, for performance considerations, it’s a good idea to use social sharing buttons or SocialCount that use lazy loading to load the actual social scripts on click. If you use a third-party plugin, such as AddThis, there are ways to make those compatible with a responsive website as well.

Recommendation

To increase referrals from social networks and to facilitate content discovery, Disney Junior should incorporate social-sharing buttons that are responsive, that add to the overall experience but that don’t significantly increase page-loading time.

Does The Website Display Content For Users’ Needs, Regardless Of The Device Used?

Those who champion “content parity” have a worthy cause in making websites accessible to everyone, regardless of device. Unfortunately, the issue is a little complicated. Making content available on all platforms is sometimes good for users, other times not.

Disney Junior is fairly representative. It deprives users of relevant content, fails to connect relevant content to users who need it, and gives users content that they have no chance in the world of ever using. Unfortunately, in my experience this is fairly typical of responsive websites, many of which don’t go far enough to make all content accessible, regardless of device, and which provide content that’s unusable on mobile devices.

Google sees one weakness so often on mobile websites that it has threatened to penalize for it in smartphone search results: unplayable videos. In June 2013, Google said this:

“We recommend using HTML5 standard tags to include videos and avoid content in formats, such as Flash, that are not supported by all mobile devices. Regardless, try your best to offer smartphone users the best experience possible to make sure that the videos are playable on as many devices as possible.”

So, when I encounter many screens like the following when trying to play Disney Junior videos, I get a little concerned:

Telling users that their device does not support something may lower your ranking in Google smartphone search (large preview).
Not supporting a piece of content for mobile users could lower your ranking in smartphone search results.

In some cases, though, Disney Junior goes too far with content parity. The four main sections of the website are games, videos, printables and a live feed. The problem is that unless a user is not only aware of Google Cloud Print but is one of the handful of people who has it installed on their device, they won’t have any way to print the coloring pages or other printables.

I have no idea how many people have actually printed these PDFs from mobile devices, but I’m guessing that the vast majority of people who attempt to access this content leave in frustration. I have no evidence that Google actively penalizes a website for unplayable videos, but it announced that it would more than four months ago, and it could start any day. Regardless of the impact on ranking, fixing this issue would make the website more usable to whoever visits.

One quarter of the content on DisneyJunior.com is devoted to content that most users will not be able to use on mobile devices (large preview).
One quarter of the content on Disney Junior is inaccessible on mobile devices.

No doubt, printables and coloring pages are a popular feature on desktops and laptops, and users actively look for such content. On mobile devices, it’s an entirely different story.

Bing Ad Intelligence Coloring Pages Search Volume by Device
According to Bing Ads Intelligence, which lets you see search volume by device, less than 5% of the total search volume for “coloring pages” comes from mobile devices. The reason is that most mobile devices aren’t connected to a printer, thus rendering such content unusable.

It’s worth noting that Disney has made huge strides on its responsive website at Disney.com since I addressed this issue in April 2013. None of its Flash games had been accessible to smartphone users, and it’s since made most of them compatible with HTML5. So, fixing these issues that plague both responsive and dedicated mobile websites is possible, but it won’t happen by itself.

Making this content available to all users is fine, but if Disney is going to do this, then it needs to explain that, before printing it from a mobile device, the user will have to download Google Chrome and/or the Google Cloud Print app and connect their device to a printer. If Disney doesn’t do that, then a lot of confused and unhappy kids will be left staring at a PDF on their mobile device, wondering how to print it. Disney could also consider making the feature less prominent on its home page for mobile users.

Finally, what’s missing from Disney Junior’s responsive website is any mention of the mobile content that is actually available. Assumptions on user behavior are always tricky, but in this case it’s quite safe to assume that mobile users would be looking more for apps and mobile games than for printables. However, the first search result for “Disney junior apps” in Google takes the user to a non-responsive page, before giving them an error message due to the absence of Flash.

disney junior apps user flow
“Dad, what is this?!” says my four-year-old son as he tries to watch Disney Junior on my smartphone but is redirected to a page that tells him to download Flash.

What’s more, when it comes to “mobile games,” a phrase that people search for more than 12,000 times a month, Disney is nowhere to be found. It’s a shame, because Disney’s games, being compatible with smartphones, are relevant to the term, but they won’t be found in relevant search results because the term isn’t included. And not appearing in relevant search results is definitely not search engine optimization.

disney mobile searches
These figures for search volume of Disney-related keywords by platform show that mobile users often search for different things and at a different frequency than users on desktops and laptops.

Recommendation

Disney Junior has three opportunities to improve the accessibility and relevance of its responsive website:

  • Ensure that all videos and games are playable on most smartphones before Google makes good on its promise to penalize websites for unplayable videos. This will also improve the user experience and reduce the number of frustrated visitors.
  • Rethink the website’s information architecture. Highlighting printables and coloring pages for mobile users doesn’t make sense if these users have no way to print them from their device.
  • Ensure that the website contains all relevant keywords, regardless of platform, so that the many people who search with platform-specific keywords, like “mobile games” and “iphone apps,” can find relevant content on Disney Junior.

Does The Website Load Quickly?

When I echoed the claim of Guy Podjarny, Akamai’s evangelist, in Search Engine Land that responsive websites are often bloated and slow, I was met with a lot of hostile comments. Many thought I was saying that responsive websites cannot be fast. Of course they can be made fast; unfortunately, most are not (PDF).

This is not just bad for performance, but can be detrimental to SEO as well. When Google pointed out common mistakes that it might soon penalize for, it put slow-loading pages on the hit list. Fortunately, Google has given developers many resources to make their websites faster, including the PageSpeed Insights tool. Following the recommendations made by this tool will go a long way to making your website load in one second or less, which is what Google recommends for optimum mobile performance.

Disney Junior takes more than seven seconds to load on average, and Google gives it a score of 71 out of 100 for page speed.

disney junior responsive mobitest
Akamai’s Mobitest tool is great for showing how long a page takes to load on mobile devices.

disney junior page speed test google
Google’s PageSpeed Insights follows the color coding of stop lights to indicate the priority of its recommendations.

Much has been written in the design community about the performance of responsive websites. At the very least, a responsive website should follow Google’s guideline of loading in less than a second.

Recommendation

Disney Junior should follow Google’s PageSpeed Insights and optimize its images, eliminate render-blocking JavaScript and CSS in content above the fold, and reduce page-loading time overall to one second or less.

Does The Website Try To Get You To Download An App Instead Of Showing What You Asked For?

WTF Mobile Web lists many websites that attempt to get users to download an app, instead of immediately showing them relevant content — a technique known as a door slam. Rotten Tomatoes is typical:

rotten tomatoes app interstitial
An app interstitial is shown to smartphone users who access Rotten Tomatoes’ website.

This used to be merely annoying to users — until June 2013, when Google added it to its list of things to penalize websites for. Now it should annoy all website owners, too.

These door slams usually happen when the website detects the user agent to be a mobile device. They’re rarer on responsive websites, in my experience, because most owners of responsive sites are happy to let users freely access their content, instead of trying to persuade them to switch to the app experience.

Disney Junior is the exception to the rule. If you attempt to watch one of its videos on your mobile device, the page will redirect you to an app. If you’re on an Android device, you will be redirected to an app that doesn’t exist:

watch disney junior mobile
Android users who click on the “Watch Disney Junior” link on the home page are greeted with this message.

If you include app interstitials on your responsive website, then now is the time to consider whether their business value outweighs the prospect of not appearing as often in Google search results for relevant queries on smartphones. As far as I can tell, Google hasn’t yet begun to penalize for this infraction; however, as with unplayable videos, it’s only a matter of time before it does.

Recommendation

Disney Junior should remove the app interstitial or at least turn it into a banner that is not disruptive. Smart banners are available on Android and on iOS as replacements for app interstitials.

Including a smart banner on iOS is as simple as adding the following meta tag to the head of the page, with the relevant App Store details:


<meta name="apple-itunes-app" content="app-id=myAppStoreID, affiliate-data=myAffiliateData, app-argument=myURL">

Adding a smart banner on Android is not quite as simple, but it can be done through AdMob. Some developers have simulated iOS’ banner on Android using jQuery.

Does The Website Contain Duplicate Content?

Duplicate content has been preventing relevant pages from ranking for given queries long before smartphones came around. Now, it gets even more complicated. In case you don’t know, duplicate content is a single piece of content that exists on more than one URL. It makes a page less competitive because its PageRank indicators are dispersed.

Mobile subdomains are usually the culprit of duplicate content, as my case studies demonstrate. However, just because a website is responsive doesn’t mean that duplicate content is not possible. In fact, checking for it is important for any website, mobile-friendly or not. Neglecting it means letting Google itself identify the canonical page for a given piece of content, which doesn’t always end well for the website owner.

We shouldn’t take chances. Instead, we should ensure that search engines can understand which pages are intended for users and which pages are just copies of canonical pages. We can give search engines as many signals as possible to alert them to canonical pages (such as by including canonical tags, by adjusting the handling of parameters in Webmaster Tools, and so on).

As mentioned earlier, Disney Junior has quite a bit of duplicate content. But according to the source code, it has added canonical tags to tell Google and Bing which pages to rank. So, we have no recommendation for Disney on this one.

If you’re not familiar with canonical tags, it’s as simple as adding the following tag to the head of any duplicate page:


<link rel="canonical" href="http://www.disneyjunior.com"/>

The URL referenced in this tag should be the canonical URL for the given content. Some of Disney Junior’s properties do have a slight problem of duplicate content. Chuggington.com, for example, has equivalent content on DisneyJunior.com/Chuggington. If Disney wanted to set Chuggington.com as the canonical URL, it would simply add the following tag to DisneyJunior.com/Chuggington:


<link rel="canonical" href="http://www.chuggington.com"/>

Does The Website Take Advantage Of Mobile Devices To Deliver Extra Value?

Disney Junior doesn’t show anything unique to mobile devices. This might or might not be intentional, but it ignores opportunities to delight the consumer with device-specific content.

Back in 2008, one analyst predicted that mobility would create an entirely new Internet, filled with voice-recognition apps, GPS and scannable content, and hosted on .mobi domains. He called it the mobile ’net and thought that it would be separate from the Internet as we knew it.

It reads a little like science fiction today, with silver mylar-clad people jetting around in flying cars and on hoverboards, but the author was spot on with one thing. He basically predicted Uber, the mobile ride-sharing app that is disrupting the taxi industry. And the article foreshadows the purchase of mobile-first photo app Instagram by Facebook for $1 billion in April 2012. Smartphones and tablets have different properties than PCs, and they are often used in situations where a PC would be impractical, thus instilling behaviors and patterns that previously did not exist on the Internet.

GPS enables users to find things around them with Google and Bing, and search data is showing that users are taking advantage of it, using terms like “navigate to,” “closest” and “near me” primarily on smartphones and tablets. Shopping-related searches are two times more likely to happen in-store now, and 70% of mobile searchers call businesses directly from search results — something they can’t really do on a PC. Granted, many people search for the same things on their mobile devices as they do on their desktop computers, and they often do it when they’re not on the go. However, 17% of them are on the go, and all of them have access to things like GPS, the camera, the accelerometer and the phone, which aren’t available on desktops and laptops.

Today, these mobile-first experiences are sometimes built on the Web (i.e. not a separate mobile Web) and more often in apps. From an SEO standpoint, they make the most sense on the Web. In its “Webmaster Guidelines,” Google says, “Think about what makes your website unique, valuable, or engaging. Make your website stand out from others in your field.” And it echoes this sentiment in its “SEO Starter Guide” (PDF), saying that “creating compelling and useful content will likely influence your website more than any of the other factors discussed here.”

In short, great content that people find useful will be linked to and shared, which will alert Google to its quality. But an app probably won’t appear in search results unless the user types the word “download” or “app” or the app’s name in their query, and Google cannot process or index deep links within apps. The way to make content as accessible to mobile users as possible is not to put it in an app, but to put it on the Web.

The problem is that doing this with a responsive Web design, as Google defines it, is difficult. Many responsive websites make great use of geolocation (such as Starbucks’), but most use JavaScript or server-side components to accomplish it, which is beyond Google’s scope of a responsive website.

Still, some websites do offer content that would be very useful to someone on a smartphone or tablet, which might differentiate those websites just enough for Google to rank them higher than competitors. One of my favorite examples is Sears, which uses dynamic serving on its mobile website to give smartphone users a scanner to compare competitors’ prices with Sears’ online prices. This empowers consumers to find the best price, while combatting the practice of “showrooming” by making it easy for people to find the best prices at Sears.

sears scanner on mobile website
Sears’ mobile website enables smartphone owners to scan items to compare against Sears’ prices.

Another great example is Lowe’s, which provides an in-store map on the location-related pages of its mobile website. It’s a small addition, but it helps users find what they’re looking for faster once they get to the store, and it potentially differentiates the brand.

lowe's mobile location pages
Lowe’s provides an in-store map on the location-related pages of its mobile website.

Recommendation

Disney Junior should consider mobile-specific opportunities to differentiate its content from that of competitors and to add value for users.

Conclusion

Unfortunately, creating a responsive website alone won’t be enough to be properly optimized for search results. If you’re keeping score, more than half of the content that Disney Junior’s responsive website highlights is unusable on smartphones. The website is beautiful, to be sure, and Disney deserves credit for trying to make its content accessible to users, regardless of device. But can a website like this really be called “responsive”? It contains all the ingredients of a responsive design; however, responsive means “reacting quickly and positively,” which this website fails to do in many ways.

User-friendly responsive Web design is preferred by Google, and Google doesn’t (intentionally) send traffic to websites that frustrate users, or to websites that don’t contain the keywords that people are including in their queries, or to websites whose content is not more compelling than that of competitors. Hopefully, this audit has taken you beyond just thinking about mobile configurations when tackling mobile SEO. And hopefully you feel equipped to really make your own responsive website competitive in search results.

Start with these basic SEO tips, which many responsive websites do not follow:

  • Make sure your responsive website has been indexed by doing a site:domain.com search in Google and Bing. If it hasn’t, then merely making the website responsive will not improve its visibility.
  • Ensure that search engine spiders can crawl your website and that they index every unique piece of content once. Responsive websites should have static URLs, without difficult-to-crawl characters such as hashbangs, if possible.
  • A site map will make your website more crawlable. But use the proper type of site map for the content, and follow protocol.
  • Include enough plain text that search engines can understand what the website is about, without needing access to image- or Flash-based content. Progressive enhancement is a useful method of building pages that are accessible to search engines. Use SEO-Browser.com to check your work.
  • Try to include relevant keywords in your URLs, and make pages easy to remember and share by avoiding long strings of human-unreadable characters.
  • Include responsive social bookmarklets to help mobile users share your content on their social networks. But keep an eye on page speed.
  • Don’t highlight content that users can’t access on their device. Content parity is great, but serving device-specific content to the wrong devices goes too far.
  • Don’t make all of your content adaptive. Users often look for device-specific content (such as mobile games, apps, mobile coupons, etc.). Including relevant keywords is the only way to get traffic from search engines.
  • Make your responsive website load in less than a second, using Google’s PageSpeed Insights and Akamai’s Mobitest. Not doing so could hurt you in search results for smartphones.
  • Use smart banners to promote your app, instead of the more disruptive interstitials.
  • Identify content that exists on more than one URL, and use canonical tags and permanent redirections to tell search engines which pages to show in search results.
  • Make sure that adaptive content and responsive Web design are really the best way to serve your users. Not every website is a traditional publication, and sometimes users have needs that are difficult to address with a single information architecture.
  • If responsive Web design truly is best for your users, then enhance your adaptive content by adding useful features (such as a scanner or GPS locator), using server-side components.

(al, ea, il)


© Bryson Meunier for Smashing Magazine, 2013.

Tags: Mobile

October 24 2013

11:12

Automate Your Responsive Images With Mobify.js


  

Responsive images are one of the biggest sources of frustration in the Web development community. With good reason, too: The average size of pages has grown from 1 MB to a staggering 1.5 MB in the last year alone. Images account for more than 60% of that growth, and this percentage will only go up.

Much of that page weight could be reduced if images were conditionally optimized based on device width, pixel density and modern image formats (such as WebP). These reductions would result in faster loading times and in users who are more engaged and who would stick around longer. But the debate isn’t about whether to optimize images for different devices, but about how to go about doing so.

In an ideal world, we would continue using the img tag, and the browser would download exactly what it needs based on the width of the device and the layout of the page. However, no functionality like that currently exists. One way to get functionality similar to that would be to change the src attribute of img elements on the fly with JavaScript, but the lookahead pre-parser (or preloader) prevents this from being a viable option.

The first step to overcoming this problem is to create a markup-based solution that allows for alternate image sources to be delivered based on a device’s capabilities. This was solved with the introduction of the picture element, created by the W3C Responsive Images Community Group (although no browser currently implements it natively).

However, the picture element introduces a whole new problem: Developers must now generate a separate asset for every image at every breakpoint. What developers really need is a solution that automatically generates small images for small devices from a single high-resolution image. Ideally, this automated solution would make only one request per image and would be 100% semantic and backwards-compatible. The Image API in Mobify.js provides that solution.

The <picture> Element As The Upcoming Best Practice

The picture element is currently the frontrunner to replace the img element because it enables developers to specify different images for different screen resolutions in order to solve the problem of both performance and art direction (although the new srcN proposal is worth looking into). The typical set-up involves defining breakpoints, generating images for each breakpoint and then writing the picture markup for the image. Let’s see how we can make the following image responsive using a workflow that includes the picture element:

President Obama and Governor Christi

We’ll use a baseline of 320, 512, 1024 and 2048 pixels.

First, we need to generate a copy of each image for those different resolutions, either by using a command-line interface (CLI) tool such as Image Optim or by saving them with Photoshop’s “Save for web” feature. Then, we would use the following markup:


<picture>
    <source src="responsive-obama-320.png">
    <source src="responsive-obama-512.png" media="(min-width: 512px)">
    <source src="responsive-obama-1024.png" media="(min-width: 1024px)">
    <source src="responsive-obama-2048.png" media="(min-width: 2048px)">
    <noscript><img src="responsive-obama-320.png"></noscript>
</picture>

One problem with this markup is that, in its current configuration, our image would not be optimized for mobile devices. Here is the same image scaled down to 320 pixels wide:

President Obama and Governor Christi

Identifying the people in this photo is difficult. To better cater to the smaller screen size, we need to use the power of art direction to crop this photo for small screens:

responsive-obama-mobile

Because this file isn’t simply a scaled-down version of the original, the name of the file should be given a different structure (so, responsive-obama-mobile.png, instead of responsive-obama-320.png):


<picture>
    <source src="responsive-obama-mobile.png">
    <source src="responsive-obama-512.png" media="(min-width: 512px)">
    <source src="responsive-obama-1024.png" media="(min-width: 1024px)">
    <source src="responsive-obama-2048.png" media="(min-width: 2048px)">
    <noscript><img src="responsive-obama-512.png"></noscript>
</picture>

But what if we want to account for high-DPI (dots per inch) devices? The picture element’s specification has a srcset attribute that allows us to easily specify different images for different pixel ratios. Below is what our markup would look like if we used the picture element.


<picture>
    <source srcset="responsive-obama-mobile.png 1x, responsive-obama-mobile-2x.png 2x">
    <source srcset="responsive-obama-512.png 1x, responsive-obama-1024.png 2x" media="(min-width: 512px)">
    <source srcset="responsive-obama-1024.png 1x, responsive-obama-1024.png 2x" media="(min-width: 1024px)">
    <source srcset="responsive-obama-2048.png 1x, responsive-obama-4096.png 2x" media="(min-width: 2048px)">
    <noscript><img src="responsive-obama-512.png"></noscript>
</picture>

Here we have introduced a couple of new files (responsive-obama-mobile-2x.png and responsive-obama-4096.png) that must also be generated. At this point, we’ll have six different copies of the same image.

Let’s take this a step further. What if we want to conditionally load our images in a more modern format, such as WebP, according to whether the browser supports it? Suddenly, the total number of files we must generate increases from 6 to 12. Let’s be honest: No one wants to generate multiple versions of every image for various resolutions and have to constantly update those versions in the markup. We need automation!

The Ideal Responsive Image Workflow

The ideal workflow is one that allows developers to upload images in the highest resolution possible while still using the img element in such a way that it automatically resizes and compresses the images for different browsers. The img element is great because it is a simple tag for solving a simple problem: displaying images for users on the Web. Continuing to use this element in a way that is performant and backwards-compatible would be ideal. Then, when the need for art direction arises and scaling down images is not enough, we could use the picture element; the branching logic built into its syntax is perfect for that use case.

This ideal workflow is possible using the responsive Image API in Mobify.js. Mobify.js is an open-source library that improves responsive websites by providing responsive images, JavaScript and CSS optimization, adaptive templating and more. The Image API automatically resizes and compresses img and picture elements and, if needed, does it without changing a single line of markup in the back end. Simply upload your high-resolution assets and let the API take care of the rest.

Automatically Make Images Responsive Without Changing The Back End

The problem of responsive images is a hard one to solve because of the lookahead pre-parser, which prevents us from changing the src attribute of an img element on the fly with JavaScript in a performant way. The pre-parser is a feature of browsers that starts downloading resources as fast as possible by spawning a separate thread outside of the main rendering thread and whose only job is to locate resources and download them in parallel. The way the pre-parser works made a lot of sense prior to responsive design, but in our multi-device world, images in the markup are not necessarily the images we want users to download; thus, we need to start thinking of APIs that allow developers to control resource loading without sacrificing the benefits of the pre-parser. For more details on this subject, consider reading Steve Souders’ “I <3 Image Bytes.”

One way that many developers avoid the pre-parser is by manually changing the src attribute of each img into data-src, which tricks the pre-parser into not noticing those images, and then changing data-src back to src with JavaScript. With the Capturing API in Mobify.js, we can avoid this approach entirely, allowing us to be performant while remaining completely semantic (no <noscript> or data-src hacks needed). The Capturing technique stops the pre-parser from initially downloading the resources in the page, but it doesn’t prevent parallel downloads. Using Mobify.js’ Image API in conjunction with Capturing, we are able to have automatic responsive images with a single JavaScript tag.

Here is what the API call looks like:


Mobify.Capture.init(function(capture){
    var capturedDoc = capture.capturedDoc;
    var images = capturedDoc.querySelectorAll('img, picture');
    Mobify.ResizeImages.resize(images, capturedDoc) 
    capture.renderCapturedDoc();
});

This takes any image on the page and rewrites the src to the following schema:


http://ir0.mobify.com/<format><quality>/<maximum width>/<maximum height>/<url>

For example, if this API was running on the latest version of Chrome for Android, with a screen 320 CSS pixels wide and a device pixel ratio of 2, then the following image…


<img src='cdn.mobify.com/mobifyjs/examples/assets/images/forest.jpg'>

… would be rewritten to this:


<img src='//ir0.mobify.com/webp/640/http://cdn.mobify.com/mobifyjs/examples/assets/images/forest.jpg'>

The image of the forest would be resized to 640 pixels wide, and, because Chrome supports WebP, we would take advantage of that in order to reduce the size of the image even further. After the first request, the image would be cached on Mobify’s CDN for the next time it is needed in that particular size and format. Because this image of the forest does not require any art direction, we can continue using the img element.

You can see an example of automatic image resizing for yourself. Feel free to open your Web inspector to confirm that the original images do not download!

Using this solution, we simply our workflow. We only upload a high-resolution asset for each image, and then sit back and let the the API take care of resizing them automatically. No proxy in the middle, no changing of any attributes — just a single line of JavaScript that is copied to the website. Go ahead and try it out by copying and pasting the following line of code at the top of your head element. (Please note that it must go before any other tag that loads an external resource.)


<script>!function(a,b,c,d,e){function g(a,c,d,e){var f=b.getElementsByTagName("script")[0];a.src=e,a.id=c,a.setAttribute("class",d),f.parentNode.insertBefore(a,f)}a.Mobify={points:[+new Date]};var f=/((; )|#|&|^)mobify=(\d)/.exec(location.hash+"; "+b.cookie);if(f&&f[3]){if(!+f[3])return}else if(!c())return;b.write('<plaintext style="display:none">'),setTimeout(function(){var c=a.Mobify=a.Mobify||{};c.capturing=!0;var f=b.createElement("script"),h="mobify",i=function(){var c=new Date;c.setTime(c.getTime()+3e5),b.cookie="mobify=0; expires="+c.toGMTString()+"; path=/",a.location=a.location.href};f.onload=function(){if(e)if("string"==typeof e){var c=b.createElement("script");c.onerror=i,g(c,"main-executable",h,mainUrl)}else a.Mobify.mainExecutable=e.toString(),e()},f.onerror=i,g(f,"mobify-js",h,d)})}(window,document,function(){var a=/webkit|msie\s10|(firefox)[\/\s](\d+)|(opera)[\s\S]*version[\/\s](\d+)|3ds/i.exec(navigator.userAgent);return a?a[1]&&+a[2]<4?!1:a[3]&&+a[4]<11?!1:!0:!1},

// path to mobify.js
"//cdn.mobify.com/mobifyjs/build/mobify-2.0.0.min.js",

// calls to APIs go here
function() {
  var capturing = window.Mobify && window.Mobify.capturing || false;

  if (capturing) {
    Mobify.Capture.init(function(capture){
      var capturedDoc = capture.capturedDoc;

      var images = capturedDoc.querySelectorAll("img, picture");
      Mobify.ResizeImages.resize(images);
        
      // Render source DOM to document
      capture.renderCapturedDoc();
    });
  }
});
</script>

(Please note that this script does not have a single point of failure. If Mobify.js fails to load, then the script will opt out and your website will load as normal. If the image-resizing servers are down or if you are in a development environment and the images are not publicly accessible, then the original images will load.)

You can also make use of the full documentation. Browser support for the snippet above is as follows: All Webkit/Blink based browsers, Firefox 4+, Opera 11+, and Internet Explorer 10+.

Resizing img elements automatically is great for the majority of use cases. But, as demonstrated in the Obama example, art direction is necessary for certain types of images. How can we continue using the picture element for art direction without having to maintain six versions of the same image? The Image API will also resize picture elements, meaning that you can use the picture element for its greatest strength (art direction) and leave the resizing up to the API.

Resizing <picture> Elements

While automating the sizes of images for different browsers is possible, automating art direction is impossible. The picture element is the best possible solution for specifying different images at different breakpoints, due to the robust branching logic built into its defined syntax (although as mentioned before, srcN is a more recent proposal that offers very similar features). But, as mentioned, writing the markup for the picture element and creating six assets for each image gets very complicated:


<picture>
    <source srcset="responsive-obama-mobile.png 1x, responsive-obama-mobile-2x.png 2x">
    <source srcset="responsive-obama-512.png 1x, responsive-obama-1024.png 2x" media="(min-width: 512px)">
    <source srcset="responsive-obama-1024.png 1x, responsive-obama-1024.png 2x" media="(min-width: 1024px)">
    <source srcset="responsive-obama-2048.png 1x, responsive-obama-4096.png 2x" media="(min-width: 2048px)">
    <noscript><img src="responsive-obama-512.png"></noscript>
</picture>

When using the Image API in conjunction with the picture element, we can simplify the markup significantly:


<picture>
    <source src="responsive-obama-mobile.png">
    <source src="responsive-obama.png" media="(min-width: 512px)">
    <img src="responsive-obama.png">
</picture>

The source elements here will be automatically rewritten in the same way that the img elements were in the previous example. Also, note that the markup above does not require noscript to be used for the fallback image to prevent a second request, because Capturing allows you to keep the markup semantic.

Mobify.js also allows for a modified picture element, which is useful for explicitly defining how wide images should be at different breakpoints, instead of having to rely on the width of devices. For example, if you have an image that is half the width of a tablet’s window, then specifying the width of the image according to the maximum width of the browser would generate an image that is larger than necessary:

tablet-example
In this case, automatically specifying a width according to the browser’s width would create an unnecessarily large image.

To solve this problem, the Image API allows for alternate picture markup that enables us to override the width of each source element, instead of specifying a different src attribute for each breakpoint. For example, we could write an element like this:


<picture data-src="responsive-obama.png">
    <source src="responsive-obama-mobile.png">
    <source media="(min-width: 512px)">
    <source media="(min-width: 1024px)" data-width="512">
    <source media="(min-width: 2048px)" data-width="1024">
    <img src="responsive-obama.png">
</picture>

Notice the use of the data-src attribute on the picture element. This gives us a high-resolution original image as a starting point, which we can use to resize into assets for other breakpoints.

Let's break down how this would actually work in the browser:

  • If the browser is between 0 and 511 pixels wide (i.e. a smartphone), then use responsive-obama-mobile.png (for the purpose of art direction).
  • If the browser is between 512 and 1023 pixels wide, then use responsive-obama.png, because src is not specified in the source element corresponding to that media query. Automatically determine the width because data-width isn't specified.
  • If the browser is between 1024 and 2047 pixels wide, then use responsive-obama.png, because src is not specified in the sourceelement corresponding to that media query. Resize to 512 pixels wide, as specified in the data-width attribute.
  • If the browser is 2048 pixels or wider, then use responsive-obama.png, because src is not specified in the source element corresponding to that media query. Resize to 1024 pixels wide, as specified in the data-width attribute.
  • If JavaScript isn't supported, then fall back to the regular old img tag.

The Image API will run on each picture element, transforming the markup into this:


<picture data-src="http://cdn.mobify.com/mobifyjs/examples/assets/images/responsive-obama-mobile.jpg">
    <source src="//ir0.mobify.com/webp/400/http://cdn.mobify.com/mobifyjs/examples/assets/images/responsive-obama-mobile.jpg">
    <source src="//ir0.mobify.com/webp/400/http://cdn.mobify.com/mobifyjs/examples/assets/images/responsive-obama.jpg" media="(min-width: 512px)">
    <source src="//ir0.mobify.com/webp/512/http://cdn.mobify.com/mobifyjs/examples/assets/images/responsive-obama.jpg" media="(min-width: 1024px)" data-width="512">
    <source src="//ir0.mobify.com/webp/512/http://cdn.mobify.com/mobifyjs/examples/assets/images/responsive-obama.jpg" media="(min-width: 2048px)" data-width="1024">
    <img src="responsive-obama.jpg">
</picture>

The Picture polyfill (included in Mobify.js) would then run and select the appropriate image according to the media queries. It will also work well when browser vendors implement the picture element natively.

See a page that uses the modified picture element markup for yourself.

Using the Image API without Capturing

One caveat with Capturing is that it requires the script to be inserted in the head element, which is a blocking JavaScript call that can delay the initial downloading of resources. The total length of delay on first load is approximately 0.5 seconds on a device with a 3G connection (i.e. including the DNS lookup and downloading and Capturing), less on 4G or Wi-Fi, and about 60 milliseconds on subsequent requests (since the library will have been cached). But the minor penalty is a small price to pay in exchange for being easy to use, backwards-compatible and semantic.

To use the Image API without Capturing to avoid the blocking JavaScript request, you need to change the src attribute of all of your img elements to x-src (you might also want to add the appropriate noscript tags if you're concerned about browsers on which JavaScript has been disabled) and paste the following asynchronous script right before the closing head tag:


<script src="//cdn.mobify.com/mobifyjs/build/mobify-2.0.0.min.js">
<script>
    var intervalId = setInterval(function(){
        if (window.Mobify) {
            var images = document.querySelectorAll('img[x-src], picture');
            if (images.length > 0) {
                Mobify.ResizeImages.resize(images);
            }
            // When the document has finished loading, stop checking for new images
            if (Mobify.Utils.domIsReady()) {
                clearInterval(intervalId)
            }
        }
    }, 100);
</script>

This script will load Mobify.js asynchronously, and when finished loading, will start to loading the images as the document loads (it does not need to wait for the entire document to finish loading before kicking off image requests).

Using The Image API For Web Apps

If you are using a client-side JavaScript model-view-controller (MVC) framework, such as Backbone or AngularJS, you could still use Mobify.js’ Image API. First, include the Mobify.js library in your app:


<script src="//cdn.mobify.com/mobifyjs/build/mobify-2.0.0.min.js"></script>

Then, rewrite image URLs with the method outlined in Mobify.js’ documentation:


Mobify.ResizeImages.getImageUrl(url)

This method accepts an absolute URL and returns the URL to the resized image. The easiest way to pass images into this method is by creating a template helper (for example, {{image_resize '/obama.png' }} in Handlebars.js) that executes the getImageUrl method in order to generate the image’s URL automatically.

Using Your Own Image-Resizing Back End

Images are resized through Mobify's Performance Suite resizing servers, which provides support for automatic resizing, WebP, CDN caching and more. There is a default limit on how many images you can convert for free per month, but if you're driving a large volume of traffic, then give Mobify a shout and we'll find a way to help. The API also allows you to use a different image-resizing service, such as Sencha.io Src, or your own backend service.

How Can Browser Vendors Better Support Responsive Images?

The Webkit team has recently implemented the src-set attribute, and so will Blink and Gecko in the coming months. This is a huge step in the right direction, as it means that browser vendors are taking the responsive image problem seriously. However, it doesn't solve the art direction problem, nor does it prevent the issue of needing to generate multiple assets at different resolutions.

The developer community recently got together to discuss the responsive image problem. One of the more interesting proposals discussed was Client Hints from Ilya Grigorik, which is a proposal that involves sending device properties such as DPR, width and height in the headers of each request. I like this solution, because it allows us to continue using the img tag as per usual, and only requires us to use the picture (or srcN) when we need branching logic to do art direction. Although valid concerns have been raised about adding additional HTTP headers and using content negotiation to solve this problem. More importantly, for established websites with thousands of images, it may not be so easy to route those images through a server that can resize using the headers provided by Client Hints. This could be solved by re-writing images at the web server level, or with a proxy, but both of those can be problematic to setup. In my opinion, this is something we should be able to handle on the client through greater control over resource loading.

If developers had greater control over resource loading, then responsive images would be a much simpler problem to tackle. The reason why so many responsive image solutions out there are proxy-based is because the images must be rewritten before the document arrives to the browser. This is to accommodate the pre-parser's attempt to download images as quickly as possible. But proxies can be very problematic in their security and scalability and, really, if we had an easy way to interact with the pre-parser, then many proxy-based solutions would redundant.

How can we get greater control over resource loading while still getting all of the benefits from the pre-parser? The key thing here is that we do not want to simply turn off the pre-parser — its ability to download assets in parallel is a huge win and one of the biggest performance improvements introduced into browsers. (Please note that the Capturing API does not prevent parallel downloads.) One idea is to provide a beforeload event that fires before each resource on a page has loaded. This event is actually available when one uses Safari browser extensions, and in some browsers it is available in a very limited capacity. If we could use this event to control resource loading in a way that works with the pre-parsing thread, then Capturing would no longer be needed. Here is a basic example of how you might use the beforeload event, if it worked as described:


function rewriteImgs(event) {
    if (event.target === "IMG") {
        var img = event.target;
        img.src = "//ir0.mobify.com/" + screen.width + "/" + img.src;
    }
}
document.addEventListener("beforeload", rewriteImgs, true);

The key challenge is to somehow get the pre-parser to play nice with JavaScript that is executed in the main rendering loop. There is currently a new system being developed in browsers called the Service Worker, which is intended to allow developers to intercept network requests in order to help build web applications that work offline. However, the current implementation does not allow for intercepting requests on the initial load. This is because loading an external script which controls resource loading would have to block the loading of other resources — but I believe it could be modified to do so in a way that does not sacrifice performance through the use of inline scripts.

Conclusion

While there are many solutions to the problem of responsive images, the one that automates as much work as possible while still allowing for art direction will be the solution that drives the future of Web development. Consider using Mobify.js to automate responsive images today if you are after a solution that does the following:

  • requires you to generate only one high-resolution image for each asset, letting the API take care of serving smaller images based on device conditions (width, WebP support, etc.);
  • makes only one request per image;
  • allows for 100% semantic and backwards-compatible markup that doesn't require changes to your back end (if using Capturing);
  • has a simplified picture element that is automatically resized, so you can focus on using it only for art direction.

(Front page image credits: Creating High-Performance Mobile Websites)

(al, ea, il)


© Shawn Jansepar for Smashing Magazine, 2013.

Tags: Mobile

October 17 2013

11:40

Best Of Both Worlds: Mixing HTML5 And Native Code


  

Much has been written recently in the ongoing debate between native and HTML5 applications. There are three principal ways to develop a mobile solution:

  • native code,
  • hybrid mobile app,
  • mobile Web app.

Developing an application in HTML5 is a way to leverage code across multiple platforms, rather than having to write the entire application from scratch for each platform. As such, much of the user interface, perhaps the entire interface, would be done in HTML.

“Hybrid application” is a term often given to applications that are developed largely in HTML5 for the user interface and that rely on native code to access device-specific features that are not readily available to Web applications. Much of this native code is non-visual in nature, simply passing data back to the HTML5 layer of the app, where it is rendered to the user. Libraries such as PhoneGap provide this capability.

html5-native-arrows_500_mini

Much of the debate is not about whether HTML5 is up to the task of powering a mobile application’s user experience. Mark Zuckerberg’s comments about some of the difficulties Facebook has encountered with HTML5 on mobile are well known. Some of Facebook’s issues could likely be addressed through the techniques outlined by Sencha in its Fastbook application and by LinkedIn in its infinite scrolling post. Despite the latter article, LinkedIn subsequently switched directions towards using more native code.

Why Not Mix It Up?

Rather than build an app entirely with native or HTML5 technology, why not mix and match the technologies? With a hybrid application, building a mobile experience that leverages both native and HTML5 code for the user interface is quite possible. This enables the developer to use the most appropriate tool for the job when developing the user interface.

Clearly, developing portions of the user experience in two or more different technologies has some downsides. Chiefly, you would need developers who can handle both native development and HTML5. The portion of the user interface in native code couldn’t be readily used on other platforms and would need to be redeveloped. Given the broader scope of knowledge required by this technique and the difficulties outlined above, why would anyone want to embark on an endeavor that mixes user-interface technologies?

HTML For Rich Document Layout

Perhaps your user experience dictates that documents be displayed in the application with rich formatting. While text formatting can be accomplished on platforms like iOS with NSAttributedString, this document structure is not readily portable to other device platforms, nor does it provide the full complement of features found in HTML. If the document requires formatted content such as tables to be displayed, then a solution like NSAttributedString won’t be up to the task.

Consider the iPad application below, which was developed for a genetic sequencing company. Here we have an example of a hybrid UI:

voyager-screenshot_500_mini
(See larger view.)

Just looking at this application, it’s not readily apparent what has been done in native code and what portions are HTML5. Users of the application would be hard pressed to tell the difference. The document along the right side is of variable length. The table contains expandable rows, complete with CSS3 animations, and other text sections in the document vary in height as well.

Documents such as this are what HTML does best. Trying to create something like this in native code would have entailed considerably greater effort, with little benefit to the user. In this application, the HTML5 is generated with a mustache template, via the GRMustache library for iOS.

voyager-screenshot-shaded_500_mini
The illustration above shows the technologies used to draw each portion of the screen. (See larger view.)

Note that as the user scrolls up, the native document header near the top on the right dynamically resizes to show more content, thus illustrating how portions of the user interface can interact with each other. The user can tap on links and buttons in the HTML document area to pop up an internal Web viewer to load related documents. As we’ll see later in this article, creating this linkage between native and HTML experiences is relatively easy.

Desire For A “Best Of Breed” Approach

As we’ve seen, a given screen may readily mix the two technologies to provide a seamless experience. You may wish to use a native UINavigationController, with its associated UINavigationBar, on iOS, or use the action bar in an Android application. The remainder of the screen may be rendered in HTML5.

Below is an application that uses HTML5 for its shopping-cart experience, but retains the native navigation controller and tab bar for moving between pages of the workflow:

gallery-prints-1_500_mini
(See larger view.)

gallery-review-order_500_mini
(See larger view.)

Here, the HTML5 content is actually loaded from a server, and it controls the workflow throughout the experience. A defined API allows the server-supplied HTML content to modify the contents of the navigation controller bar to suit the page in the workflow.

You may find that native controls, such as MapKit on iOS, provide superior performance to what can be achieved in HTML5. In this case, your HTML5 code could communicate with the iOS native layer and ask it to present the native content over or adjacent to the HTML content.

gallery-map-pin_500_mini
(See larger view.)

gallery-loc-selected_500_mini
(See larger view.)

The examples above illustrate a native store-locator experience, displayed over top of the HTML5 workflow. Upon the user selecting a store location, the data would populate the HTML5 content, and the user would continue their workflow.

Once the user has completed interacting with the map, the results of their interaction may be passed back to the HTML5 document for further handling. Depending on how you wish to structure the application, native “widgets” may be implemented for each platform (iOS, Android, Windows, etc.) and then invoked from the common core HTML5 application. In this strategy, the HTML5 code would provide the workflow for the application, and the native content would sit in the background, waiting to provide the supplementary experience when needed.

If done properly, the user would have little indication of which portions are done in native code and which in HTML.

Ease Of App Updates

Most people are aware that in order to update the native code on an iOS or Android app, it is necessary to resubmit the application through the normal publishing channels for that platform. In the case of iOS, it can take as long as seven days to get app updates approved by Apple. This makes it more difficult to time app updates to marketing requirements and to rapidly respond to changing business needs. Exposing a portion of the user experience through HTML and JavaScript means that some of the experience can be served from the Web and thus be dynamic.

As we saw earlier, it’s possible for this Web-served content to invoke native code in your application via a “bridge” that you create. This is much the same as the bridging mechanism that allows HTML code to communicate with native code in PhoneGap via the PhoneGap plugin interface. The example application shown earlier uses this technique. This approach was chosen chiefly so that the purchasing workflow could be altered without having to redeploy the native application. A page served from the Web may now seamlessly display a natively rendered map using whatever mapping capability is present on the device. A well-defined interface between your Web page and the surrounding native container makes this possible.

Watch That Content!

For content coming from the Web, it’s generally best to restrict the traffic that can flow over this bridge. The built-in functionality of something like PhoneGap provides access to a wide range of device information, some of which may be well beyond your application’s purpose. If that Web content is compromised for some reason, through something like cross-site scripting, you might suddenly find this content invoking native code in a manner you never intended.

For this reason, when bridging from Web-sourced content to native code, use your own bridge code in order to tightly control exactly what the dynamically loaded content may invoke. Some platforms have additional controls to limit what native code may be invoked from a Web view. On Android, classes must explicitly register themselves as being JavaScript-accessible. On Android 4.2, this can be further restricted to the individual methods by adding an @JavaScriptInterface annotation on the methods that you wish to access from a Web view.

Things to consider

  • Pulling content directly from a Web server is fine if the user is online. Remember that these are mobile applications with the potential for connectivity issues. What happens if network connectivity is interrupted? Some offline capability could be retained through the use of AppCache, but this needs to be planned for. Another approach will be covered in an upcoming section where updates to the HTML content are stored locally.
  • Hosting content on a Web server requires some infrastructure. There are now more “components” that must be considered as part of your complete app solution. It’s no longer just some native code that’s been deployed via the AppStore and potentially a set of backend services. Care must be taken that site updates do not conflict with the existing interface in the mobile applications. Keep in mind that if site changes necessitate an update to the mobile app itself, not everyone will necessarily update their copy of your app quickly, if at all. For this reason it may become necessary to version the site content so that it may be consumed with older versions of your mobile app. This is very much like the versioning built into some REST based interfaces in order to support compatibility with older client applications.

  • Apple takes a dim view of applications that are merely a Web view wrapped in a native app. Submissions to the iOS AppStore will likely not be approved if you’ve done nothing more than wrap some Web content in a native app. Apple expects your application will make use of some device features that cannot be otherwise accomplished in Mobile Safari. Consider if your application can benefit by integrating with the camera, contact list, local data storage, offline operation, etc. Also remember that the content served in these Web views needs to remain consistent with the original submitted intent of the application.

    If you create a word-search game and then suddenly start serving totally unrelated content in your webviews this will violate Apple’s terms and the app could be pulled from the AppStore. In accordance with the iOS developer agreement, updates may not change the primary purpose of the application by significantly altering the features or functionality of the application as submitted to the App Store.

Pushing Updates To Your App Experience

A variation on the loading of Web content as outlined above is to refresh local content with updates from the Web. In this approach, the Web content (HTML, CSS and JavaScript) is bundled on the server and downloaded to the mobile app client whenever the content is updated. This updated code can now change the user interface, application flow and so on, without the developer needing to update the app via the iOS App Store or Google Play.

Note that you can only update the HTML, CSS and JavaScript in this manner; native code cannot be updated. Apple allows JavaScript code to be downloaded and executed as long as it runs within the context of the UIWebView control in your iOS application. This is the only situation in which code “downloading” is allowed in iOS.

amway-android_500_mini
(See larger view.)

The application above is running on Android. The HTML content has been downloaded to the client as a bundle of content and is now being served locally. Note the use of the action bar to provide a navigation experience that is familiar to Android users. Below is the same application running on iOS, where a UINavigationController is used:

amway-ios_500_mini
(See larger view.)

In addition to speeding up the updating process, this manner of transmitting app changes ensures that users will be running the latest code. In the traditional approach, native app updates are distributed via an application store, and the user is informed that an update is available, although they are not required to install the latest update. Not everyone updates apps regularly. Updating local HTML, CSS and JavaScript via a dynamically deployed update bundle mitigates this issue.

The ability to dynamically update local content is actually part of a commercial hybrid mobile-development platform called Trigger.IO. The Trigger.IO Reload feature permits you to push updates to the HTML portion of your application. Note that native code cannot be updated in this manner. However, if much of your application’s flow is controlled with JavaScript and augmented with native code, then making substantial changes to your application’s workflow dynamically is possible.

Keep in mind that, according to the iOS developer agreement, these updates may not change the primary purpose of the application by significantly altering the features or functionality of the application as submitted to the App Store. As such, major releases would likely still have to be carried out in the traditional manner.

Dynamically Reloading Content For Development And Testing

The ability to dynamically load updates to an app could also accelerate the development process. The native development process on both iOS and Android generally requires application code to be compiled on a desktop computer and then transferred to the device for testing. Some development programs, such as Icenium, with its LiveSync and Ion tools, compile app updates in a cloud environment and then simply download the update to a native app that is already running on your device.

icenium-ion_500_mini
Icenium Ion is reloading updated content. (See larger view.)

As code changes are made and saved on the server, a simple three-finger tap-and-hold gesture will update the app on an iOS device. Icenium makes a similar capability available for Android. Adobe’s PhoneGap Build makes updates possible via a feature known as Hydration. In this case, a developer can instantly reload any changes they make in their IDE onto a variety of devices without having to physically connect the devices to a development machine and then push the code onto the devices.

Each time the application is started with Hydration enabled, the application will check the PhoneGap Build server to see whether a new version is available. If so, the user is prompted to update to the new version. This can make quality assurance of your application easier as well. You won’t have to worry about signing code updates, provisioning and so on if the person testing your app already has the native “shell” application installed.

Be Careful

While the appeal of dynamic application updates is strong, keep in mind that for content sourced from the Web, you must implement some controls to ensure that downloaded code cannot simply call any native method in your app. Failing to do so could expose your user’s data to a third party and violate your agreement with Apple or Google. In addition, updates loaded dynamically must not significantly change the purpose or functionality of the app. Failing to adhere to this would likely result in your application being removed from the iOS App Store or Google Play. Treat these techniques as a way to enhance the user experience and to deliver bug fixes and incremental features quickly, not to thwart the App Store’s approval process.

A Bridge Between Two Worlds

Both iOS and Android offer a Web view control that you can place in a native application. In the section below, I’ll offer some code examples to illustrate how JavaScript in a Web view can communicate with native code and how native code can invoke methods on the JavaScript in the Web view.

phonegap-logo_500_mini
(See larger view.)

Before we look at code, keep in mind that you also have the option to use PhoneGap (Apache Cordova) within an existing application with native controls. Starting with PhoneGap 1.4 for iOS and PhoneGap 1.9 for Android, placing a CordovaWebView into a native ViewController on iOS or into an activity on Android is possible. This permits your Web code to access the entire PhoneGap API and to access any PhoneGap plugins. You could, of course, follow the PhoneGap plugin API to expose your own native code to the CordovaWebView.

Assuming you don’t need the full capabilities of PhoneGap and you simply need to expose a small portion of functionality between the two worlds, implementing your own bridge with a relatively small amount of code is possible. The three applications referenced earlier in this document were all built in this manner using a “private API.” This further ensures that you tightly control the API that governs which portions of native functionality may be accessed by the Web view.

Bridging From iOS To A UIWebView

The UIWebView in iOS and its associated UIWebViewDelegate provide the mechanism that allows native code and JavaScript to communicate with each other. I have prepared a simple iOS application to show how a form in a UIWebView can communicate with a form in native code:

ios-bridge-example_500_mini
(See larger view.)

Note that the upper portion of this screen, with the light-gray background, is rendered by the UIWebView. The lower portion, with the white background, is rendered natively.

This application simply passes a JSON object from the Web view to native code, and vice versa. The code for this and an Android version are available in a GitHub repository.

Invoking Native iOS Code From JavaScript

My standard technique for moving data from the JavaScript to native code is to invoke a native execution method with a command name and pass along data in the form of a JSON string containing the arguments associated with the command.


var nativeBridge = {
   invoke: function (commandName, args) {
      console.log(commandName + ": " + JSON.stringify(args, null, 2));
      window.location = 'js-call:' + commandName + ':' +
                      encodeURIComponent(JSON.stringify(args));
  }
};

This code attempts to navigate the browser to a new URL, but not one with a normal http:// or file:// URL scheme. In this case, we are using a custom protocol scheme of js-call:. On the Objective-C side of things, we’ll trap this attempt to navigate in our UIWebViewDelegate:


(BOOL)webView:(UIWebView *)webView shouldStartLoadWithRequest:(NSURLRequest *)request navigationType (UIWebViewNavigationType)navigationType

Inside the delegate for this call, we can interrogate the incoming URL to see whether it’s a call from our JavaScript:


NSString *requestURLString = [[request URL] absoluteString];
    if ([requestURLString hasPrefix:@"js-call:"]) {

Now we can pull apart the request to determine the command name and the JSON arguments string:


NSArray *components = [requestURLString componentsSeparatedByString:@":"];
NSString *commandName = (NSString*)[components objectAtIndex:1];
NSString *argsAsString = [ (NSString*)[components objectAtIndex:2] stringByReplacingPercentEscapesUsingEncoding:NSUTF8StringEncoding ];

Now that we have the arguments as a string, we can turn them into an NSDictionary object:


NSError *error = nil;
NSData *argsData = [argsAsString dataUsingEncoding:NSUTF8StringEncoding];
NSDictionary *args = (NSDictionary*)[NSJSONSerialization JSONObjectWithData:argsData options:kNilOptions error:&error];

With everything decoded, it’s now simple to invoke the right native method to properly take the data that we’ve received from JavaScript and update the native UI:


if ([commandName isEqualToString:@"updateNames"]) {
   [self updateNativeNameValuesWithFirstName:[args objectForKey:@"fname"]
                                    LastName:[args objectForKey:@"lname"]];
}

When successfully handling a call via the js-call URL scheme, we must remember to return a boolean NO value from our delegate to prevent the Web view from actually trying to visit the desired URL.

Invoking JavaScript From iOS Native Code

This approach is a bit more straightforward because we don’t have to use a delegate method. We can simply call the desired JavaScript function by passing the appropriate JavaScript method’s name and required arguments as a string value to the UIWebView’s stringByEvaluatingJavaScriptFromString method.

Here, we take some values from the native view and turn them into a JSON object via an NSDictionary:


NSDictionary *namesDict = @{@"fname": self.fNameTextField.text, 
                            @"lname": self.lNameTextField.text};
NSError *error;
NSData *namesData = [NSJSONSerialization dataWithJSONObject:namesDict 
                                                    options:0 
                                                      error:&error];
NSString *namesJSON = [ [NSString alloc] initWithData:namesData
                                             encoding:NSUTF8StringEncoding] ;

Now that we have the JSON values as a string, we can simply format a command and volley it over the wall to the Web view for execution:


NSString *jsCommand = [NSString stringWithFormat:@"setNames(%@)", namesJSON];
[self.webView stringByEvaluatingJavaScriptFromString:jsCommand];

As simple as that, we’ve roundtripped between JavaScript and Objective-C. This basic technique could, of course, be leveraged to build more complex interactions.

Bridging From Android To A Web View

Below is a screenshot of the Android application running the same HTML and JavaScript code that we used in the native iOS example.

android-bridge-example_500_mini
(See larger view.)

As with the iOS example, the upper portion of this screen, with the light-gray background, is rendered by the Web view. The lower portion, with the white background, is rendered natively. The appearances of the iOS example and the Android example differ because we are using the default component styles for each platform. In addition, the Android example contains an action bar, which Android users would expect to find in an application.

Invoking Native Android Code From JavaScript

Android Web views have the benefit of being able to expose an entire Java class to JavaScript. There’s no need to trap an attempt to navigate off a Web view via some sort of custom URL scheme, as we did with iOS. First, we must signal to the Android Web view that it’s OK for JavaScript to execute on the Web view. I do this when loading the Web view content:


@Override
public void onStart() {
   super.onStart();
   WebSettings webSettings = webview.getSettings();
   webSettings.setJavaScriptEnabled(true);
   webview.loadUrl("file:///android_asset/webviewContent.html");
   webview.addJavaScriptInterface(new WebViewInterface(this), "Android");
}

Note the last line in the code above. We are registering a particular class, in this case WebViewInterface, to be accessed via JavaScript. In addition, this class is being namespaced by the string Android. Thus, in our JavaScript we can test whether the namespace Android is defined and, if so, invoke methods on this class, like so:


if (window.Android) {
   Android.updateNames(JSON.stringify(nameData));
}

On the Java side, the WebViewInterface is a standard Java class. If you are targeting the Android 4.2 SDK, then all methods to be exposed to JavaScript must be decorated with the @JavaScriptInterface attribute:


public class WebViewInterface {
   Context mContext;

   /** Instantiate the interface and set the context */
   WebViewInterface(Context c) {
       mContext = c;
   }

   @JavaScriptInterface
   public void updateNames(String namesJsonString) {
      Log.d(getPackageName(), "Sent from webview: " + namesJsonString);
      try {
         JSONObject namesJson = new JSONObject(namesJsonString);
         final String firstName = namesJson.getString("fname");
         final String lastName = namesJson.getString("lname");

         // When invoked from JavaScript, this is executed on a thread 
         // other than the UI thread.
         // Because we want to update the native UI controls, we must 
         // create a runnable for the main UI thread.
         runOnUiThread(new Runnable() {
            public void run() {
               fnameEditText.setText(firstName);
               lnameEditText.setText(lastName);
             }
         });
      } catch (JSONException e) {
         Log.e(getPackageName(), 
            "Failed to create JSON object from Web view data");
      }
   }
}

Note that when the methods on this class are invoked by JavaScript, they are not executed on the main UI thread. Thus, if our intention is to update any part of the UI, we must create a new runnable and post it for execution on the main UI thread, like so:


runOnUiThread(new Runnable() {
   public void run() {
      fnameEditText.setText(firstName);
      lnameEditText.setText(lastName);
   }
});

Invoking JavaScript from native Android code is essentially similar to how we call JavaScript from Objective-C. We load the JSON representation of the data into a string and then form a JavaScript function call. One difference is that the function reference must be prefixed with the JavaScript: protocol scheme:


public void sendNamesToWebView() {
   JSONObject namesJson = new JSONObject();
   try {
      namesJson.put("fname", fnameEditText.getText().toString());
      namesJson.put("lname", lnameEditText.getText().toString());
      webview.loadUrl( "JavaScript:setNames(" + namesJson.toString() + ")" );
   } catch (JSONException e) {
      Log.e(getPackageName(), 
        "Failed to create JSON object for Web view");
   }
}

That’s all there is to roundtripping JSON data between JavaScript and native Android code. Both sample applications are available in the GitHub repository for your reference.

Web Views: What’s The Downside?

Like most things in software development, there must be some caveats to developing with Web views. Keep these in mind when considering the hybrid approach and determining which portions of your application to do with which technologies.

  • Not all Web views render the same
    The capability of Web views has improved markedly in the last two years. However, your user base may not be running the latest and greatest version of Android or iOS. The Android platform has generally lagged behind the iOS platform in the capability and performance of its Web view. Google appears to be putting in a renewed effort to improve that situation, but users stuck on old versions of Android (especially releases before Ice Cream Sandwich 4.x) might find that the performance of Web views, especially those with complex layouts, do not match what is achievable on iOS. Developers will need to test the HTML5 code on Android 2.2, 2.3 and 4.x to ensure that content renders consistently. Reserve sufficient time in your project to perform adequate testing across all mobile OS versions that you intend to support.
  • Performance
    Web views are typically not as performant as native code. However, in many situations, you may find the performance to be perfectly acceptable, particularly on current hardware. Keep in mind that bridging calls between the native and Web view portions of your code will exact a performance toll. You won’t want to make calls across this bridge at sub-second intervals, but for most applications this is not an issue.
  • Scrolling behavior
    Users might be able to distinguish between the way Web views scroll and the way native portions of the app scroll. Some of this can be mitigated in new versions of WebKit with the CSS3 -webkit-overflow-scrolling: touch property. You might also wish to look at solutions such as FTScroller and iScroll.
  • Complexity
    As mentioned earlier, if you will be serving some content from Web servers you must ensure you have the infrastructure necessary to support this. Because your application now spans native code, Web servers, and potentially backend services there are more points of failure. When making Web content changes you must perform backwards compatibility testing against earlier deployed versions of the application to ensure existing versions of your application still running on users devices will continue to perform as expected.
  • Developer Agreement Compliance
    Familiarize yourself with the restrictions each app marketplace might impose on content provided from the Web. As this article identified there are some notable restrictions, particularly on the iOS platform, on what your app is permitted to do with content sourced from the Web. Failure to take these restrictions into account could result in your app not being approved for release, or an existing app being pulled from the AppStore.
  • Security
    As stated earlier content sourced from the Web should be treated with some suspicion, particularly if it will be interacting with an interface to your native code. Ensure you’ve exposed only those portions of your native code that are truly required for interaction with the Web layer. Failure to do so could result in some consequences should compromised Web content be able to freely into call your native code. Make sure you take standard Web security practices into account to ensure your Web content cannot be compromised to begin with.

Given the flexibility afforded by a hybrid solution, you have the freedom to write portions of your application in native code if you find that the performance of the Web view is insufficient for a given interaction. Of course, the more native code you write, the more code will have to be rewritten for every platform you intend to support. It’s a trade-off; however, a hybrid solution puts you in control of the application’s flexibility and performance.

Summary

The use of HTML5 in mobile app development is a somewhat controversial subject. However, it’s important to understand the benefits each potential development strategy affords. As an app developer you have the ultimate decision on what strategy best suits the needs of your application. Despite some of the negative publicity that mobile HTML5 solutions have received in recent months, this article shows how HTML5 is still a valuable part of app development.

Keep performance, and security in mind when using any application development platform. Test early, and test often, across a variety of devices. As we’ve seen in this article, HTML5 brings with it many benefits that extend far beyond the convenience of sharing code across platforms. Used properly, the hybrid approach of marrying native and HTML5 code can create some truly unique and flexible solutions that would not otherwise be possible. Keep your technology choices open and flexible to reap the rewards of a hybrid experience.

(al ea)


© Peter Traeg for Smashing Magazine, 2013.

Tags: Mobile

October 08 2013

12:56

Lightening Your Responsive Website Design With RESS


  

Editor’s Note: This article features just one of the many solutions for creating high-performance mobile websites. We suggest that you review different approaches such as Building A Responsive Web App, Improving Mobile Support and Making Your Websites Faster before choosing a particular solution.

This article explains how to use RESS (responsive design with server-side components) to make significant performance and reach improvements to a website for both mobile and desktop devices alike. This technique requires just a few lines of code, some simple configuration and no ongoing maintenance.

Your website will change from one that works on desktops, tablets and smartphones to one that works on almost anything anywhere and loads faster in all cases. It’s hard to over-emphasize the importance of this, but if you need a good case study, read about what happened to YouTube when Google lightened its pages (spoiler: entire new territories opened up to it).

The following three screenshots show a sample website with increasing levels of RESS optimization being applied to it and the resulting overall page sizes:


Original: 1,027 KB (large preview)


Partial optimization: 253 KB (large preview)


Full optimization: 153 KB (large preview)

The discerning reader will note that the screenshots all look the same, and that’s the whole point: The optimization procedure we’ll be applying ensures that the optics of the website remain intact, while improving the user experience significantly due to the smaller page size. In fact, the screenshots are not exactly the same (click to view them at native size), but the slight differences in the quality of the images tend to go unnoticed on phone screens.

Language issues aside, making a website truly accessible worldwide usually comes with two problems:

  • Diversity of devices
    Remember what the first two W’s in WWW stand for? A truly global website should work on any devices that your customers use, with varying connectivity standards, world-wide.
  • Constrained connectivity
    Many places are not well connected to the Internet, whether in terms of capacity, data plans, etc. This problem affects people living in modern well-connected cities as much as people in less developed rural areas.

This article will help your website break through this glass ceiling (or floor) and enable everyone everywhere to reach your content with equal ease. Note that we’ll be using open-source software in conjunction with a device-detection library to achieve this goal; there are many device-detection solutions out there (such as WURFL, OpenDDR and DeviceAtlas — I was involved in developing the latter one).

We’ll demonstrate three levels of optimization, each building on the previous one, but each of which can be implemented separately, too. These optimizations will take on the following:

  1. Reduce image payload (the biggest effect),
  2. Reduce JavaScript and CSS payload,
  3. Further optimize based on bandwidth detection.

Before we begin, let’s confirm why this process is necessary.

Background On Responsive Web Design And RESS

Responsive Web design is fast becoming the preferred method for making a website mobile-friendly, and with good reason. For many websites, it achieves a reasonable balance between mobile-friendliness and ease of implementation.

But design isn’t just about appearance, and the main problem that we all are struggling with in Responsive Web design is that by “default” (when using the simplest way to display and hide containers with display: none;) all devices are sent the same payload. It means that devices with low-resolution screens are sent the same images as those sent to high-resolution devices, even though they can’t show them at their native resolution. This is inefficient at best and market-limiting at worst. The result is a page that works well on high-end well-connected devices with generous data plans.

Many attempts have been made to solve this problem, with varying levels of effectiveness. Probably the most promising approach has been to have the browser determine the most appropriate image version to fetch. The W3C has an ongoing initiative to standardize an approach, but this likely won’t be commonplace in browsers within the next year or so. The candidate solutions — the new <picture> element and new srcset attribute for the <img> element — each has its own issues and benefits. Rather than discuss them here, I suggest reading Jason Grigsby’s “The Real Conflict Behind <picture> and @srcset.” In the meantime, polyfills such as Boris Smussrcset-polyfill will mimic some of the proposed attribute’s behavior.

We’ll describe a relatively straightforward approach to minimizing page weight in RWD using a touch of server-side magic. We’ll use device detection on the server to optimize images when they’re requested by the browsers by reducing their dimensions to the optimum size.

Some people use Sencha.io Src (neé TinySRC) and other image-resizing libraries, but doing this yourself is just as easy, and then you’ll have total control over the result. Yes, this does exchange one external dependancy for another, but it also enables you to optimize much more than images. We’ll use PageSpeed for this purpose, an open-source project from Google.

PageSpeed is a Web server module that applies multiple best practices to a website without forcing the developer to change their workflow. These optimizations include combining and minifying JavaScript and CSS files, inlining small resources, removing unused meta data from each file, and re-encoding images to the most efficient format accessible to the user. PageSpeed is available for both Apache and NGINX. We’ll take advantage of a little-used image-resizing feature of PageSpeed.

The resulting weight savings are dramatic and require very little work. If you follow the steps in this article, you should be able to cut down on image weight many times over with three easy steps, only four lines of code and one line of configuration.

This article assumes that you are using the Apache Web server and are comfortable with some light PHP coding, but the techniques will work with NGINX also, and any programming language.

The Website

Our sample website is based on Twitter’s open-source Bootstrap Web framework, to save the world from my design skills. I created a single-page “website” for a fictional mobile phone store. The page is visually rich, with an industry-average breakdown of HTML, images and JavaScript. This page is based on a lightly modified version of Bootstrap’s carousel template. Here is the page in its entirety, as you would see it in a desktop browser:

full website image

The website has an approximately industry-average payload that breaks down as follows:

Component Size on disk HTML 12 KB Image 941 KB (73%) JavaScript (mostly minified) 159 KB CSS 170 KB Total 1,281 KB

Instructions

Step 1

Install PageSpeed. This is best done by following Google’s instructions. The installation process will usually activate the module for the default website, but you might need to ensure that it works with your virtual hosts, if they’re configured. Basically, you just have to add a line to each one, or get them all to inherit from the default configuration server-wide.

Next, restart your Web server.

Step 2

Get a device detection library set up. In our case, if you are using DeviceAtlas (which we’ll use just as an example here), you’ll need to enter your license key in the DeviceAtlasCloud/Client.php file after you unpack the ZIP file. We then can use DeviceAtlas to determine the optimal size for images.

Step 3

Copy DeviceAtlas’ PHP file to a directory where it is executable by the Web server. In this case, I’ve created a directory in the root of the website, named DeviceAtlasCloud. Enter the following code at the top of your HTML file or website template to set up a couple of variables that we can use throughout the page. If you’re using a different solution, then the syntax will vary but the same properties should be available.


<?php
  include 'DeviceAtlasCloud/Client.php'; // instantiate client
  $results = DeviceAtlasCloudClient::getDeviceData(); // fetch props 
  $props = $results['properties']; // store result
  // set $width to correct width or "" if unknown 
  $width = (isset($props['displayWidth'])) ? $props['displayWidth'] :""; 
?>

The impact of this fetching of properties should be no more than a few milliseconds if everything is set up correctly as most solutions will cache data.

Step 4

The final step is to give all of your images that might need resizing a width attribute, set to use the $width variable:


<img src="http://media.smashingmagazine.com/wp-content/uploads/2013/07slide-01.jpg" width="<?php echo $width; ?>" alt="image description" />

Thus, images will now have their width attribute set from the $width variable, which is set automatically to the maximum display width for each device. Then, PageSpeed will notice the width="…" tag for each image and scale it down if necessary, replacing the image source’s attribute with a reference to a resized version of the same thing. There is no need to set the height attribute because PageSpeed automatically retains the aspect ratio. Resized images are cached, so there isn’t really any significant impact on the server. Refer to the PageSpeed configuration notes below for more fine-grained control of this cache.

Add this variable width tag only to images that will need to be resized for each device; that is, don’t add it to images that are already small enough (such as bullets and icons). Manually resizing images that require special attention might also be wise. Also, be aware that the width attribute for each image will need to play nice with any CSS used for image styling. Anyone using a content management system (CMS) might have to use a different technique for this, depending on how much access the CMS gives them to the underlying HTML.

Background images might require a different approach, depending on how they are implemented on the website. Still, PageSpeed will read inline style="…" tags.

Result

With those changes made, it’s time to see how the website looks. To measure the impact of this step, I tested the download speed of our website for different devices and network speeds, using the ever-useful Charles Proxy as well as real devices that I forced into various network configurations to reduce available bandwidth.

Before making the changes, the page’s overall size was 1,027 KB, regardless of device (i.e. a dynamic range of exactly 1.0). The breakdown is as follows:

Component Size on disk Size over network HTML 12 KB 3 KB Images 941 KB 941 KB (73%) JavaScript 159 KB 55 KB CSS 170 KB 27 KB Total 1,281 KB 1,027 KB

This is the over-the-network size of the page, thanks to gzip compression from Apache. The RWD makes the page look OK on small screens, but it isn’t an efficient use of network resources because the original images are about five times wider (in pixels) than the average phone screen and, hence, impossible to display at full resolution without panning. In fact, the sheer size of the page means that it doesn’t even finish loading on some devices.

After following the above steps, the payload breakdown for an iPhone is this:

Component Size over network HTML 3 KB Images 177 KB JavaScript 51 KB CSS 21 KB Total 253 KB

So, the page has dropped to one fourth of its original size, with image sizes accounting for the majority of this reduction. On devices with lower-resolutions screens, further gains are possible; the same page on a Nokia feature phone (Nokia 6230) now has a total image weight of just 89 KB, a large saving compared to the original. Importantly, the before and after websites have no perceptible difference to the user; the image data that was removed cannot easily be seen by the mobile user.

After following the four steps outlined here, the page’s weight now varies between 1,027 KB and about 150 KB. In other words, the page has gone from having a dynamic range factor of exactly 1.0 to a decent 6.8, with the image payload shrinking from 941 KB to just 80 KB. This will have a huge impact on real-world customers:

  • Much faster page-loading times lead to better engagement and fewer drop-offs.
  • A smaller impact on the user’s data plan leads to more return visits.
  • Wider device and network compatibility leads to improved reach.

Here’s the loading-time effect on a Retina iPhone using 3G and 2.5G networks:

Device and network Before After iPhone 3G 14s 6s (2.3× faster) iPhone GPRS 2m 30s 35s (4.3× faster)

The results on Android devices are similar. On lower-end devices with smaller screens, more dramatic improvements are possible because the image-resizing gains are greater.

Going Further, Part 1: JavaScript And CSS

So far, we’ve looked only at the main source of bloat on pages: images. But screen size shouldn’t be the sole factor in our methods — user contexts and constraints demand more of a multi-device publishing strategy because many additional optimizations are to be made. For example, if you know that the requesting device doesn’t support JavaScript or rich CSS, then ditching them might make sense. This is quite straightforward to accomplish.

If we add another line of PHP to the top of our HTML file, we can do a little more.


$highEndDevice = (isset($properties['browserRenderingEngine']) && in_array($properties['browserRenderingEngine'], array('Gecko', 'Trident', 'WebKit', 'Presto')));

We are making the determination of a low-end device based on its rendering engine. This is a crude rule but good enough to demonstrate a point. Based on this variable, we can now selectively include some resources only if they’re beneficial. In this case, if the device appears to be low end, we’ll jettison the CSS and JavaScript, because low-end phones will have problems with both the file size and the rendering of this CSS, and they usually won’t run the JavaScript:


<?php if ($highEndDevice): ?>
    <link href="css/bootstrap.css" rel="stylesheet">
    <link href="css/bootstrap-responsive.css" rel="stylesheet">
    <link href="css/additional.css" rel="stylesheet">
<?php endif; ?> 

And now the JavaScript:


<?php if ($highEndDevice): ?>
    <!-- Le javascript
    ================================================== -->
    <!-- Placed at the end of the document so the pages load faster -->
    <script src="js/jquery.js"></script>
    .
    .
<?php endif; ?>

This saves a further 72 KB (quite a reduction from 253 KB!) and actually makes the page look better on low-end devices (see the screenshot at the end of this article), in addition to being quicker to render. Now, when viewed on a low-end device, the page’s weight is as follows:

Component Size over network HTML 3 KB Images 50 KB JavaScript 0 KB CSS 0 KB Total 53 KB

This means that our simple RWD page has gone from a fixed size of about 1 MB to a highly varying one that goes as low as about 50 KB, or 20 times smaller than the page we started with. Not a bad result for fewer than 10 lines of code. The net result is that our page is now viewable on almost anything, anywhere, from televisions to feature phones — and quickly! You might not be targeting televisions and feature phones, but now they will come to you.

Note: In testing this approach on real devices, HTML5’s doctype tag did not cause problems. The page loaded on pretty much every device I tried.

Going Further, Part 2: Connectivity Detection

Most of the weight savings so far hinge on certain low-end mobile devices not needing full-resolution images, rich styling or JavaScript. This set of techniques is highly effective, but we can do more to cater to our users and extend our reach. Desktop devices sometimes need help, too. Anyone who has used a laptop over airport Wi-Fi or in a poorly connected country knows just how frustrating viewing large pages is.

Connectivity detection comes to the rescue here. The idea is that, if you can detect the connectivity available to the requesting browser, then you can apply similar image-compression techniques dynamically according to the client’s available bandwidth. This can make a huge difference to the browsing experience, at some cost to page fidelity: If we detect a poor connection, then images can be aggressively compressed without reducing their pixel size. The result is a page that loads much faster, with only a slight impact on the experience — the page’s layout and overall appearance will be preserved. Depending on the compression levels, many people won’t even notice.

The W3C is working on making bandwidth information available to the browser, but the Network Information API is still in draft status and so won’t be widely deployed in the near future.

In the meantime, we can use some server-side capabilities. DeviceAtlas incorporates a feature to do exactly this, enabling the developer to make useful decisions about what to send to the client when bandwidth is limited. For this example, we’ll do something simple yet effective: If the bandwidth available to the device is detected to fall below a certain threshold, then we will redirect the browser to a different virtual host. This virtual host will be served by the same Web server and will serve exactly the same page, but it will trigger a different set of options for PageSpeed. In this case, we’ll change the image-compression level from its default to something much lower — say, 20%. At this level, images will still be very recognizable and will fit the page’s layout but will be many times smaller in bytes.

What follows is a simple way to achieve this, explained merely as a quick example of what is possible rather than as a definitive technique. First, add a new virtual host to your server’s configuration, and configure PageSpeed to use different settings for it. Then, restart your Web server. I am using site.com and lo.site.com as my virtual hosts. Note that the DocumentRoot is the same in each case — the exact same HTML is being served for both virtual hosts.


<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    ServerName site.com
    DocumentRoot /var/www/site.com
    ModPagespeed on
</VirtualHost>

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    ServerName lo.site.com
    DocumentRoot /var/www/site.com
    ModPagespeed on
    ModPagespeedImageRecompressionQuality 20
</VirtualHost>

Next, add the connectivity-checking code to your website’s template. This is what switches the virtual host if connectivity appears to be poor:


require_once 'DeviceAtlasNPC.php';        // check network performance
session_start();
$deviceAtlasNPC = new DeviceAtlasNPC();   // instantiate NPC
$quality = $deviceAtlasNPC->getQuality(); // test network performance
$path = $_SERVER['SCRIPT_NAME'];

switch($quality) {
    case DeviceAtlasNPC::HIGH_QUALITY:
        if ($_SERVER['HTTP_HOST'] == 'lo.site.com') {
            header("Location: http://site.com".$path );
        }
        break;
    case DeviceAtlasNPC::MEDIUM_QUALITY:
        if ($_SERVER['HTTP_HOST'] == 'lo.site.com') {
            header("Location: http://site.com".$path );
        }
        break;
    case DeviceAtlasNPC::LOW_QUALITY:
        if ($_SERVER['HTTP_HOST'] == 'site.com') {
            header("Location: http://lo.site.com".$path );
        }
        break;
    default:
}

Yes, the initial redirection to the low-bandwidth website does affect the loading time, but this penalty is far outweighed by the net savings in doing so, particularly for users with constrained bandwidth; high-bandwidth users should be almost unaffected. The cost of this redirection on a slow GPRS connection is approximately one second, but the resulting savings can add up to minutes.

Overall Results

So, let’s look at how this all comes together with the various optimizations, device types and connectivity options. The following table sums it all up.

Original website Add image resizing Add adaptive JS and CSS Page size Loading time Page size Loading time Page size Loading time Desktop (high-speed) 1027 KB 3s 921 KB 3s 921 KB 3s Desktop (56k modem) 1027 KB 3m 27s 921 KB 3m 14s 921 KB 2m 05s iPhone 3G 1027 KB 14s 253 KB 7s 253 KB 6s iPhone GPRS 1027 KB 2m 30s 253 KB 40s 253 KB 40s Feature phone (2G) 1027 KB ∞ 203 KB 35s 87 KB 25s Original website Add network adaptivity Page size Loading time Page size Loading time Desktop (high-speed) 1027 KB 3s 921 KB 3s Desktop (56k modem) 1027 KB 3m 27s 227 KB 40s iPhone 3G 1027 KB 14s 153 KB 5s iPhone GPRS 1027 KB 2m 30s 153 KB 25s Feature phone (2G) 1027 KB ∞ 25 KB 12s

These loading times are all worst-case scenarios, tested with an empty cache. Subsequent page loads as you traverse the website will feel much faster.

With all of these optimizations in place, we now have a more sensible responsive design that scales dynamically in richness and size, from 1 MB all the way down to 25 KB, all in. The website incorporates the best of RWD and server-side optimizations to yield a dynamic range factor of over 40.

The website is now responsive to multiple factors, not just one: screen size, device capabilities and network performance. As a result, the reach of this page has extended from desktop and smart devices in well-connected locations to almost everything, everywhere, regardless of connection type. Apart from getting our image size tags to populate dynamically, we didn’t have to do much work to make this happen.

Website before optimization on various devices:


Desktop: 1360 × 768 pixels, page size 1,027 KB


iPhone: 320 × 480 pixels, page size 1,027 KB


Nokia 6300: 240 × 320 pixels, page size 1,027 KB

Website after optimization on various devices:


Desktop: 1360 × 768 pixels, page size 1,027 KB


iPhone: 320 × 480 pixels, page size 153 KB


Nokia 6300: 240 × 320 pixels, page size 25 KB

Circling back to the page linked at the top of this article, Chris Zacharias, speaking of his experience optimizing YouTube’s page weight, says the following:

“[Previously], entire populations of people simply could not use YouTube because it took too long to see anything… By keeping your code small and lightweight, you can literally open your product up to new markets.”

By using some of the techniques outlined in this article, you might be able to achieve similar results for your website.

Notes

PageSpeed has a couple of other useful tricks up its sleeve.

Many thanks to Jonathan Heron of McCannBlue for reviewing this article.

(al, il)


© Ronan Cremin for Smashing Magazine, 2013.

Tags: Mobile

September 24 2013

11:06

Responsive Image Container: A Way Forward For Responsive Images?


  

The aim of republishing the original article by Yoav is to raise awareness and support the discussion about solutions for responsive images. We look forward to your opinions and thoughts in the comments section! – Ed.

It’s been a year since I last wrote about it, but the dream of a “magical” image format that will solve world hunger and/or the responsive images problem (whichever comes first) lives on. A few weeks back, I started wondering if such an image format could be used to solve both the art direction and resolution-switching use cases.

I had a few ideas on how this could be done, so I created a prototype to prove its feasibility. The prototype is now available, ready to be tinkered with. In this post, I’ll explain what this prototype does, what it cannot do, how it works, and its advantages and disadvantages relative to markup solutions. I’ll also try to de-unicorn the concept of a responsive-image format and make it more tangible and less magical.

“Got Something Against Markup Solutions?”

No, I don’t! Honestly! Some of my best friends are markup solutions.

I’ve been participating in the Responsive Images Community Group for a while now, prototyping, promoting and presenting markup solutions. Current markup solutions (picture and srcset) are great and can cover all of the important use cases for responsive images. And if it was up to me, I’d vote to start shipping both picture and srcset (i.e. its resolution-switching version) in all browsers tomorrow.

But the overall markup-based solution has some flaws. Here’s some of the criticism I’ve been hearing for the last year or so when talking about markup solutions for responsive images.

Too Verbose

Markup solution are, by definition, verbose because they must enumerate all of the various resources. When art direction is involved, they must also list the breakpoints, which add to that verbosity.

Mixing Presentation and Content

A markup solution that is art direction-oriented needs to keep layout breakpoints in the markup. This mixes presentation and content and means that any layout changes would force changes to the markup.

Constructive discussions have taken place on how this can be resolved — particularly by bringing back the media query definitions into CSS — but it’s not certain when any of this will be defined and implemented.

Define Viewport-Based Breakpoints

This one is proposed often by developers. For performance reasons, markup-based solutions are based on viewport size, rather than on image dimensions. Because the layout dimensions of images are not yet known to the browser by the time it starts fetching images, it cannot rely on them to decide which resources to fetch.

This means that developers will need to store some sort of table of viewports and dimensions on the server side, or maintain one in their head, in order to create images that are ideally sized for certain viewport dimensions and layouts.

While the addition of a build step could resolve this issue in many cases, it can get complicated in cases where a single component is used over multiple pages, with varying dimensions on each.

Results in Excessive Downloading in Some Cases

OK, this one I hear mostly in my head (and from other Web performance freaks on occasion).

From a performance perspective, any solution that’s based on separate resources for different screen sizes and dimensions will require the entire images to be redownloaded if the screen size or dimensions change to a higher resolution. Because most of that image data will very likely already be in the browser’s memory or cache, having to redownload everything from scratch would be sad.

All of the above makes me wonder (again) how wonderful life would be if we had a solution based on a file format that addresses these issues.

Why Would A File Format Be Better?

A solution based on a file format would do better for the following reasons:

  • The burden is put on the image encoder. The markup would remain identical to what it is today: a single tag with a single resource.
  • Automatically converting websites to such a responsive-image solution might be easier because the automation layer would focus only on the images themselves, rather than on the page’s markup and layout.
  • Changes to image layout (as a result of changes to the viewport’s dimensions) could be handled by downloading only the difference between current image and the higher-resolution one, without having to redownload the data that the browser already has in memory.
  • Web developers would not need to maintain multiple versions of each image resource, although they would have to keep a non-responsive version of the image for the purpose of content negotiation.

This is my attempt at a simpler solution based on file format that would relieve Web developers of much grunt work and would avoid useless image data from having to be downloaded (even when conditions change), while keeping preloaders working.

Why Not Progressive JPEG?

Progressive JPEG could serve this role for the resolution-switching case, but it’s extremely rigid. It comes with strict limits on the lowest image quality and, from what I’ve seen, is often too data-heavy. Also, the minimal difference between resolutions is limited and doesn’t give enough control to encoders that want to do better. Furthermore, progressive JPEG cannot handle art direction at all.

What Would It Look Like?

We’re talking about a responsive image container, containing internal layers that could be WebP or JPEG-XR or any future format. It uses resizing and cropping operations to cover both the resolution-switching and the art direction use cases.

The decoder (i.e. the browser) would then be able to download just the number of layers it needs (and their bytes) in order to show a certain image. Each layer would enhance the layer before it, giving the decoder the data it needs to show it properly at a higher resolution.

How Does It Work?

  1. The encoder takes the original image, along with a description of the required output resolutions and, optionally, directives on art direction.
  2. It then outputs one layer per resolution that the final image should be perfectly rendered in.
  3. Each layer represents the difference in image data between the previous layer (when “stretched” on the current layer’s canvas) and the current layer’s “original” image. This way, the decoder can construct the layers one by one, each time using the previous layer to recreate the current one, creating a higher resolution image as it goes.

Support for resolution-switching is obvious in this case, but art direction could also be supported by positioning the previous layer on the current one and being able to give it certain dimensions. Let’s look at some examples.

Art Direction

Here’s a photo that is often used in discussions about the art direction use case:

Obama in a jeep factory - original with context

Let’s see what the smallest layer would look like:

Obama in a jeep factory - cropped to show only Obama

That’s just a cropped version of the original. Nothing special.

Now, one layer above that:

Obama in a jeep factory - some context + diff from previous layer

You can see that pixels that don’t appear in the previous layer are shown as normal, while pixels that do appear there contain only the difference between them and the equivalent ones in the previous layer.

And here’s the third, final layer:

Obama in a jeep factory - full context + diff from previous layer

Resolution-Switching

A high-resolution photo of a fruit:

iPhone - original resolution

Here is the first layer, showing a significantly downsized version:

iPhone - significantly<br />
downsized

The second layer shows the difference between a medium-sized version and the
“stretched” previous layer:

iPhone - medium sized<br />
diff

The third layer shows the difference between the original and the “stretched” previous layer:

iPhone - full sized<br />
diff

If you’re interested in more detail, you can go to the repository. More detail on the container’s structure is also there.

“But I Need More From Art Direction”

I’ve seen cases where rotation and image repositioning are required for art direction, usually in order to add a logo at different locations around the image itself, depending on the viewport’s dimensions.

This use case is probably better served by CSS. CSS transforms can handle rotation, while CSS positioning, along with media-specific background images, could probably handle the rest.

Note: If your art direction is special and can’t be handled by either one of these, I’d love to hear about it.

How Is It Fetched?

This is where things get tricky. A special fetching mechanism must be created to fetch this type of image. I can’t say that I have figured that part out, but here’s my rough idea on how it could work.

My proposed mechanism relies on HTTP ranges, similar to the fetching mechanisms of the <video> element, when seeks are involved.

More specifically:

  • Resources that should be fetched progressively should be flagged as such. One possibility is to add a progressive attribute to the element that describes the resource.
  • Once the browser detects an image resource with a progressive attribute on it, it picks the initial requested range for that resource. The initial range request could be:
    • a relatively small fixed range for all images (like 8 KB);
    • specified by the author (for example, as a value of the progressive attribute);
    • some heuristic;
    • based on a manifest (which we’ll get to later).
  • The browser can fetch this initial range at the same time that it requests the entire resource today, or even sooner, because the chances of starving critical path resources (including CSS and JavaScript) are slimmer once the payloads are of a known size.
  • Once the browser has downloaded the image’s initial range, it has the file’s offset table box, which links byte offset to resolution. This means that once the browser has calculated the page’s layout, it will know exactly which byte range it needs in order to display the image correctly.
  • Assuming that the browser sees fit, it could heuristically fetch follow-up layers (i.e. of higher resolutions) even before it knows for certain that they are needed.
  • Once the browser has the page’s layout, it can complete fetching all of the required image layers.

The mechanism above will increase the number of HTTP requests, which in an HTTP/1.1 world would introduce some delay in many cases.

This mechanism may be optimized by defining a manifest that describes the byte ranges of the image resources to the browser. The idea of adding a manifest was proposed by Cyril Concolato at the W3C’s Technical Plenary / Advisory Committee meetings week last year, and it makes a lot of sense, borrowing from our collective experience with video streaming. It enables browsers to avoid fetching an arbitrary initial range (at least once the manifest is downloaded itself).

Adding a manifest will prevent these extra requests for everything requested after the layout, and it might help to prevent them (using heuristics) even before the layout.

Creating a manifest could be easily delegated either to development tools or to the server-side layer, so that developers don’t have to manually deal with these image-specific details.

“Couldn’t We Simply Reset the Connection?”

In theory, we could address this by fetching the entire image, and then reset the connection once the browser has all the necessary data, but that would most likely introduce serious performance issues.

Here are the problems with reseting a TCP connection during a browsing session:

  • It terminates an already connected, warmed-up TCP connection, whose set-up had a significant performance cost and which could have been reused for future resources.
  • It sends at least a round-trip time’s worth of data down the pipe, the time it takes for the browser’s reset to reach the server. That data is never read by the browser, which means wasted bandwidth and slower loading times.

Downsides To This Approach

There are a few downsides to this approach:

  • It involves touching and modifying many pieces of the browser stack, which means that standardization and implementation could be painful and take a while.
  • The monochrome and print use case cannot be addressed by this type of a solution.
  • The decoding algorithm involves per-layer upscaling, which could be processing-heavy. Therefore, decoding performance could be an issue. Moving this to the GPU might help, but I don’t know that area well enough to judge. If you have an opinion on the subject, I’d appreciate your comments.
  • Introducing a new file format is a long process. As we have seen with the introduction of previous image formats, the lack of a client-side mechanism makes this a painful process for Web developers. Because new file formats start out being supported in some browsers but not others, a server-side mechanism must be used (hopefully based on the Accept header, rather than on the User-Agent header). I’m hoping that this new file format’s simplicity and reliance on other file formats to do the heavy lifting help here, but I’m not sure they would.
  • As discussed, it would likely increase the number of requests, and could introduce some delay in HTTP/1.1.
  • This solution cannot address the need for “pixel-perfect” images, which is mainly needed to improve decoding speed. Even if it did, it’s not certain that decoding speed would benefit from it.
  • Relying on HTTP ranges for the fetching mechanism could result in some problem with intermediate cache servers, which don’t support it.

So, Should We Dump Markup-Based Solutions?

Not at all. This is a prototype, showing how most of the responsive-image use cases would have been solved by such a container.

Reaching consensus on this solution, defining it in detail and implementing it in an interoperable way could be a long process. The performance implications on HTTP/1.1 websites and decoding speed still need to be explored.

I believe this might be a way to simplify responsive images in the future, but I don’t think we should wait for the ideal solution.

To Sum Up

If you’ve just skipped to here, that’s OK. This is a long post.

To sum it up, I’ve demonstrated (along with a prototype) how a responsive-image format could work and how it could resolve most responsive-image use cases. I also went into some detail about which other bits would have to be added to the solution in order to make it a viable solution.

I consider this to be a long-term solution because some key issues need to be addressed before it can be made practical. In my opinion, the main issue is decoding performance, with the impact of downloading performance on HTTP/1.1 being a close second.

Continuing to explore this option is worthwhile, but let’s not wait for it. Responsive images need an in-browser, real-life solution two years ago today, not two years from now.

(al, ea, il)


© Yoav Weiss for Smashing Magazine, 2013.

Tags: Mobile

September 19 2013

10:29

Building An Online Magazine App For Windows 8, Part 1: The HTML5 App


  

Back in 2010, Microsoft shifted its focus from propriety Web technology to open Web technology. The first fruits of this refocus materialized a few years later — in Internet Explorer, the Windows operating system, its developer tools and its cloud software.

Things have changed for the better so far:

  • With version 10, Internet Explorer has finally grown up and become a fast modern browser.
  • You can build native Windows apps with JavaScript, HTML5 and CSS — apps that look and feel solid and have modern user interfaces.
  • On the server side, ASP.NET lets you own your own markup and HTTP services (finally), with Web developer-friendly tools like Razor and Web API.
  • At the cloud level, Microsoft has thrown its full weight behind Windows Azure, hosting HTTP servers en masse.

Across the board, Web developers should see significant improvements, making Windows an HTML5-friendly platform.

Windows 8 Modern UI layering
Windows 8 apps that reflect Microsoft’s “modern UI” can be built using a variety of technologies — HTML5, JavaScript and CSS being among them. Access to the native API enables you to do a lot more than you could from a browser. (Larger view)

A Two-Part Whirlwind Tour

In this first article in our two-part series, we’ll look at an HTML5 app that runs natively on Windows 8. In the second part, we’ll tie it up with the server and cloud. It will be a whirlwind tour through Microsoft-based Web development in 2013, with a hands-on case study and some code.

If you code along with us, you’ll end up with an online magazine app, built with Web technology. The Windows 8 app will present articles fetched in JSON format from a content host running on Windows Azure. You will have a personally branded touch-friendly magazine app that you can submit to the Windows Store, that integrates with the Search and Share charms and that displays articles you manage in a cloud-based content management system (CMS).

In this first part, we’ll explain how you can launch and customize a pre-baked HTML5-based Windows Store app. In the second part, we’ll guide you through setting up your own content host, from where you can publish articles.

Surface RT running the Magazine App
Our app will run on any Windows 8 machine, including Windows RT devices such as the Microsoft Surface RT. (Larger view)

The source code presented here, as well as the CMS back end, are all open source and easy to adapt, and the hosting and tools referred to here are available free of charge.

Throughout, you will find code snippets and links to relevant in-depth articles, and I’ll pass along some of the gotchas that I learned while building this app.

What You’ll Need To Get Started

To develop and debug a Windows Store app, you’ll need to be running Windows 8, either natively or via a virtual machine. To edit the HTML, CSS and so on, you can use your favorite editor and compile the app via the command line; here, I’ll be using Visual Studio 2012. You can download Visual Studio Express 2012 for Windows 8 for free.

A Barebones App

Before diving into the pre-baked app, we should get to know the playing field a bit. So, here is what the code for a barebones Windows Store app might look like:


<!DOCTYPE html>
<html>
    <head>
    </head>
    <body>
        <img src="https://upload.wikimedia.org/wikipedia/commons/2/28/HelloWorld.svg" />
    </body>
</html>

To compile this into an app, Visual Studio will also need a project file and a manifest file (which contains meta data about your app). However, the HTML document above is all you really need to get a “Hello World.”

What you add is up to you, as long as it runs in Internet Explorer (IE) 10. (You can safely use CSS3, canvas, video, SVG, your favorite JavaScript libraries, etc.) In addition, you will also have the option to use the Windows Store API from JavaScript, to take advantage of Windows 8-specific features.

In our pre-baked app, we will use the Windows Store API. Even though it is optional, we will need it if we want things to feel native and to integrate with Windows 8.

Downloading And Launching The Magazine App

The source code for the magazine app is available on GitHub. If you don’t have Git on your machine, simply download the ZIP file and extract the source code.

Once it’s downloaded, launch the project file, Composite.AppFeed.Client.Template.jsproj. When Visual Studio has launched, hit F5 to run the app.

The app main view
The app’s main view shows articles within tiles that are familiar from Windows 8’s UI. (Larger view)

Our app has the following basic features:

  • It’s a simple navigation app, with grouped items, a group view and an item view.
  • Clicking a group’s name will display the group view. Clicking an item’s tile will show the item.
  • Search and Share charms, device rotation, docking and the app bar all work.
  • Offline mode is supported (cached content is shown).
  • Rich HTML content is supported, including things like video.
  • It has the modern UI navigation experience expected in Windows 8 apps.

The Search and Share charms are part of a canonical charms menu in Windows 8, invoked by swiping in from the right edge (or moving your mouse to one of the right corners). The Search charm, as the name suggests, lets the user search, in our case, through articles in the app. The Share charm integrates with the user’s mail and social media accounts, enabling the user to share the URL of a given article.

In the following few sections, we will customize the design.

The sample content you see in the app is downloaded from the Web from various sources (including an existing blog, CodePlex and StackOverflow) and gives an idea of what existing raw Web content could look like when presented via this app. We will replace this default feed with one you control in part two.

Project Explorer
The project explorer shows files that make up the HTML5 app.

Making The App Your Own

The project’s structure is pretty straightforward. The css, images and js folders contain global resources for the app, while the individual pages (including HTML, CSS and JavaScript) are defined below the pages folder. In the root, we have default.html, which is the app’s starting point, and package.appxmanifest, which contains meta data used by the operating system (OS) to present the app.

Once you know your way around the sources, you can customize as you wish. To customize the UI quickly, focus on the following files.

/package.appxmanifest

Edit the package.appxmanifest file. In the “Application UI” tab, specify your app’s title, language and description. In the “Packaging” tab, set a unique package name and publisher display name.

You can customize other things, such as the appearance of the tile for your app. The images for your tiles can be replaced in the /images folder (see below).

The app manifest file
The app’s manifest file is where you control how the app shows up in the OS. (Larger view)

/images Folder

Right-click the /image folder, select Open Folder in File Explorer, and replace the various-sized logos with your own. These logos will be used to create the tiles for the Windows 8 start screen and your app’s splash screen. You can change the background on the main page by replacing background_mini.jpg. Change inapplogo.png to customize the footer.

File Explorer
Use PNG and JPEG images in your app and to represent the app on the start screen. (Larger view)

/css/default.css

Set the colors for the app’s bar and background at the top of default.css.

/js/options.js

In options.js, you can set the app’s title and details such as the “About” page’s URL, the privacy page’s URL and the alert text for offline mode.

At a minimum, customize the title and websiteUrl. Later on, when you’re running your own content host, change dataHostUrl, but keep it as is for now.

options.js
The app’s global settings are controlled via JavaScript. (Larger view)

If you will be submitting your app to the Windows Store, pay attention to the value of privacyUrl; this is the URL that gets loaded when the user invokes yours app’s privacy policy, via the Settings charm. Because your app will be contacting a server on the Internet to fetch articles, you will need to explain your privacy policy to the user in order for your app to be approved by Microsoft. More on that in part two.

Diving Deeper Into The App

If you’ve updated the colors, titles, URLs and images, as explained above, then the major design elements of your magazine app should be branded to your liking. The next step is, obviously, to get your own content host up and running. But before we cover this, let’s cover a few pointers on further customizing the app.

IE 10 and the Windows Store App API

When building your app, you can safely use the HTML5 and CSS3 features of IE 10, including canvas and SVG. You can use any JavaScript library out there, such as AngularJS, jQuery or LESS. You are not required to test your pages in other browsers or in older versions of IE.

The Windows Store app API is by far the biggest change from the Web developer’s traditional experience. If you wish to interact with the OS, go through this API using JavaScript. The API also contains a host of other features that make JavaScript even more powerful. The API is unobtrusive in that you don’t have to care about it until you need to interact with the OS or want to use one of its features.

Here are a few APIs used by this app:

  • WinJS enables more advanced JavaScript features (such as the Promise object and XHR), which are excellent for writing asynchronous network interactions.
  • Windows.UI, Windows.ApplicationModel and Windows.Storage let you plug into the native OS and UI to do things such as interact with the charms menu and read and write to local files.
  • WinJS.UI contains presentation controls that act in accordance with Windows 8’s modern UI, including the scrollable list view that you see when launching the application.

For more on this, see “Windows API for Windows Store Apps.”

Single-Page Navigation

This app uses a single-page navigation model, meaning that HTML, CSS and JavaScript for individual subpages are loaded into /default.html as you invoke links. This enables you to preserve state across pages and to get smooth page transitions; but it also changes how page-specific CSS and scripts are written.

Below the /pages folder, you will find subfolders with HTML, CSS and JavaScript, making up the individual pages of the app, and this is where you can customize the individual views to your heart’s content.

Due to the single-page navigation model, expect the HTML, CSS and JavaScript from /default.html to stay with you all the time. When you navigate a subpage, scripts and CSS are loaded dynamically, and the HTML of a subpage is loaded into the /default.html div with the ID contenthost.

When you navigate from one page to another, CSS rules will not be unloaded, which means that any global selectors you style in a CSS file for a subpage will stick. If this is not desirable, you can prefix subpage-specific CSS selectors with a unique class name that identifies the subpage.

An example is this rule from /pages/groupedItems/groupedItems.css:


.groupeditemspage .groupeditemslist .group-header .group-title, .groupeditemspage .pagetitle {     
    color: #eeeeee; 
}

The class name groupeditemspage is set in /pages/groupedItems/gropuedItems.html and only there. This enables us to set a light color for title text, but only for this particular subpage. When we navigate to one of the other subpages, the groupeditemspage class will no longer be in the DOM, and titles will fall back to a dark color, even though the CSS rules above still load.

It is worth mentioning that I’m using JavaScript (see /js/navigation.js) to lift this page-specific class name up to the body element on the current page every time the user navigates; this enables me to style all elements on the current DOM. If I didn’t do this, I would only be able to style elements below contenthost in default.html. If you study groupedItems.css, you will see a background image being set, and it only works because of this “move the subpage class up the DOM” trick — a small generic hack that lets us do more with CSS.

Single-Page Navigation Requires You To Unregister Resources

You also need to care about unregistering resources, such as removing event listeners when a subpage is unloaded. Because the DOM is always kept alive, failing to unregister subpage resources on unloading can lead to undesirable results.

An example is this handler from /pages/groupedItems/groupedItems.js:


unload: function () {
   var dataTransferManager = Windows.ApplicationModel.DataTransfer.DataTransferManager.getForCurrentView();
   dataTransferManager.removeEventListener("datarequested", shareDataRequested);
}

Earlier in the subpage’s life, we subscribed to a data request event (which happens when a user invokes the Share charm), and we need to unsubscribe from this event when the user navigates away from that particular subpage.

A Few Words About CSS Media Queries

Your app can run on both classic Windows 8 PCs and ARM-based devices, such as the Surface RT tablet. In addition to a multitude of screen resolutions, be prepared for things such as horizontal to vertical rotation of the device and your app being snapped next to another one.

App running in snapped view
Windows 8 let you run two apps side by side, with one being “snapped.” Here, our app is snapped next to the Web browser, which lets users follow links in articles within the app without losing focus. (Larger view)

This you can do by using CSS media queries for Windows 8 apps. Below are a few sample declarations.


@media screen and (-ms-view-state: snapped) {
    .groupdetailpage .itemslist .win-groupheader {
        visibility: hidden;     
    } 
} 
@media screen and (-ms-view-state: fullscreen-portrait) {
    .groupdetailpage .itemslist .win-horizontal.win-viewport .win-surface {
        margin-left: 100px;     
    }
}

You can also respond to rotation and snap events in scripting. For example, see the updateLayout handler in groupedItems.js.

Testing and Debugging

The coding and debugging experience is slightly different than in normal Web development. For starters, your HTML app will run in a 100% chromeless full-screen environment, so the developer toolbar is not where you are used to seeing it. You’ll find it inside Visual Studio instead, which means you’ll have to jump between Visual Studio and your running app to inspect elements and so on.

DOM Explorer in Visual Studio
The DOM explorer in Visual Studio 2012 looks and feels like most other developer toolbars. (Larger view)

On the bright side, you’ll have a nice spacious development canvas and excellent JavaScript debugging capabilities. And if you are running dual monitors, the experience is quite nice and seamless.

While your app is running, you will find a “DOM Explorer” tab in Visual Studio, which represents the page you are currently viewing. The “Select Element” button will bring focus to your HTML app and let you select an element, just like in the developer toolbars you know.

Visual Studio can launch your app locally, via a simulator and on a remote device. The simulator lets you emulate different device sizes and test things such as touch events, geo-location services and changes in orientation, even though you are sitting in front of a classic PC.

App Simulator
The simulator can launch a new log-in session on your Windows 8 machine, displaying everything as if it were running on the device you have selected. This lets you test how your app will behave (including with touch actions) on different device sizes and orientations. (Larger view)

Remote machine debugging is surprisingly easy to get up and running. Apart from letting you instantly test your app on a device like Surface RT, it is also an easy way to get your app onto a device. Once the app has been launched this way, it will stay on the device and can be used for demoing and so forth.

A Few Common Problems and How to Avoid Them

During development, I ran into two issues that took me way too much time to figure out. To save you from repeating my experience, I’ll leave you with two pointers as you polish the app.

If you ever find yourself developing a Windows Store app that fetches data from your local network, via a virtual private network or the like, you might need to edit package.appxmanifest and enable “Private Network (Client & Server)” in the “Capabilities” tab. If this setting is not enabled and you are trying to read data from a local test server, then network communication will be blocked (with no usable hints). Uncheck this again before submitting the app, unless you expect users to need this feature.

Capabilities of your app
You have to explicitly opt in to access things such as location, the webcam, private networks and the microphone. Users will be informed of this usage. To keep users from getting suspicious, select only what you need. (Larger view)

If you write code that interacts with the Share charm and your code fails inside the datarequested handler or you stop a debugging session there, then you will experience a bug whereby the Share charm stops working, even if you restart the debugging session. The easiest way to fix this is to launch the Task Manager and restart explorer.exe.

What To Expect In Part 2

In the second and last part of this series, we’ll walk through setting up the CMS back end, getting it online and having the app read content from it. With the customized app and the back end online, you’ll have something of real value to publish, and we’ll briefly touch on how to do just that.

If you plan to submit your app to the Microsoft Store, now is a good time to sign up for a developer account. If you choose the “individual account,” you will get up and running a whole lot faster; the “company account” requires verification but lets you publish apps using more features. Make sure you know the differences between the two account types before signing up. There is a $49 annual registration fee, while submitting apps is free.

(al) (ea)


© Marcus Wendt for Smashing Magazine, 2013.

Tags: Mobile

September 16 2013

11:35

Addressing The Responsive Images Performance Problem: A Case Study


  

Five-inch mobile devices are on the market that have the same screen resolution as 50-inch TVs. We have users with unlimited high-speed broadband as well as users who pay money for each megabyte transferred. Responsive design for images is about optimizing the process of serving images to users. In this article, we will share our responsive image technique, the “padding-bottom” technique, which we researched and implemented on the mobile version of the Swedish news website Aftonbladet.

The techniques presented here are the result of a few weeks of research that we did in October 2012. We were fortunate enough to be a part of the team that built a new responsive mobile website for Aftonbladet. Aftonbladet is Sweden’s largest website, and its mobile version gets about 3 million unique users and up to 100 million page views per week.

With that amount of users, we felt it was our responsibility to make a fast and well-optimized website. Saving just 100 KB of image data per page view would translate into a lot of terabytes of data traffic saved in Sweden per year. We started out by researching other responsive image techniques, but because none of them was a perfect match, we ended up combining some of the best hacks into our own solution. Please note that this project covered only a responsive mobile website; but do not worry — the technique presented here applies to all types of responsive websites.

The Specification

We started out by creating a simple specification in order to select a suitable responsive image solution. The solution had to:

  • be easy to cache,
  • multiserve images.

Let’s go through these requirements and see what they mean.

Easy to Cache

With a website that gets traffic peaks of over 10,000 requests per second, we wanted to keep the server logic as simple as possible. This means that we didn’t want to use server-side device detection or a cookie-based solution that serves multiple versions of the HTML. We needed a single HTML file to be served to all users, although manipulating the HTML with JavaScript after it has loaded is acceptable. The same rules apply to images; we needed to be able to put the images on a content delivery network (CDN), and we did not want any dynamics in the image-serving logic.

Multiserving Images

We wanted to serve different image versions to different devices. One big complaint about our previous mobile website was that high-DPI iPhones and Android devices did not get the high-resolution images they deserved. So, we wanted to improve image quality, but only for the devices that were capable of displaying it.

Loading Images With JavaScript

JavaScript, if placed in the footer where it should be, will load after the HTML and CSS has been parsed. This means that, if JavaScript is responsible for loading images, we can’t take advantage of the browser’s preloader, and so an image will start downloading a fair bit later than normal. This is not good, of course, and it reveals another problem: The page might reflow every time the JavaScript inserts an image into the DOM.

Reflowing happens when the browser recalculates the dimensions of the elements on the page and redraws them. We have set up a demo page and a video that demonstrate this effect. Note that the demo page has an inserted delay of 500 milliseconds between each image in order to simulate a slow connection speed.

As you can see from the video, another very annoying feature is that the user will likely get lost in the reflowing when returning to a page with the “Back” button. This is actually a serious problem for websites such as Aftonbladet. Having a functional “Back” button will keep users longer on the website.

The reflowing problem would not really be present on a website that is not responsive because we would be able to set a width and height in pixels on the image tag:


<img src="img.jpg" width="60" height="60"/>

One important aspect of responsive Web design is to remove those hardcoded attributes and to make images fluid, with CSS:


img {
	max-width: 100%;
}

No More max-width: 100%

We needed to find a solution whereby we could reserve space for an image with only HTML and CSS and, thus, avoid reflowing. That is, when the JavaScript inserts an image into the page, it would just be inserted in the reserved area and we would avoid reflowing. So, we threw out one of the cornerstones of responsive Web design, img { max-width: 100% }, and searched for another solution that could reserve space for a responsive image. What we needed was something that specifies just the aspect ratio of the image and lets the height shrink with the width. And we found a solution.

The Padding-Bottom Hack

This technique is based on something called intrinsic ratios, but because none of our team’s members could remember, understand or pronounce the term “intrinsic” we just called it the “padding-bottom hack.”

Many people learned about this feature back in 2009 in A List Apart’s article “Creating Intrinsic Ratios for Video,” by Thierry Koblentz, and the technique is often used to embed third-party content and media on responsive websites. With the technique, we define the height as a measure relative to the width. Padding and margin have such intrinsic properties, and we can use them to create aspect ratios for elements that do not have any content in them.

Because padding has this capability, we can set padding-bottom to be relative to the width of an element. If we also set height to be 0, we’ll get what we want.


.img-container {
	padding-bottom: 56.25%; /* 16:9 ratio */
	height: 0;
	background-color: black;
}

The next step is to place an image inside the container and make sure it fills up the container. To do this, we need to position the image absolutely inside the container, like so:


.img-container {
	position: relative;
	padding-bottom: 56.25%; /* 16:9 ratio */
	height: 0;
	overflow: hidden;
}

.img-container img {
	position: absolute;
	top: 0;
	left: 0;
	width: 100%;
	height: 100%;
}


The container reserves the space needed for the image.

Now we can tweak our demo, applying the padding-bottom hack, and the user will no longer get lost in the reflowing that we saw earlier. Also, the “Back” button functions as expected. See the new video and demo.

This technique improves the user experience of the website quite a bit over the traditional max-width approach, but the experienced reader will by now have noticed two things:

  1. We need to know the aspect ratio of the image before we load an image.
  2. Images could be scaled up larger than their original size.

To handle the aspect ratios, you need either to have a content management system with which you can control the templates or to have a limited, fixed set of aspect ratios for images. If you have something in between, whereby you cannot affect how image tags are rendered, then this method will probably be hard to use.

At Aftonbladet, we decided to calculate the padding-bottom percentage on the server and print it out as an inline style in the HTML, as you will see in the following code snippets. For the second problem, we found that, for our use case, letting the image scale up if needed (and losing some quality) was actually better than setting a fixed maximum width for the image.

Choosing An Image-Loading Technique

Now that we’ve allowed ourselves to load images with JavaScript, because we’ve minimized the reflowing, we can set up the requirements for this:

  • The resulting HTML should be a single img tag.
  • The DOM elements should be minimal.
  • It should execute as quickly as possible.
  • It should not break when JavaScript is disabled.

Based on this simple specification, we created an inline vanilla JavaScript, based on the “noscript” technique. The idea is to add the information about different image sizes to the HTML as data attributes in a noscript tag. The content of the noscript tag would be an img tag and would be shown to browsers that have JavaScript turned off. Let’s look at the markup:


<noscript data-src-small="img-small.jpg" 
    data-src-medium="img-medium.jpg" 
    data-src-high="img-high" 
    data-src-x-high="img-x-high.jpg">
        <img src="img-small.jpg">
</noscript>

The job of the JavaScript, then, is to parse the content of the page, identify images that should be lazy-loaded, check the size of the device’s screen and pick the correct image. The following code would look for images to load and insert them into the DOM. It is important that the JavaScript be inline and load as soon as possible after the HTML. The script would also retrieve the alt tag from the noscript tag and insert it into the newly created img tag.


	var lazyloadImage = function (imageContainer) {

		var imageVersion = getImageVersion();

		if (!imageContainer || !imageContainer.children) {
			return;
		}
		var img = imageContainer.children[0];

		if (img) {
			var imgSRC = img.getAttribute("data-src-" + imageVersion);
			var altTxt = img.getAttribute("data-alt");
			if (imgSRC) {
				var imageElement = new Image();
				imageElement.src = imgSRC;
				imageElement.setAttribute("alt", altTxt ? altTxt : "");
				imageContainer.appendChild(imageElement);
				imageContainer.removeChild(imageContainer.children[0]);
			}
		}
	},
	lazyLoadedImages = document.getElementsByClassName("lazy-load");

	for (var i = 0; i < lazyLoadedImages.length; i++) {
		lazyloadImage(lazyLoadedImages[i]);
	}

Picking The Perfect Image

So far, the techniques described here generally apply to any website that implements responsive images. The last step, selecting the image to send to the browser, is different in the way that it has to be adapted to the needs of the website.

Many factors need to to be considered when choosing the optimal image to send to a particular device, such as screen size, network speed, cacheability, overall page weight and the user’s preference. The website we built for Aftonbladet mainly targets mobile browsers, and we were lucky enough to have a lot of statistics on the average user’s behavior. By analyzing the numbers, we could identify some trends.

First, the vast majority hold their device in portrait mode. For reading and browsing articles, portrait mode is the natural choice. And while screen size varies a lot, over 99% of the traffic we analyzed represent devices with a viewport width of either 320 or 360 pixels.

Secondly, most of the visiting devices have high-density screens, with a native width of 480, 640, 720 or 1080 pixels. The highest resolutions come from newer phones, such as the Galaxy S4 and Xperia Z; while a 1080 pixel-wide image looks great on those phones, tests showed that a 720 pixel-wide image would look good enough, with less of a bandwidth cost.

After analyzing the data, we settled on three versions for each image:

  • small (optimized for a 320 pixel-wide screen),
  • medium (optimized for a 640 pixel-wide screen),
  • large (optimized for a 720 pixel-wide screen).

(Devices without JavaScript would get the small image.) We believe these settings are reasonable for a mobile website, but a fully responsive website that targets all kinds of devices would probably benefit from different settings.

We give the versions logical names, instead of specifying media queries in the markup. We choose to do it this way for flexibility. This makes it easier to evolve the JavaScript in the future to, for example, adapt to network speed or enable a user setting that overrides which image to use. In its simplest form, the engine for selecting image versions could be implemented as in the following example (although, to support Internet Explorer, we’d need another function as a workaround for the absence of window.devicePixelRatio).


var getImageVersion = function() {
        	var devicePixelRatio = getDevicePixelRatio(); /* Function defined elsewhere.*/
        	var width = window.innerWidth * devicePixelRatio;
        	if (width > 640) {
                    	return "high";
        	} else if (width > 320) {
                    	return "medium";
        	} else {
                    	return "small"; // default version
        	}
};

We also tried to take the screen’s height into account when selecting the right image. In theory, this would have been a nice complement to make sure that the image is well suited to the device. But when we tested the theory in the real world, we soon found too many edge cases that made it impossible to detect the height in a way that was good enough.

Apps that embed Web views inside scroll views reported a seemingly random height between 0 and 3000 pixels, and features such as Samsung’s TouchWiz system, which has a split-screen mode, made us abandon the screen’s height as a reliable value for choosing image sizes.

We created a simple demo page that has all of the JavaScript and CSS needed. But keep in mind that our code is targeted at mobile devices, so it doesn’t work out of the box in, say, old Internet Explorer browsers.

Making Smaller Images

Large beautiful images use up a lot of bandwidth. Downloading a lot of 720 pixel-wide images on a cellular network can be both slow and costly for the user. While a new image format such as WebP would help some users, it is not supported by enough browsers to be viable as the only solution. Fortunately, thanks to research by Daan Jobsis, we can take advantage of the fact that a high compression rate doesn’t affect the perceived quality of an image very much if the image’s dimensions are larger than displayed or if the image is displayed at its native size on a high-density screen.

With aggressive JPEG compression, it is, therefore, possible to maintain a reasonable download size while still having images look beautiful on high-density displays. We already had an image server that could generate scaled, cropped and compressed images on the fly, so it was just a matter of choosing the right settings.

This is also one reason why we didn’t include an image version for 480-pixel screens. Scaling down a 640 pixel-wide image with a high compression level made for a better-looking image at a smaller size than we could achieve with an image that had the native resolution of the 480-pixel screen. In this case, we decided that making the device scale the image to fit was worth it.

Red Areas Don’t Compress Well

A high compression rate is no silver bullet, though. Some images look terrible when compressed, especially ones with prominent bright-red areas, in which JPEG compression artifacts would be clearly visible and could spoil the overall impression of the image. Unfortunately, the editors at Aftonbladet have a fondness for images with prominent bright-red areas, which made our task just a little more challenging.

artifacts
These two images are saved with a 30% quality setting. While the image on the left might be passable even on a normal screen, the red circle in the right image looks bad even on a high-density screen.

Finding a Compromise

We could solve this problem in a few ways. We could increase the dimensions and compression of the images even more, which would make the artifacts less visible. This would mean that the browser has to scale images while rendering the page, which could have a performance impact. It would also require larger source images, which in practice are not always available. Another solution would be to let the editors choose how compression should be handled for each image, but this would add another step to the editors’ workflow, and we would need to educate them on the intricacies of how image compression, size, performance and mobile devices work together.

In the end, we settled on a compromise and avoided using high compression rates for important images and image galleries, instances where the image is the center of attention. In these cases, we also make sure to load only the visible images, to not waste the user’s bandwidth.

quality
The teaser images on Aftonbladet’s section pages (left) work really well with high compression levels, while a full-screen image gallery (right) benefits from higher-quality images.

Generating the different images can be problematic. We have the luxury of an existing back end that can scale, compress and crop images on demand. Rolling your own might entail a lot of work, but implementing it as, for example, a WordPress plugin (using the WP_Image_Editor class) would actually be pretty straightforward.

The Bottom Line

Three years after Ethan Marcotte introduced responsive Web design, we’re still struggling to find a good solution to the problem of how to handle images. All responsive image solutions are more or less a hack, and so is this one. But we have managed to escape the excessive reflowing problem, we have not introduced a lot of unneeded DOM elements, we have a cacheable website, and we prevent images that aren’t used from being downloaded.

Aftonbladet’s home page on a mobile device has around 40 images and, with this technique, ends up being around 650 KB on a “large screen,” 570 KB on a medium screen and 450 KB on a small screen (although the size varies according to the content). Without the high compression rate, the large and medium versions would be over a megabyte in size. And compared to the old website, we’ve managed to move from blurry low-resolution images to high-quality images tailored to each device, with just a 25% increase in download size.

So, we are still waiting for the perfect solution to responsive images. In the meantime, what we have outlined above has been a success for Aftonbladet’s new responsive website and hybrid apps. The new website (whose perceived loading time is twice as fast as that of the old one) has led to a huge boost in traffic and user interaction; and all through the summer, traffic to Aftonbladet’s mobile version has been higher than traffic to the desktop and tablet versions.

(al) (ea)


© Anders Andersen for Smashing Magazine, 2013.

Tags: Mobile
Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl