Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

January 07 2014

05:00

28 Beginner Tutorials for Modern API Web Development

Advertise here with BSA


Web development has grown far beyond the typical HTML/CSS/JS code structure. Modern web services like Facebook and Twitter are built with something called an Application Programming Interface. This allows developers to connect into 3rd party services and pull out data to be displayed on another website.

For this post I’ve collected a number of free online tutorials which delve into API development. These are perfect for anybody new to the scene who wants to learn a bit more about typical web services. Some will require an API key while others may simply return XML/JSON data on command. Either way these articles detail many popular development techniques in an easy-to-understand fashion.

Facebook’s Graph API Explorer

facebooks graph api explorer to grab insights data

Using the Pinterest API

pinterest api howto tutorial development

DeviantArt API Instant Search App

deviantart api instant search howto jquery ajax php tutorial

Google Places API

google places api tutorial jquery howto development

Login and Register using LinkedIn

linkedin oauth api login register howto tutorial

Guide to the HTML5 APIs

Using the Etsy API

using the etsy api development tutorial howto

A Look at the WordPress HTTP API

http api wordpress practical wp remote post tutorial

How To Create a Twitter Widget

creating a twitter widget howto tutorial api

Beginner’s Guide To jQuery-Based JSON API Clients

json based clients api development howto learning article

The Reddit API in Python

reddit api python howto tutorial open source

Objective-C JSON and the Flickr API

objectivec json flickr api tutorial howto

Vine’s Undocumented API

vine.co vine api social video sharing howto tutorial

Basic YouTube API Tutorial

pulling videos youtube api tutorial web development

Integrate the Mailchimp API in PHP

integrate mailchimp api development php tutorial

Google Maps API and Custom Overlays

google maps api custom overlays

Create a Twitter-Like “Load More” Widget

create twitter style load more widget tutorial

Creating an API-Centric Web Application

api centric web application tutorial howto

Build an Integrated Facebook App

design code howto tutorial integrated facebook application

Facebook Invite Friends API

facebook invite friends api howto coding tutorial

Dead-Simple Twitter Feed with jQuery

dead simple twitter feed using jquery tutorial

REST API Integration with iPhone App

rest api into iphone application objective-c tutorial

Coding a Dynamic IMDb Webapp

dynamic imdb webapp my movie api howto tutorial jquery

Simple Github API Webapp

coding jquery ajax howto tutorial github api

Game Audio with the Web Audio API

browser game audio web api htmnl5 howto tutorial

Access Dropbox Using PHP

access tutorial api php dropbox howto coding

Google Analytics API

access google api analytics howto tutorial coding

Simple PHP REST API

simple php rest api tutorial open source


Advertise here with BSA

May 22 2013

16:00

Laravel 4: A Start at a RESTful API (Updated)

RESTful API's are hard! There are a lot of aspects to designing and writing a successful one. For instance, some of the topics that you may find yourself handling include authentication, hypermedia/HATEOS, versioning, rate limits, and content negotiation. Rather than tackling all of these concepts, however, let's instead focus on the basics of REST. We'll make some JSON endpoints behind a basic authentication system, and learn a few Laravel 4 tricks in the process.


The App

Let's build an API for a simple Read-It-Later app. Users will be able to create, read, update and delete URLs that they wish to read later.
Ready to dive in and get started?

Install Laravel 4

Create a new install of Laravel 4. If you're handy with CLI, try this quickstart guide. Otherwise, we have a video tutorial here on Nettuts+ that covers the process.

We're going to first create an encryption key for secure password hashing. You can do this easily by running this command from your project root:

$ php artisan key:generate

Alternatively, you can simple edit your app/config/app.php encryption key:

/*
|--------------------------------------------------------------------------
| Encryption Key
|--------------------------------------------------------------------------
|
| This key is used by the Illuminate encrypter service and should be set
| to a random, long string, otherwise these encrypted values will not
| be safe. Make sure to change it before deploying any application!
|
*/

'key' => md5('this is one way to get an encryption key set'),

Database

Once you have a working install of Laravel 4, we can get started with the fun. We'll begin by creating the app's database.

This will only require two database tables:

  1. Users, including a username and password
  2. URLs, including a url and description

We'll use Laravel's migrations to create and populate the database.

Configure Your Database

Edit app/config/database.php and fill it with your database settings. Note: this means creating a database for this application to use. This article assumes a MySQL database.

'connections' => array(

    'mysql' => array(
        'driver'    => 'mysql',
        'host'      => 'localhost',
        'database'  => 'read_it_later',
        'username'  => 'your_username',
        'password'  => 'your_password',
        'charset'   => 'utf8',
        'collation' => 'utf8_unicode_ci',
        'prefix'    => '',
    ),
),

Create Migration Files

$ php artisan migrate:make create_users_table --table=users --create
$ php artisan migrate:make create_urls_table --table=urls --create

These commands set up the basic migration scripts that we'll be using to create the database tables. Our job now is to fill them with the correct table columns.

Edit app/database/migrations/SOME_DATE_create_users_table.php and add to the up() method:

public function up()
{
    Schema::create('users', function(Blueprint $table)
    {
        $table->increments('id');
        $table->string('username')->unique();
        $table->string('password');
        $table->timestamps();
    });
}

Above, we're setting a username (which should be unique), a password, as well as the timestamps. Save that, and now edit app/database/migrations/SOME_DATE_create_urls_table.php, and add to the up() method:

public function up()
{
    Schema::create('urls', function(Blueprint $table)
    {
        $table->increments('id');
        $table->integer('user_id');
        $table->string('url');
        $table->string('description');
        $table->timestamps();
    });
}

The only important note in this snippet is that we're creating a link between the url and users table, via the user_id field.

Add Sample Users

We can use Laravel's seeds to create a few sample users.

Create a file within the app/database/seeds folder that has the same name as the table that it corresponds to; in our case, UserTableSeeder.php. Add:

<?php

class UserTableSeeder extends Seeder {

    public function run()
    {
        DB::table('users')->delete();

        User::create(array(
            'username' => 'firstuser',
            'password' => Hash::make('first_password')
        ));

        User::create(array(
            'username' => 'seconduser',
            'password' => Hash::make('second_password')
        ));
    }

}

Next, make sure that seeder class gets run when the database is seeded. Edit app/database/seeds/DatabaseSeeder.php:

public function run()
{
    Eloquent::unguard();

    // Add or Uncomment this line
    $this->call('UserTableSeeder');
}

Run the Migrations

Here’s how to create those two tables, and insert our sample users.

// Create the two tables
$ php artisan migrate

// Create the sample users
$ php artisan db:seed

Models

Laravel 4 continues to use the excellent Eloquent ORM. This will make the process of handling database calls a snap. We'll require one model per table.

Luckily, Laravel comes with a User model setup, so let's create a model for our urls table.

Create and edit file app/models/Url.php.

<?php

class Url extends Eloquent {

    protected $table = 'urls';

}

Authentication

Laravel's filters can handle authentication for us. In particular, Laravel now comes with a Basic Authentication filter, which we can use as a quick authentication model to be used with our API requests.

If you open app/filters.php, you'll see what it looks like:

Route::filter('auth.basic', function()
{
    return Auth::basic();
});

We just need to make one adjustment. By default, this filter looks for an "email" field to identify the user. Since we're using usernames instead of emails, we just need to adjust that preference. Change the Auth::basic() call by giving it our username field as a parameter:

Route::filter('auth.basic', function()
{
    return Auth::basic("username");
});

Routes

Let's test this out. Create a route, called testauth, and make sure that our auth.basic filter runs before it.

Edit app/routes.php:

Route::get('/authtest', array('before' => 'auth.basic', function()
{
    return View::make('hello');
}));

We can test this with a curl request. From your terminal, try pointing to your build of Laravel. In mine, it looks like this (Your URL will likely be different!):

$ curl -i localhost/l4api/public/index.php/authtest
HTTP/1.1 401 Unauthorized
Date: Tue, 21 May 2013 18:47:59 GMT
WWW-Authenticate: Basic
Vary: Accept-Encoding
Content-Type: text/html; charset=UTF-8

Invalid credentials

As you can see, an unauthorized request is detected and a "Invalid Credentials" message is returned with a 401 status code. Next, try including basic authentication.

$ curl --user firstuser:first_password localhost/l4api/public/index.php/authtest
HTTP/1.1 200 OK
Date: Tue, 21 May 2013 18:50:51 GMT
Vary: Accept-Encoding
Content-Type: text/html; charset=UTF-8

<h1>Hello World!</h1>

It worked!

At this point, the baseline work of our API is done. We have:

  • Installed Laravel 4
  • Created our database
  • Created our models
  • Created an authentication model

Creating Functional Requests

You may be familiar with Laravel's RESTful controllers. They still exist in Laravel 4; however, we can also use Laravel's Resourceful Controllers, which set up some paradigms that we can use to make a consistent API interface. We'll be using a Resourceful controller.

Here's a breakdown of what each method in the resourceful controller will handle. Please note that you can remove the /resource/create and /resource/{id}/edit routes, since we won't be needing to show 'create' or 'edit' forms in an API.

Create a Resourceful Controller

$ php artisan controller:make UrlController

Next, setup a route to use the controller, and require each route to be authenticated.

Edit app/routes.php and add:

// Route group for API versioning
Route::group(array('prefix' => 'api/v1', 'before' => 'auth.basic'), function()
{
    Route::resource('url', 'UrlController');
});

A few things are happening there.

  1. This is going to respond to requests made to http://example.com/api/v1/url.
  2. This allows us to add extra routes, if we need to expand our API. For instance, if you add a user end-point, such as /api/v1/user.
  3. There is also a naming mechanism in place for versioning our API. This gives us the opportunity to roll out new API versions without breaking older versions – We can simply create a v2 route group, and point it to a new controller!

Note: You may want to consider more advanced API versioning techniques, such as using an Accept header or subdomain which can help you point different API versions separate code bases.

Add the Functionality

Edit the new app/controllers/UrlController.php file:

// Edit this:
public function index()
{
    return 'Hello, API';
}

Let’s test it:

$ curl -i localhost/l4api/public/index.php/api/v1/url
HTTP/1.1 401 Unauthorized
Date: Tue, 21 May 2013 19:02:59 GMT
WWW-Authenticate: Basic
Vary: Accept-Encoding
Content-Type: text/html; charset=UTF-8

Invalid credentials.

$ curl --user firstuser:first_password localhost/l4api/public/index.php/api/v1/url
HTTP/1.1 200 OK
Date: Tue, 21 May 2013 19:04:19 GMT
Vary: Accept-Encoding
Content-Type: text/html; charset=UTF-8

Hello, API

We now have a resourceful controller with authentication working, and are ready to add functionality.

Create a URL

Edit app/controllers/UrlController.php:

/**
 * Store a newly created resource in storage.
 *
 * @return Response
 */
public function store()
{
    $url = new Url;
    $url->url = Request::get('url');
    $url->description = Request::get('description');
    $url->user_id = Auth::user()->id;

    // Validation and Filtering is sorely needed!!
    // Seriously, I'm a bad person for leaving that out.

    $url->save();

    return Response::json(array(
        'error' => false,
        'urls' => $urls->toArray()),
        200
    );
}

It's time to test this with another curl request. This one will send a POST request, which will correspond to the store() method created above.

$ curl -i --user firstuser:first_password -d 'url=http://google.com&description=A Search Engine' localhost/l4api/public/index.php/api/v1/url
HTTP/1.1 201 Created
Date: Tue, 21 May 2013 19:10:52 GMT
Content-Type: application/json

{"error":false,"message":"URL created"}

Cool! Let's create a few more, for both of our users.

$ curl --user firstuser:first_password -d 'url=http://fideloper.com&description=A Great Blog' localhost/l4api/public/index.php/api/v1/url

$ curl --user seconduser:second_password -d 'url=http://digitalsurgeons.com&description=A Marketing Agency' localhost/l4api/public/index.php/api/v1/url

$ curl --user seconduser:second_password -d 'url=http://www.poppstrong.com/&description=I feel for him' localhost/l4api/public/index.php/api/v1/url

Next, let's create methods for retrieving URLs.

/**
 * Display a listing of the resource.
 *
 * @return Response
 */
public function index()
{
    //Formerly: return 'Hello, API';

    $urls = Url::where('user_id', Auth::user()->id)->get();

    return Response::json(array(
        'error' => false,
        'urls' => $urls->toArray()),
        200
    );
}

/**
 * Display the specified resource.
 *
 * @param  int  $id
 * @return Response
 */
public function show($id)
{
    // Make sure current user owns the requested resource
    $url = Url::where('user_id', Auth::user()->id)
            ->where('id', $id)
            ->take(1)
            ->get();

    return Response::json(array(
        'error' => false,
        'urls' => $url->toArray()),
        200
    );
}

Let's test them out:

$ curl --user firstuser:first_password localhost/l4api/public/index.php/api/v1/url
{
    "error": false,
    "urls": [
       {
            "created_at": "2013-02-01 02:39:10",
            "description": "A Search Engine",
            "id": "2",
            "updated_at": "2013-02-01 02:39:10",
            "url": "http://google.com",
            "user_id": "1"
        },
        {
            "created_at": "2013-02-01 02:44:34",
            "description": "A Great Blog",
            "id": "3",
            "updated_at": "2013-02-01 02:44:34",
            "url": "http://fideloper.com",
            "user_id": "1"
        }
    ]
}

$ curl --user firstuser:first_password localhost/l4api/public/index.php/api/v1/url/1
{
    "error": false,
    "urls": [
        {
            "created_at": "2013-02-01 02:39:10",
            "description": "A Search Engine",
            "id": "2",
            "updated_at": "2013-02-01 02:39:10",
            "url": "http://google.com",
            "user_id": "1"
        }
    ]
}

Almost done. Let's now allow users to delete a url.

/**
 * Remove the specified resource from storage.
 *
 * @param  int  $id
 * @return Response
 */
public function destroy($id)
{
    $url = Url::where('user_id', Auth::user()->id)->find($id);

    $url->delete();

    return Response::json(array(
        'error' => false,
        'message' => 'url deleted'),
        200
        );
}

Now, we can delete a URL by using a DELETE request:

$ curl -i -X DELETE --user firstuser:first_password localhost/l4api/public/index.php/api/v1/url/1
HTTP/1.1 200 OK
Date: Tue, 21 May 2013 19:24:19 GMT
Content-Type: application/json

{"error":false,"message":"url deleted"}

Lastly, let's allow users to update a url.

/**
 * Update the specified resource in storage.
 *
 * @param  int  $id
 * @return Response
 */
public function update($id)
{
    $url = Url::where('user_id', Auth::user()->id)->find($id);

    if ( Request::get('url') )
    {
        $url->url = Request::get('url');
    }

    if ( Request::get('description') )
    {
        $url->description = Request::get('description');
    }

    $url->save();

    return Response::json(array(
        'error' => false,
        'message' => 'url updated'),
        200
    );
}

To test URL updates, run:

$ curl -i -X PUT --user seconduser:second_password -d 'url=http://yahoo.com' localhost/l4api/public/index.php/api/v1/url/4
HTTP/1.1 200 OK
Date: Tue, 21 May 2013 19:34:21 GMT
Content-Type: application/json

{"error":false,"message":"url updated"}

// View our changes
$ curl --user seconduser:second_password localhost/l4api/public/index.php/api/v1/url/4
{
    "error": false,
    "urls": [
        {
            "created_at": "2013-02-01 02:44:34",
            "description": "I feel for him",
            "id": "3",
            "updated_at": "2013-02-02 18:44:18",
            "url": "http://yahoo.com",
            "user_id": "1"
        }
    ]
}

And That’s It

We now have the beginnings of a fully-functioning API. I hope that you've learned a lot about how to get an API underway with Laravel 4.

To recap, we achieved the following in this lesson:

  1. Install Laravel
  2. Create the database, using migrations and seeding
  3. Use Eloquent ORM models
  4. Authenticate with Basic Auth
  5. Set up Routes, including versioning the API
  6. Create the API functionality using Resourceful Controllers

The Next Steps

If you’d like to push your API up a notch, you might consider any of the following as a next step.

  1. Validation (Hint: Laravel has a Validation library).
  2. API-request error handling – It's still possible to receive HTML response on API requests (Hint: Laravel Error Handling, plus Content Negotiation.)
  3. Content Negotiation – listening for the Accept header. (Hint: Laravel's Request Class will give you the request headers).
  4. Check out the API Craft Google Group
  5. Learn about the different types caching and how Validation Caching can improve your API
  6. Unit test your code
  7. Check out Apigee's great API resources

January 28 2012

23:18

Writing an API Wrapper in Ruby with TDD

Sooner or later, all developers are required to interact with an API. The most difficult part is always related to reliably testing the code we write, and, as we want to make sure that everything works properly, we continuosly run code that queries the API itself. This process is slow and inefficient, as we can experience network issues and data inconsistencies (the API results may change). Let’s review how we can avoid all of this effort with Ruby.


Our Goal

“Flow is essential: write the tests, run them and see them fail, then write the minimal implementation code to make them pass. Once they all do, refactor if needed.”

Our goal is simple: write a small wrapper around the Dribbble API to retrieve information about a user (called ‘player’ in the Dribbble world).
As we will be using Ruby, we will also follow a TDD approach: if you’re not familiar with this technique, Nettuts+ has a good primer on RSpec you can read. In a nutshell, we will write tests before writing our code implementation, making it easier to spot bugs and to achieve a high code quality. Flow is essential: write the tests, run them and see them fail, then write the minimal implementation code to make them pass. Once they all do, refactor if needed.

The API

The Dribbble API is fairly straightforward. At the time of this it supports only GET requests and doesn’t require authentication: an ideal candidate for our tutorial. Moreover, it offers a 60 calls per minute limit, a restriction that perfectly shows why working with APIs require a smart approach.


Key Concepts

This tutorial needs to assume that you have some familiarity with testing concepts: fixtures, mocks, expectations. Testing is an important topic (especially in the Ruby community) and even if you are not a Rubyist, I’d encourage you to dig deeper into the matter and to search for equivalent tools for your everyday language. You may want to read “The RSpec book” by David Chelimsky et al., an excellent primer on Behavior Driven Development.

To summarize here, here are three key concepts you must know:

  • Mock: also called double, a mock is “an object that stands in for another object in an example”. This means that if we want to test the interaction between an object and another, we can mock the second one. In this tutorial, we will mock the Dribbble API, as to test our code we don’t need the API, itself, but something that behaves like it and exposes the same interface.
  • Fixture: a dataset that recreates a specific state in the system. A fixture can be used to create the needed data to test a piece of logic.
  • Expectation: a test example written the from the point of view of the result we want to achieve.

Our Tools

“As a general practice, run tests every time you update them.”

WebMock is a Ruby mocking library that is used to mock (or stub) http requests. In other words, it allows you to simulate any HTTP request without actually making one. The primary advantage to this is being able to develop and test against any HTTP service without needing the service itself and without incurring in related issues (like API limits, IP restrictions and such).
VCR is a complementary tool that records any real http request and creates a fixture, a file that contains all the needed data to replicate that request without performing it again. We will configure it to use WebMock to do that. In other words, our tests will interact with the real Dribbble API just once: after that, WebMock will stub all the requests thanks to the data recorded by VCR. We will have a perfect replica of the Dribbble API responses recorded locally. In addition, WebMock will let us test edge cases (like the request timing out) easily and consistently. A wonderful consequence of our setup is that everything will be extremely fast.

As for unit testing, we will be using Minitest. It’s a fast and simple unit testing library that also supports expectations in the RSpec fashion. It offers a smaller feature set, but I find that this actually encourages and pushes you to separate your logic into small, testable methods. Minitest is part of Ruby 1.9, so if you’re using it (I hope so) you don’t need to install it. On Ruby 1.8, it’s only a matter of gem install minitest.

I will be using Ruby 1.9.3: if you don’t, you will probably encounter some issues related to require_relative, but I’ve included fallback code in a comment right below it. As a general practice, you should run tests every time you update them, even if I won’t be mentioning this step explicitly throughout the tutorial.


Setup

Setup

We will use the conventional /lib and /spec folder structure to organize our code. As for the name of our library, we’ll call it Dish, following the Dribbble convention of using basketball related terms.

The Gemfile will contain all our dependencies, albeit they’re quite small.

source :rubygems

gem 'httparty'

group :test do
  gem 'webmock'
  gem 'vcr'
  gem 'turn'
  gem 'rake'
end

Httparty is an easy to use gem to handle HTTP requests; it will be the core of our library. In the test group, we will also add Turn to change the output of our tests to be more descriptive and to support color.

The /lib and /spec folders have a symmetrical structure: for every file contained in the /lib/dish folder, there should be a file inside /spec/dish with the same name and the ‘_spec’ suffix.

Let’s start by creating a /lib/dish.rb file and add the following code:

require "httparty"
Dir[File.dirname(__FILE__) + '/dish/*.rb'].each do |file|
  require file
end

It doesn’t do much: it requires ‘httparty’ and then iterates over every .rb file inside /lib/dish to require it. With this file in place, we will be able to add any functionality inside separate files in /lib/dish and have it automatically loaded just by requiring this single file.

Let’s move to the /spec folder. Here’s the content of the spec_helper.rb file.

#we need the actual library file
require_relative '../lib/dish'
# For Ruby < 1.9.3, use this instead of require_relative
# require(File.expand_path('../../lib/dish', __FILE__))

#dependencies
require 'minitest/autorun'
require 'webmock/minitest'
require 'vcr'
require 'turn'

Turn.config do |c|
 # :outline  - turn's original case/test outline mode [default]
 c.format  = :outline
 # turn on invoke/execute tracing, enable full backtrace
 c.trace   = true
 # use humanized test names (works only with :outline format)
 c.natural = true
end

#VCR config
VCR.config do |c|
  c.cassette_library_dir = 'spec/fixtures/dish_cassettes'
  c.stub_with :webmock
end

There’s quite a few things here worth noting, so let’s break it piece by piece:

  • At first, we require the main lib file for our app, making the code we want to test available to the test suite. The require_relative statement is a Ruby 1.9.3 addition.
  • We then require all the library dependencies: minitest/autorun includes all the expectations we will be using, webmock/minitest adds the needed bindings between the two libraries, while vcr and turn are pretty self-explanatory.
  • The Turn config block merely needs to tweak our test output. We will use the outline format, where we can see the description of our specs.
  • The VCR config blocks tells VCR to store the requests into a fixture folder (note the relative path) and to use WebMock as a stubbing library (VCR supports some other ones).

Last, but not least, the Rakefile that contains some support code:

require 'rake/testtask'

Rake::TestTask.new do |t|
  t.test_files = FileList['spec/lib/dish/*_spec.rb']
  t.verbose = true
end

task :default => test

The rake/testtask library includes a TestTask class that is useful to set the location of our test files. From now on, to run our specs, we will only type rake from the library root directory.

As a way to test our configuration, let’s add the following code to /lib/dish/player.rb:

module Dish
  class Player
  end
end

Then /spec/lib/dish/player_spec.rb:

require_relative '../../spec_helper'
# For Ruby < 1.9.3, use this instead of require_relative
# require (File.expand_path('./../../../spec_helper', __FILE__))

describe Dish::Player do

  it "must work" do
    "Yay!".must_be_instance_of String
  end

end

Running rake should give you one test passing and no errors. This test is by no means useful for our project, yet it implicitly verifies that our library file structure is in place (the describe block would throw an error if the Dish::Player module was not loaded).


First Specs

To work properly, Dish requires the Httparty modules and the correct base_uri, i.e. the base url of the Dribbble API. Let’s write the relevant tests for these requirements in player_spec.rb:

...
describe Dish::Player do

  describe "default attributes" do

    it "must include httparty methods" do
      Dish::Player.must_include HTTParty
    end

    it "must have the base url set to the Dribble API endpoint" do
      Dish::Player.base_uri.must_equal 'http://api.dribbble.com'
    end

  end

end

As you can see, Minitest expectations are self-explanatory, especially if you are an RSpec user: the biggest difference is wording, where Minitest prefers “must/wont” to “should/should_not”.

Running these tests will show one error and one failure. To have them pass, let’s add our first lines of implementation code to player.rb:

module Dish

  class Player

    include HTTParty

    base_uri 'http://api.dribbble.com'

  end

end

Running rake again should show the two specs passing. Now our Player class has access to all Httparty class methods, like get or post.


Recording our First Request

As we will be working on the Player class, we will need to have API data for a player. The Dribbble API documentation page shows that the endpoint to get data about a specific player is http://api.dribbble.com/players/:id

As in typical Rails fashion, :id is either the id or the username of a specific player. We will be using simplebits, the username of Dan Cederholm, one of the Dribbble founders.

To record the request with VCR, let’s update our player_spec.rb file by adding the following describe block to the spec, right after the first one:

  ...

  describe "GET profile" do

  before do
    VCR.insert_cassette 'player', :record => :new_episodes
  end

  after do
    VCR.eject_cassette
  end

  it "records the fixture" do
    Dish::Player.get('/players/simplebits')
  end

  end

end

After running rake, you can verify that the fixture has been created. From now on, all our tests will be completely network independent.

The before block is used to execute a specific portion of code before every expectation: we use it to add the VCR macro used to record a fixture that we will call ‘player’. This will create a player.yml file under spec/fixtures/dish_cassettes. The :record option is set to record all new requests once and replay them on every subsequent, identical request. As a proof of concept, we can add a spec whose only aim is to record a fixture for simplebits’s profile. The after directive tells VCR to remove the cassette after the tests, making sure that everything is properly isolated. The get method on the Player class is made available, thanks to the inclusion of the Httparty module.

After running rake, you can verify that the fixture has been created. From now on, all our tests will be completely network independent.


Getting the Player Profile

Dribbble

Every Dribbble user has a profile that contains a pretty extensive amount of data. Let’s think about how we would like our library to be when actually used: this is a useful way to flesh out our DSL will work. Here’s what we want to achieve:

simplebits = Dish::Player.new('simplebits')
simplebits.profile
  => #returns a hash with all the data from the API
simplebits.username
  => 'simplebits'
simplebits.id
  => 1
simplebits.shots_count
  => 157

Simple and effective: we want to instantiate a Player by using its username and then get access to its data by calling methods on the instance that map to the attributes returned by the API. We need to be consistent with the API itself.

Let’s tackle one thing at a time and write some tests related to getting the player data from the API. We can modify our "GET profile" block to have:

describe "GET profile" do

  let(:player) { Dish::Player.new }

  before do
    VCR.insert_cassette 'player', :record => :new_episodes
  end

  after do
    VCR.eject_cassette
  end

  it "must have a profile method" do
    player.must_respond_to :profile
  end

  it "must parse the api response from JSON to Hash" do
    player.profile.must_be_instance_of Hash
  end

  it "must perform the request and get the data" do
    player.profile["username"].must_equal 'simplebits'
  end

end

The let directive at the top creates a Dish::Player instance available in the expectations. Next, we want to make sure that our player has got a profile method whose value is a hash representing the data from the API. As a last step, we test a sample key (the username) to make sure that we actually perform the request.

Note that we’re not yet handling how to set the username, as this is a further step. The minimal implementation required is the following:

...
class Player

  include HTTParty

  base_uri 'http://api.dribbble.com'

  def profile
    self.class.get '/players/simplebits'
  end

end
...

A very little amount of code: we’re just wrapping a get call in the profile method. We then pass the hardcoded path to retrieve simplebits’s data, data that we had already stored thanks to VCR.

All our tests should be passing.


Setting the Username

Now that we have a working profile function, we can take care of the username. Here are the relevant specs:

describe "default instance attributes" do

  let(:player) { Dish::Player.new('simplebits') }

  it "must have an id attribute" do
    player.must_respond_to :username
  end

  it "must have the right id" do
    player.username.must_equal 'simplebits'
  end

end

describe "GET profile" do

  let(:player) { Dish::Player.new('simplebits') }

  before do
    VCR.insert_cassette 'base', :record => :new_episodes
  end

  after do
    VCR.eject_cassette
  end

  it "must have a profile method" do
    player.must_respond_to :profile
  end

  it "must parse the api response from JSON to Hash" do
    player.profile.must_be_instance_of Hash
  end

  it "must get the right profile" do
    player.profile["username"].must_equal "simplebits"
  end

end

We’ve added a new describe block to check the username we’re going to add and simply amended the player initialization in the GET profile block to reflect the DSL we want to have. Running the specs now will reveal many errors, as our Player class doesn’t accept arguments when initialized (for now).

Implementation is very straightforward:

...
class Player

  attr_accessor :username

  include HTTParty

  base_uri 'http://api.dribbble.com'

  def initialize(username)
    self.username = username
  end

  def profile
    self.class.get "/players/#{self.username}"
  end

end
...

The initialize method gets a username that gets stored inside the class thanks to the attr_accessor method added above. We then change the profile method to interpolate the username attribute.

We should get all our tests passing once again.


Dynamic Attributes

At a basic level, our lib is in pretty good shape. As profile is a Hash, we could stop here and already use it by passing the key of the attribute we want to get the value for. Our goal, however, is to create an easy to use DSL that has a method for each attribute.

Let’s think about what we need to achieve. Let’s assume we have a player instance and stub how it would work:

player.username
  => 'simplebits'
player.shots_count
  => 157
player.foo_attribute
  => NoMethodError

Let’s translate this into specs and add them to the GET profile block:

...
describe "dynamic attributes" do

  before do
    player.profile
  end

  it "must return the attribute value if present in profile" do
    player.id.must_equal 1
  end

  it "must raise method missing if attribute is not present" do
    lambda { player.foo_attribute }.must_raise NoMethodError
  end

end
...

We already have a spec for username, so we don’t need to add another one. Note a few things:

  • we explicitly call player.profile in a before block, otherwise it will be nil when we try to get the attribute value.
  • to test that foo_attribute raises an exception, we need to wrap it in a lambda and check that it raises the expected error.
  • we test that id equals 1, as we know that that is the expected value (this is a purely data-dependent test).

Implementation-wise, we could define a series of methods to access the profile hash, yet this would create a lot of duplicated logic. Moreover, the would rely on the API result to always have the same keys.

“We will rely on method_missing to handle this cases and ‘generate’ all those methods on the fly.”

Instead, we will rely on method_missing to handle this cases and ‘generate’ all those methods on the fly. But what does this mean? Without going into too much metaprogramming, we can simply say that every time we call a method not present on the object, Ruby raises a NoMethodError by using method_missing. By redefining this very method inside a class, we can modify its behaviour.

In our case, we will intercept the method_missing call, verify that the method name that has been called is a key in the profile hash and in case of positive result, return the hash value for that key. If not, we will call super to raise a standard NoMethodError: this is needed to make sure that our library behaves exactly the way any other library would do. In other words, we want to guarantee the least possible surprise.

Let’s add the following code to the Player class:

def method_missing(name)
  if profile.has_key?(name.to_s)
    profile[name.to_s]
  else
    super
  end
end

The code does exactly what described above. If you now run the specs, you should have them all pass. I’d encorage you to add some more to the spec files for some other attribute, like shots_count.

This implementation, however, is not really idiomatic Ruby. It works, but it can be streamlined into a ternary operator, a condensed form of an if-else conditional. It can be rewritten as:

def method_missing(name, *args, &block)
  profile.has_key?(name.to_s) ? profile[name.to_s] : super
end

It’s not just a matter of length, but also a matter of consistency and shared conventions between developers. Browsing source code of Ruby gems and libraries is a good way to get accustomed to these conventions.


Caching

As a final step, we want to make sure that our library is efficient. It should not make any more requests than needed and possibly cache data internally. Once again, let’s think about how we could use it:

player.profile
  => performs the request and returns a Hash
player.profile
  => returns the same hash
player.profile(true)
  => forces the reload of the http request and then returns the hash (with data changes if necessary)

How can we test this? We can by using WebMock to enable and disable network connections to the API endpoint. Even if we’re using VCR fixtures, WebMock can simulate a network Timeout or a different response to the server. In our case, we can test caching by getting the profile once and then disabling the network. By calling player.profile again we should see the same data, while by calling player.profile(true) we should get a Timeout::Error, as the library would try to connect to the (disabled) API endpoint.

Let’s add another block to the player_spec.rb file, right after dynamic attribute generation:

describe "caching" do

  # we use Webmock to disable the network connection after
  # fetching the profile
  before do
    player.profile
    stub_request(:any, /api.dribbble.com/).to_timeout
  end

  it "must cache the profile" do
    player.profile.must_be_instance_of Hash
  end

  it "must refresh the profile if forced" do
    lambda { player.profile(true) }.must_raise Timeout::Error
  end

end

The stub_request method intercepts all calls to the API endpoint and simulates a timeout, raising the expected Timeout::Error. As we did before, we test the presence of this error in a lambda.

Implementation can be tricky, so we’ll split it into two steps. Firstly, let’s move the actual http request to a private method:

...
def profile
  get_profile
end

...

private

def get_profile
  self.class.get("/players/#{self.username}")
end
...

This will not get our specs passing, as we’re not caching the result of get_profile. To do that, let’s change the profile method:

...
def profile
  @profile ||= get_profile
end
...

We will store the result hash into an instance variable. Also note the ||= operator, whose presence makes sure that get_profile is run only if @profile returns a falsy value (like nil).

Next we can add the forced reload directive:

...
def profile(force = false)
  force ? @profile = get_profile : @profile ||= get_profile
end
...

We’re using a ternary again: if force is false, we perform get_profile and cache it, if not, we use the logic written in the previous version of this method (i.e. performing the request only if we don’t have already an hash).

Our specs should be green now and this is also the end of our tutorial.


Wrapping Up

Our purpose in this tutorial was writing a small and efficient library to interact with the Dribbble API; we’ve laid the foundation for this to happen. Most of the logic we’ve written can be abstracted and requesed to access all the other endpoints. Minitest, WebMock and VCR have proven to be valuable tools to help us shape our code.

We do, however, need to be aware of a small caveat: VCR can become a double-edged sword, as our tests can become too much data-dependent. If, for any reason, the API we’re building against changes without any visible sign (like a version number), we may risk having our tests perfectly working with a dataset, which is no longer relevant. In that case, removing and recreating the fixture is the best way to make sure that our code still works as expected.


Tags: Ruby api TDD

January 18 2012

00:39

The Principles of Web API Usage

Not too long ago, I wrote an article about “The Increasing Importance of APIs in Web Development.” As a follow-up, today, I’ll cover the basics of using Web APIs.


Before we begin: this article does not detail the process of API design or development. That’s for a different article!


Basic Principles

There are a handful of basic principles that should be followed when using Web APIs:

  • Build applications that are social and engaging
  • Give users choice and control whenever applicable
  • Don’t surprise, mislead or confuse users
  • Don’t create or distribute spam; always encourage genuine communication
  • Respect the user’s privacy
  • Be a good partner to Web API providers

Most of these guidelines are listed in the API “terms of service” for Twitter and Facebook, but can be applied to all Web APIs.


Evaluate Your Coding Skills

There are thousands of Web APIs available and most can be either integrated into a website, or to create a mashup. Some Web APIs are fairly simple and require minimal coding skills to implement, while others can be extremely complex and require advanced programming skills.

The programming language you use to integrate a Web API into your website certainly depends on which ones you’re most comfortable with. You’ll find that many Web APIs, such as Flickr and Last.fm offer language specific toolkits.


Read the Terms and Conditions

Every Web API has specific terms regarding its usage; so it’s incredibly important that the “terms and conditions” are read carefully. They will detail many aspects of Web API usage, including:

  • Copyright Infringement
  • Logo Identity, Branding & Trademarks
  • Content Usage & Guidelines
  • General Restrictions
  • Usage Limitations
  • Use of cache files
  • Privacy Policy

There may be additional “terms of use” information located on separate web pages. For example, there are “Display Guidelines” described in detail on separate web pages for Twitter, Foursquare, LinkedIn and Facebook.


Read the Documentation

Most Web APIs have detailed documentation available, which usually contains important information such as:

  • Features list
  • Available data formats
  • How to request an API key
  • How to use authentication (OAuth)
  • Examples of API calls & data requests
  • Tutorials and sample code
  • API reference page

Use the Latest Version / Versioning

Some Web API Providers recommend the use of a versioning parameter in the API call.

Many Web API providers frequently release new versions of their APIs. These new releases may include the following:

  • Added functionality
  • Increased speed
  • Improved stability
  • Improved accuracy
  • Bug fixes

Some Web API Providers recommend the use of a versioning parameter in the API call. Foursquare for example, accepts the param “v=YYYYMMDD” in API calls. This parameter will help indicate that the client is up to date as of the specified date.


Periodically Check the Change Log

It is a smart idea to periodically check the change log (if one exists) when using Web APIs. Many popular APIs have change logs available, such as Facebook, Google Maps, Google Charts and Foursquare.

Plenty of API providers offer additional ways to keep track of their API revisions:

  • Subscribing to the API Developer blog
  • Following the API on Twitter
  • Joining an API User Group
  • Subscribing to an API email newsletter

Usage Limits

Nearly all web APIs are subject to rate limits; some API providers even charge fees if a user exceeds the API rate limit!

Usage of most (if not all) APIs is subject to rate limits and each API provider has its own API rate limiting policy. Some API providers even charge fees if a user exceeds the API rate limit! Google recently introduced a new rate limiting policy for the Maps API which includes usage fees. Google has also provided additional details regarding the Maps API usage limits on their Geo Developers Blog. Here are links to rate limiting information for a few of the more popular APIs:

It’s incredibly important that you not abuse the API. If you’re website is making an API call for every single page load, and for every single visitor, ask yourself, “Why”? Implement proper caching techniques to improve your website’s performance, and potentially save money.


Caching Data

Using a “cache file” will prevent problems with API Rate Limiting.

Using a “cache file” will prevent problems with API Rate Limiting. In the Nettuts+ post “How to Create an Advanced Twitter Widget,”, there are instructions on how to create a PHP “cache file” script. This script retrieves the Twitter API “GET statuses/user_timeline” information and stores it in a “TXT” file located in a cache directory on the server. This script can be easily modified to work with most APIs. All you would have to do is change the txt file name and the API call. Note that the call in the script retrieves the data in JSON format.

It is important that you read the APIs “terms of use” for any reference to using cache files. For example, the Amazon Product Advertising API has restrictions on caching. Facebook recommends that if you use caching you should try to keep the data up to date.


Data Formats

The two most popular data formats for Web APIs are XML and JSON.

The two most popular data formats for Web APIs are XML and JSON; many providers offer both formats. However, JSON has become so popular among web developers to the point that some API providers, such as Foursquare, Twitter and Google+ offer the JSON data format exclusively.

API data formats are covered in detail in a previous post.


Sign-up for the API

Most API providers require users to sign up for a user account and/or an API Key. For example, Google asks its users to request an API key here. This is usually a fairly simple process and should only take a few minutes to do. You may also be asked to use a developer/application ID when using an API.


OAuth Token

Many APIs require the use of OAuth for specific API functionality.

OAuth (Open Authorization) is an open standard for authorization that allows users to share data and/or files stored on one website with another website.

Many APIs require the use of OAuth for specific API functionality, such as logging in and out of an application. The Google+ API requires ALL API calls to use an OAuth 2.0 token or an API key. Google recommends using OAuth 2.0 as the authorization protocol for Google APIs. Other examples of APIs that require the use of OAuth are LinkedIn, Twitter and Foursquare.


API Calls

An API “call” needs to be used in order for your website application to access a Web API. Depending on the type of application you are creating, you may need to use multiple API calls. Most APIs can be used with a variety of programming languages. The call may need to be modified for different programming languages. For example, in the tutorial, “How to Create an Advanced Twitter Widget”, a PHP “cache file” script is created using the following Twitter API call:


http://api.twitter.com/1/statuses/user_timeline/twitter_screen_name.json?count=3&include_rts=true&include_entities=true

If you’re using JavaScript or jQuery instead of PHP, the call will need to be changed to:


http://api.twitter.com/1/statuses/user_timeline/envatowebdev.json?count=3&include_rts=true&include_entities=true&callback=?

When using JavaScript or jQuery, “&callback=?” needs to be included in the API call, but if using PHP, it needs to be removed or the data will not be generated in a valid JSON format.


Test the API Call

There are several API testing tools available to help you test your API calls:


Platform Status

If you are having unexpected problems with an API, some API providers like Foursquare and Twitter offer platform status updates on status blogs.

Facebook offers a nice “Live Status Tool” that includes:

  • Current platform status
  • Issue history
  • Average API response time chart
  • Error count chart

Twitter also has an up to date “API Status Page” that includes:

  • Known hot issues
  • Recently closed issues
  • Live Current Performance and Availability Status
  • Performance and Availability History

Conclusion

Web APIs are becoming increasingly important in web development, and their popularity and usage has increased exponentially in the past few years. Hopefully, this article has detailed the basics of using Web APIs. If there are additional notes that I have not included, please mention them in the comments.


December 30 2011

17:28

Creating an API-Centric Web Application

Planning to start working on a new web application? In this tutorial, we’ll discuss how to create an API-centric web application, and explain why this is essential in today’s multi-platform world.


Introduction

API?

For those who are unfamiliar with the term, API is short for Application Programming Interface. According to Wikipedia:

An application programming interface (API) is a source code based specification intended to be used as an interface by software components to communicate with each other. An API may include specifications for routines, data structures, object classes, and variables.

In simpler terms, an API refers to a set of functions built into an application, which can be used by other applications (or by itself, as we’ll see later), to interact with the application. An API is a great way to expose an application’s functionality to external applications safely and securely, since all functionality that these external applications can do is limited with what functionality is exposed in the API.

What’s an “API-Centric” Web Application?

An API-Centric Web Application is a web application that basically executes most, if not, all its functionality through API calls.

An API-Centric Web Application is a web application that basically executes most, if not, all its functionality through API calls. For example, if you were to log in a user, you would send his credentials to the API, and the API would return to you a result saying if the user provided the correct user-password combination.

Another characteristic of an API-Centric Web Application is that the API will always be stateless, meaning it can’t recognize API calls by session. Since API calls will be made by usually via the backend code, it will be hard to implement session handling, since there are usually no cookies involved in that. This limitation is actually good — this “forces” a developer to build an API that works not based on the state of the current user, but rather on functionality, which in turn, makes it easier to test, since the current state of a user doesn’t need to be recreated.

Why go through all this trouble?

As web developers, we’ve seen technology evolve first hand. It’s common knowledge that people today don’t just use applications via a browser, but through other gadgets, like mobile phones and tablets. For example, this article on Mashable, entitled “Consumers Now Spending More Time on Mobile Apps Than the Web”, states:

Consumers are spending more time on mobile apps than on the web for the first time, a new report claims.

Flurry compared its mobile data to stats from comScore and Alexa, and found that in June, consumers spent 81 minutes per day using mobile apps, compared to 74 minutes of web surfing.

Here’s a more recent article from ReadWriteWeb, entitled “More People Browse On Mobile Than Use IE6 & IE7 Combined:

The latest data on browser trends from Sitepoint show that more people browse the Web on smartphones than use Internet Explorer 6 and 7 combined. Those two old clunkers have been the bugbears of Web developers for years, requiring sites to degrade as nicely as possible to that least common denominator of browsers. But it’s a new world now; 6.95% of Web activity in November 2011 was on mobile browsers, and only 6.49% was on IE 6 or 7.

As we can clearly see, more and more people get their news from alternative venues, specifically mobile devices.

What does this have to do with me creating an API-Centric Web Application?

This would inevitably lead to more usage of our application, since it can be used anywhere a person wants.

One of the main advantages of creating an API-centric application is that it helps you build functionality that can be used by ANY device, be it a browser, a mobile phone, a tablet, or even a desktop app. All you need to do is to create the API in such a way that all these devices can communicate with it, and voila! You’ll have built a centralized application that can take input and execute functionality from any device that a person has!

API-Centric Application Diagram

API-Centric Application Diagram

By creating an application in this manner, we’re able to easily take advantage of the different mediums used by different people. This would inevitably lead to more usage of an application, since it can be used anywhere a person wants.

To drive the point home, here’s an article about Twitter’s new redesigned website, which tells us about how they now use their API to power Twitter.com, essentially making it API-centric:

One of the most important architectural changes is that Twitter.com is now a client of our own API. It fetches data from the same endpoints that the mobile site, our apps for iPhone, iPad, Android, and every third-party application use. This shift allowed us to allocate more resources to the API team, generating over 40 patches. In the initial page load and every call from the client, all data is now fetched from a highly optimized JSON fragment cache.

In this tutorial, we’ll be creating a simple TODO list application that is API-Centric and create one front-end client on the browser that interacts with our TODO list application. By the end, you’ll know the integral parts of an API-Centric application, and at the same time, how to facilitate secure communication between the two. With that in mind, let’s begin!


Step 1: Plan the Application’s Functions

The TODO application we’ll be building in this tutorial will have the basic CRUD functions:

  • Create TODO Items
  • Read TODO Items
  • Update TODO Items (rename, mark as done, mark as undone)
  • Delete TODO Items

Each TODO item will have:

  • a Title
  • a Date Due
  • a Description
  • a flag to tell if the TODO Item Is Done
  • Let’s mockup the application as well so we have a guide on how it should look like afterwards:

    SimpleTODO Mockup

    SimpleTODO Mockup

    Step 2: Create the API Server

    Since we’re developing an API-Centric application, we’ll be creating two “projects”: the API Server, and the Front-end Client. Let’s begin by creating the API server first.

    On your web server’s folder, create a folder named simpletodo_api, and create an index.php file. This index.php file will act as a front controller for the API, so all requests to the API server will be made through this file. Open it up and put the following code inside:

    <?php
    // Define path to data folder
    define('DATA_PATH', realpath(dirname(__FILE__).'/data'));
    
    //include our models
    include_once 'models/TodoItem.php';
    
    //wrap the whole thing in a try-catch block to catch any wayward exceptions!
    try {
    	//get all of the parameters in the POST/GET request
    	$params = $_REQUEST;
    
    	//get the controller and format it correctly so the first
    	//letter is always capitalized
    	$controller = ucfirst(strtolower($params['controller']));
    
    	//get the action and format it correctly so all the
    	//letters are not capitalized, and append 'Action'
    	$action = strtolower($params['action']).'Action';
    
    	//check if the controller exists. if not, throw an exception
    	if( file_exists("controllers/{$controller}.php") ) {
    		include_once "controllers/{$controller}.php";
    	} else {
    		throw new Exception('Controller is invalid.');
    	}
    
    	//create a new instance of the controller, and pass
    	//it the parameters from the request
    	$controller = new $controller($params);
    
    	//check if the action exists in the controller. if not, throw an exception.
    	if( method_exists($controller, $action) === false ) {
    		throw new Exception('Action is invalid.');
    	}
    
    	//execute the action
    	$result['data'] = $controller->$action();
    	$result['success'] = true;
    
    } catch( Exception $e ) {
    	//catch any exceptions and report the problem
    	$result = array();
    	$result['success'] = false;
    	$result['errormsg'] = $e->getMessage();
    }
    
    //echo the result of the API call
    echo json_encode($result);
    exit();
    

    What we’ve essentially built here is a simple front controller that does the following:

    • Accept an API call with any number of parameters
    • Extract the Controller and Action for the API call
    • Make the necessary checks to ensure that the Controller and Action exist
    • Execute the API call
    • Catch errors, if any
    • Send back a result to the caller

    Besides the index.php file, create three folders: a controllers, models and data folder.

    API server folders
    • The controllers folder will contain all the controllers we’ll be using for the API server. We’ll be building it using the MVC architecture to make the structure of the API server cleaner and more organized.
    • The models folder will contain all the data models for the API server.
    • The data folder will be where the API server saves any data

    Go into the controllers folder and create a file called Todo.php. This will be our controller for any TODO list related tasks. With the functions we’ll be needing for our TODO application in mind, create the necessary methods for the Todo controller:

    
    <?php
    class Todo
    {
    	private $_params;
    
    	public function __construct($params)
    	{
    		$this->_params = $params;
    	}
    
    	public function createAction()
    	{
    		//create a new todo item
    	}
    
    	public function readAction()
    	{
    		//read all the todo items
    	}
    
    	public function updateAction()
    	{
    		//update a todo item
    	}
    
    	public function deleteAction()
    	{
    		//delete a todo item
    	}
    }
    

    Now, add the necessary functionality to each action. I’ll provide the code for the createAction method and I’ll leave it up to you to create the code for the other methods. If you’re not in the mood though, you can just download the source code for the demo and copy it from there.

    
    public function createAction()
    {
    	//create a new todo item
    	$todo = new TodoItem();
    	$todo->title = $this->_params['title'];
    	$todo->description = $this->_params['description'];
    	$todo->due_date = $this->_params['due_date'];
    	$todo->is_done = 'false';
    
    	//pass the user's username and password to authenticate the user
    	$todo->save($this->_params['username'], $this->_params['userpass']);
    
    	//return the todo item in array format
    	return $todo->toArray();
    }
    

    Create TodoItem.php inside the models folder so we can create the “item creation” code. Take note that I won’t be connecting to a database, rather, I’ll be saving the information into files. It should be relatively easy though to make this work with any database.

    
    <?php
    class TodoItem
    {
    	public $todo_id;
    	public $title;
    	public $description;
    	public $due_date;
    	public $is_done;
    
    	public function save($username, $userpass)
    	{
    		//get the username/password hash
    		$userhash = sha1("{$username}_{$userpass}");
    		if( is_dir(DATA_PATH."/{$userhash}") === false ) {
    			mkdir(DATA_PATH."/{$userhash}");
    		}
    
    		//if the $todo_id isn't set yet, it means we need to create a new todo item
    		if( is_null($this->todo_id) || !is_numeric($this->todo_id) ) {
    			//the todo id is the current time
    			$this->todo_id = time();
    		}
    
    		//get the array version of this todo item
    		$todo_item_array = $this->toArray();
    
    		//save the serialized array version into a file
    		$success = file_put_contents(DATA_PATH."/{$userhash}/{$this->todo_id}.txt", serialize($todo_item_array));
    
    		//if saving was not successful, throw an exception
    		if( $success === false ) {
    			throw new Exception('Failed to save todo item');
    		}
    
    		//return the array version
    		return $todo_item_array;
    	}
    
    	public function toArray()
    	{
    		//return an array version of the todo item
    		return array(
    			'todo_id' => $this->todo_id,
    			'title' => $this->title,
    			'description' => $this->description,
    			'due_date' => $this->due_date,
    			'is_done' => $this->is_done
    		);
    	}
    }
    

    The createAction method calls two functions on the TodoItem model:

    • save() – this saves the TodoItem into a file, as well as set the todo_id for the TodoItem if necessary
    • toArray() – this returns an array version of the TodoItem, where the variables are the array’s indexes

    Since the API is called via HTTP requests, let’s test that API call by calling it through the browser:

    http://localhost/simpletodo_api/?controller=todo&action=create&title=test%20title&description=test%20description&due_date=12/08/2011&username=nikko&userpass=test1234

    If everything worked, you should see a new folder inside the data folder, and inside that folder, you should see a file with the following content:

    createAction() result

    createAction() result

    Congratulations! You’ve successfully created an API server and made an API call!


    Step 3: Secure the API Server with an APP ID and APP SECRET

    Currently, the API server is set to accept ALL API requests. We’ll need to limit it to our own applications only, to ensure that only our own front-end clients are able to make API requests. Alternatively, you can actually create a system wherein users can create their own applications that have access to your API server, similar to how Facebook and Twitter applications work.

    Begin by creating a set of id-key pairs for the clients that will be using the API server. Since this is just a demo, we can use any random, 32 character string. For the APP ID, let’s say it’s application APP001.

    Open the index.php file again, and then update it with the following code:

    
    <?php
    // Define path to data folder
    define('DATA_PATH', realpath(dirname(__FILE__).'/data'));
    
    //Define our id-key pairs
    $applications = array(
    	'APP001' => '28e336ac6c9423d946ba02d19c6a2632', //randomly generated app key
    );
    //include our models
    include_once 'models/TodoItem.php';
    
    //wrap the whole thing in a try-catch block to catch any wayward exceptions!
    try {
    	//*UPDATED*
    	//get the encrypted request
    	$enc_request = $_REQUEST['enc_request'];
    
    	//get the provided app id
    	$app_id = $_REQUEST['app_id'];
    
    	//check first if the app id exists in the list of applications
    	if( !isset($applications[$app_id]) ) {
    		throw new Exception('Application does not exist!');
    	}
    
    	//decrypt the request
    	$params = json_decode(trim(mcrypt_decrypt(MCRYPT_RIJNDAEL_256, $applications[$app_id], base64_decode($enc_request), MCRYPT_MODE_ECB)));
    
    	//check if the request is valid by checking if it's an array and looking for the controller and action
    	if( $params == false || isset($params->controller) == false || isset($params->action) == false ) {
    		throw new Exception('Request is not valid');
    	}
    
    	//cast it into an array
    	$params = (array) $params;
    	...
    	...
    	...
    

    What we’ve done here is actually implement a very simple way of authenticating our front-end clients using a system similar to public-private key authentication. Basically, here is the step-by-step breakdown of how the authentication happens:

    Public-key encryption

    Public-key encryption
    • an API call is made, in it an $app_id and $enc_request is provided.
    • the $enc_request value is the API call parameters, encrypted using APP KEY. The APP KEY is NEVER sent to the server, it’s only used to hash the request. Additionally, the request can only be decrypted using the APP KEY.
    • once the API call arrives to the API server, it will check its own list of applications for the APP ID provided
    • when found, the API server attempt to decrypt the request using the key that matches the APP ID sent
    • if it was successful in decrypting it, then continue on with the program

    Now that the API server is secured with an APP ID and APP SECRET, we can begin programming a front-end client to use the API server.


    Step 4: Create the Browser Front-end Client

    We’ll begin by setting up a new folder for the front-end client. Create a folder called simpletodo_client_browser on your web server’s folder. When that’s done, create an index.php file and put this code inside:

    
    <!DOCTYPE html>
    <html>
    <head>
    	<title>SimpleTODO</title>
    
    	<link rel="stylesheet" href="css/reset.css" type="text/css" />
    	<link rel="stylesheet" href="css/bootstrap.min.css" type="text/css" />
    
    	<script src="js/jquery.min.js"></script>
    	<script src="js/jquery-ui-1.8.16.custom.min.js"></script>
    
    	<style>
    	body {
    		padding-top: 40px;
    	}
    	#main {
    		margin-top: 80px;
    		text-align: center;
    	}
    	</style>
    </head>
    <body>
    	<div class="topbar">
    		<div class="fill">
    			<div class="container">
    				<a class="brand" href="index.php">SimpleTODO</a>
    			</div>
    		</div>
    	</div>
    	<div id="main" class="container">
    		<form class="form-stacked" method="POST" action="login.php">
    			<div class="row">
    				<div class="span5 offset5">
    					<label for="login_username">Username:</label>
    					<input type="text" id="login_username" name="login_username" placeholder="username" />
    
    					<label for="login_password">Password:</label>
    					<input type="password" id="login_password" name="login_password" placeholder="password" />
    
    				</div>
    			</div>
    			<div class="actions">
    				<button type="submit" name="login_submit" class="btn primary large">Login or Register</button>
    			</div>
    		</form>
    	</div>
    </body>
    </html>
    

    That should look something like this:

    SimpleTODO Login Page

    SimpleTODO Login Page

    Take note that I’ve included 2 JavaScript files and 2 CSS files here:

    Next, let’s create the login.php file so we store the username and password inside a session on the client.

    
    <?php
    //get the form values
    $username = $_POST['login_username'];
    $userpass = $_POST['login_password'];
    
    session_start();
    $_SESSION['username'] = $username;
    $_SESSION['userpass'] = $userpass;
    header('Location: todo.php');
    exit();
    

    Here, we simply start a session for the user, based on the username and password combination the user will provide. This acts as a simple combination key, which will allow a user to access stored TODO items for a specific combination of both the username and password. We then redirect to todo.php, where we start interacting with the API server. Before we start coding the todo.php file though, let’s first create an ApiCaller class, which will encapsulate all the API calling methods we’ll need, including encrypting the requests.

    Create apicaller.php and put the following inside:

    
    <?php
    class ApiCaller
    {
    	//some variables for the object
    	private $_app_id;
    	private $_app_key;
    	private $_api_url;
    
    	//construct an ApiCaller object, taking an
    	//APP ID, APP KEY and API URL parameter
    	public function __construct($app_id, $app_key, $api_url)
    	{
    		$this->_app_id = $app_id;
    		$this->_app_key = $app_key;
    		$this->_api_url = $api_url;
    	}
    
    	//send the request to the API server
    	//also encrypts the request, then checks
    	//if the results are valid
    	public function sendRequest($request_params)
    	{
    		//encrypt the request parameters
    		$enc_request = base64_encode(mcrypt_encrypt(MCRYPT_RIJNDAEL_256, $this->_app_key, json_encode($request_params), MCRYPT_MODE_ECB));
    
    		//create the params array, which will
    		//be the POST parameters
    		$params = array();
    		$params['enc_request'] = $enc_request;
    		$params['app_id'] = $this->_app_id;
    
    		//initialize and setup the curl handler
    		$ch = curl_init();
    		curl_setopt($ch, CURLOPT_URL, $this->_api_url);
    		curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    		curl_setopt($ch, CURLOPT_POST, count($params));
    		curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
    
    		//execute the request
    		$result = curl_exec($ch);
    
    		//json_decode the result
    		$result = @json_decode($result);
    
    		//check if we're able to json_decode the result correctly
    		if( $result == false || isset($result['success']) == false ) {
    			throw new Exception('Request was not correct');
    		}
    
    		//if there was an error in the request, throw an exception
    		if( $result['success'] == false ) {
    			throw new Exception($result['errormsg']);
    		}
    
    		//if everything went great, return the data
    		return $result['data'];
    	}
    }
    

    We’ll be using the ApiCaller class to send requests to our API server. This way, all the necessary encryption and cURL initialization code will be in one place, and we won’t have to repeat our code.

    • the __construct function takes in three parameters:
      1. $app_id – the APP ID for the client (which is APP001 for the browser client)
      2. $app_key – the APP KEY for the client (which is 28e336ac6c9423d946ba02d19c6a2632 for the browser client)
      3. $api_url – the URL of the API server, which is http://localhost/simpletodo_api/
    • the sendRequest() function:
      1. encrypts the request parameters using the mcrypt library in the same manner that the API server decrypts it
      2. generates the $_POST parameters to be sent to the API server
      3. executes the API call via cURL
      4. checks the result of the API call was successful or not
      5. returns the data when everything went according to plan

    Now, let’s begin with the todo.php page. First off, let’s create some code to retrieve the current list of todo items for the user nikko with the password test1234 (this is the user/password combination we used earlier to test the API server).

    
    <?php
    session_start();
    include_once 'apicaller.php';
    
    $apicaller = new ApiCaller('APP001', '28e336ac6c9423d946ba02d19c6a2632', 'http://localhost/simpletodo_api/');
    
    $todo_items = $apicaller->sendRequest(array(
    	'controller' => 'todo',
    	'action' => 'read',
    	'username' => $_SESSION['username'],
    	'userpass' => $_SESSION['userpass']
    ));
    
    echo '';
    var_dump($todo_items);
    

    Go to the index.php page, login as nikko/test1234, and you should see a var_dump() of the TODO item we created earlier.

    TODO item var_dump()

    Congratulations, you’ve successfully made an API call to the API server! In this code, we’ve:

    • started the session so we have access to the username and userpass in the $_SESSION
    • instantiated a new ApiCaller class, giving it the APP ID, APP KEY and the URL of the API server
    • send a request via the sendRequest() method

    Now, let’s reformat the data so it looks better. Add the following HTML to the todo.php code. Don’t forget to remove the var_dump()!

    
    <!DOCTYPE html>
    <html>
    <head>
    	<title>SimpleTODO</title>
    
    	<link rel="stylesheet" href="css/reset.css" type="text/css" />
    	<link rel="stylesheet" href="css/bootstrap.min.css" type="text/css" />
    	<link rel="stylesheet" href="css/flick/jquery-ui-1.8.16.custom.css" type="text/css" />
    
    	<script src="js/jquery.min.js"></script>
    	<script src="js/jquery-ui-1.8.16.custom.min.js"></script>
    
    	<style>
    	body {
    		padding-top: 40px;
    	}
    	#main {
    		margin-top: 80px;
    	}
    
    	.textalignright {
    		text-align: right;
    	}
    
    	.marginbottom10 {
    		margin-bottom: 10px;
    	}
    	#newtodo_window {
    		text-align: left;
    		display: none;
    	}
    	</style>
    
    	<script>
    	$(document).ready(function() {
    		$("#todolist").accordion({
    			collapsible: true
    		});
    		$(".datepicker").datepicker();
    		$('#newtodo_window').dialog({
    			autoOpen: false,
    			height: 'auto',
    			width: 'auto',
    			modal: true
    		});
    		$('#newtodo').click(function() {
    			$('#newtodo_window').dialog('open');
    		});
    	});
    	</script>
    </head>
    <body>
    	<div class="topbar">
    		<div class="fill">
    			<div class="container">
    				<a class="brand" href="index.php">SimpleTODO</a>
    			</div>
    		</div>
    	</div>
    	<div id="main" class="container">
    		<div class="textalignright marginbottom10">
    			<span id="newtodo" class="btn info">Create a new TODO item</span>
    			<div id="newtodo_window" title="Create a new TODO item">
    				<form method="POST" action="new_todo.php">
    					<p>Title:<br /><input type="text" class="title" name="title" placeholder="TODO title" /></p>
    					<p>Date Due:<br /><input type="text" class="datepicker" name="due_date" placeholder="MM/DD/YYYY" /></p>
    					<p>Description:<br /><textarea class="description" name="description"></textarea></p>
    					<div class="actions">
    						<input type="submit" value="Create" name="new_submit" class="btn primary" />
    					</div>
    				</form>
    			</div>
    		</div>
    		<div id="todolist">
    			<?php foreach($todo_items as $todo): ?>
    			<h3><a href="#"><?php echo $todo->title; ?></a></h3>
    			<div>
    				<form method="POST" action="update_todo.php">
    				<div class="textalignright">
    					<a href="delete_todo.php?todo_id=<?php echo $todo->todo_id; ?>">Delete</a>
    				</div>
    				<div>
    					<p>Date Due:<br /><input type="text" id="datepicker_<?php echo $todo->todo_id; ?>" class="datepicker" name="due_date" value="12/09/2011" /></p>
    					<p>Description:<br /><textarea class="span8" id="description_<?php echo $todo->todo_id; ?>" class="description" name="description"><?php echo $todo->description; ?></textarea></p>
    				</div>
    				<div class="textalignright">
    					<?php if( $todo->is_done == 'false' ): ?>
    					<input type="hidden" value="false" name="is_done" />
    					<input type="submit" class="btn" value="Mark as Done?" name="markasdone_button" />
    					<?php else: ?>
    					<input type="hidden" value="true" name="is_done" />
    					<input type="button" class="btn success" value="Done!" name="done_button" />
    					<?php endif; ?>
    					<input type="hidden" value="<?php echo $todo->todo_id; ?>" name="todo_id" />
    					<input type="hidden" value="<?php echo $todo->title; ?>" name="title" />
    					<input type="submit" class="btn primary" value="Save Changes" name="update_button" />
    				</div>
    				</form>
    			</div>
    			<?php endforeach; ?>
    		</div>
    	</div>
    </body>
    </html>
    

    It should now look something like this:

    TODO Home

    Pretty cool huh? But this currently does nothing, so let’s begin adding some functionality. I’ll provide the code for new_todo.php, which will call the todo/create API call to create a new TODO item. Creating the other pages (update_todo.php and delete_todo.php) should be very similar to this one, so I’ll leave it up to you to create those. Open up new_todo.php and add the following code:

    
    <?php
    session_start();
    include_once 'apicaller.php';
    
    $apicaller = new ApiCaller('APP001', '28e336ac6c9423d946ba02d19c6a2632', 'http://localhost/simpletodo_api/');
    
    $new_item = $apicaller->sendRequest(array(
    	'controller' => 'todo',
    	'action' => 'create',
    	'title' => $_POST['title'],
    	'due_date' => $_POST['due_date'],
    	'description' => $_POST['description'],
    	'username' => $_SESSION['username'],
    	'userpass' => $_SESSION['userpass']
    ));
    
    header('Location: todo.php');
    exit();
    ?>
    

    As you can see, the new_todo.php page uses the ApiCaller again to facilitate the sending the todo/create request to the API server. This basically does the same thing as before:

    • start a session so it has access to the $username and $userpass saved in the $_SESSION
    • instantiate a new ApiCaller class, giving it the APP ID, APP KEY and the URL of the API server
    • send the request via the sendRequest() method
    • redirect back to todo.php
    New TODO!

    Congratulations, it works! You’ve successfully created an API-centric application!


    Conclusion

    There are so many advantages to developing an application that’s built around an API. Want to create an Android application version of SimpleTODO? All the functionality you would need is already in the API server, so all you need to do is just create the client! Want to refactor or optimize some of the classes? No problem — just make sure the output is the same. Need to add more functionality? You can do it wihtout affecting any of the client’s code!

    Though there are some disadvantages like longer development times or more complexity, the advantages of developing a web application in this manner greatly outweight the disadvantages. It’s up to us to leverage on this kind of development today so we can reap the benefits later on.

    Are you planning to use an API server for your next web application, or have you already used the same technique for a project in the past? Let me know in the comments!


December 01 2011

18:00

Wrangling with the Facebook Graph API

Have you ever wanted to learn how to make your applications more social with Facebook? It’s much easier than you think!

In this tutorial, we’ll be building an application that reads and publishes data to and from Facebook using Facebook’s Graph API. Interested? Join me after the jump!


A Short Introduction to the Open Graph Protocol

Facebook Open Graph

Open Graph Protocol

Image courtesy of http://ogp.me

The Open Graph protocol enables any web page to become a rich object in a social graph. For instance, this is used on Facebook to allow any web page to have the same functionality as any other object on Facebook.

While many different technologies and schemas exist and could be combined together, there isn’t a single technology which provides enough information to richly represent any web page within the social graph. The Open Graph protocol builds on these existing technologies and gives developers one thing to implement. Developer simplicity is a key goal of the Open Graph protocol which has informed many of the technical design decisions.

In a nutshell, the Open Graph Protocol turns any web page into an object in a huge graph. Each object, at least in Facebook’s graph objects, can have many other objects linked to it. For example, a Facebook Page can have multiple Post objects, which are posts made by that page. In turn, each Post object can have multiple Comment objects attached to it, referring to comments written by people on the post. This relationship between graph objects is the basis for Facebook’s Graph API, which in turn allows us to do CRUD operations on these objects.

In this tutorial, we’ll be learning how to use and integrate the Facebook Graph API into an application. We’ll also learn how to use data from the Graph API to do operations like logging in a user via their Facebook account. Ultimately, we’ll be creating a small application than allows people to create and read posts from a Facebook Page they’re managing, similar to HootSuite or TweetDeck.


Step 1: Create a Facebook Application

The first thing you should do when you’re planning to use the Facebook Graph API is to create a Facebook application. This doesn’t necessarily mean that we’ll be putting the application on Facebook (although we can); we just need a Facebook application (specifically an APP ID and APP SECRET) to access the API.

Open http://developers.facebook.com and click on the Apps link in the navigation bar.

Facebook's Developer Site

Facebook’s Developer Site

You’ll be prompted to log in (if you’re not) and allow the Developer application to access your account. Just click on Allow and you’ll be redirected to the Developer App homepage.

Allow the Developer application access

Allow the Developer application access

On the Developer App homepage, click on Create New App in the upper right corner of the page.

Developer App homepage

Developer App homepage

You’ll be greeted with a modal window asking for an App Display Name and App Namespace. Provide anything you want here, but for the purposes of this tutorial, I’ll be using Nettuts+ Page Manager and nettuts-page-manager respectively. Click on Continue.

Birth of the Nettuts+ Page Manager

Birth of the Nettuts+ Page Manager

After some obligatory captcha checking, you’ll be redirected to your newly-minted application’s page. Here you’ll see the APP ID and APP SECRET that we need. Copy and paste these values somewhere for later use.

The APP ID and APP SECRET

The APP ID and APP SECRET

When that’s done, go to the lower part of the page and click on the “Website” box, and below it should appear a form that asks for a Site URL. Since I’m just using my local machine to build the application, I’ll use http://localhost/nettuts/pagemanager. When you’re done, click on the Save Changes button below.

Nettuts+ Page Manager Settings

Nettuts+ Page Manager Settings

Step 2: Download and Set Up Facebook’s PHP SDK

Facebook's GitHub page

Facebook’s GitHub page

Our next task is to download and set up Facebook’s PHP SDK. The best location to get it would be on Facebook’s GitHub page, since this is where the latest and greatest version of the PHP SDK will be.

Downloading Facebook's PHP SDK

Downloading Facebook’s PHP SDK

Point your browser to https://github.com/facebook/php-sdk and click on the “ZIP” button. This should prompt a download for the latest version of the SDK. Save it anywhere you like.

Now, extract this into PHP’s include_path to make it accessible by any application. Alternatively, if you’re just using this for one application, extract it inside the application’s folder — just make sure to take note where, since we’ll have to include facebook.php in our code later.


Step 3: Read from Facebook via the Graph API

Let’s start creating our project and using the Facebook Graph API to read information from Facebook. For starters, create an index.php file where a user can log in via Facebook. The index.php file should contain the following code:

<!DOCTYPE html>
<html>
<head>
	<title>Nettuts+ Page Manager</title>
	<link rel="stylesheet" href="css/reset.css" type="text/css" />
	<link rel="stylesheet" href="css/bootstrap.min.css" type="text/css" />
	<script src="js/jquery.min.js"></script>

	<style>
	body {
		padding-top: 40px;
	}
	#main {
		margin-top: 80px;
		text-align: center;
	}
	</style>
</head>
<body>
	<div class="topbar">
		<div class="fill">
		<div class="container">
			<a class="brand" href="/">Nettuts+ Page Manager</a>
		</div>
	</div>
	<div id="main" class="container">
		<a href="connect.php" class="btn primary large">Login via Facebook</a>
	</div>
</body>
</html>

If you’re wondering, reset.css is just your standard stylesheet reset, and bootstrap.min.css is Twitter Bootstrap. I’ve also added jQuery into the mix to make it easier to do client-side stuff. Now, if you refresh the page, it should look something like this:

Nettuts+ Page Manager First Run
Nettuts+ Page Manager First Run

Now let’s create our connect.php file, which will enable us to connect a user’s Facebook account and the pages that he or she manages. Let’s start by including the Facebook library we downloaded earlier. Instantiate it using the APP ID and APP SECRET:


//include the Facebook PHP SDK
include_once 'facebook.php';

//instantiate the Facebook library with the APP ID and APP SECRET
$facebook = new Facebook(array(
	'appId' => 'REPLACE WITH YOUR APP ID',
	'secret' => 'REPLACE WITH YOUR APP SECRET',
	'cookie' => true
));

The $facebook variable can now be used to make API calls to Facebook on behalf of the application.

  • The appID setting tells Facebook which application we’re using.
  • The secret setting “authenticates” our API calls, telling Facebook that they came from someone who owns the application. This should never be shown to the public, which is why it’s named the “Application Secret.”
  • The cookie setting tells the library to store the user’s session using cookies. Without it, we won’t be able to know whether the user is logged in via Facebook or not.

Now, we check if the current user has already allowed access to the application. If not, the application has to redirect them to Facebook’s “Allow Permissions” page.


//Get the FB UID of the currently logged in user
$user = $facebook->getUser();

//if the user has already allowed the application, you'll be able to get his/her FB UID
if($user) {
	//do stuff when already logged in
} else {
	//if not, let's redirect to the ALLOW page so we can get access
	//Create a login URL using the Facebook library's getLoginUrl() method
	$login_url_params = array(
		'scope' => 'publish_stream,read_stream,offline_access,manage_pages',
		'fbconnect' =>  1,
		'redirect_uri' => 'http://'.$_SERVER['HTTP_HOST'].$_SERVER['REQUEST_URI']
	);
	$login_url = $facebook->getLoginUrl($login_url_params);

	//redirect to the login URL on facebook
	header("Location: {$login_url}");
	exit();
}

Essentially, this is all that happens here:

  • The application makes a simple API call to get the user’s Facebook User ID (also known as FB UID) via the $facebook->getUser() method.
  • If the $user variable has a value, it means that the user has already allowed basic permissions for the application.
  • If not, then it means the user hasn’t allowed the application permissions yet, and the application needs to redirect to Facebook’s permissions page to get the necessary permissions.
  • It then generates a Login URL, which is where the application should redirect the user to show Facebook’s permissions page for the application. The getLoginUrl() method takes in the following parameters:
    • scope – this is a comma-delimited list of permissions the application needs
    • fbconnect – this should be 1 to tell Facebook that the application will be using Facebook to authenticate the user
    • redirect_uri – this is the page Facebook redirects to after the user has gone through the Facebook permissions page

    You can read more about the getLoginUrl() method here: https://developers.facebook.com/docs/reference/php/facebook-getLoginUrl/

  • Afterwards, the application redirects the user to the Login URL and the user sees Facebook’s permissions page.
Facebook's Permissions Page

Facebook’s Permissions Page

Facebook Permissions

Let’s take a minute to talk about Facebook permissions. Similar to when you install an application on your Android phone, Facebook permissions are various levels of access a Facebook application can do on behalf of a user. For example, if we wanted access to user’s email address, we can add the email permission to the scope setting we use to generate the login URL.

Remember, with great power comes great responsibility, so make sure to only use your permissions for good, not for evil.

It’s important that your application only asks a user for permissions that it actually needs. If you only need to authenticate a user via Facebook, you don’t even need to ask any permissions at all! Remember, with great power comes great responsibility, so make sure to only use your permissions for good, not for evil.

In the app, we use the following permissions:

  • publish_stream – allows the application to publish updates to Facebook on the user’s behalf
  • read_stream – allows the application to read from the user’s News Feed
  • offline_access – converts the access_token to one that doesn’t expire, thus letting the application make API calls anytime. Without this, the application’s access_token will expire after a few minutes, which isn’t ideal in this case
  • manage_pages – lets the application access the user’s Facebook Pages. Since the application we’re building deals with Facebook Pages, we’ll need this as well.

There are a lot more permissions Facebook requires for different things. You can read all about them in Facebook’s documentation for permissions.

Going back to the application, now that the user has allowed the permissions required by the application, we can do some API calls to Facebook! Place this inside the if-else block from the code above:

<pre>
//start the session if needed
if( session_id() ) {

} else {
	session_start();
}

//get the user's access token
$access_token = $facebook->getAccessToken();

//check permissions list
$permissions_list = $facebook->api(
	'/me/permissions',
	'GET',
	array(
		'access_token' => $access_token
	)
);
</pre>

The first thing we do is get the $access_token for the user that the application just authenticated. This is crucial, as the application will need this access token for almost everything we do. To get it, we use the getAccessToken() method. The $access_token acts as a ‘password’ for the user. It’s always unique for every user on every application, so make sure to store this when needed.

Afterwards, we can see how to make API calls to Facebook using the api() method. This method takes in the following parameters:

  • $graph_path – this is the Facebook graph path, which is essentially the “URL” to the open graph object we want to access. This can be any graph object on Facebook, like a Post (e.g. ‘/<post_id>’), a Comment (e.g. ‘/<comment_id>’), or even a User (‘/me’ is a shortcut for the current user whom the $access_token belongs to. It can also be ‘/<user_name>’ or ‘/<fb_uid>’, but the $access_token you’re using must have access to that user, or you’ll only be able to get the user’s public information).
  • $method – this is the kind of method you want to do. It’s usually GET when you’re trying to “get” information, and POST when you’re trying to “post” or update information and DELETE if you want to remove information. If you’re not sure, the Graph API documentation tells you which method to use for specific API calls.
  • $params – these are the parameters that come with your API request. Usually for GET methods you only need to supply the user’s $access_token. For POST methods though, you’ll also need to provide other parameters. For example, if you want to post a new status update, you’ll provide the status update message here.

Alternatively, we can also use the api() method to execute FQL (Facebook Query Language) queries, which lets us get data via SQL-style language. This is great for retrieving information not available in the Graph API, as well as running multiple queries in one call. For example, we can get a user’s name and other details through this FQL API call:

$fql = 'SELECT name from user where uid = ' . $fb_uid;
$ret_obj = $facebook->api(array(
   'method' => 'fql.query',
   'query' => $fql,
 ));

We won’t need this for this tutorial, but it’s good to keep this in mind when you come across something the Graph API can’t get.

Now that we have the list of permissions the user has allowed for the application, we need to check if the user gave the application all the necessary ones. Since Facebook’s permissions page allows users to revoke certain permissions, we need to make sure that all of them have been allowed to make sure the application works. If the user has revoked one of the permissions, we’ll redirect them back to the permissions page.

//check if the permissions we need have been allowed by the user
//if not then redirect them again to facebook's permissions page
$permissions_needed = array('publish_stream', 'read_stream', 'offline_access', 'manage_pages');
foreach($permissions_needed as $perm) {
	if( !isset($permissions_list['data'][0][$perm]) || $permissions_list['data'][0][$perm] != 1 ) {
		$login_url_params = array(
			'scope' => 'publish_stream,read_stream,offline_access,manage_pages',
			'fbconnect' =>  1,
			'display'   =>  "page",
			'next' => 'http://'.$_SERVER['HTTP_HOST'].$_SERVER['REQUEST_URI']
		);
		$login_url = $facebook->getLoginUrl($login_url_params);
		header("Location: {$login_url}");
		exit();
	}
}
var_dump() of the $permissions_list variable

var_dump() of the $permissions_list variable

When that’s done, it means we’re all set and we’ll be able to run the application without any problems. Let’s start by doing another API call, this time to retrieve the list of Facebook Pages the user has administrative rights to, and saving these into a session variable. Afterwards, we’ll redirect the user to the manage.php page, where the page management code will be.

//if the user has allowed all the permissions we need,
//get the information about the pages that he or she managers
$accounts = $facebook->api(
	'/me/accounts',
	'GET',
	array(
		'access_token' => $access_token
	)
);

//save the information inside the session
$_SESSION['access_token'] = $access_token;
$_SESSION['accounts'] = $accounts['data'];
//save the first page as the default active page
$_SESSION['active'] = $accounts['data'][0];

//redirect to manage.php
header('Location: manage.php');
var_dump() of the $accounts variable

var_dump() of the $accounts variable

The /me/accounts graph path gives us a list of the pages that a user has administrative rights to. This returns an array of all of the pages, plus specific $access_tokens for each of these pages. With those $access_tokens, we’ll be able to do API calls on behalf of the Facebook Page as well!

Save these in the $_SESSION array and redirect to the manage.php page.

Let’s start working on our manage.php file. Remember that we’ve saved the user’s $accounts list into the $_SESSION array, as well as set the first account on the list as the default active page. Let’s GET that account’s news feed and display it:

//include the Facebook PHP SDK
include_once 'facebook.php';

//start the session if necessary
if( session_id() ) {

} else {
	session_start();
}

//instantiate the Facebook library with the APP ID and APP SECRET
$facebook = new Facebook(array(
	'appId' => 'APP ID',
	'secret' => 'APP SECRET',
	'cookie' => true
));

//get the news feed of the active page using the page's access token
$page_feed = $facebook->api(
	'/me/feed',
	'GET',
	array(
		'access_token' => $_SESSION['active']['access_token']
	)
);

Again, the application does the intialization of the Facebook library, and then another api() call, this time to /me/feed. Take note that instead of using the user's access token, we use the active page's access token (via $_SESSION['active']['access_token']). This tells Facebook that we want to access information as the Facebook Page and not as the user.

var_dump() of the $page_feed variable

var_dump() of the $page_feed variable

Wow, that’s a lot of information about a Facebook Post. Let’s dissect a single Facebook Post object and see what it’s made of.

The Facebook Post Object

Facebook Post Cheat Sheet

Facebook Post Cheat Sheet
  • id – this is the Post‘s ID
  • from – contains information about the user who posted the message
  • message (red) – the message component of the post
  • picture (orange) – a link to the photo attached to the post
  • name (blue) – the “title” of the Facebook post
  • link (also blue) – the link to where the name goes to when clicked
  • caption (purple) – a few words to describe the link
  • description (pink) – more than a few words to describe the link
  • icon (grey) – a link to the icon image used
  • actions – Facebook actions you can do to the post, such as Liking it or Commenting on it.
  • privacy – the privacy of the post
  • type – the type of the post. A post can be a status, link, photo or video.
  • created_time – the time when the post was created
  • updated_time – the time when the post was updated
  • comments – comments on the post

It would be wise to keep a copy of the cheat sheet above, since we’ll be using that again when we publish Post objects to Facebook.

Moving on, let’s display the news feed in a more pleasing way. Add the following HTML below the PHP code:

<!DOCTYPE html>
<html>
<head>
	<title>Nettuts+ Page Manager</title>
	<link rel="stylesheet" href="css/reset.css" type="text/css" />
	<link rel="stylesheet" href="css/bootstrap.min.css" type="text/css" />
	<script src="js/jquery.min.js"></script>

	<style>
	body {
		padding-top: 40px;
		background-color: #EEEEEE;
	}
	img {
		vertical-align: middle;
	}
	#main {
		text-align: center;
	}

	.content {
		background-color: #FFFFFF;
		border-radius: 0 0 6px 6px;
		box-shadow: 0 1px 2px rgba(0, 0, 0, 0.15);
		margin: 0 -20px;
		padding: 20px;
	}
	.content .span6 {
		border-left: 1px solid #EEEEEE;
		margin-left: 0;
		padding-left: 19px;
		text-align: left;
	}
	.page-header {
		background-color: #F5F5F5;
		margin: -20px -20px 20px;
		padding: 20px 20px 10px;
		text-align: left;
	}
	</style>

</head>
<body>
<div id="main" class="container">
	<div class="content">
		<div class="page-header">
			<h1>
				<img width="50" src="http://graph.facebook.com/<?php echo $_SESSION['active']['id']; ?>/picture" alt="<?php echo $_SESSION['active']['name']; ?>" />
				<?php echo $_SESSION['active']['name']; ?>
				<small><a href="http://facebook.com/profile.php?id=<?php echo $_SESSION['active']['id']; ?>" target="_blank">go to page</a></small>
			</h1>
		</div>
		<div class="row">
			<div class="span10">
				<ul id="feed_list">
					<?php foreach($page_feed['data'] as $post): ?>
					<?php if( ($post['type'] == 'status' || $post['type'] == 'link') && !isset($post['story'])): ?>
					<?php //do some stuff to display the post object ?>
					<?php endif; ?>
					<?php endforeach; ?>
				</ul>
			</div>
			<div class="span6">
				<h3>Post a new update</h3>
			</div>
		</div>
	</div>
</div>
</body>
</html>

We make use of a simple Graph API trick here: the Picture Graph API object. Basically, you can take the Graph path of any User or Page object (e.g. http://graph.facebook.com/293518907333589), add /picture in the end and you’ll get a 50×50 photo of that object. For example:

http://graph.facebook.com/293518907333589/picture

Becomes…



Demo Picture Graph API Object

Going back, when we refresh the manage.php page now, it should look something like this:

Add this inside the <?php foreach($page_feed['data'] as $post): ?> to display the feed from the page:

<?php if( ($post['type'] == 'status' || $post['type'] == 'link') && !isset($post['story'])): ?>
<li>
	<div class="post_photo">
		<img src="http://graph.facebook.com/<?php echo $post['from']['id']; ?>/picture" alt="<?php echo $post['from']['name']; ?>"/>
	</div>

	<div class="post_data">
		<p><a href="http://facebook.com/profile.php?id=<?php echo $post['from']['id']; ?>" target="_blank"><?php echo $post['from']['name']; ?></a></p>
		<p><?php echo $post['message']; ?></p>
		<?php if( $post['type'] == 'link' ): ?>
		<div>
			<div class="post_picture">
				<?php if( isset($post['picture']) ): ?>
				<a target="_blank" href="<?php echo $post['link']; ?>">
					<img src="<?php echo $post['picture']; ?>" width="90" />
				</a>
				<?php else: ?>
				&nbsp;
				<?php endif; ?>
			</div>
			<div class="post_data_again">
				<p><a target="_blank" href="<?php echo $post['link']; ?>"><?php echo $post['name']; ?></a></p>
				<p><small><?php echo $post['caption']; ?></small></p>
				<p><?php echo $post['description']; ?></small></p>
			</div>
			<div class="clearfix"></div>
		</div>
		<?php endif; ?>
	</div>
	<div class="clearfix"></div>
</li>
<?php endif; ?>

When you refresh the page again, it should look something like this:

Looks good, right? We’ll have to thank the Twitter Bootstrap CSS for that!

Now, let’s add a top navigation bar to help us switch from one page to another. Add the following HTML after the <body> tag and before the <div id="main" class="container"> tag:

<div class="topbar">
	<div class="fill">
	<div class="container">
		<a class="brand" href="/">Nettuts+ Page Manager</a>
		<ul class="nav secondary-nav">
			<li class="dropdown" data-dropdown="dropdown">
				<a class="dropdown-toggle" href="#">Switch Page</a>
				<ul class="dropdown-menu">
					<?php foreach($_SESSION['accounts'] as $page): ?>
					<li>
						<a href="switch.php?page_id=<?php echo $page['id']; ?>">
							<img width="25" src="http://graph.facebook.com/<?php echo $page['id']; ?>/picture" alt="<?php echo $page['name']; ?>" />
							<?php echo $page['name']; ?>
						</a>
					</li>
					<?php endforeach; ?>
				</ul>
			</li>
		</ul>
	</div>
	</div>
</div>

Don’t forget to load the dropdown JavaScript library from Twitter Bootstrap’s JavaScript page. You can download it here.

<script src="js/bootstrap-dropdown.js"></script>
<script>
$('.topbar').dropdown()
</script>

Lastly, let’s create the switch.php file, where we’ll set another page as the active page:

<?php
session_start();
$page_id = $_GET['page_id'];

foreach($_SESSION['accounts'] as $page) {
	if( $page['id'] == $page_id ) {
		$_SESSION['active'] = $page;
		header('Location: manage.php');
		exit();
	}
}
exit();
?>

Refresh the manage.php page again and you should see something like this:

And now, by using our dropdown switcher, we’re able to switch pages.


Step 4: Publish to Facebook via the Graph API

Now we’re ready to publish new updates to our page! First of all, let’s create the HTML form where we can set what we’ll publish. Add this below the <h3>Post a new update</h3> HTML:

<img src="post_breakdown.png" alt="Facebook Post Cheat Sheet" width="340" /><br />
<form method="POST" action="newpost.php" class="form-stacked">
	<label for="message">Message:</label>
	<input class="span5" type="text" id="message" name="message" placeholder="Message of post" />
	<label for="picture">Picture:</label>
	<input class="span5" type="text" id="picture" name="picture" placeholder="Picture of post" />
	<label for="link">Link:</label>
	<input class="span5" type="text" id="link" name="link" placeholder="Link of post" />
	<label for="name">Name:</label>
	<input class="span5" type="text" id="name" name="name" placeholder="Name of post" />
	<label for="caption">Caption:</label>
	<input class="span5" type="text" id="caption" name="caption" placeholder="Caption of post" />
	<label for="description">Description:</label>
	<input class="span5" type="text" id="description" name="description" placeholder="Description of post" />

	<div class="actions">
		<input type="submit" class="btn primary" value="Post" />
		<input type="reset" class="btn" value="Reset" />
	</div>
</form>

Refresh to see how it looks:



Loving Twitter Bootstrap’s default form styles

The Post cheat sheet should come in handy when we’re figuring out what to put in the fields. Now, let’s create the newpost.php file where we’ll actually use the Facebook Graph API to publish to the page’s feed.

Start by creating the file and initializing the Facebook library like we did for the other pages:

<?php
//include the Facebook PHP SDK
include_once 'facebook.php';

//start the session if necessary
if( session_id() ) {

} else {
	session_start();
}

//instantiate the Facebook library with the APP ID and APP SECRET
$facebook = new Facebook(array(
	'appId' => 'APP ID',
	'secret' => 'APP SECRET',
	'cookie' => true
));
?>

Afterwards, let’s get all of the content we received from the POST request:

//get the info from the form
$parameters = array(
	'message' => $_POST['message'],
	'picture' => $_POST['picture'],
	'link' => $_POST['link'],
	'name' => $_POST['name'],
	'caption' => $_POST['caption'],
	'description' => $_POST['description']
);

Let’s not forget to add the $access_token for the page to the parameters:

//add the access token to it
$parameters['access_token'] = $_SESSION['active']['access_token'];

Now that we have our $parameters array, let’s build and send our Graph API request!

//build and call our Graph API request
$newpost = $facebook->api(
	'/me/feed',
	'POST',
	$parameters
);

Take note of what difference the $method can make. A GET API call to /me/feed will return the news feed of that particular object, but a POST API call to /me/feed will post a new status update to the object.

To publish a new Post object to Facebook, the application needs to make a call to the /<page_id or /me>/feed graph path, and it needs to supply the following in the $parameters array:

  • access_token – the access token of the account we are publishing for
  • message – the message to publish
  • picture (optional) – a link to the photo of the post
  • name (optional) – the “title” of the post
  • link (optional) – a link to where the name will go to when clicked
  • caption (optional) – a few words to describe the link/name
  • description (optional) – more than a few words to describe the link/name

You can see that the only parameters that are required are the access_token and the message parameters. By providing only these two, we can publish a simple status message, like so:



Status Update Example

Everything else is optional. However, if you provide a name, link, caption, or decription, you have to provide all four. As for picture, if a parameter value is not provided or not accessible, the post’s picture will be blank.

Finally, let’s try publishing to Facebook using the application! Go back to the manage.php page and fill up the form, then press Post:

Afterwards, you should be redirected back to the manage.php page, but there should be a new post on the feed!



Publishing to Facebook successful!

You can also check the Facebook Page itself. You should see the new post there:



Publishing to Facebook Page

Conclusion

By now, you should already have a clear grasp on how to use the Facebook Graph API to both read and publish to Facebook. In this tutorial, we only cover the basics — there’s a whole lot more you can do using the Graph API. Stuff like real-time updates, which lets you subscribe to events that happen to your application’s users (e.g. when a user changes his/her profile photo) or the Insights Graph object, which lets you get the statistics of an application, page, or domain the user has access to.

Of course, the best resource for learning more about the Graph API is undoubtedly Facebook’s Graph API documentation, but I also suggest taking a look at FBDevWiki, which is a third-party hosted wiki for Facebook Development documentation.



FBDevWiki.com

Additionally, there’s also a special Facebook version of StackOverflow you can visit at http://facebook.stackoverflow.com. The questions here are all about Facebook and Facebook Development, so it’s well worth visiting if you need any help or have any questions on the subject.



Facebook StackOverflow

Hopefully, you’ve learned from this tutorial how easy it is to use the Facebook Graph API to make your applications more social.

Have you used the Facebook Graph API in a previous project, or are you planning to use it now? Tell me all about it in the comments below and thank you so much for reading!


September 05 2011

14:37

Getting Started With The PayPal API

Advertisement in Getting Started With The PayPal API
 in Getting Started With The PayPal API  in Getting Started With The PayPal API  in Getting Started With The PayPal API

PayPal is the most popular platform for receiving online payments today. The ease of opening a PayPal account and receiving payments compared to opening a merchant account with a traditional payment gateway is probably the number one reason for its popularity, with a close second being the comprehensive API that PayPal provides for its payment services.

Disclaimer: PayPal’s API is among the worst I’ve ever had to deal with. Inconsistencies, sometimes poor or conflicting documentation, unpredictable failures and account changes, and major differences between the live and sandbox versions all conspire to make the PayPal API quite a pain in the arse to work with. Over the years, I’ve taken my lumps from working quite a bit with the PayPal API, and I’ve published the results of my hard-learned lessons as a commercial PHP PayPal API component on the source-code marketplace Binpress.

In this post, I will break down some of the techniques and approaches to working with the PayPal API, in order to make integration and troubleshooting simpler and easier.

 in Getting Started With The PayPal API

The Different Payment Options

PayPal offers a variety of payment options, which might be confusing at first:

  • Express Checkout
    The premier PayPal service. Express Checkout allows you to receive payments without having a merchant account and without having to meet special requirements other than verifying your account (either via a bank account or a credit card). Previously, you could receive Express Checkout payments from PayPal users only, but PayPal has since added a credit-card option for non-PayPal users, making this service accessible to practically anyone with a major credit card. Note that the Express Checkout process occurs on PayPal’s platform and thus can never be fully integrated in your website’s experience.
  • Direct Payment
    The Direct Payment method allows you to receive credit-card payments directly through an API call. This enables you to host the payment process on your website in full, which might make for a more complete shopping experience for your customers. The Direct Payment method has several variations that enable you to authorize a payment and complete it at a later date: the appropriately named Authorization and Capture methods. These variations are a part of the Website Payments Pro API, which is available only to US, Canadian and UK accounts.
  • Recurring Payments
    This allows you to set up a recurring transaction (i.e. a subscription payment).
  • Mass Payments
    This allows you to transfer money to multiple accounts at once.
  • Adaptive Payments
    Here is another API for sending funds to multiple recipients, with some differences from the Mass Payments API. (Did I mention that the PayPal API is confusing and a bit redundant?)

This list is not comprehensive, but it covers the main payment options (see the API documentation for more).

Making API Requests

PayPal supports two main formats over HTTP: NVP and SOAP. NVP is short for Name-Value Pair, and SOAP stands for Simple Object Access Protocol. I will cover the NVP approach, which I prefer to SOAP’s relatively verbose and complex syntax.

Each of the API methods has different parameters, but they all share some basic parameters, which are used to identify the API account and sign the transaction. These include:

  • USER
    Your PayPal API user name.
  • PWD
    Your PayPal API password.
  • VERSION
    The version number of the NVP API service, such as 74.0 (the most recent as of this writing).
  • SIGNATURE
    Your PayPal API signature string. This parameter is optional if you use a certificate to authenticate.

The last required parameter is METHOD, which declares which API method we are calling.

Requests are made over HTTPS. We’ll use cURL to build our basic request, and then encapsulate the process in a class:

class Paypal {
   /**
    * Last error message(s)
    * @var array
    */
   protected $_errors = array();

   /**
    * API Credentials
    * Use the correct credentials for the environment in use (Live / Sandbox)
    * @var array
    */
   protected $_credentials = array(
      'USER' => 'seller_1297608781_biz_api1.lionite.com',
      'PWD' => '1297608792',
      'SIGNATURE' => 'A3g66.FS3NAf4mkHn3BDQdpo6JD.ACcPc4wMrInvUEqO3Uapovity47p',
   );

   /**
    * API endpoint
    * Live - https://api-3t.paypal.com/nvp
    * Sandbox - https://api-3t.sandbox.paypal.com/nvp
    * @var string
    */
   protected $_endPoint = 'https://api-3t.sandbox.paypal.com/nvp';

   /**
    * API Version
    * @var string
    */
   protected $_version = '74.0';

   /**
    * Make API request
    *
    * @param string $method string API method to request
    * @param array $params Additional request parameters
    * @return array / boolean Response array / boolean false on failure
    */
   public function request($method,$params = array()) {
      $this -> _errors = array();
      if( empty($method) ) { //Check if API method is not empty
         $this -> _errors = array('API method is missing');
         return false;
      }

      //Our request parameters
      $requestParams = array(
         'METHOD' => $method,
         'VERSION' => $this -> _version
      ) + $this -> _credentials;

      //Building our NVP string
      $request = http_build_query($requestParams + $params);

      //cURL settings
      $curlOptions = array (
         CURLOPT_URL => $this -> _endPoint,
         CURLOPT_VERBOSE => 1,
         CURLOPT_SSL_VERIFYPEER => true,
         CURLOPT_SSL_VERIFYHOST => 2,
         CURLOPT_CAINFO => dirname(__FILE__) . '/cacert.pem', //CA cert file
         CURLOPT_RETURNTRANSFER => 1,
         CURLOPT_POST => 1,
         CURLOPT_POSTFIELDS => $request
      );

      $ch = curl_init();
      curl_setopt_array($ch,$curlOptions);

      //Sending our request - $response will hold the API response
      $response = curl_exec($ch);

      //Checking for cURL errors
      if (curl_errno($ch)) {
         $this -> _errors = curl_error($ch);
         curl_close($ch);
         return false;
         //Handle errors
      } else  {
         curl_close($ch);
         $responseArray = array();
         parse_str($response,$responseArray); // Break the NVP string to an array
         return $responseArray;
      }
   }
}

Note that I use a CA certificate file for SSL certificate validation. You can obtain the file from the cURL website or any trusted source. Update the path to the certificate file according to where you’ve placed it.

The response returned will be in NVP format as well, and I reformat it into an array before returning it. A parameter named ACK signifies the status of the request: Success or SuccessWithWarning when the request succeeds, and Error or Warning when the request fails.

A request could fail for many reasons, and there are different reasons for each API method, which are covered in detail in the manual. We’ll go over some further down in this article and look at ways to handle them. Keep in mind that the parameter values are case-sensitive, so code against them accordingly.

Express Checkout

One of the most popular APIs is the Express Checkout API, which enables you to receive payments without opening a Website Payments Pro account (which is available only to verified US accounts) or hosting the actual transaction yourself (which requires additional security).

The Express Checkout process works as follows:

  1. We request a checkout token from PayPal using the transaction details;
  2. If successful, we redirect the user to the PayPal endpoint using the received token;
  3. The user completes or cancels the payment on the PayPal platform and is redirected back to our website;
  4. We complete the payment either when the user is redirected back or via an Instant Payment Notification (IPN).

Scr Ukexpresscheckout-cut in Getting Started With The PayPal API

1. Getting the Checkout Token: SetExpressCheckout

We initiate the Express Checkout process by passing the order details to the PayPal API, and we receive a token string that identifies it. This token would be used in the next step to redirect to PayPal.

Here are the required parameters:

  • METHOD
    This is the API method that we’re using (i.e. SetExpressCheckout).
  • RETURNURL
    The URL that the user will be redirected to after the payment process is completed.
  • CANCELURL
    The URL that the user will be redirected to after having cancelled the payment process.
  • PAYMENTREQUEST_0_AMT
    The transaction’s total amount. This must have two decimal places, with the decimal separator being a period (.). The optional thousands separator must be a comma (,).
  • PAYMENTREQUEST_0_ITEMAMT
    The total cost of the items in the order, excluding shipping, taxes and other costs. If there are no extra costs, then it should be the same value as PAYMENTREQUEST_0_AMT.

We can pass additional parameters to add more information about the order, some of which have default values:

  • PAYMENTREQUEST_0_CURRENCYCODE
    The payment’s currency, as a three-letter code. The default is USD.
  • PAYMENTREQUEST_0_SHIPPINGAMT
    The total shipping costs for this order.
  • PAYMENTREQUEST_0_TAXAMT
    The total tax amount for this order. This is required if per-item tax is specified (see below).
  • PAYMENTREQUEST_0_DESC
    The order’s description.

We can also add details about individual items in the order:

  • L_PAYMENTREQUEST_0_NAMEm
    The item’s name.
  • L_PAYMENTREQUEST_0_DESCm
    The item’s description.
  • L_PAYMENTREQUEST_0_AMTm
    The item’s cost.
  • L_PAYMENTREQUEST_0_QTYm
    The quantity of an item.

The variable index m identifies the item. (Use the same variable for all details of the same item.)

There are many other optional parameters, which can be found in the API documentation.

We’ll use the function that we wrote above to build the SetExpressCheckout request:

//Our request parameters
$requestParams = array(
   'RETURNURL' => 'http://www.yourdomain.com/payment/success',
   'CANCELURL' => 'http://www.yourdomain.com/payment/cancelled'
);

$orderParams = array(
   'PAYMENTREQUEST_0_AMT' => '500',
   'PAYMENTREQUEST_0_SHIPPINGAMT' => '4',
   'PAYMENTREQUEST_0_CURRENCYCODE' => 'GBP',
   'PAYMENTREQUEST_0_ITEMAMT' => '496'
);

$item = array(
   'L_PAYMENTREQUEST_0_NAME0' => 'iPhone',
   'L_PAYMENTREQUEST_0_DESC0' => 'White iPhone, 16GB',
   'L_PAYMENTREQUEST_0_AMT0' => '496',
   'L_PAYMENTREQUEST_0_QTY0' => '1'
);

$paypal = new Paypal();
$response = $paypal -> request('SetExpressCheckout',$requestParams + $orderParams + $item);

2. Redirecting to PayPal Using the Checkout Express Token

If the request is successful, we’ll receive a checkout token in the TOKEN parameter of the response.

if(is_array($response) && $response['ACK'] == 'Success') { //Request successful
      $token = $response['TOKEN'];
      header( 'Location: https://www.paypal.com/webscr?cmd=_express-checkout&token=' . urlencode($token) );
}

The user now goes through the purchase process on PayPal’s website. When they confirm or cancel it, they will return to one of the URLs that we’ve specified in the request.

3. Completing the Transaction

Assuming the user confirms the transaction, they will be redirected to our website by PayPal. At this point, we should use two relevant API methods: DoExpressCheckoutPayment will complete the transaction, but before that we might want to get additional information on the buyer using GetExpressCheckoutDetails.

PayPal will redirect the user back from the purchase with the checkout token, which we will use to call those methods. The token will be available in the URL query parameters via the token parameter. We will check for its existence in the confirmation URL and then send our API requests if we find it.

The GetExpressCheckoutDetails method requires only the checkout token. DoExpressCheckoutPayment requires a couple of additional parameters:

  • PAYMENTREQUEST_0_PAYMENTACTION
    This is the payment action. It should be set to Sale unless we’ve specified a different action in the SetExpressCheckout method (possible values include Authorization and Capture).
  • PAYERID
    This is the unique identification for the PayPal account. This, too, is returned in the URL query parameters (in the PayerID parameter) and can also be retrieved from the details returned by GetExpressCheckoutDetails.
if( isset($_GET['token']) && !empty($_GET['token']) ) { // Token parameter exists
   // Get checkout details, including buyer information.
   // We can save it for future reference or cross-check with the data we have
   $paypal = new Paypal();
   $checkoutDetails = $paypal -> request('GetExpressCheckoutDetails', array('TOKEN' => $_GET['token']));

   // Complete the checkout transaction
   $requestParams = array(
      'PAYMENTREQUEST_0_PAYMENTACTION' => 'Sale',
      'PAYERID' => $_GET['PayerID']
   );

   $response = $paypal -> request('DoExpressCheckoutPayment',$requestParams);
   if( is_array($response) && $response['ACK'] == 'Success') { // Payment successful
      // We'll fetch the transaction ID for internal bookkeeping
      $transactionId = $response['PAYMENTINFO_0_TRANSACTIONID'];
   }
}

Direct Payment

The Direct Payment API allows you to receive payments directly on your website or application, giving you complete control over the checkout process. PayPal tends to push users to register and use a PayPal account, which is understandable, but this conflicts somewhat with our interest to make the payment process as simple and clear as possible for our customers. For this reason, full control over the checkout process is preferred and gives us more options to optimize sales and generate more sales.

Scr Ukdirectpayment-cut in Getting Started With The PayPal API

The process is a bit simpler than that of Express Checkout, because the entire interaction occurs on our website, and we need to perform just one API call to process a normal payment: DoDirectPayment.

A couple of more API requests are required if you want to perform a transaction that is billed at a later date (for example, when you ship the product or confirm availability). These would be the Authorization & Capture API methods, which I will not cover in this post, but be aware that this option exists.

DirectPayment Parameters

DirectPayment requires different parameters than Express Checkout, as to be expected. While the transaction details parameters are similar (with different key names, to make it more interesting), the method also requires credit-card and address information.

DirectPayment’s basic parameters:

  • METHOD
    This is DoDirectPayment.
  • IPADDRESS
    This is the IP address of the payer. In PHP, we can retrieve it using the superglobal $_SERVER['REMOTE_ADDR']. You’ll have to do a bit more work to get the IP when dealing with set-ups that have a proxy between the PHP process and the outside network (such as nginx).
  • PAYMENTACTION
    This is the type of action that we want to perform. A value of Sale indicates an immediate transaction. A value of Authorization indicates that this transaction will not be performed immediately, but rather will be captured later using the Authorization & Capture API mentioned earlier.

Credit-card details:

  • CREDITCARDTYPE
    The credit-card type (Visa, MasterCard, etc.). See the API documentation for the full list.
  • ACCT
    The credit-card number. (Don’t you love these abbreviated key names?) This must conform to the particular format of the card’s type.
  • EXPDATE
    The expiration date, in MMYYYY format (i.e. a two-digit month and a four-digit year, as one string).
  • CVV2
    The “card verification value,” or security code, as it’s sometimes known.

Payer information and address parameters:

  • FIRSTNAME, LASTNAME
    The payer’s first name and last name, respectively (in separate fields). You can also provide an email address in an EMAIL parameter, but it’s not required.
  • CITY, STATE, COUNTRYCODE, ZIP
    The city, state, country code (as a two-letter code) and zip code parts of the address, all required.
  • STREET, STREET2
    Two lines for the address (only the first is required).

This address will be used in the address verification system (AVS). You’ll receive a specific error code if a transaction has failed due to an address verification failure.

The payment details parameters are the same as the ones for Express Checkout, but with slightly different names (AMT, ITEMAMT, CURRENCYCODE, SHIPPINGAMT, TAXAMT and DESC) and without the PAYMENTREQUEST_0_ prefix. Refer to the previous section or the API documentation for specific details on those.

Similarly, the item details parameters are similar to those of Express Checkout. These include L_NAMEm, L_DESCm, L_AMTm and L_QTYm, giving you granular control of item details in the order summary. The m integer variable is used to account for multiple items (replace with 0, 1 and so on for numbered items in the order). See the API documentation for a comprehensive list of item details.

Performing the Transaction

Sending the request using our function is very similar to GetExpressCheckoutToken. We pass all of the parameters into the request function as before, with the method set to DoDirectPayment.

$requestParams = array(
   'IPADDRESS' => $_SERVER['REMOTE_ADDR'],
   'PAYMENTACTION' => 'Sale'
);

$creditCardDetails = array(
   'CREDITCARDTYPE' => 'Visa',
   'ACCT' => '4929802607281663',
   'EXPDATE' => '062012',
   'CVV2' => '984'
);

$payerDetails = array(
   'FIRSTNAME' => 'John',
   'LASTNAME' => 'Doe',
   'COUNTRYCODE' => 'US',
   'STATE' => 'NY',
   'CITY' => 'New York',
   'STREET' => '14 Argyle Rd.',
   'ZIP' => '10010'
);

$orderParams = array(
   'AMT' => '500',
   'ITEMAMT' => '496',
   'SHIPPINGAMT' => '4',
   'CURRENCYCODE' => 'GBP'
);

$item = array(
   'L_NAME0' => 'iPhone',
   'L_DESC0' => 'White iPhone, 16GB',
   'L_AMT0' => '496',
   'L_QTY0' => '1'
);

$paypal = new Paypal();
$response = $paypal -> request('DoDirectPayment',
   $requestParams + $creditCardDetails + $payerDetails + $orderParams + $item
);

if( is_array($response) && $response['ACK'] == 'Success') { // Payment successful
   // We'll fetch the transaction ID for internal bookkeeping
   $transactionId = $response['TRANSACTIONID'];
}

There are plenty of parameters, but all relatively simple.

Error Handling

In a perfect world, this section would not exist. In reality, you will be referring to it quite a lot. PayPal can fail a transaction for a multitude of reasons, not all of which you can control.

The $response variable we returned from our paypalApiRequest() function could contain a different value than Success for the ACK parameter. That value could be:

  • Success
    Indicates a successful operation.
  • SuccessWithWarning
    Indicates a successful operation, and that messages were returned in the response that you should examine.
  • Failure
    Indicates a failed operation, and that the response contains one or more error messages explaining the failure.
  • FailureWithWarning
    Indicates a failed operation, and that messages were returned in the response that you should examine.

This gives us two success statuses and two failure statuses. The mock code above tests for the Success value only, but we could change it to check for SuccessWithWarning as well; and keep in mind that we need to find out what the warning is. A common scenario is that a Direct Payment charge will have been performed successfully, but the credit-card company responds that the transaction has failed, for whatever reason.

Errors from PayPal are returned in four parameters in the response:

  • L_ERRORCODE0
    A numeric error code, which can referenced against PayPal’s error code list (there are quite a few).
  • L_SHORTMESSAGE0
    A short error message describing the problem.
  • L_LONGMESSAGE0
    A longer error message describing the problem.
  • L_SEVERITYCODE0
    The severity code. (I couldn’t find any useful documentation on this, and it doesn’t really matter, so let’s put it aside.)

The 0 part of these parameters is an incrementing integer for multiple error message (1, 2, etc.).

Here are some common errors you’ll run into:

  • 10002
    Authentication or authorization failed. This usually indicates invalid API credentials, or credentials that do not match the type of environment you are working in (such as a live or sandbox environment).
  • 81***
    Missing parameter. There are quite a few of these, all starting with 81. Each refers to a specific required parameter that is missing from the request.
  • 104**
    Invalid argument. This indicates that one of the supplied parameters has an invalid value. Each argument has specific errors, all starting with 104. A common one is 10413, which means that the total cost of the cart items does not match the order’s amount (i.e. the total amount parameter, AMT, does not equal the items’ total plus shipping, handling, taxes and other charges).

How Do We Handle These Errors in Practice?

PayPal error messages vary and could contain private information that you do not want your users to see (such as an invalid merchant configuration). That being the case, showing PayPal error messages directly to users is not advisable, even though some of them might be useful.

In most cases, I would do the following:

  1. Set up a white-list array of errors that can be shown safely (such as a missing credit-card number and expiration date);
  2. Check the response code against that array;
  3. If the error message is not white-listed, then display a generic message, such as “An error has occurred while processing your payment. Please try again in a few minutes, or contact us if this is a recurring issue.”

If an error falls outside of the white-listed array, I will also log it to a file on the server and send an email to the administrator, with the full details so that someone is up to speed on payment failures. In fact, logging PayPal requests and responses is good practice regardless of errors, so that you can monitor and troubleshoot payment failures (I provide this option in the commercial component that I mentioned at the beginning of this article).

Ready To Get Started With The PayPal API?

In this post, I’ve covered two of the most widely used API methods, as well as error handling with the PayPal API. This should be enough for you to get started using the most popular payment platform online.

The PayPal API has many more methods and processes, more than can be covered in any one post. Once you get up to speed on the basic methods, learning the others should be relatively straightforward (even if somewhat exhausting). I hope this guide has given you a good head start on using the API. If you have questions or comments, I would love to hear from you in the comments!

Front cover: Image source

(al)


© Eran Galperin for Smashing Magazine, 2011.

December 21 2010

13:10

YQL: Using Web Content For Non-Programmers

Advertisement in YQL: Using Web Content For Non-Programmers
 in YQL: Using Web Content For Non-Programmers  in YQL: Using Web Content For Non-Programmers  in YQL: Using Web Content For Non-Programmers

Building a beautiful design is a great experience. Seeing the design break apart when people start putting in real content, though, is painful. That’s why testing it as soon as possible with real information to see how it fares is so important. To this end, Web services provide us with a lot of information with which to fill our products. In recent years, this has been a specialist’s job, but the sheer amount of information available and the number of systems to consume it makes it easier and easier to use Web services, even for people with not much development experience.

On Programmable Web, you can find (to date) 2580 different application programming interfaces (or APIs). An API allows you to get access to an information provider’s data in a raw format and reformat it to suit your needs.

Programmable in YQL: Using Web Content For Non-Programmers

The Trouble With APIs

The problem with APIs is that access to them varies in simplicity, from just having to load data from a URL all the way up to having to authenticate with the server and give all kinds of information about the application you want to build before getting your first chunk of information.

Each API is based on a different idea of what information you need to provide, what format it should be in, what data it will give back and in what format. All this makes using third-party APIs in your products very time-consuming, and the pain multiplies with each one you use. If you want to get photos from Flickr and updates from Twitter and then show the geographical information in Twitter on a map, then you have quite a trek ahead.

Simplifying API Access

Yahoo uses APIs for nearly all of its products. Instead of accessing a database and displaying the information live on the screen, the front end calls an API, which in turn gets the information from the back end, which talks to databases. This gives Yahoo the benefit of being able to scale to millions of users and being able to change either the front or back end without disrupting the other.

Because the APIs have been built over 10 years, they all vary in format and the way in which you access them. This cost Yahoo too much time, which is why it built Yahoo Pipes — to ease the process.

Pipes-500px in YQL: Using Web Content For Non-Programmers
Large view

Pipes is amazing. It is a visual way to mix and match information from the Web. However, as people used Pipes more, they ran into limitations. Versioning pipes was hard; to change the functionality of the pipe just slightly, you had to go back to the system, and it tended to slow down with very complex and large conversions. This is why Yahoo offers a new system for people’s needs that change a lot or get very complex.

YQL is both a service and a language (Yahoo Query Language). It makes consuming Web services and APIs dead simple, both in terms of access and format.

Retrieving Data With YQL

The easiest way to access YQL is to use the YQL console. This tool allows you to preview your YQL work and play with the system without having to know any programming at all. The interface is made up of several components:

Yqlcons in YQL: Using Web Content For Non-Programmers
Large view

  1. The YQL statement section is where you write your YQL query.
    YQL has a very simple syntax, and we’ll get into its details a bit later on. Now is the time to try it out. Enter your query, define the output format (XML or JSON), check whether to have diagnostics reporting, and then hit the “Test” button to see the information. There is also a permalink; click it to make sure you don’t lose your work in case you accidentally hit the “Back” button.
  2. The results section shows you the information returned from the Web service.
    You can either read it in XML or JSON format or click the “Tree view” to navigate the data in an Explorer-like interface.
  3. The REST query section gives you the URL of your YQL query.
    You can copy and paste this URL at any time to use it in a browser or program. Getting information from different sources with YQL is actually this easy.
  4. The queries section gives you access to queries that you previously entered.
    You can define query aliases for yourself (much as you would bookmark websites), get a history of the latest queries (very useful in case you mess up) and get some sample queries to get started.
  5. The data tables section lists all the Web services you can access using YQL.
    Clicking the name of a table will in most cases open a demo query in the console. If you hover over the link, you’ll get two more links — desc and src — which give you information about the parameters that the Web service allows and which show the source of the data table itself. In most cases, all you need to do is click the name. You can also filter the data table list by typing what you’re looking for.

Using YQL Data

By far the easiest way to use YQL data is to select JSON as the output format and define a callback function. If you do that, you can then copy and paste the URL from the console and write a very simple JavaScript to display the information in HTML. Let’s give that a go.

As a very simple example, let’s get some photos from Flickr for the search term “cat”:

select * from flickr.photos.search where text="cat"

Type that into the YQL console, and hit the “Test” button. You will get the results in XML — a lot of information about the photos:

Yql-statement in YQL: Using Web Content For Non-Programmers
Large view

Instead of XML, choose JSON as the output format, and enter myflickr as the callback function name. You will get the same information as a JSON object inside a call to the function myflickr.

Yql-statement2 in YQL: Using Web Content For Non-Programmers
Large view

You can then copy the URL created in the “REST query” field:

Rest-query in YQL: Using Web Content For Non-Programmers
Large view

Write a JavaScript function called myflickr with a parameter data, and copy and paste the URL as the src of another script block:

<script>
  function myflickr(data){
    alert(data);
  }
</script>
<script src="http://query.yahooapis.com/v1/public/yql?
q=select%20*%20from%20flickr.photos.search%20where%20tex
t%3D%22cat%22&format=json&env=store%3A%2F%2Fdatatables.org
%2Falltableswithkeys&callback=myflickr"></script>

If you run this inside a browser, the URL you copied will retrieve the data from the YQL server and send it to the myflickr function as the data parameter. The data parameter is an object that contains all the returned information from YQL. To make sure you have received the right information, test whether the data.query.results property exists; then you can loop over the result set:

<script>function myflickr(data){
  if(data.query.results){
    var photos = data.query.results.photo;
    for(var i=0,j=photos.length;i<j;i++){
      alert(photos[i].title);
    }
  }
}</script>
<script src="http://query.yahooapis.com/v1/public/yql?
q=select%20*%20from%20flickr.photos.search%20where%20text%3D%22cat%22
&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&
callback=myflickr"></script>

You can easily get the structure of the information and know what is loop-able by checking the tree view of the results field in the console:

Datatree in YQL: Using Web Content For Non-Programmers

Right now, all this does is display the titles of the retrieved photos as alerts, which is nothing but annoying. To display the photos in the right format, we need a bit more — but no magic either:

<div id="flickr"></div>
<script>function myflickr(data){
  if(data.query.results){
    var out = '<ul>';
    var photos = data.query.results.photo;
    for(var i=0,j=photos.length;i<j;i++){
      out += '<li><img src="http://farm' + photos[i].farm +
             '.static.flickr.com/' + photos[i].server + '/' + photos[i].id +
             '_' + photos[i].secret + '_s.jpg" alt="' + photos[i].title +
             '"></li>';
    }
    out += '</ul>';
  }
  document.getElementById('flickr').innerHTML = out;
}</script>
<script src="http://query.yahooapis.com/v1/public/yql?
q=select%20*%20from%20flickr.photos.search%20where%20text%3D%22cat%22&
format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&
callback=myflickr"></script>

Flickrphotos in YQL: Using Web Content For Non-Programmers

Put this into action and you’ll get photos of cats, live from Flickr and without having to go through any painful authentication process.

The complexity of the resulting HTML for display differs from data set to data set, but in essence the main trick remains the same: define a callback function, write it, copy and paste the URL you created in the console, test that data has been returned, and then go nuts.

Using YQL To Reuse HTML Content

One other very powerful use of YQL is to access HTML content on the Web and filter it for reuse. This is usually called “scraping” and is a pretty painful process. YQL makes it easier because of two things: it cleans up the HTML retrieved from a website by running it through HTML Tidy, and it allows you to filter the result with XPATH. As an example, let’s retrieve the list of my upcoming conferences and display it.

Go to http://icant.co.uk/ to see my upcoming speaking engagements:

Upcoming in YQL: Using Web Content For Non-Programmers

You can then use Firebug in Firefox to inspect this section of the page. Simply open Firebug, click the box with the arrow icon next to the bug, and move the cursor around the page until the blue border is around the element you want to inspect:

Fb in YQL: Using Web Content For Non-Programmers
Large view

Right-click the selection, and select “Copy XPath” from the menu:

Xpath in YQL: Using Web Content For Non-Programmers
Large view

Go to the YQL console, and type in the following:

select * from html where url="http://icant.co.uk" and xpath=''

Copy the XPath from Firebug into the query, and hit the “Test” button.

select * from html where url="http://icant.co.uk" and
xpath='//*[@ID="travels"]'

Select-statement in YQL: Using Web Content For Non-Programmers
Large view

As you can see, this gets the HTML of the section that we want inside some XML. The easiest way to reuse this in HTML is by requesting a format that YQL calls JSON-P-X. This will return a simple JSON object with the HTML as a string. To use this, do the following:

  1. Copy the URL from the REST field in the console.
  2. Add &format=xml&callback=travels to the end of the URL.
  3. Add this as the src to a script block, and write this terribly simple JavaScript function:
<div id="travels"></div>
<script>function travels(data){
  if(data.results){
    var travels = document.getElementById('travels');
    travels.innerHTML = data.results[0];
  }
}</script>
<script src="http://query.yahooapis.com/v1/public/yql?
q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Ficant.co.uk%22%20
and%20xpath%3D'%2F%2F*%5B%40id%3D%22travels%22%5D'&
diagnostics=true&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&
format=xml&callback=travels"></script>

The result is an unordered list of my events on your website:

Yql-demo in YQL: Using Web Content For Non-Programmers

Debugging YQL Queries

Things will go wrong, and having no idea why is terribly frustrating. The good news with YQL is that you will get error messages that are actually human-readable. If something fails in the console, you will see a big box under the query telling you what the problem was:

Select-statement2 in YQL: Using Web Content For Non-Programmers
Large view

Furthermore, you will see a diagnostics block in the data returned from YQL that tells you in detail what happened “under the hood.” If there are any problems accessing a certain service, it will show up there.

Select-statement3 in YQL: Using Web Content For Non-Programmers
Large view

YQL Syntax

The basic syntax of YQL is very easy:

select {what} from {source} where {conditions}

You can filter your results, cut the information down only to the bits you want, paginate the results and nest queries in others. For all the details of the syntax and its nuances, check the extensive YQL documentation.

YQL Examples

You can do quite amazing things with YQL. By nesting statements in parentheses and filtering the results, you can reach far and wide across the Web of data. Simply click the following examples to see the results as XML documents. Copy and paste them into the console to play with them.

This is just a taste of the power of YQL. Check out some of my presentations on the subject.

YQL’s Limits

YQL has a few (sensible) limits:

  • You can access the URL 10,000 times an hour; after that you will be blocked. It doesn’t matter in our case because the blocking occurs per user, and since we are using JavaScript, this affects our end users individually and not our website. If you use YQL on the back end, you should cache the results and also authenticate to the service via oAuth to be allowed more requests.
  • The language allows you to retrieve information; insert, update and delete from data sets; and limit the amount of data you get back. You can get paginated data (0 to 20, 20 to 40 and so on), and you can sort and find unique entries. What you can’t do in the YQL syntax is more complex queries, like “Get me all data sets in which the third character in the title attribute is x,” or something like that. You could, however, write a JavaScript that does this kind of transformation before YQL returns the data..
  • You can access all open data on the Web, but if a website chooses to block YQL using the robots.txt directive, you won’t be allowed to access it. The same applies to data sources that require authentication or are hosted behind a firewall.

There Is More To YQL

This article covers how to use YQL to access information. If you have an interesting data set and want it to become part of the YQL infrastructure, you can easily do that, too. We’ll cover that in the next article.

Documentation and Related Links

(al)(vf)


© Christian Heilmann for Smashing Magazine, 2010. | Permalink | Post a comment | Add to del.icio.us | Digg this | Stumble on StumbleUpon! | Tweet it! | Submit to Reddit | Forum Smashing Magazine
Post tags: api, Coding, yql

Tags: Coding api yql

September 20 2010

10:00

Creating an Interactive Travel Map using the Google Maps API

This is my third tutorial on the Google Maps API v3. I’m going to show you how to use some other google maps features by building an interactive travel map. Everyone will be able to add new locations on the map and view the locations added by others. We will also be using MySQL databases to store the values of different locations so let’s get started.

What google maps features you’ll learn?

  • Adding a map to your site
  • Adding controls for the map and settings the options for the controls
  • Responding to events
  • Adding markers, settings options for markers and dynamically changing marker settings
  • Geocoding a location
  • Creating tabbed infowindows (using jquery, jquery ui and some sample code from google code)

Prerequisites

The application is built using html, php and javascript and also saves the locations in a mysql database. It would be best if you had some basic knowledge of these.

What are we going to build?

We’re going to build an interactive travel map. Everyone will be able to see the locations on the map and also add new ones easily. We’ll save some info about each location (coordinates, address, a description, a photo and data about the person who added the location). Users will be able to add locations by clicking on the map to find out the coordinates and address and by filling in the rest of the info. The users will also be able to view all the locations from the database. For viewing them easier, we’ll add a location list the user can browse and we will highlight a location on the map when the user goes with the mouse pointer over the location name.

You can view the app here! And also download the source code from here.

Creating the database table

First we have to create a table in our database to save the data about the locations on the map. We’ll save the following data about each location:

  • A name
  • A description
  • A photo
  • Coordinates: latitude and longitude
  • Address
  • Info about the user who added the location: name and location

Here’s how the code for creating the table looks like:

CREATE TABLE IF NOT EXISTS `locations` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` text NOT NULL,
`description` text NOT NULL,
`photo` text NOT NULL,
`latitude` double NOT NULL,
`longitude` double NOT NULL,
`address` text NOT NULL,
`user_name` text NOT NULL,
`user_location` text NOT NULL,
PRIMARY KEY (`id`));

Creating the layout for our app

We’ll use a simple layout. We’ll show a title and some info about our app at the top of the page. Below that we’ll add three columns, one for adding locations (with a form for adding new locations), one for the map (a div called ‘map_canvas’) and one for the locations list.

Since the code is pretty long I’ve decided not to post it here, but you can see it in the source code of the app.

Connecting to the database and getting the info about the locations

To get the data from the database we’ll first have to connect to it. We’ll create a small php script for this and save it in the ‘connect.php’ file. The script looks like this:

$server = "WRITE_SERVER_NAME_HERE";
$username = "WRITE_USERNAME_HERE";
$password = "WRITE_PASSWORD_HERE";
$database = "WRITE_DATABASE_NAME_HERE";
$connId = mysql_connect($server,$username,$password) or die("Cannot connect to server");
$selectDb = mysql_select_db($database,$connId) or die("Cannot connect to database");

Don’t forget to fill in your data!

We’ll include this script in our main file and the database connection is done! We’ll add the following lines at the beginning of our main file:

<?php
include "connect.php";
?>

Now we need to find out the number of locations and the information about all the locations in the database.

Here’s how:

$locations = mysql_query("select * from locations");
$nr_locations = mysql_num_rows($locations);

We have done a simple query to the database and retrieved the info about the locations in the $locations variable. We have also found out the total number of saved locations. We will print this in the text we’ll show on top of our page:

Click on the map to add a new location! Total locations added: <?php echo $nr_locations;?>

We also have to print the location list, like this:

<?php
// find locations from db
// save data for locations
$lats = "";          // string with latitude values
$lngs = "";          // string with longitude values
$addresses = "";     // string with address values
$names = "";         // string with names
$descrs = "";        // string with descriptions
$photos = "";        // string with photo names
$user_names = "";    // string with user names
$user_locs = "";     // string with user locations
$i=0;
// take the locations from the db one by one
while ($locat = mysql_fetch_array($locations))
{
  // add location data to info strings
  $lats .= $locat['latitude'].";;";
  $lngs .= $locat['longitude'].";;";
  $addresses .= $locat['address'].";;";
  $names .= $locat['name'].";;";
  $descrs .= $locat['description'].";;";
  $photos .= $locat['photo'].";;";
  $user_names .= $locat['user_name'].";;";
  $user_locs .= $locat['user_location'].";;";
  // show the location name in the right of the map with link that calls the highlightMarker function
?>
<a onmouseover="highlightMarker(<?php echo $i;?>)"><?php echo $locat['name'];?></a> <br/>
<?php
  $i++;
}
// hidden inputs for saving the info for all the locations in the db
?>
<input type="hidden" value="<?php echo $lats;?>" id="lats" name="lats"/>
<input type="hidden" value="<?php echo $lngs;?>" id="lngs" name="lngs"/>
<input type="hidden" value="<?php echo $addresses;?>" id="addresses" name="addresses"/>
<input type="hidden" value="<?php echo $names;?>" id="names" name="names"/>
<input type="hidden" value="<?php echo $descrs;?>" id="descrs" name="descrs"/>
<input type="hidden" value="<?php echo $photos;?>" id="photos" name="photos"/>
<input type="hidden" value="<?php echo $user_names;?>" id="user_names" name="user_names"/>
<input type="hidden" value="<?php echo $user_locs;?>" id="user_locs" name="user_locs"/>

We have shown the location names as links. When the user goes with the mouse over the links, the highlightMarker JavaScript function is called. We’ll take a look at it a bit later.

We have also saved the information about the locations in hidden input fields. We will use the info from the hidden fields to show the locations on the map.

Showing the map

To add the map we will first have to include the google maps api, like so:

<script src="http://maps.google.com/maps/api/js?sensor=true"></script>

Now, we’ll define a javascript function called initialize to show the map in the div created for it. We will call this function after the page is loaded.

Here’s how to call the function when the page loads:

<body onload="initialize();">

And here’s the function code:

<script>
var geocoder;
var map;
// initializing the map
function initialize()
{
  // define map center
  var latlng = new google.maps.LatLng(57.279043,29.355469);
  // define map options
  var myOptions =
  {
    zoom: 3,
    center: latlng,
    mapTypeId: google.maps.MapTypeId.HYBRID,
    scaleControl: true,
    navigationControl: true,
    navigationControlOptions: {
      style: google.maps.NavigationControlStyle.SMALL
    },
    mapTypeControl: true,
    mapTypeControlOptions: {
      style: google.maps.MapTypeControlStyle.DROPDOWN_MENU
    }
  };
  // initialize map
  map = new google.maps.Map(document.getElementById("map_canvas"), myOptions);
  // add event listener for when a user clicks on the map
  google.maps.event.addListener(map, 'click', function(event) {
    findAddress(event.latLng);
  });
}
</script>

To create a map, we first have to define the options for showing the map. We have set the zoom level, the center of the map. Next, we have set the controls to show on the map: scale control, navigation and map type and we have defined some options for it. We have chosen to show a small navigation control and to show the map type control as a drop down menu. You can view all the options available for controls in the API reference.

Then, we initialize the map object and add an event for when the user clicks on the map. When this event is fired, the findAddress function will be called. The function takes as an argument the coordinates of the point where the user clicked on the map.

// finds the address for the given location
function findAddress(loc)
{
  geocoder = new google.maps.Geocoder();
  if (geocoder)
  {
  geocoder.geocode({'latLng': loc}, function(results, status)
  {
    if (status == google.maps.GeocoderStatus.OK)
    {
      if (results[0])
      {
        address = results[0].formatted_address;
        // fill in the results in the form
        document.getElementById('lat').value = loc.lat();
        document.getElementById('long').value = loc.lng();
        document.getElementById('address').value = address;
      }
  }
  else
  {
    alert("Geocoder failed due to: " + status);
  }
  });
  }
}

This function uses the geocoding service from google maps to find the address corresponding to the coordinates where the user clicked on the map. When the function get the results from google, it shows the results in our form:

// fill in the results in the form
document.getElementById('lat').value = loc.lat();
document.getElementById('long').value = loc.lng();
document.getElementById('address').value = address;

The user will just have to fill in the rest of the data to add a new location. I’ve not included here the php code to add the location to the database. You can see that in the source code!

Showing the markers

We will also have to show the locations on the map when it loads. We will call one more function when the page loads:

<body onload="initialize(); addMarkers()">

Let’s see how the function looks like:

// initialize the array of markers
var markers = new Array();
// the function that adds the markers to the map
function addMarkers()
{
  // get the values for the markers from the hidden elements in the form
  var lats = document.getElementById('lats').value;
  var lngs = document.getElementById('lngs').value;
  var addresses = document.getElementById('addresses').value;
  var names = document.getElementById('names').value;
  var descrs = document.getElementById('descrs').value;
  var photos = document.getElementById('photos').value;
  var user_names = document.getElementById('user_names').value;
  var user_locs = document.getElementById('user_locs').value;
  var las = lats.split(";;")
  var lgs = lngs.split(";;")
  var ads = addresses.split(";;")
  var nms = names.split(";;")
  var dss = descrs.split(";;")
  var phs = photos.split(";;")
  var usrn = user_names.split(";;")
  var usrl = user_locs.split(";;")
  // for every location, create a new marker and infowindow for it
  for (i=0; i<las.length; i++)
  {
  if (las[i] != "")
  {
    // add marker
    var loc = new google.maps.LatLng(las[i],lgs[i]);
    var marker = new google.maps.Marker({
      position: loc,
      map: window.map,
      title: nms[i]
    });
    markers[i] = marker;
    var contentString = [
      '<div>',
      '<ul>',
      '<li><a href="#tab-1"><span>photo</span></a></li>',
      '<li><a href="#tab-2"><span>description</span></a></li>',
      '<li><a href="#tab-3"><span>location</span></a></li>',
      '</ul>',
      '<div>',
      '<p><h1>'+nms[i]+'</h1></p>',
      '<p><img src="./photos/'+phs[i]+'"/></p>'+
      '</div>',
      '<div>',
      '<p><h1>'+nms[i]+'</h1></p>',
      '<p>Added by: '+usrn[i]+' from '+usrl[i]+'</p>'+
      '<p>'+dss[i]+'</p>'+
      '</div>',
      '<div>',
      '<p><h1>'+nms[i]+'</h1></p>',
      '<p>Address: '+ads[i]+'</p>'+
      '<p>Location: '+loc+'</p>'+
      '</div>',
      '</div>'
      ].join('');
    var infowindow = new google.maps.InfoWindow;
    bindInfoWindow(marker, window.map, infowindow, contentString);
  }
  }
}
// make conection between infowindow and marker (the infowindow shows up when the user goes with the mouse over the marker)
function bindInfoWindow(marker, map, infoWindow, contentString)
{
  google.maps.event.addListener(marker, 'mouseover', function() {
    map.setCenter(marker.getPosition());
    infoWindow.setContent(contentString);
    infoWindow.open(map, marker);
    $("#tabs").tabs();
  });
}

We will also save the markers in an array.

We will take the information about the locations from the hidden input fields we have created earlier. We will create a new marker for each of the locations. The title of the marker will be the location name. We’ll also create some infowindows to show when the user goes with the mouse over the markers. As you can see, we have used another function to bind each marker to its infowindow. We’ve also centered the map on the marker when the user goes with the mouse over it.

To show the information about the markers in the infowindow nicely, I have decided to use some sample code I found here. It uses jquery and jqueryui. We will also have to include these for the script to work:

<script src="jquery-1.4.2.min.js"></script>
<script src="jquery-ui-1.8rc3.custom.min.js"></script>
<link href="jquery-ui-1.8rc3.custom.css" rel="stylesheet" />

And we’re almost done!

All there’s left to do now is the highlightMarker function, the one called when the users goes with the mouse pointer over a location name in the location list. Let’s take a look at the function:

// highlighting a marker
// make the marker show on top of the others
// change the selected markers icon
function highlightMarker(index)
{
  // change zindex and icon
  for (i=0; i<markers.length; i++)
  {
    if (i == index)
    {
      markers[i].setZIndex(10);
      markers[i].setIcon('http://www.google.com/mapfiles/arrow.png');
    }
    else
    {
      markers[i].setZIndex(2);
      markers[i].setIcon('http://www.google.com/mapfiles/marker.png');
    }
  }
}

The function takes as an argument the index of the marker in the markers array we have created.  The highlightMarker function will make the selected marker show up on top of the other markers and change its icon. To make the marker on top of the others, we have to change the zIndex value of the marker.  The marker with the biggest value will show on top. We have set the selected markers value to 10 and the values of the rest of them to 2 (using the setZIndex function). We have also changed the icon of the marker with the setIcon function.

You might have noticed I also added a small javascript function to check if the user filled in all the information from the add form. The function is called when the form is submitted. It just checks the values filled in the form and if one of them is missing, it will show and error message and prevent the form from submitting.

And that’s it! We now have an interactive travel map!

August 04 2010

16:38

The launch of rebbbound: tracking for dribbble’s api

Last night Dave Rupert and I literally had an idea, designed it, coded it, Twitterized it and made it live… all within less than 24 hours. We are happy to release rebbbounds: Tracking the Dribbble API and allowing others to submit their own projects using the API.

Follow along on Twitter or subscribe to the RSS feed if you like. We are also excepting submissions for great ideas, sites, and work utilizing Dribbble’s API. Now go make amazing things happen! :)

July 02 2010

10:00

How to Build a Distance Finder with Google Maps API (Part-2)

In the first part of this tutorial we’ve built a distance finder using google maps. To build our app we’ve used the newest google maps api version, v3. Our distance finder lets the user write two addresses, shows them on a map, shows the route between them and computes the route’s length. Today, we will make some improvements to our distance finder and learn some other features of google maps!

What are we going to add?

We’re going to add custom markers to our map and a new feature: draggable markers. The user will be able to drag the markers around the map and the route will be computed again. We will also let the user choose what type of route to show on the map (the available options are driving, walking and bicycling). Another thing we’ll add is saving the map type selected by the user. This way, the selection will be saved when the user presses the “show” button next time and the user won’t have to select the desired map type each time he refreshes the map.

You can check out the new version here and also download the source code.

Prerequisites

I recommend reading the first part of this tutorial if you haven’t already. In this one I’ll only describe the new features.

Apart from adding the new features, I’ve made a small change to the programs structure: I’ve made a separate function for showing the route between the two points (called drawRoutes()). I’ve done this because we’ll have to call this function each time one of the markers is dragged by the user.

Adding custom markers

The first thing we’ll add are custom markers. For this, we need a small image, I’ve used a silly one as an example. The code to set the image as the marker looks like this:

var rabbit = new google.maps.MarkerImage('distance-finder-custom-marker-image.png');
// create the markers for the two locations
var marker1 = new google.maps.Marker({
   map: map,
   position: location1,
   title: "First location",
   icon: rabbit,
   draggable: true
});
var marker2 = new google.maps.Marker({
   map: map,
   position: location2,
   title: "Second location",
   icon: rabbit,
   draggable: true
});

We’ve defined a new MarkerImage object and given it the location of the image for the marker. Also, we’ve changed the options a bit when creating the markers: we’ve added the icon parameter.

Making the markers draggable

As you can see from the previous piece of code, we’ve also set the “draggable” parameter to true, making our markers draggable. The user can now drag the markers anywhere on the map. But nothing happens when the markers are dragged! We’ll have to add the code to find the address of the new locations and show the new route. But first, we’ll have to add another action listener to the markers, to define the function to be called when the user stops dragging the marker.

Here’s what we have to add to our code:

// add action events for dragging the markers
google.maps.event.addListener(marker1, 'dragend', function() {
   location1 = marker1.getPosition();
   drawRoutes(location1, location2);
});
google.maps.event.addListener(marker2, 'dragend', function() {
   location2 = marker2.getPosition();
   drawRoutes(location1, location2);
});

We’ve added one listener for each marker. Both of them save the new location of the marker (we can get that using the getPosition() function from the google maps api – the function returns the marker’s coordinates) and call the drawRoutes() function giving it the new locations as parameters.

In the drawRoutes() function we’ll have to find the addresses of the new locations and show the new line between the points and compute a new route.

To find the addresses of the points we’ll use the reverse geocoding feature provided by google maps. Geocoding means finding the coordinates of a given address, and reverse geocoding means finding the address of the points when you know the coordinates.

Here’s  the code for this:

geocoder = new google.maps.Geocoder(); // creating a new geocode object
if (geocoder)
{
  geocoder.geocode({'latLng': location1}, function(results, status)
  {
    if (status == google.maps.GeocoderStatus.OK)
    {
      if (results[0])
      {
        address1 = results[0].formatted_address;
        document.getElementById("address1").value = address1;
      }
    }
    else
    {
      alert("Geocoder failed due to: " + status);
    }
  });
}

if (geocoder)
{
  geocoder.geocode({'latLng': location2}, function(results, status)
  {
    if (status == google.maps.GeocoderStatus.OK)
    {
      if (results[0])
      {
        address2 = results[0].formatted_address;
        document.getElementById("address2").value = address2;
        continueShowRoute(location1, location2);
      }
    }
    else
    {
      alert("Geocoder failed due to: " + status);
    }
  });
}

The code is very similar to the code we’ve used to geocode the addresses. We’ve created a new geocoder object and called the geocode function as before. This time, the parameter we sent to the function is the location of the points. We’ve saved the result from the google server and the address1 and address2 variables. The address has the following format: “275-291 Bedford Ave, Brooklyn, NY 11211, USA”. We’ve written this address in the text fields from the form, so the user will know the exact address of the plotted points. We’ve used the document.getElementById() to set the values of the text fields. When both addresses are found, we call the continueShowRoute() function which shows the routes.

We’ll first show the line that connects the points. The difference from the first tutorial is that we now have to delete the last drawn line. Like this:

// hide last line
if (line)
{
   line.setMap(null);
}

Then we’ll draw the new one, just like before.

We’ll then compute the distance on a straight line between the points and show it on the screen, as I’ve shown in the first tutorial. The code to compute the quickest route and show the results on the screen also remains unchanged. The only thing we need to do is update the text on the infowindows with the new address values.

// update text in infowindows
var text1 = '
<div>'+</div>
   '<h1 id="firstHeading">First location'+
   '<div id="bodyContent">'+
   'Coordinates: '+location1+'

'+
   'Address: '+address1+'

'+
   '</div>'+
   '</div>';
var text2 = '
<div>'+</div>
   '<h1 id="firstHeading">Second location</h1>'+
   '<div id="bodyContent">'+
   'Coordinates: '+location2+'

'+
   'Address: '+address2+'

'+
   '</div>'+
   '</div>';
infowindow1.setContent(text1);
infowindow2.setContent(text2);

We’ve created new variables for holding the new texts to be shown in the infoboxes and used the setContent() function to change the texts on them.

Choosing the travel mode

We’ll also add the option to change the driving mode for the route shown on the map. The available travel modes are: driving, walking and bicycling (this option is only available for routes in the US).

We’ll first have to add a drop down box for the user to select the desired travel mode. We’ll add a new row to our previously created table:

<tr>
  <td>Route type:
    <select id="<span class="></select>
      <option value="driving">driving</option>
      <option value="walking">walking</option>
      <option value="bicycling">bicycling (only in US)</option>
    </select>
  </td>
</tr>

Now, let’s see what we have to do to show the selected travel mode correctly. We’ll have to change the options for the request we send to the directions service from google.

var travelmode = document.getElementById("travelMode").value;
// get the selected travel mode
if (travelmode == "driving")
   travel = google.maps.DirectionsTravelMode.DRIVING;
else if (travelmode == "walking")
   travel = google.maps.DirectionsTravelMode.WALKING;
else if (travelmode == "bicycling")
   travel = google.maps.DirectionsTravelMode.BICYCLING;
// find and show route between the points
var request = {
   origin:location1,
   destination:location2,
   travelMode: travel
};

We’ve first saved the selected option from the drop down box in the travelmode variable. We’ve checked the value of this variable and set the travel variable that holds the google.map.travelmode. When setting the options for the request, we’ve used this variable to set the travel mode.

Saving the selected map setting

The map types the user can choose from are: roadmap, hybrid, satellite and terrain.

We’re going to use a hidden field in the form for holding the selected map

<input id="<span class=" />maptype" type="hidden" value="roadmap"/>

When creating the new map object, we’ll first check the value of this field and set the map type accordingly.

// get the map type value from the hidden field
var maptype = document.getElementById("maptype").value;
var typeId;
if (maptype == "roadmap")
   typeId = google.maps.MapTypeId.ROADMAP;
else if (maptype == "hybrid")
   typeId = google.maps.MapTypeId.HYBRID;
else if (maptype == "satellite")
   typeId = google.maps.MapTypeId.SATELLITE;
else if (maptype == "terrain")
   typeId = google.maps.MapTypeId.TERRAIN;

// set map options
var mapOptions =
{
   zoom: 1,
   center: latlng,
   mapTypeId: typeId
};

The last thing we need to do is update the hidden field value when the user changes the map type. For this, we need to add a listener that will be triggered when the user changes the map type. The listener will update the value of the hidden field.

// event listener to update the map type
google.maps.event.addListener(map, 'maptypeid_changed', function() {
   maptype = map.getMapTypeId();
   document.getElementById('maptype').value = maptype;
});

We’ve used the getMapTypeId() function to get the new map type and the document.getElementById() function to set the value for the hidden field.

And that’s it! We’ve finished improving our distance finder!

June 30 2010

08:15

IE9 gets a Web Timing API to measure performance

Web site performance is a very important topic. We should not let our end users wait for our sites and optimizing them for load time and rendering can save us thousands of dollars in traffic. There is a lot of great content out there on performance (spearheaded by Yahoo a few years back). When it comes to testing the performance after the page has loaded though there is a lot of things you can do wrong as you need to test things with timers and hope nothing else happening to your test machine interferes with your results.

The IE9 team wants to make it easier for developers and added a new Web Timing API in the browser. Web Timing is a W3C working draft and the API implemented the NavigationTiming part of the spec in window.msPerformance.timing and offers you a few sets of information without having to hack your own solution:

JAVASCRIPT:
interface MSPerformanceTiming{
     readonly attribute unsigned longlong navigationStart;
     readonly attribute unsigned longlong fetchStart;
     readonly attribute unsigned longlong unloadStart;
     readonly attribute unsigned longlong unloadEnd;
     readonly attribute unsigned longlong domainLookupStart;
     readonly attribute unsigned longlong domainLookupEnd;
     readonly attribute unsigned longlong connectStart;
     readonly attribute unsigned longlong connectEnd;
     readonly attribute unsigned longlong requestStart;
     readonly attribute unsigned longlong requestEnd;
     readonly attribute unsigned longlong responseStart;
     readonly attribute unsigned longlong responseEnd;
     readonly attribute unsigned longlong domLoading;
     readonly attribute unsigned longlong domInteractive;
     readonly attribute unsigned longlong domContentLoaded;
     readonly attribute unsigned longlong domComplete;
     readonly attribute unsigned longlong loadStart;
     readonly attribute unsigned longlong loadEnd;
     readonly attribute unsigned longlong firstPaint;
     readonly attribute unsigned longlong fullyLoaded;
}

You have even more granular control in timingMeasures

JAVASCRIPT:
interface MSPerformanceTimingMeasures{
     readonly attribute unsigned longlong navigation;
     readonly attribute unsigned longlong fetch;
     readonly attribute unsigned longlong unload;
     readonly attribute unsigned longlong domainLookup;
     readonly attribute unsigned longlong connect;
     readonly attribute unsigned longlong request;
     readonly attribute unsigned longlong response;
     readonly attribute unsigned longlong domLoading;
     readonly attribute unsigned longlong domInteractive;
     readonly attribute unsigned longlong domContentLoaded;
     readonly attribute unsigned longlong domComplete;
     readonly attribute unsigned longlong load;
     readonly attribute unsigned longlong firstPaint;
     readonly attribute unsigned longlong fullyLoaded;
}

Read the original post on MSDN and check out the demo on IE Test Drive

April 22 2010

10:36

Desktop Notifications with Webkit

Mohit Muthanna has a nice blog post explaining how you can create desktop notifications with the latest Webkit/Chrome.

desktop-notifications

There are essentially three API calls you use:

  • window.webkitNotifications.requestPermission(callback) - Request access to Desktop Notifications for this domain.
  • window.webkitNotifications.checkPermission() - Returns 0 if this domain has Desktop Notification access.
  • window.webkitNotifications.createNotification(icon,title,body) - Returns a popup notification instance, which you can display by calling show() on it.

The demo code is pretty straight forward:

JAVASCRIPT:
  1.  
  2. function Notifier() {}
  3.  
  4. // Request permission for this page to send notifications. If allowed,
  5. // calls function "cb" with "true" as the first argument.
  6. Notifier.prototype.RequestPermission = function(cb) {
  7.   window.webkitNotifications.requestPermission(function() {
  8.     if (cb) { cb(window.webkitNotifications.checkPermission() == 0); }
  9.   });
  10. }
  11.  
  12. // Popup a notification with icon, title, and body. Returns false if
  13. // permission was not granted.
  14. Notifier.prototype.Notify = function(icon, title, body) {
  15.   if (window.webkitNotifications.checkPermission() == 0) {
  16.     var popup = window.webkitNotifications.createNotification(
  17.       icon, title, body);
  18.     popup.show();
  19.     return true;
  20.   }
  21.  
  22.   return false;
  23. }
  24.  

The user request happens the same way the W3C geo location API requests access - a toolbar popping up asking you if you are OK to see Desktop Notifications. This is good as I can see this being used by people to fake IM messages - much like they do now in the browser.

The notifications animate in from top right and have the originating URL and a "block" link - which makes it hard to use them for phishing.

Interesting concept and if it could use native systems like Growl it would spell the end of awkward alerts and hand-rolled notification windows in the document.

January 29 2010

15:17

Addmap.js – automatically analyse a text for geo locations and add a map

As part of an upcoming article on geo location I am putting together a few Geo Toys for myself and here is the first one. Addmap.js is a JavaScript that analyses an elements text content, finds geographical locations and links them to Google Maps. It also adds a map preview and a list of the found locations to the element.

See addmap.js in action below - all the content in the green box is generated from the paragraph of text above it. You can try it out for yourself by clicking the screenshot.

Demonstration screenshot of addmap.js

Using addmap.js is easy - sign up for a Google Maps Key and provide it as a configuration parameter. Then call the analyse function with the ID of the element to analyse as the parameter:

XML:
  1. <script>
  2. addmap.config.mapkey = 'YOUR_API_KEY';
  3. addmap.analyse('content');
  4. </script>

The script uses YQL and Yahoo PlaceMaker under the hood, for more info and updates on this, check the blog.

January 25 2010

15:47

An API for the Web: Learning YQL

The Yahoo Query Language is a great tool that’s guaranteed to speed up your web development time. The more complex your project, the more time YQL will save you. So, is it a framework, an application, a beverage? Today, you’ll find out what it is and how to use it!

Prefer a Video Tutorial? Join Plus!

If you’re more of a visual learner, you can watch a video version of this article instead. Simply help give back to Nettuts+ by signing up for a Plus membership, which grants you access to the Plus content for ALL of our sites – all for $9 per month.

Already a Member?

Watch the video version of this tutorial.

What is YQL?

Web apps and web services multiply like rabbits. They’re all fun to play with (like rabbits) and fun to integrate into other projects (unlike rabbits). But learning a new API every other day isn’t feasible or fun. And that’s the problem the Yahoo Query Language (YQL) is out to solve.

Think of YQL as the API for the web, the one API to rule them all. It’s not a hard one to learn, so let’s get you up to speed right now!

How do I use it?

Yahoo has put together a pretty nice console for us to flex our muscles with YQL. Load up that console, and let’s explore it.

In the right sidebar, you can choose a “table” in the “database”; a sample query will show up in the statement box at the top. To the right of the statement box, you can see what the corresponsing REST query would. Below, you have the data returned from the query; you can receive data in either XML or JSON.

So, let’s try a query!

select * from flickr.photos.interestingness(20)

Here’s one of the sample queries; This will return twenty images from the Flickr’s interestingness group. The results of the query look like this:

Let’s try another one.

select * from feed where url='http://rss.news.yahoo.com/rss/topstories'

This query returns each of the recent items in a feed, in this case, the Yahoo News Top Stories. Of course, we could handle it ourselves, but this will be quicker and easier.

You’ll notice that both these queries are for Yahoo sites; out of the box YQL only offers table for Yahoo properties. But they have made it extendable, and many people have written other tables. To get those, click the “Show Community Tables” in the sidebar. Now we can leverage everything from Netflix to the New York Times, from GitHub to Instapaper.

So how can we use YQL in our own projects? Most often, you’ll implement it using cURL, but we could do it in JavaScript as well. Let’s look at the cURL now, and what we get from it.

Let’s take that flickr interestingness query we just looked at; here’s what we do:

$query = 'select * from flickr.photos.interestingness(20)';

// insert the query into the full URL
$url = 'http://query.yahooapis.com/v1/public/yql?format=json&q=' . urlencode($query);

// set up the cURL
$c = curl_init();
curl_setopt($c, CURLOPT_URL, $url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($c, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($c, CURLOPT_SSL_VERIFYHOST, false);

// execute the cURL
$rawdata = curl_exec($c);
curl_close($c);

// Convert the returned JSON to a PHP object
$data = json_decode($rawdata);

// Show us the data
echo '<pre>';
print_r($data);
echo '</pre>';

It’s not too complicated; if you’ve not familiar with cURL, check out Burak Guzel’s great tut here on Nettuts earlier this month. We assign the cURL return value to $rawdata and then convert it to PHP. Calling the print_r function gives us these results.

As you can see, our $data object has one property: query. That property parents all the tasty bits. You can see from the $data->query->count that we received 20 objects, matching our query. But it’s $data->query->results that we’re really interested in; that’s where our data is. The results object has one array in it, called photos.

Armed with this information, we could display the twenty latest interesting photos from flickr (we would use the url http://www.flickr.com/photos/$owner/$id, taking those variable from each photo object.)

I should note here that not all queries will display their results in the same way; some aren’t quite as developer friendly as this one. It’s a good idea to use the YQL console (or just print_r) to check out the results format before proceeding.

So you’ve got an idea of what YQL is and how you can use it. Now, let’s use YQL in a small project!

Tuts+ Tweets

Let’s build a Twitter panel that will show the latest tweets from each of the Tuts+ sites’ Twitter accounts. We’ll start by going to the YQL console and looking at our options. Make sure you’re viewing the community tables. Under the Twitter section, choose twitter.user.profile (which will include the latest tweet), or type this query into the statement box:

select * from twitter.user.profile where id="nettuts"

As we can see from the results in the tree view, the object we’ll get back isn’t formatted quite as nicely as the Flickr ones; however, we’ll make it!

Let’s begin by replacing the Flickr query in the above example with this one. Here’s what we get:

What’s wrong? Since the twitter datatable isn’t one of Yahoo’s built-in tables, we need to tell YQL to use the community tables as well. How do we do that? We’ll add a the key/value env=store://datatables.org/alltableswithkeys to our base URL; now the $url variable should look like this:

$url = 'http://query.yahooapis.com/v1/public/yql?format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&q=' . urlencode($query);

Now if we try it …

We’ve got the twitter data!

Now that we’re successfully getting Nettuts’ twitter profile, let’s consider the others. We need to get the profiles of the following accounts:

  • Psdtuts
  • Vectortuts
  • Audiotuts
  • Aetuts
  • Activetuts
  • Cgtuts
  • Phototuts
  • Tutsplus

So do we need to do eight more cURLs to YQL to get all the data we need? Thankfully, YQL has our back here; we can use this:

SELECT * FROM query.multi where queries="QUERIES GO HERE"

Armed with this knowledge, we’re ready to build our widget. We’ll begin with an array of the twitter queries:

$twitter = array (
'tutsplus'   => 'SELECT * FROM twitter.user.profile WHERE id=\'tutsplus\'',
'nettuts'    => 'SELECT * FROM twitter.user.profile WHERE id=\'nettuts\'',
'phototuts'  => 'SELECT * FROM twitter.user.profile WHERE id=\'phototuts\'',
'audiotuts'  => 'SELECT * FROM twitter.user.profile WHERE id=\'audiotuts\'',
'psdtuts'    => 'SELECT * FROM twitter.user.profile WHERE id=\'psdtuts\'',
'aetuts'     => 'SELECT * FROM twitter.user.profile WHERE id=\'aetuts\'',
'cgtuts'     => 'SELECT * FROM twitter.user.profile WHERE id=\'cgtutsplus\'',
'vectortuts' => 'SELECT * FROM twitter.user.profile WHERE id=\'vectortuts\'',
'activetuts' => 'SELECT * FROM twitter.user.profile WHERE id=\'activetuts\''
);

Let’s create our full query now:

$query ='SELECT * FROM query.multi where queries="' . implode(';', $twitter) . '"';

Since it’s getting a bit complicated, we’ll put the root URL in its own variable, and then put everything together. Note that I added diagnostics=false to the root URL; this prevents YQL from returning a bit of extra data with our results. Why? It will just make it easier when we inspect the results in a moment.

$root = 'http://query.yahooapis.com/v1/public/yql?format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&diagnostics=false';
$url = $root . '&q=' . urlencode($query);

Now that we’ve got our complete URL, let’s create our cURL, just as we already did:

$c = curl_init();

curl_setopt($c, CURLOPT_URL, $url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($c, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($c, CURLOPT_SSL_VERIFYHOST, false);

And like last time, we’ll catch the results, convery the JSON to a PHP object, and output them for inspection.

$data = json_decode(curl_exec($c));

curl_close($c);

echo '<pre>';
print_r($data);
echo '</pre>';

I won’t show them to you here, but you should be able to stroll through them and see the pieces of data we want to pull out. Notice that the results object has a results array inside it; that’s a bit unexpected, but I believe it has something to do with the fact that we’re executing multiple queries. When you’re done, head back to the PHP and create a $results variable (and don’t forget to remove the printr code):

$results = $data->query->results->results;

The HTML

Now that we’ve got our data, its time for some HTML. Let’s throw in a basic template under the PHP:

<!DOCTYPE html>
<html>
<head>
    <meta charset='utf-8' />
    <title>Yahoo Query Language Introduction</title>
<link rel="stylesheet" href="default.css" />
</head>
<body>
<h1>The Tuts+ Network: Latest Tweet</h1>
<ul>

</ul>
</body>
</html>

Now, inside the list, we’ll use some PHP to iterate over each of the items in the $results array we extracted and build an entry for it. First, set up the loop:

<?php for ($i = 0; i$i < count($twitter); $i++) : ?>
	<li>

	</li>
<?php endfor; ?>

Inside that list item, open a PHP codeblock; we should begin by setting up a few variables.

	$meta = $results[$i]->item->meta;
	$item = $results[$i]->item->item;
	$link = $results[$i]->item->resource;

Unfortunately, the author of the twitter table didn’t make the return object too easy to work with; instead of using key/value pairs, each key and value are entries in their own array. So it won’t be incredible obvious what we’re each object reference is when we’re done. However, remember that this is all subject to the author. The flickr table we looked at earlier—or the RSS table that you should check out—is a much more usable API.

So what do we want in our twitter widget? Let’s show the user avatar on the left and their name, username, latest tweet, and time of latest tweet on the right. To do so, let’s add this below those variables:

<?php echo "<a class='img' href='$link'><img src='" .$item[0]->resource ."' alt='" . $meta[0]->content ."' /></a>"; ?>
	<div>
		<?php echo "<a href='$link'>" .$meta[0]->content . "</a> "; ?>
		<small>(<?php echo $meta[1]->content ?>)</small>
		<small> <?php echo $item[1]->meta[2]->content; ?> </small>
		<?php echo '<a href="' . $item[1]->resource . '">' . $item[1]->meta[1]->content . '</a>'; ?>
	</div>

I know it’s a bit cryptic, but if you look at this and then look at the object we printed out to the browser, you’ll see that it works out nicely. We start with an anchor, and put the avatar image in it. After that, inside a div, we make another link for the name, which links to their twitter page. Then we put their username and time of last tweet in small tags (and if we wanted to, we could convert the time to something a little more viewer-friendly). Finally, we put their latest tweet in an anchor; cliking it will take you to the tweet’s page.

Here’s what this looks like:

Not pretty yet, but we’ve got some good hooks for our CSS.

The CSS

Nothing complicated here; we start by evening out the landscape:

body {
	font: 13px/1.5 'Helvetica Neue', Arial, 'Liberation Sans', FreeSans, sans-serif;
	background:#ececec;
	padding:10px;
}
img {
	border:0;
}

Then we’ll give out list its look and feel:

ul {
	margin:0;
	padding:0;
	border:1px solid #474747;
	border-radius:5px;
	-moz-border-radius:5px;
	-webkit-border-radius:5px;
	background:#ccc;
	width:50%;
}
li {
	min-height:50px;
	padding:10px 5px;
	list-style-type:none;
	border-bottom:1px solid #474747;
	border-top:1px solid #ececec;
}
li div {
	padding-left:58px;
}
li a.img {
	float:left;
	padding-right:5px;
}
li a {
	display:block;
}
li:first-child {
	border-radius:5px 5px 0 0;
	-moz-border-radius:5px 5px 0 0;
	-webkit-border-radius:5px 5px 0 0;
}
li:last-child {
	border-bottom:0;
}

As a final touch, we’ll give each list item a shadow on hover:

li:hover {
	box-shadow: 0px 0px 15px #000;
	-moz-box-shadow: 0px 0px 15px #000;
	-webkit-box-shadow: 0px 0px 15px #000;
}

There you have it! Behold our completed twitter widget:

Doing it with JavaScript

If you’d prefer, you can use jQuery to execute a YQL statement. You can get the plugin—called jquery.queryYQL—on GitHub. It’s pretty simple to use; here’s a modification of the example query:

$.queryYQL("select * from feed where url='http://feeds.feedburner.com/nettuts?format=xml'", function (data) {

	var ul = $("<ul/>");

	$.each(data.query.results.item, function () {
		$("<li/>").append(this.title).appendTo(ul);
	});

	ul.appendTo($("#content"));
});

Will you use it?

YQL is a pretty powerful tool; it should save you a lot of time by giving you a single, common API to access content all over the web. You really should browse through the list of available tables; you’ll probably find something that will same you a lot of time. Some tables even provide authentication and writing.

Is YQL a tool you’ll use in the future? Let me know in the comments!

Write a Plus Tutorial

Did you know that you can earn up to $600 for writing a PLUS tutorial and/or screencast for us? We’re looking for in depth and well-written tutorials on HTML, CSS, PHP, and JavaScript. If you’re of the ability, please contact Jeffrey at nettuts@tutsplus.com.

Please note that actual compensation will be dependent upon the quality of the final tutorial and screencast.

Write a PLUS tutorial


January 15 2010

13:19

Pure CSS bar charts as a simple API

I am right now working on a high-traffic project that will run in a sandbox that doesn't allow me to pull third party JavaScript or use canvas/Flash. Yet I need to generate bar charts from a set of data.

Pure CSS bar charts

The solution to me was to create the charts from HTML using CSS. There have been some solutions for this problem already but I wanted something very simple and easy to maintain. Hence all the markup I am using is this:

XML:
  1. <ul>
  2.   <li><span>400</span></li>
  3.   <li><span>20</span></li>
  4.   <li><span>30</span></li>
  5.   <li><span>233</span></li>
  6. </ul>

Instead of hard-coding any of the trickery necessary I wrote a PHP script to generate the HTML, the styles and do all the math. So all that is needed to render one of the charts is the following code:

PHP:
  1. <?php
  2.   $values = '400,20,30,233,312,78,20,67';
  3.   $height = 100;
  4.   $width = 600;
  5.   $bargap = 0;
  6.   include('csscharts.php');
  7. ?>

Of course, this can also be turned into a web service - you can get the chart with the following URL:

http://icant.co.uk/csscharts/csscharts.php?values=400,20,30,233,312,78,20,67&height=100&width=600&bargap=0

And if you specify JSON as the format you get it with a callback to a function called csscharts:

http://icant.co.uk/csscharts/csscharts.php?values=400,20,30,233,312,78,20,67&format=json

JAVASCRIPT:
  1. csscharts(
  2.   {
  3.     chart:"<ul class=\"barchart\" [… the rest of the html …]</ul>"
  4.   }
  5. )

That way you can use it in JavaScript:

JAVASCRIPT:
  1. <script>
  2. function csscharts(o){
  3.   var container = document.getElementById('container');
  4.   c.innerHTML = o.chart + c.innerHTML;
  5. }
  6. </script>

You can see some demos here, get detailed info about the CSS trickery used and of course download the code on GitHub.

December 15 2009

13:54

Downloadify – create downloads on the fly without server interaction

Downloadify is an interesting solution to offer files as downloads without having to create temporary files and loop through a script that sets a force-download header.

As shown in the demo page offering a file for download and setting its content is as easy as this:

JAVASCRIPT:
  1. Downloadify.create('downloadify',{
  2.   filename: function(){
  3.     return document.getElementById('filename').value;
  4.   },
  5.   data: function(){
  6.     return document.getElementById('data').value;
  7.   },
  8.   onComplete: function(){
  9.     alert('Your File Has Been Saved!');
  10.   },
  11.   onCancel: function(){
  12.     alert('You have cancelled the saving of this file.');
  13.   },
  14.   onError: function(){
  15.     alert('You must put something in the File Contents or there will be nothing to save!');
  16.   },
  17.   swf: 'media/downloadify.swf',
  18.   downloadImage: 'images/download.png',
  19.   width: 100,
  20.   height: 30,
  21.   transparent: true,
  22.   append: false
  23. });

As you can see you have callbacks for all the possibilities and several parameters to style the flash movie overlaying the real "download this" button. The only issue I can see is that it doesn't work by sending the form via keyboard. This needs fixing.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl