Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

October 22 2013

06:30

Bento: One-Stop Shop for Online-Courses on Web Development


  

It’s merely a few weeks ago, that we brought you a comprehensive overview of 22 of the online learning resources the planet has to offer. Just today we stumbled upon another effort for web designers, developers and programmers. An offering by the name of Bento curates the best learning resources in its field. The collection is community-based and as such more helpful than any Google search result could ever be ;-)

June 18 2013

00:20

Advanced Python Data Structures

The aim of this tutorial is to show off Python’s data structures and the best way to use them. Depending on what you need from a data structure, whether it’s fast lookup, immutability, indexing, etc, you can choose the best data structure for the job and most of the time, you will be combining data structures together to obtain a logical and easy to understand data model.


Introduction

Python data structures are very intuitive from a syntax point of view and they offer a large choice of operations. This tutorial tries to put together the most common and useful information about each data structure and offer a guide on when it is best to use one structure or another. You can choose different kinds of data structures depending on what the data involves, if it needs to be modified, or if it’s fixed data, and even what access type you would like, such as at the beginning/end/random etc.


Lists

A List represents the most versatile type of data structure in Python. It can contain items of different types and it has no rule against unicity. List indices start from zero, the elements can be sliced, concatenated, and so on. Lists also have a lot of similarities with strings, supporting the same kind of operations but unlike strings, lists are mutable.

How to Construct a List

A list can be built using the keyword list or using square brackets: [], both of which accept comma separated values. Here’s an example:

>>> l = ['a', 'b', 123]
>>> l
['a', 'b', 123]

How to Retrieve an Element From a List

>>> l[0]
'a'
>>> l[10]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range

As seen above, in order to access the data within your list, you must know what index position the element is at, otherwise you get an “index out of range” error.

How to Slice a List

All slicing operations return a shallow copy of the list. The slicing indexes are optional and they work in the same way as slicing indexes for strings.

Here’s an example of how to slice a list:

>>> l = ['a', 'b', 123]
>>> l[:]
['a', 'b', 123]
>>> new_l = l[1:]
>>> new_l
['b', 123]
>>> l
['a', 'b', 123]

Let’s take a look at some other common list operations.

Inserting and Removing Elements

>>> l = ['a', 'b', 123]
>>> l.append(234) #inserts an element at the end of the list
>>> l
['a', 'b', 123, 234]
>>> l.insert(2, 'c') #inserts an element into the third position
>>> l
['a', 'b', 'c', 123, 234]
>>> l.insert(-1, 111) #inserts an element into the second from last position of the list (negative indices start from the end of the list)
>>> l
['a', 'b', 'c', 123, 111, 234]
>>> l.remove(111) #removes an element based on value
>>> l
['a', 'b', 'c', 123, 234]
>>> l.remove('does not exist in the list') 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: list.remove(x): x not in list

Retrieving and Looking Up Elements

Lists can also be used as stacks or queues because of how easy it is to add and remove elements from the beginning or end of the list.

>>> last_element = l.pop() #returns the last element, modifying the list
>>> last_element
234
>>> l
['a', 'b', 'c', 123]
>>> third_element = l.pop(2) #returns the third element, modifying the list
>>> third_element
'c'
>>> l
['a', 'b', 123]
>>> l.index('a') 
0
>>> l.index('does not exist in the list')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: 'does not exist in the list' is not in list
>>> l.count('a') #returns the number of occurrences of an element 
1

Whole List Operations

>>> l.extend ([1, 2]) # concatenates a list on to the existing list
>>> l
['a', 'b', 123, 1, 2]
>>> l.sort()
>>> l
[1, 2, 123, 'a', 'b']
>>> l.reverse()
>>> l
['b', 'a', 123, 2, 1]

As you can see, it’s very easy to extend, sort, and reverse lists using the above methods.

List Comprehensions

A list comprehension means, constructing a list in a way that is very natural from a mathematical point of view. The code for doing this is brief and very easy to read. A simple example of when you would use list comprehensions is when you want to construct a new list based on the elements from another list. Let’s have a look at how you can multiply all of the elements of a numeric list by two, in a simple one line construct:

>>> l = [1, 2, 3]
>>> new_l = [x*2 for x in l]
>>> new_l 
[2, 4, 6]

In the example above, the comprehension is represented by the multiplication expression that will be applied to every x element in the original, l list.

When to Use Lists

As shown in the examples above, lists are best used in the following situations:

  • When you need a mixed collection of data all in one place.
  • When the data needs to be ordered.
  • When your data requires the ability to be changed or extended. Remember, lists are mutable.
  • When you don’t require data to be indexed by a custom value. Lists are numerically indexed and to retrieve an element, you must know its numeric position in the list.
  • When you need a stack or a queue. Lists can be easily manipulated by appending/removing elements from the beginning/end of the list.
  • When your data doesn’t have to be unique. For that, you would use sets.

Sets

A set is an unordered collection with no duplicate values. A set can be created by using the keyword set or by using curly braces {}. However, to create an empty set you can only use the set construct, curly braces alone will create an empty dictionary.

Use the set keyword to create an empty set, curly brackets {} will create an empty dictionary.

How to Construct a Set

The set construct accepts one argument, a list.

>>> l = [1, 2, 3]
>>> s = set(l)
>>> s
set([1, 2, 3])

The {} construct is straight forward as well:

>>> s = {1, 2, 3}
>>> s

Sets are used to eliminate duplicate values from within a list:

>>> l = [1, 2, 3, 3]
>>> s = set(l)
>>> s
set([1, 2, 3])

The way a set detects if a clash between non-unique elements has occurred is by indexing the data in memory, creating a hash for each element. This means that all elements in a set must be hashable.

Here is an example:

>>> set ([1, [1,2]])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'

Note that a hashable object doesn’t necessarily mean that an object has the __hash__ method available to it. It is important that the hash of the object doesn’t change during its lifetime, which is obviously not the case with lists, sets, or dictionaries (dictionaries will be discussed later in this tutorial).

Set Operations

The set structure also supports mathematical operations like:

  • Union
  • Intersection
  • Difference
  • Symmetric Difference

Here are a few examples:

>>> set1 = set([1, 2, 3])
>>> set2 = set([3, 4, 5])
>>> set1 | set2 #union
set([1, 2, 3, 4, 5])
>>> set1 & set2 #intersection
set([3])
>>> set1 -  set2 #difference
set([1, 2])
>>> set1 ^ set2 #symmetric difference (elements that are in the first set and the second, but not in both)
set([1, 2, 4, 5])
>>> 

Set Comprehensions

Just like lists, sets also support comprehensions. Here is an example of how to use set comprehensions to find the unique consonants within a word:

>>> vowels = ['a', 'e', 'i', 'o', 'u']
>>> {x for x in 'maintenance' if x not in vowels }
set(['c', 'm', 't', 'n'])

Frozensets

A frozenset is basically just like a regular set, except that is immutable. It is created using the keyword frozenset, like this:

>>> frozen = frozenset([1, 2, 3])

When to Use Sets

You should choose to use a set in the following situations:

  • When you need a unique set of data: Sets check the unicity of elements based on hashes.
  • When your data constantly changes: Sets, just like lists, are mutable.
  • When you need a collection that can be manipulated mathematically: With sets it’s easy to do operations like difference, union, intersection, etc.
  • When you don’t need to store nested lists, sets, or dictionaries in a data structure: Sets don’t support unhashable types.

Tuples

Tuples are immutable, but can hold mutable objects.

A tuple is represented by a number of values separated by commas. Unlike lists, tuples are immutable and the output is surrounded by parentheses so that nested tuples are processed correctly. Additionally, even though tuples are immutable, they can hold mutable data if needed.

How to Construct a Tuple

Constructing an empty tuple requires parentheses. Here’s an example:

>>> my_empty_tuple = ()
>>> my_empty_tuple 
()

Constructing a tuple with one element requires a trailing comma. Example:

>>> one_elem_tuple = 'a',
>>> one_elem_tuple 
('a',)
>>> one_elem_tuple = ('a',)
>>> one_elem_tuple
('a',)

If the trailing comma is not there when creating a single element tuple, regardless if you use parentheses or not, Python will just interpret the value as a literal and will not create the tuple.

Constructing a tuple with multiple elements requires a list of values separated by commas. Here’s an example:

>>> s = 'a', 'b', [1, 2, 3]
>>> s
('a', 'b', [1, 2, 3])
>>> 

Tuples, from a performance point of view are great because of their immutability. Python will know exactly how much memory to allocate for the data to be stored.

When to Use Tuples

  • When you need to store data that doesn’t have to change.
  • When the performance of the application is very important. In this situation you can use tuples whenever you have fixed data collections.
  • When you want to store your data in logical immutable pairs, triples etc.

Dictionaries

Dictionaries are represented by a key:value pair. In other words, they are maps or associative collections. The keys, unlike lists where they are numeric, can be of any immutable type and must be unique. The values can be of any type, mutable or immutable.

How to Construct a Dictionary

There are several ways to construct a dictionary. The most efficient way in terms of performance is to use curly braces {}:

>>> vowels = {1: 'a', 2: 'e', 3: 'i', 4: 'o', 5:'u'}
>>> vowels
{1: 'a', 2: 'e', 3: 'i', 4: 'o', 5: 'u'}

Another way to create a dictionary is by using comprehensions. By using an input collection you can create the key:value pairs in one simple construct.

Here’s an example:

>>> {x:x*x for x in (1, 2, 3)}
{1: 1, 2: 4, 3: 9}

You can also use the keyword dict and get the same result. The dict construct takes as an argument, a list of key:value pairs. Here’s one in action:

>>> dict([(1,'a'), (2,'e'), (3,'i'), (4,'o'), (5,'u')])
{1: 'a', 2: 'e', 3: 'i', 4: 'o', 5: 'u'}

If the keys are strings, you can use the following keyword expression:

>>> dict(a=1, e=2, i=3, o=4, u=5)
{'i': 3, 'u': 5, 'e': 2, 'a': 1, 'o': 4}

However, it does not work the other way around when the keywords are numeric:

>>> dict(1=a, 2=e, 3=i, 4=o, 5=u)
  File "<stdin>", line 1
SyntaxError: keyword can't be an expression

Dictionary Operations

Accessing data in a dictionary is very straight forward, just wrap its key name within square brackets:

>>> vowels = {1: 'a', 2: 'e', 3: 'i', 4: 'o', 5:'u'}
>>> vowels[1]
'a'
>>> vowels[10]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 10
>>> 

As seen above, a KeyError occurs if the key doesn’t exist in the dictionary.

Also, a key:value pair can be deleted using the del keyword, like so:

>>> vowels = {1: 'a', 2: 'e', 3: 'i', 4: 'o', 5:'u'}
>>> del(vowels[1])
>>> vowels
{2: 'e', 3: 'i', 4: 'o', 5: 'u'}
>>> del(vowels[10])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 10
>>> 

Built-in Methods

Dictionaries support many built-in methods, but some of the most useful ones are: keys(), values(), iteritems(), itervalues(), and has_key().

Here’s an example:

>>> vowels = {1: 'a', 2: 'e', 3: 'i', 4: 'o', 5:'u'}
>>> vowels.keys()
[1, 2, 3, 4, 5]
>>> vowels.values()
['a', 'e', 'i', 'o', 'u']

The methods iteritems() and itervalues() return iterators, so they can be used in for loops. Here’s an example:

>>> for k, v in vowels.iteritems():
...     print k, v
... 
1 a
2 e
3 i
4 o
5 u
>>> for v in vowels.itervalues():
...     print v
... 
a
e
i
o
u
>>> 

When to Use a Dictionary

  • When you need a logical association between a key:value pair.
  • When you need fast lookup for your data, based on a custom key.
  • When your data is being constantly modified. Remember, dictionaries are mutable.

Conclusion

General Python Data Structure Usage

  • Use lists if you have a collection of data that does not need random access. For random access, you need to have knowledge of the element’s numeric index. Try to choose lists when you need a simple, iterable collection that is modified frequently. Lists are very useful in comprehension expressions for constructing sets or dictionaries.
  • Use a set if you need unicity for the elements and you don’t need a nested dictionary or list. Also remember that a set cannot hold any unhashable data types.
  • Use tuples when your data cannot change. Many times, a tuple is used in combination with a dictionary, for example, a tuple might represent a key, because it’s immutable.
  • Use frozensets if you need both unique data and immutability.

Other Tips

  • Every data structure has a multitude of built in methods and capabilities. If you go through them carefully you’ll learn how the data is meant to be used. For example, if you call dir and pass in a set object, you will see that it holds a lot of methods that can perform mathematical operations.
  • Most of the time, your data needs to adapt to the operations that you want to perform. So if you know your object will be hashed, create an immutable structure.
  • Most data structures have multiple ways of constructing or accessing its data. If you are concerned about the performance of your application always read the documentation or read the source code of the construct implementation to find out more about its performance. For example, creating a dictionary by using dict offers less performance than using the curly braces ({}) syntax.
  • While debugging, many developers find it useful to learn how to identify and connect the error that they are receiving to a specific data structure. For example, you’ll find that lists throw IndexError and ValueError and dictionaries throw KeyError. This technique should help you with detecting which of your data structures is causing the error.

June 12 2013

23:53

Intro to Flask: Signing In and Out

Many web applications require users to sign in and out in order to perform important tasks (like administration duties). In this article, we’ll create an authentication system for our application.

In the previous article, we built a contact page using the Flask-WTF and Flask-Mail extensions. We’ll use Flask-WTF, once again, this time to validate a user’s username and password. We’ll save these credentials into a database using yet another extension called Flask-SQLAlchemy.

You can find the source code for this tutorial on GitHub. While following along with this tutorial, when you see a caption, such as Checkpoint: 13_packaged_app, it means that you can switch to the GIT branch named “13_packaged_app” and review the code at that point in the article.


Growing the Application

So far, our Flask app is a fairly simple application. It consists of mostly static pages; so, we’ve been able to organize it as a Python module. But now, we need to reorganize our application to make it easier to maintain and grow. The Flask documentation recommends that we reorganize the app as a Python package, so let’s start there.

Our app is currently organized like this:

flaskapp/
└── app/
        ├── static/
        ├── templates/
        ├── forms.py
        ├── routes.py
        └── README.md

To restructure it as a package, let’s first create a new folder inside app/ named intro_to_flask/. Then move static/, templates/, forms.py and routes.py into intro_to_flask/. Also, delete any .pyc files that are hanging around.

flaskapp/
└── app/
        ├── intro_to_flask/
        │      ├── static/
        │      ├── templates/
        │      ├── forms.py
        │      ├── routes.py
        └── README.md

Next, create a new file named __init__.py and place it inside intro_to_flask/. This file is required to make Python treat the intro_to_flask/ folder as a package.

flaskapp/
└── app/
        ├── intro_to_flask/
        │      ├── __init__.py
        │      ├── static/
        │      ├── templates/
        │      ├── forms.py
        │      ├── routes.py
        └── README.md

When our app was a Python module, the application-wide imports and configurations were specified in routes.py. Now that the app is a Python package, we’ll move these settings from routes.py into __init__.py.

app/intro_to_flask/__init__.py

from flask import Flask

app = Flask(__name__)

app.secret_key = 'development key'

app.config["MAIL_SERVER"] = "smtp.gmail.com"
app.config["MAIL_PORT"] = 465
app.config["MAIL_USE_SSL"] = True
app.config["MAIL_USERNAME"] = 'contact@example.com'
app.config["MAIL_PASSWORD"] = 'your-password'

from routes import mail
mail.init_app(app)

import intro_to_flask.routes

The top of routes.py now looks like this:

app/intro_to_flask/routes.py

from intro_to_flask import app
from flask import render_template, request, flash
from forms import ContactForm
from flask.ext.mail import Message, Mail

mail = Mail()
.
.
.
# @app.route() mappings start here

We previously had app.run() inside of routes.py, which allowed us to type $ python routes.py to run the application. Since the app is now organized as a package, we need to employ a different strategy. The Flask docs recommend adding a new file named runserver.py and placing it inside app/. Let’s do that now:

flaskapp/
└── app/
        ├── intro_to_flask/
        │      ├── __init__.py
        │      ├── static/
        │      ├── templates/
        │      ├── forms.py
        │      ├── routes.py
        ├── runserver.py        
        └── README.md

Now take the app.run() call from routes.py and place it inside of runserver.py.

app/runserver.py

from intro_to_flask import app

app.run(debug=True)

Now you can type $ python runserver.py and view the app in the browser. From the top, here’s how you’ll enter your development environment and run the app:

$ cd flaskapp/
$ . bin/activate
$ cd app/
$ python runserver.py

The app is now organized as a package, we’re ready to move on and install a database to manage user credentials.

Checkpoint: 13_packaged_app


Flask-SQLAlchemy

We’ll use MySQL for our database engine and the Flask-SQLAlchemy extension to manage all of the database interaction.

Flask-SQLAlchemy uses Python objects instead of SQL statements to query the database. For example, instead of writing SELECT * FROM users WHERE firstname = "lalith", you would write User.query.filter_by(username="lalith").first().

The moral of this aside is to not completely rely on, or abandon a database abstraction layer like Flask-SQLAlchemy, but to be aware of it, so that you can determine when it’s useful for your needs.

But why can’t we just write raw SQL statements? What’s the point of using this weird syntax? As with most things, using Flask-SQLAlchemy, or any database abstraction layer, depends on your needs and preferences. Using Flask-SQLAlchemy allows you to work with your database by writing Python code instead of SQL. This way you don’t have SQL statements scattered amidst your Python code, and that’s a good thing, from a code quality perspective.

Also, if implemented correctly, using Flask-SQLAlchemy will help make your application to be database-agnostic. If you start building your app on top of MySQL and then decide to switch to another database engine, you shouldn’t have to rewrite massive chunks of sensitive database code. You could simply switch out Flask-SQLAlchemy with your new database abstraction layer without much of an issue. Being able to easily replace components is called modularity, and it’s a sign of a well designed application.

On the other hand, it might be more intuitive and readable if you write raw SQL statements instead of learning how to translate it into Flask-SQLAlchemy’s Expression Language. Fortunately, it’s possible to write raw SQL statements in Flask-SQLAlchemy too, if that’s what you need.

The moral of this aside is to not completely rely on, or abandon a database abstraction layer like Flask-SQLAlchemy, but to be aware of it, so that you can determine when it’s useful for your needs. For the database queries in this article, I’ll show you both the Expression Language version and the equivalent SQL statement.

Installing MySQL

Check to see if your system already has MySQL by running the following command in your terminal:

$ mysql --version

If you see a version number, you can skip to the “Creating a Database” section. If the command was not found, you’ll need to install MySQL. With the large variety of different operating systems out there, I’ll defer to Google to provide installation instructions that work for your OS. The installation usually consists of running a command or an executable. For example, the Linux command is:

$ sudo apt-get install mysql-server mysql-client

Creating a Database

Once MySQL is installed, create a database for your app called ‘development‘. You can do this from a web interface like phpMyAdmin or from the command line, as shown below:

$ mysql -u username -p
Enter password:

mysql> CREATE DATABASE development;

Installing Flask-SQLAlchemy

Inside the isolated development environment, install Flask-SQLAlchemy.

$ pip install flask-sqlalchemy

When I tried to install Flask-SQLAlchemy, I received an error stating that the installation had failed. I searched the error and found that others had resolved the problem by installing libmysqlclient15-dev, which installs MySQL’s development files. If your Flask-SQLAlchemy installation fails, Google the error for solutions or leave a comment and we’ll try to help you figure it out.

Configuring Flask-SQLAlchemy

Just as we did with Flask-Mail, we need to configure Flask-SQLAlchemy so that it knows where the development database lives. First, create a new file named models.py, along with adding in the following code

app/intro_to_flask/models.py

from flask.ext.sqlalchemy import SQLAlchemy

db = SQLAlchemy()

Here we import the SQLAlchemy class from Flask-SQLAlchemy (line one) and create a variable named db, containing a usable instance of the SQLAlchemy class (line three).

Next, open __init__.py and add the following lines after mail.init_app(app) and before import intro_to_flask.routes.

app/intro_to_flask/__init__.py

app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://your-username:your-password@localhost/development'

from models import db
db.init_app(app)

Let’s go over this:

  1. Line one tells the Flask app to use the ‘development‘ database. We specify this through a data URI which follows the pattern of: mysql://username:password@server/database. The server is ‘localhost’ because we’re developing locally. Make sure to fill in your MySQL username and password.
  2. db, the usable instance of the SQLAlchemy class we created in models.py, still doesn’t know what database to use. So we import it from models.py (line three) and bind it to our app (line four), so that it also knows to use the ‘development‘ database. We can now query the ‘development‘ database through our db object.

Now that our configuration is complete, let’s ensure that everything works. Open routes.py and create a temporary URL mapping so that we can perform a test query.

app/intro-to-flask/routes.py

from intro_to_flask import app
from flask import Flask, render_template, request, flash, session, redirect, url_for
from forms import ContactForm, SignupForm, SigninForm
from flask.ext.mail import Message, Mail
from models import db
.
.
.
@app.route('/testdb')
def testdb():
  if db.session.query("1").from_statement("SELECT 1").all():
    return 'It works.'
  else:
    return 'Something is broken.'

First we import the database object (db) from models.py (line five). We then create a temporary URL mapping (lines 9-14) wherein we issue a test query to ensure that the Flask app is connected to the ‘development‘ database. Now when we visit the URL /testdb, a test query will be issued (line 11); this is equivalent to the SQL statement SELECT 1;. If all goes well, we’ll see “It works” in the browser. Otherwise, we’ll see an error message stating what went wrong.

I received an error when I visited the /testdb URL: ImportError: No module named MySQLdb. This meant that I didn’t have the mysql-python library installed, so I tried to install it by typing the following:

$ pip install mysql-python

That installation failed, too. The new error message suggested that I first run easy_install -U distribute and then try the mysql-python installation again. So I did, just like below:

$ easy_install -U distribute
$ pip install mysql-python

This time the mysql-python installation succeeded, and then I received the “It works” success message in the browser. Now the reason I’m recounting the errors I’ve received and what I did to solve them is because installing and connecting to databases can be a tricky process. If you get an error message, please don’t get discouraged. Google the error message or leave a comment, and we’ll figure it out.

Once the test query works, delete the temporary URL mapping from routes.py. Make sure to retain the “from models import db“” part, because we’ll need it next.

Checkpoint: 14_db_config


Create a User Model

It’s not a good idea to store passwords in plain text, for security reasons.

Inside the ‘development‘ database, we need to create a users table where we can store each user’s information. The information we want to collect and store are the user’s first name, last name, email, and password.

It’s not a good idea to store passwords in plain text, for security reasons. If an attacker gains access to your database, they would be able to see each user’s login credentials. One way to defend against such an attack is to encrypt passwords with a hash function and a salt (some random data), and store that encrypted value in the database instead of the plain text password. When a user signs in again, we’ll collect the password that was submitted, hash it, and check if it matches the hash in the database. Werkzeug, the utility library on which Flask is built, provides the functions generate_password_hash and check_password_hash for these two tasks, respectively.

With this in mind, here are the columns we’ll need for the users table:

ColumnTypeConstraintsuidintPrimary Key, Auto Incrementfirstnamevarchar(100)lastnamevarchar(100)emailvarchar(120)Uniquepasswordvarchar(54)

Just like before, you can create this table from a web interface such as phpMyAdmin or from the command line, as shown below:

mysql> CREATE TABLE users (
uid INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
firstname VARCHAR(100) NOT NULL,
lastname VARCHAR(100) NOT NULL,
email VARCHAR(120) NOT NULL UNIQUE,
pwdhash VARCHAR(100) NOT NULL
);

Next, in models.py, let’s create a class to model a user with attributes for a user’s first name, last name, email, and password.

app/intro_to_flask/models.py

from flask.ext.sqlalchemy import SQLAlchemy
from werkzeug import generate_password_hash, check_password_hash

db = SQLAlchemy()

class User(db.Model):
  __tablename__ = 'users'
  uid = db.Column(db.Integer, primary_key = True)
  firstname = db.Column(db.String(100))
  lastname = db.Column(db.String(100))
  email = db.Column(db.String(120), unique=True)
  pwdhash = db.Column(db.String(54))
  
  def __init__(self, firstname, lastname, email, password):
    self.firstname = firstname.title()
    self.lastname = lastname.title()
    self.email = email.lower()
    self.set_password(password)
    
  def set_password(self, password):
    self.pwdhash = generate_password_hash(password)
  
  def check_password(self, password):
    return check_password_hash(self.pwdhash, password)

We use the set_password() function to set a salted hash of the password, instead of using the plain text password itself.

Lines one and four already existed in models.py, so we’ll start on line two by importing the generate_password_hash and check_password_hash security functions from Werkzeug. Next, we create a new class named User, inheriting from the database object db‘s Model class (line six.)

Inside of our User class, we create attributes for the table’s name, primary key, and the user’s first name, last name, email, and password (lines 10-14). We then write a constructor which sets the class attributes (lines 17-20). We save names in title case and email addresses in lowercase to ensure a match regardless of how a user types in his credentials on subsequent sign ins.

We use the set_password function (lines 22-23) to set a salted hash of the password, instead of using the plain text password itself. Lastly, we have a function named check_password that uses check_password_hash, to check a user’s credentials on any subsequent sign ins (lines 25-26).

Checkpoint: 15_user_model

Sweet! We’ve created a users table and a user model, thereby laying down the foundation of our authentication system. Now let’s build the first user-facing component of the authentication system: the signup page.


Building a Signup Page

Planning

Take a look at Fig. 1 below, to see how everything will fit together.

The Sign up process.

Fig. 1

Implement SSL site-wide so that passwords and session tokens cannot be intercepted.

Let’s go over the figure from above:

  1. A user visits the URL /signup to create a new account. The page is retrieved through an HTTP GET request and loads in the browser.
  2. The user fills in the form fields with his first name, last name, email, and password.
  3. The user clicks the “Create account” button, and the form submits to the server with an HTTP POST request.
  4. On the server, a function validates the form data.
  5. If one or more fields do not pass validation, the signup page reloads with a helpful error message, prompting the the user to try again.
  6. If all fields are valid, a new User object will be created and saved into the database. The user will then be signed in and redirected to a profile page.

This sequence of steps should look familiar, as it’s identical to the sequence of steps we took to create a contact form. Here, instead of sending an email at the end, we save a user’s credentials to the database. The previous article already explained creating a form in detail, I’ll move more quickly in this section so that we can get to the more exciting parts, faster.

Creating a Signup Form

We installed Flask-WTF in the previous article, so let’s proceed with creating a new form inside forms.py.

app/intro_to_flask/forms.py

from flask.ext.wtf import Form, TextField, TextAreaField, SubmitField, validators, ValidationError, PasswordField
from models import db, User
.
.
.
class SignupForm(Form):
  firstname = TextField("First name",  [validators.Required("Please enter your first name.")])
  lastname = TextField("Last name",  [validators.Required("Please enter your last name.")])
  email = TextField("Email",  [validators.Required("Please enter your email address."), validators.Email("Please enter your email address.")])
  password = PasswordField('Password', [validators.Required("Please enter a password.")])
  submit = SubmitField("Create account")

  def __init__(self, *args, **kwargs):
    Form.__init__(self, *args, **kwargs)

  def validate(self):
    if not Form.validate(self):
      return False
    
    user = User.query.filter_by(email = self.email.data.lower()).first()
    if user:
      self.email.errors.append("That email is already taken")
      return False
    else:
      return True

We start by importing one more Flask-WTF class named PasswordField (line one), which is like TextField except that it generates a password textbox. We’ll need the db database object and the User model to handle some custom validation logic inside the SignupForm class; so we import them too (line two).

Then we create a new class named SignupForm containing a field for each piece of user information we wish to collect (lines 7-11). There’s a presence validator on each field to ensure it’s filled in, and a format validator which requires that email addresses match the pattern: user@example.com.

Next, we write a simple constructor for the class that just calls the base class’ constructor (lines 13-14).

So we’ve added some presence and format validators to our form fields, but we need an additional validator that ensures an account does not already exist with the user’s email address. To do this we hook into Flask-WTF’s validation process (lines 16-25).

Now inside of the validate() function, we first ensure the presence and format validators run by calling the base class’ validate() method; if the form is not filled in properly, validate() returns False (lines 16-17).

Next we define the custom validator. We start by querying the database with the email that the user submitted (line 18). If you remember from our models.py file, the email address is converted to lowercase to ensure a match regardless of how it was typed in. This Flask-SQLAlchemy expression corresponds to the following SQL statement:

SELECT * FROM users 
  WHERE email = self.email.data.lower() 
  LIMIT 1

If a user record already exists with the submitted email, validation fails giving the following error message: “That email is already taken” (lines 21-22).

Using the Signup Form

Let’s now create a new URL mapping and a new web template for the signup form. Open routes.py and import the newly created signup form so that we can use it.

app/intro_to_flask/routes.py

from intro_to_flask import app
from flask import render_template, request, flash
from forms import ContactForm, SignupForm

Next, create a new URL mapping.

app/intro_to_flask/routes.py

@app.route('/signup', methods=['GET', 'POST'])
def signup():
  form = SignupForm()
  
  if request.method == 'POST':
    if form.validate() == False:
      return render_template('signup.html', form=form)
    else:   
      return "[1] Create a new user [2] sign in the user [3] redirect to the user's profile"
  
  elif request.method == 'GET':
    return render_template('signup.html', form=form)

Inside the signup() function, we create a variable named form that contains a usable instance of the SignupForm class. If a GET request has been issued, we’ll return the signup.html web template containing the signup form for the user to fill out.

Otherwise, we’ll see just a temporary placeholder string. For now, the temp string lists the three actions that should take place when the form has been successfully submitted. We’ll come back and replace this string with real code in “The First Signup” section below.

Now that we’ve created a URL mapping, the next step is to create the web template signup.html and place it inside the templates/ folder.

app/intro_to_flask/templates/signup.html

{% extends "layout.html" %}

{% block content %}
  <h2>Sign up</h2>

  {% for message in form.firstname.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  {% for message in form.lastname.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  {% for message in form.email.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  {% for message in form.password.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  <form action="{{ url_for('signup') }}" method=post>
    {{ form.hidden_tag() }}
    
    {{ form.firstname.label }}
    {{ form.firstname }}
    
    {{ form.lastname.label }}
    {{ form.lastname }}
    
    {{ form.email.label }}
    {{ form.email }}
    
    {{ form.password.label }}
    {{ form.password }}
    
    {{ form.submit }}
  </form>
    
{% endblock %}

This template looks just like contact.html. We first loop through and display any error messages if necessary. We then let Jinja2 generate most of the HTML form for us. Remember how in the Signup form class we appended the error message “That email is already taken” to self.email.errors? That’s the same object that Jinja2 loops through in this template.

The one difference from the contact.html template is the omission of the if...else logic.

In this template, we want to register and sign in the user on a successful form submission. This takes place on the back-end, so the if...else statement is not needed here.

Finally, add in these CSS rules to your main.css file so that the signup form looks nice and pretty.

app/intro_to_flask/static/css/main.css

/* Signup form */
form input#firstname,
form input#lastname,
form input#password {
  width: 400px;
  background-color: #fafafa;
  -webkit-border-radius: 3px;
     -moz-border-radius: 3px;
          border-radius: 3px;
  border: 1px solid #cccccc;
  padding: 5px;
  font-size: 1.1em;
}

form input#password {
  margin-bottom: 10px;
}

Let’s check out the newly created signup page by typing:

$ cd app/
$ python runserver.py

And browse to http://localhost:5000/signup in your favorite web browser.

The sign up page.

Excellent! We just created a signup form from scratch, handled complex validation, and created a good looking signup page with helpful error messages.

Checkpoint: 16_signup_form

If any of these steps were unclear, please take a moment to review the previous article. It covers each step in greater detail, and I followed the same steps from that article, to create this signup form.


The First Signup

Let’s start by replacing the temporary placeholder string in routes.py‘s signup() function with some real code. Upon a successful form submission, we need to create a new User object, save it to the database, sign the user in, and redirect to the user’s profile page. Let’s take this step by step, starting with creating a new User object and saving it to the database.

Saving a New User Object

Add in lines five and 17-19 to

routes.py.

app/intro_to_flask/routes.py

from intro_to_flask import app
from flask import render_template, request, flash, session, url_for, redirect
from forms import ContactForm, SignupForm
from flask.ext.mail import Message, Mail
from models import db, User
.
.
.
@app.route('/signup', methods=['GET', 'POST'])
def signup():
  form = SignupForm()
  
  if request.method == 'POST':
    if form.validate() == False:
      return render_template('signup.html', form=form)
    else:
      newuser = User(form.firstname.data, form.lastname.data, form.email.data, form.password.data)
      db.session.add(newuser)
      db.session.commit()
      
      return "[1] Create a new user [2] sign in the user [3] redirect to the user's profile"
  
  elif request.method == 'GET':
    return render_template('signup.html', form=form)

First, we import the User class from models.py so that we can use it in the signup() function (line five). Then we create a new User object called newuser and populate it with the signup form's field data (line 17).

Next, we add newuser to the database object's session (line 18), which is Flask-SQLAlchemy's version of a regular database transaction. The add() function generates an INSERT statement using the User object's attributes. The equivalent SQL for this Flask-SQLAlchemy expression is:

  INSERT INTO users (firstname, lastname, email, pwdhash)
  VALUES (form.firstname.data, form.lastname.data, form.email.data, form.password.data)  

Lastly, we update the database with the new user record by committing the transaction (line 19).

Signing in the User

Next, we need to sign in the user. The Flask app needs to know that subsequent page requests are coming from the browser of the user who has successfully signed up. We can accomplish this by setting a cookie in the user's browser containing some sort of ID and associating that key with the user's credentials in the Flask app.

This way, the ID in the browser's cookie will be passed to the app on each subsequent page request, and the app will look up the ID to determine whether it maps to valid user credentials.

If it does, the app allows access to the parts of the website that you need to be signed in for. This combination of having a key stored on the client and a value stored on the server is called a session.

Flask has a session object that accomplishes this functionality. It stores the session key in a secure cookie on the client and the session value in the app. Let's use it in our signup() function.

app/intro_to_flask/routes.py

from flask import render_template, request, flash, session
.
.
.
@app.route('/signup', methods=['GET', 'POST'])
def signup():
  form = SignupForm()
  
  if request.method == 'POST':
    if form.validate() == False:
      return render_template('signup.html', form=form)
    else:
      newuser = User(form.firstname.data, form.lastname.data, form.email.data, form.password.data)
      db.session.add(newuser)
      db.session.commit()
      
      session['email'] = newuser.email
      
      return "[1] Create a new user [2] sign in the user [3] redirect to the user's profile"
  
  elif request.method == 'GET':
    return render_template('signup.html', form=form)

We start by importing Flask's session object on line one. Next, we associate the key 'email' with the value of the newly registered user's email (line 17). The session object will take care of hashing 'email' into an excrypted ID and storing it in a cookie on the user's browser. At this point, the user is signed in to our app.

Redirecting to a Profile page

The last step is to redirect the user to a Profile page after signing in. We'll use the url_for function (which we've seen in layout.html and contact.html) in conjunction with Flask's redirect() function.

app/intro_to_flask/routes.py

from intro_to_flask import app
from flask import render_template, request, flash, session, url_for, redirect
.
.
.
@app.route('/signup', methods=['GET', 'POST'])
def signup():
  form = SignupForm()
  
  if request.method == 'POST':
    if form.validate() == False:
      return render_template('signup.html', form=form)
    else:
      newuser = User(form.firstname.data, form.lastname.data, form.email.data, form.password.data)
      db.session.add(newuser)
      db.session.commit()
      
      session['email'] = newuser.email
      return redirect(url_for('profile'))
  
  elif request.method == 'GET':
    return render_template('signup.html', form=form)

on line two, we import Flask's url_for() and redirect() functions. Then on line 19, we replace our temporary placeholder string with a redirect to the URL /profile. We don't have a URL mapping for /profile yet, so let's create that next.

app/intro_to_flask/routes.py

@app.route('/profile')
def profile():

  if 'email' not in session:
    return redirect(url_for('signin'))

  user = User.query.filter_by(email = session['email']).first()

  if user is None:
    return redirect(url_for('signin'))
  else:
    return render_template('profile.html')

Here we can finally see sessions in action. We start on line four by fetching the browser's cookie and checking if it contains a key named 'email'. If it doesn't exist, that means the user is not authenticated, so we redirect the user to a signin page (we'll create this in the next section).

If the 'email' key does exist, we look up the server-side user email value associated with the key using session['email'], and then query the database for a registered user with this same email address (line seven). The equivalent SQL for this Flask-SQLAlchemy expression is:

SELECT * FROM users WHERE email = session['email'];

If no registered user exists, we'll redirect to the signup page. Otherwise, we render the profile.html template. Let's create profile.html now.

app/intro_to_flask/templates/profile.html

{% extends "layout.html" %}
{% block content %}
  <div class="jumbo">
    <h2>Profile<h2>
    <h3>This is {{ session['email'] }}'s profile page<h3>
  </div>
{% endblock %}

I've kept this profile template simple. If we focus in on line five — you'll see that we can use Flask's session object inside Jinja2 templates. Here, I've used it to create a user-specific string, but you could use this ability to pull other types of user-specific information instead.

We're finally ready to see the result of all our hard work. Type the following into your terminal:

$ python runserver.py

Go to http://localhost:5000/ in your favorite web browser, and complete the sign up process. You should be greeted with a profile page that looks like the following screenshot:

A successful sign up!

Signing up users is a huge milestone for our app. We can adapt the code in our /signup() function and round out our authentication system by allowing users to sign in and out of the app.

Checkpoint: 17_profile_page


Building a Signin Page

Creating a signin page is similar to creating a signup page — we'll need to create a signin form, a URL mapping, and a web template. Let's start by creating the SigninForm class in forms.py.

app/intro_to_flask/forms.py

class SigninForm(Form):
  email = TextField("Email",  [validators.Required("Please enter your email address."), validators.Email("Please enter your email address.")])
  password = PasswordField('Password', [validators.Required("Please enter a password.")])
  submit = SubmitField("Sign In")
  
  def __init__(self, *args, **kwargs):
    Form.__init__(self, *args, **kwargs)

  def validate(self):
    if not Form.validate(self):
      return False
    
    user = User.query.filter_by(email = self.email.data.lower()).first()
    if user and user.check_password(self.password.data):
      return True
    else:
      self.email.errors.append("Invalid e-mail or password")
      return False

The SigninForm class is similar to the SignupForm class. To sign a user in, we need to capture their email and password, so we create those two fields with presence and format validators (lines 2-3). Then we define our custom validator inside the validate() function (lines 10-15). This time the validator needs to make sure the user exists in the database and has the correct password. If a record does exist with the supplied information, we check to see if the password matches (line 14). If it does, the validation check passes (line 15), otherwise they get an error message.

Next, let's create a URL mapping in routes.py.

app/intro_to_flask/routes.py

...
from forms import ContactForm, SignupForm, SigninForm
.
.
.
@app.route('/signin', methods=['GET', 'POST'])
def signin():
  form = SigninForm()
  
  if request.method == 'POST':
    if form.validate() == False:
      return render_template('signin.html', form=form)
    else:
      session['email'] = form.email.data
      return redirect(url_for('profile'))
                
  elif request.method == 'GET':
    return render_template('signin.html', form=form) 

Once again, the signin() function is similar to the signup() function. We import SigninForm (line two), so that we can use it in the signin() function. Then in signin(), we return the signin.html template for GET requests (lines 17-18).

If the form has been POSTed and any validation check fails, the signin form reloads with a helpful error message (lines 11-12). Otherwise, we sign in the user by creating a new session and redirecting to their profile page (lines 14-15).

Lastly, let's create the web template signin.html.

app/intro_to_flask/templates/signin.html

{% extends "layout.html" %}

{% block content %}
  <h2>Sign In</h2>

  {% for message in form.email.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  {% for message in form.password.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  <form action="{{ url_for('signin') }}" method=post>
    {{ form.hidden_tag() }}
    
    {{ form.email.label }}
    {{ form.email }}
    
    {{ form.password.label }}
    {{ form.password }}
    
    {{ form.submit }}
  </form>
    
{% endblock %}

Similar to signup.html template, we first loop through and display any error messages, then we let Jinja2 generate the form for us.

And that does it for the signin page. Visit http://localhost:5000/signin to check it out. Go ahead and sign in, you should get redirected to your profile page.

A successful sign in.

Checkpoint: 18_signin_form


Signing Out

In "The First Signup" section above, we saw that "signing in" meant setting a cookie in the user's browser containing an ID and associating that ID with the user's data in the Flask app. Therefore, "signing out" means clearing the cookie in the browser and dissociating the user data.

This can be accomplished in one line: session.pop('email', None).

We don't need a form or even a web template to sign out. All we need is a URL mapping in routes.py, which terminates the session and redirects to the Home page. The mapping, therefore, is short and sweet:

app/intro_to_flask/routes.py

@app.route('/signout')
def signout():

  if 'email' not in session:
    return redirect(url_for('signin'))
    
  session.pop('email', None)
  return redirect(url_for('home'))

The user is not authenticated if the browser's cookie does not contain a key named 'email', in that case, we just redirect to the signin page (lines 4-5). Otherwise, we terminate the session (line seven) and redirect back to the home page (line eight).

You can test the sign out functionality by visiting http://localhost:5000/signout. If you're signed in, the app will sign you out and redirect to http://localhost:5000/. Once you've signed out, try visiting the profile page http://localhost:5000/profile. You shouldn't be allowed to see the profile page if you're not signed in, and the app should redirect you back to the Signin page.

Checkpoint: 19_signout


Tidying Up

Now we need to update the site header with navigation links for "Sign Up", "Sign In", "Profile", and "Sign Out". The links should change based on whether the user is signed in or not. If the user is signed out, links for "Sign Up" and "Sign In" should be visible. When the user is signed in, we want links for "Profile" and "Sign Out" to appear, while hiding the "Sign Up" and "Sign In" links.

So how can we do this? Think back to the profile.html template where we used Flask's session object. We can use the session object to show navigation links based on the user's authentication status. Let's open layout.html and make the following changes:

app/intro_to_flask/templates/layout.html

<!DOCTYPE html>
<html>
  <head>
    <title>Flask App</title>
    <link rel="stylesheet" href="{{ url_for('static', filename='css/main.css') }}">
  </head>
  <body>

  <header>
    <div class="container">
      <h1 class="logo">Flask App</h1>
      <nav>
        <ul class="menu">
          <li><a href="{{ url_for('home') }}">Home</a></li>
          <li><a href="{{ url_for('about') }}">About</a></li>
          <li><a href="{{ url_for('contact') }}">Contact</a></li>
          {% if 'email' in session %}
          <li><a href="{{ url_for('profile') }}">Profile</a></li>
          <li><a href="{{ url_for('signout') }}">Sign Out</a></li>
          {% else %}
          <li><a href="{{ url_for('signup') }}">Sign Up</a></li>
          <li><a href="{{ url_for('signin') }}">Sign In</a></li>
          {% endif %}
        </ul>
      </nav>
    </div>
  </header>

    <div class="container">
      {% block content %}
      {% endblock %}
    </div>

  </body>
</html>

Starting on line 17, we use Jinja2's if...else syntax and the session() object to check if the browser's cookie contains the 'email' key. If it does, then the user is signed in and therefore should see the "Profile" and "Sign Out" navigation links. Otherwise, the user is signed out and should see links to "Sign Up" and "Sign In".

Now give it a try! Check out how the navigation links appear and disappear by signing in and out of the app.

The last task remaining is a similar issue: when a user is signed in, we don't want him to be able to visit the signup and signin pages. It makes no sense for a signed in user to authenticate themselves again. If signed in users try to visit these pages, they should instead be redirected to their profile page. Open routes.py and add the following piece of code to the beginning of the signup() and signin() functions:

if 'email' in session:
  return redirect(url_for('profile'))

Here's what your routes.py file will look like after adding in that snippet of code:

app/intro_to_flask/routes.py

@app.route('/signup', methods=['GET', 'POST'])
def signup():
  form = SignupForm()

  if 'email' in session:
    return redirect(url_for('profile')) 
.
.
.
@app.route('/signin', methods=['GET', 'POST'])
def signin():
  form = SigninForm()

  if 'email' in session:
    return redirect(url_for('profile'))  
  ...

And with that we're finished! Try visiting the the "signup" or "signin" pages while you are currently signed in, to test it out.

Checkpoint: 20_visibility_control


Conclusion

We've accomplished a lot in this article. We've taken our Flask app from being a simple Python module and turned it into a well organized application, capable of handling user authentication.

There are several directions in which you can take this app from here. Here are some ideas:

  • Let users sign in with an existing account, such as their Google account, by adding support for OpenID.
  • Give users the ability to update their account information, as well as delete their account.
  • Let users reset their password if they forget it.
  • Implement an authorization system.
  • Deploy to a production server. Note that when you deploy this app to production, you will need to implement SSL site-wide so that passwords and session tokens cannot be intercepted. If you deploy to Heroku, you can use their SSL certificate.

So go forth and continue to explore Flask, and build your next killer app! Thanks for reading.

Tags: Python flask

May 03 2013

16:00

Python Power Tools: virtualenvwrapper

In this series of videos, I’ll introduce you to several tools that you can add to your arsenal to become a better, more productive, programmer. In this episode, we’ll review a powerful companion app, virtualenvwrapper, that wraps virtualenv in a user friendly set of shell functions.

Download Video

Hoping to dig into Django? Let me teach you in my Tuts+ Premium course: Django Unchained!

Tags: Python Videos

May 02 2013

16:42

Python Power Tools: virtualenv

In this series of videos, I’ll introduce you to several tools that you can add to your arsenal to become a better, more productive, programmer. We’ll begin with virtualenv, which allows you to create isolated development environments for installing and upgrading software, without needing to worry about overrides or dependency issues.

Download Video

Hoping to dig into Django? Let me teach you in my Tuts+ Premium course: Django Unchained!

Tags: Python Videos

March 22 2013

20:12

Intro to Flask: Adding a Contact Page

In the previous article in this mini-series, we leveraged Flask to build a simple website that contains “Home” and “About” pages using a generalized workflow that we can apply to other Flask-based web apps. In this lesson, I’ll demonstrate how to add a “Contact” page that allow users to send you messages.

The code used in this article can be found on GitHub. Captions, such as Checkpoint: 05_contact_form, mean that you can switch to the branch named “05_contact_form” and review the code at that point in the article.


Flask Extensions

You can find a full list of extensions in the Flask Extension Registry.

Flask doesn’t come with many features off the shelf, making it easy to pick up and learn. There is no object-relational mapper for database interaction or admin interfaces to add and update content. It only offers a small set of functions, two of which we’ve already used — url_for() and render_template().

Instead of shipping with extra functionality, Flask’s extension model allows you to add functionality as needed. A Flask extension is a package that adds specific functionality to your app. For example, Flask-SQLAlchemy adds database support to your app, whereas Flask-Login adds login/logout support. You can find a full list of extensions in the Flask Extension Registry.

To create a Contact page, we’ll use Flask-WTF to handle and validate form data and Flask-Mail to email the form data to you.


Flask-WTF

Flask-WTF is an exension that handles and validates form data. What does that mean? Look at the following figure:

Fig. 1

  1. A user issues a GET request for a web page that contains a form.
  2. The user fills in the form.
  3. The user clicks the “Send” button, submitting it to the server via a POST request.
  4. The server validates the information.
  5. If one or more fields do not validate, the web page containing the form loads again with a helpful error message, prompting the user to try again.
  6. If all fields validate, the form information is used in the next step in the pipeline.

A contact page will have fields for the user’s name, email, subject, and message. In Flask, we’ll POST the form to a function inside routes.py. This function is called the form handler. We’ll run a few validation checks, and if any of the input does not pass muster, we’ll refresh the page to display a message that describes the error. Once all validation checks pass, we’ll use the form data for the next step: emailing the message to you, the website owner.

Flask extensions are simple, powerful tools that extend the functionality of your Flask-based app.

That’s how form handling and validation works. Now where do we actually define the form? We could write HTML using the <form> element and set its action attribute to a Python script. The Python script would mirror the form in order to capture each form field and validate the form field data. If we use this strategy, however, we’d essentially define the form twice — once for the front-end and once for the back-end.

It would be great to define the form only once: in the Python script. This is exactly what Flask-WTF allows us to do. We’ll define the form just once in a Python script, and then we’ll let Flask-WTF generate the form’s HTML for us. The point of all of this is to separate presentation from content.

Enough chatter. Let’s code.

Creating a Form

As a first step, let’s get back into the isolated development environment we created last time.

$ cd flaskapp
$ . bin/activate

Now that we’ve entered and activated our development environment, we can safely install Flask-WTF:

$ pip install flask-wtf

Let’s now define the form in a Python script. We already have routes.py, which maps URLs to functions. Let’s not clutter it with unrelated code. Instead, create a new file called forms.py, and place it inside the app/ folder.

app/forms.py

from flask.ext.wtf import Form, TextField, TextAreaField, SubmitField

class ContactForm(Form):
  name = TextField("Name")
  email = TextField("Email")
  subject = TextField("Subject")
  message = TextAreaField("Message")
  submit = SubmitField("Send")

We just created a form. What did we do? First, we imported a few useful classes from Flask-WTF — the base Form class, a text field, a textarea field for multi-line text input, and a submit button. Next, we created a new class named ContactForm, inheriting from the base Form class. Then we created each field that we want to see in the contact form. Instead of writing <input type="text">Name</input> in an HTML file, you write name = TextField("Name").

Using the Form

Now let’s use our form. We want it to appear when a user visits the contact page. In Flask terms, we want the form to show up in a web template and map a URL to that web template so we can visit it in the browser. This means we need to create a new web template and a new URL mapping. Let’s start by creating a new URL mapping.

This is an action-packed section, and it may be a little confusing. But stick with me and we’ll get through it.

As a first step, open routes.py and import our newly created form by adding from forms import ContactForm at the beginning of the script.

app/routes.py

from flask import Flask, render_template
from forms import ContactForm

You can prevent a CSRF attack by making sure that the form submission originates from your web app.

Next, configure Flask-WTF to handle a security exploit known as cross-site request forgery (CSRF). In a perfect world, your server would only process forms that belong to your web app. In other words, your server would only handle and validate the forms that you created. However, it is possible for an attacker to create a form on his own website, fill it in with malicious information, and submit it to your server. If your server accepts this malicious information, all sorts of bad things can happen next.

You can prevent a CSRF attack by making sure that the form submission originates from your web app. One way to do this is to keep a unique token hidden inside your HTML <form> tag that cannot be guessed by attackers. When the form POSTs to your server, the token is checked first. If the token does not match, your server rejects the form submission and does not touch the form data. If the token matches, the server proceeds with form handling and validation.

Flask-WTF does all of this with an easy one-liner. Just configure Flask-WTF with a secret key, and Flask-WTF takes care of generating and managing unique tokens for your forms.

app/routes.py

from flask import Flask, render_template, request, flash
from forms import ContactForm

app = Flask(__name__) 

app.secret_key = 'development key'

Here in line six, I set the secret key to ‘development key’. Feel free to make yours more complex, longer, and alphanumeric.

Now that we’ve imported and configured our contact form, we can use it in a URL mapping in routes.py. Let’s go ahead and create that URL mapping.

app/routes.py

@app.route('/contact')
def contact():
  form = ContactForm()
  return render_template('contact.html', form=form)

Now when someone visits the URL /contact, the function contact() will execute. Inside contact(), we first create a new instance of our contact form in line three and sent it to a web template named contact.html in line four. We will create this web template shortly.

We still have some work to do here though. Figure 1 showed that if a GET request is sent to the server, the web page containing the form should be retrieved and loaded in browser. If the server receives a POST request, a function should capture the form field data and check if it’s valid. In Python terms, this logic can be expressed in an if...else statement. There is a Flask class for distinguishing between GET and POST requests, so let’s start by importing that class at the beginning of routes.py and add the if...else logic to the contact() function.

app/routes.py

from flask import Flask, render_template, request
.
.
.
@app.route('/contact', methods=['GET', 'POST'])
def contact():
  form = ContactForm()

  if request.method == 'POST':
    return 'Form posted.'

  elif request.method == 'GET':
    return render_template('contact.html', form=form)

We already imported the Flask class and render_template() in the previous article, so here we import one more Flask class named request. request determines whether the current HTTP method is a GET or a POST. Next is the if...else logic to the contact() function (lines 9-13).

In the case of a POST request, a string indicating that the form has been posted will be returned.

This string is a temporary placeholder, and we’ll replace it with real code in the final step of this article. Otherwise, if the request uses GET, we return the web template contact.html that contains the form.

The next step is to create the web template contact.html and put it inside the templates/ folder.

app/templates/contact.html

{% extends "layout.html" %}

{% block content %}
  <h2>Contact</h2>
  <form action="{{ url_for('contact') }}" method=post>
    {{ form.hidden_tag() }}

    {{ form.name.label }}
    {{ form.name }}

    {{ form.email.label }}
    {{ form.email }}

    {{ form.subject.label }}
    {{ form.subject }}

    {{ form.message.label }}
    {{ form.message }}

    {{ form.submit }}
  </form>
{% endblock %} 

As with home.html and about.html, the contact.html template extends layout.html and fills the ‘content’ block with its own text. We first specify where to send the form data on submission by setting the <form> element’s action attribute to the contact() function we created in routes.py (line five). Next, we let the Jinja2 template engine generate the bulk of the form for us (lines 6-20). We start by inserting a hidden tag in line six to protect against CSRF exploits. Lastly, we add each label and field of the form.

We are now ready to see the result of all our work. Just type the following:

$ python routes.py

Then go to http://localhost:5000/contact in your favorite web browser.

The contact page containing the form has loaded. Fill in the form fields and click the "Send" button. You’ll see a page that looks like this:

Awesome! Form submission is working.

Let’s quickly review everything we did in this section:

  • We type in the URL http://localhost:5000/contact into the browser’s address bar.
  • The GET request hits routes.py, where the URL /contact is mapped to the function contact().
  • The function contact() executes, where a variable named form containing a usable instance of the ContactForm class is sent to the web template contact.html.
  • contact.html generates the contact form’s HTML.
  • Rendered HTML is sent back to routes.py.
  • routes.py sends the HTML back to the browser and we see the contact page containing the form.
  • We fill in the contact form and submit it by clicking the “Send” button.
  • The POST request hits routes.py, where the URL /contact is mapped to the function contact().
  • The function contact() executes once more, this time following the if...else control flow for the HTTP POST request.
  • The string 'Form posted.' is sent back to the browser, giving us the screen above.

Checkpoint: 05_contact_form

This is cool, but the contact form looks ugly. Let’s make it look better by adding some CSS. Open up main.css and add these rules:

static/css/main.css

/* Contact form */
form label {
  font-size: 1.2em;
  font-weight: bold;
  display: block;
  padding: 10px 0;
}

form input#name,
form input#email,
form input#subject {
  width: 400px;
  background-color: #fafafa;
  -webkit-border-radius: 3px;
     -moz-border-radius: 3px;
          border-radius: 3px;
  border: 1px solid #cccccc;
  padding: 5px;
  font-size: 1.1em;
}

form textarea#message {
  width: 500px;
  height: 100px;
  background-color: #fafafa;
  -webkit-border-radius: 3px;
     -moz-border-radius: 3px;
          border-radius: 3px;
  border: 1px solid #cccccc;
  margin-bottom: 10px;
  padding: 5px;
  font-size: 1.1em;
}

form input#submit {
  display: block;
  -webkit-border-radius: 3px; 
     -moz-border-radius: 3px;
          border-radius: 3px;
  border:1px solid #d8d8d8;
  padding: 10px; 
  font-weight:bold; 
  text-align: center; 
  color: #000000; 
  background-color: #f4f4f4;
  background-image: -webkit-gradient(linear, left top, left bottom, color-stop(0%, #f4f4f4), color-stop(100%, #e5e5e5));
  background-image: -webkit-linear-gradient(top, #f4f4f4, #e5e5e5);
  background-image: -moz-linear-gradient(top, #f4f4f4, #e5e5e5);
  background-image: -ms-linear-gradient(top, #f4f4f4, #e5e5e5);
  background-image: -o-linear-gradient(top, #f4f4f4, #e5e5e5);
  background-image: linear-gradient(top, #f4f4f4, #e5e5e5);filter:progid:DXImageTransform.Microsoft.gradient(GradientType=0,startColorstr=#f4f4f4, endColorstr=#e5e5e5);
}

form input#submit:hover{
  cursor: pointer;
  border:1px solid #c1c1c1; 
  background-color: #dbdbdb;
  background-image: -webkit-gradient(linear, left top, left bottom, color-stop(0%,#dbdbdb), color-stop(100%, #cccccc));
  background-image: -webkit-linear-gradient(top, #dbdbdb, #cccccc);
  background-image: -moz-linear-gradient(top, #dbdbdb, #cccccc);
  background-image: -ms-linear-gradient(top, #dbdbdb, #cccccc);
  background-image: -o-linear-gradient(top, #dbdbdb, #cccccc);
  background-image: linear-gradient(top, #dbdbdb, #cccccc);filter:progid:DXImageTransform.Microsoft.gradient(GradientType=0,startColorstr=#dbdbdb, endColorstr=#cccccc);
}

Switch back to the browser and refresh http://localhost:5000/contact to see the result of the CSS.

This looks much better. Let’s move on to form validation.

Checkpoint: 06_contact_styling

Validating Form Data

A user can now visit the URL /contact and fill in the form. But what happens if the user does not properly fill out the form? We need to validate the user input so that it won’t cause problems in later steps.

Form validation is performed by using form validators. Fortunately, Flask-WTF comes with many useful, built-in validators that we can use right away. We’ll put these validators in the ContactForm class definition in forms.py.

The most basic validator is presence, which simply ensures that all form fields are filled in, so let’s start here.

app/forms.py

from flask.ext.wtf import Form, TextField, TextAreaField, SubmitField, validators, ValidationError

class ContactForm(Form):
  name = TextField("Name",  [validators.Required()])
  email = TextField("Email",  [validators.Required()])
  subject = TextField("Subject",  [validators.Required()])
  message = TextAreaField("Message",  [validators.Required()])
  submit = SubmitField("Send")

We start by importing validators and ValidationError from Flask-WTF. This gives us access to Flask-WTF’s built-in validators. Next we add [validators.Required()] to each form field in order to validate its presence. Notice that this validator is inside a Python list, meaning that we can easily add more validators to this list.

Next, let’s require email addresses to match the pattern user@example.com by adding the Email validator to the email field.

app/forms.py

from flask.ext.wtf import Form, TextField, TextAreaField, SubmitField, validators, ValidationError

class ContactForm(Form):
  name = TextField("Name",  [validators.Required()])
  email = TextField("Email",  [validators.Required(), validators.Email()])
  subject = TextField("Subject",  [validators.Required()])
  message = TextAreaField("Message",  [validators.Required()])
  submit = SubmitField("Send")

That does it for our form validations.

Checkpoint: 07_form_validations

Flashing Error Messages

Looking back at Figure 1, if any validation check fails, the contact page should reload with an error message so that the user can fix the mistake and try again. This error message must only appear when validation fails and disappear when the mistake has been fixed.

Our next step is to send this sort of temporary error message to the user when validation fails. Flask makes this really easy by using its flash() function. Let’s start by opening routes.py and importing Flask’s flash() function at the beginning of the script.

app/routes.py

from flask import Flask, render_template, request, flash

After the contact form POSTs to the server, any validation failure should reload the form with a helpful error message. Otherwise, the input data can be used for future processing. Once again, this logic can be expressed in an if...else statement. Let’s add this if...else logic to the contact() function inside the if request.method == 'POST': block.

app/routes.py

@app.route('/contact', methods=['GET', 'POST'])
def contact():
  form = ContactForm()

  if request.method == 'POST':
    if form.validate() == False:
      flash('All fields are required.')
      return render_template('contact.html', form=form)
    else:
      return 'Form posted.'

  elif request.method == 'GET':
    return render_template('contact.html', form=form)

If any validation check fails, form.validate() will be False. The error message All fields are required will be sent to contact.html. Otherwise, we’ll see the temporary placeholder string Form posted, indicating the form has been successfully submitted.

Next, let’s modify contact.html so that it can receive and display these temporary error messages. See the following block:

{% for message in get_flashed_messages() %}
  <div class="flash">{{ message }}</div>
{% endfor %}

The function get_flashed_messages() pulls all flashed messages and returns them. We then simply display each flashed message by using a Jinja2 for loop. Add this code block to contact.html after <h2>Contact</h2> and before the <form> tag.

app/templates/contact.html

{% extends "layout.html" %}

{% block content %}
  <h2>Contact</h2>

  {% for message in get_flashed_messages() %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  <form action="{{ url_for('contact') }}" method=post>
    {{ form.hidden_tag() }}

    {{ form.name.label }}
    {{ form.name }}

    {{ form.email.label }}
    {{ form.email }}

    {{ form.subject.label }}
    {{ form.subject }}

    {{ form.message.label }}
    {{ form.message }}

    {{ form.submit }}
  </form>
{% endblock %}

Lastly, let’s add a CSS rule in main.css so that flashed error messages look pretty.

main.css

/* Message flashing */
.flash {
  background-color: #FBB0B0;
  padding: 10px;
  width: 400px;
}

Open your browser and visit http://localhost:5000/contact. Leave all the fields blank and click “Send” to test whether form validation and error message flashing work.

This is sweet! We have successfully sent an error message to our contact form if a validation check fails.

Checkpoint: 08_error_message_flashing

But we’re not done; we can actually do a little better. Instead of having one generic error message for all failed validation checks, it would be better to have a specific error message for each failed validation check. For example, if the user forgets to fill in the subject field, a specific error message that says Please enter a subject would be flashed. Likewise, if the user forgets to fill in their name, we’d flash a specific error message that says Please enter your name. We can accomplish this pretty easily, so let’s start by writing our specific error messages inside each validator in forms.py.

app/forms.py

from flask.ext.wtf import Form, TextField, TextAreaField, SubmitField, validators, ValidationError

class ContactForm(Form):
  name = TextField("Name",  [validators.Required("Please enter your name.")])
  email = TextField("Email",  [validators.Required("Please enter your email address."), validators.Email("Please enter your email address.")])
  subject = TextField("Subject",  [validators.Required("Please enter a subject.")])
  message = TextAreaField("Message",  [validators.Required("Please enter a message.")])
  submit = SubmitField("Send")

We simply write specific error messages inside each validator. Next, let’s modify contact.html to receive and display these specific error messages. Earlier, we relied on the function get_flashed_messages() to pull flashed error messages, and looped over them to display them. Let’s replace that block with this one:

{% for message in form.name.errors %}
  <div class="flash">{{ message }}</div>
{% endfor %}

{% for message in form.email.errors %}
  <div class="flash">{{ message }}</div>
{% endfor %}

{% for message in form.subject.errors %}
  <div class="flash">{{ message }}</div>
{% endfor %}

{% for message in form.message.errors %}
  <div class="flash">{{ message }}</div>
{% endfor %}

Here we use the errors attribute for each form field to pull the specific error messages and loop over them using the Jinja2 for loop to display them.

Putting it all together, contact.html now look like this:

app/templates/contact.html

{% extends "layout.html" %}

{% block content %}
  <h2>Contact</h2>

  {% for message in form.name.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}

  {% for message in form.email.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}

  {% for message in form.subject.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}

  {% for message in form.message.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  <form action="{{ url_for('contact') }}" method=post>
    {{ form.hidden_tag() }}

    {{ form.name.label }}
    {{ form.name }}

    {{ form.email.label }}
    {{ form.email }}

    {{ form.subject.label }}
    {{ form.subject }}

    {{ form.message.label }}
    {{ form.message }}

    {{ form.submit }}
  </form>
{% endblock %} 

Switch back to the browser, go to http://localhost:5000/contact, and click “Send”. Be sure to leave all form fields blank.

Perfect! The user now has helpful error messages if he makes a mistake.

Checkpoint: 09_specific_message_flashing

We accomplished a lot in this section. We created a contact form from scratch, learned how to protect against CSRF attacks, distinguished between GET and POST requests, enforced form validations, and flashed specific error messages if necessary. We now need to email the message.


Flask-Mail

Flask-Mail is a Flask exension that enables you to send emails from your Flask app. The steps below are similar to those we took to use Flask-WTF.

Let’s start by installing Flask-Mail.

$ pip install flask-mail

Configuring Flask-Mail

Next, lets import Flask-Mail into routes.py and configure it so that we can start using it.

app/routes.py

from flask import Flask, render_template, request, flash
from forms import ContactForm
from flask.ext.mail import Message, Mail

mail = Mail()

app = Flask(__name__)

app.secret_key = 'development key'

app.config["MAIL_SERVER"] = "smtp.gmail.com"
app.config["MAIL_PORT"] = 465
app.config["MAIL_USE_SSL"] = True
app.config["MAIL_USERNAME"] = 'contact@example.com'
app.config["MAIL_PASSWORD"] = 'your-password'

mail.init_app(app)

First, we import the Message and Mail classes from Flask-Mail (line three). We’ll use the Message class to compose a new email and the Mail class to send the email. Next, we create the mail variable that contain a usable instance of the Mail class (line five).

We then configure Flask-Mail with few SMTP server settings (lines 11-15). I used Gmail’s SMTP server settings here, but you can easily use your favorite email provider. Just search for its SMTP settings and you’ll be set.

For example, if you want to use Yahoo! Mail, just search for “yahoo mail smtp server settings” and update the configuration.

Make sure to enter a real email and password in app.config["MAIL_USERNAME"] and app.config["MAIL_PASSWORD"], respectively. This will be the account from which you’ll send email.

Finally, we attach mail to our Flask app so that we can start using it (line 17).

You’ve probably seen groups use contact email addresses like contact@example.com or support@example.com. If you own your own domain and can create a new contact email address, go ahead and put that email address in app.config["MAIL_USERNAME"]. Otherwise, you can use your personal email address just to see how this works.

Sending an Email

Now that the configuration is complete, let’s compose a new email containing the contact form data and send it. We should only send an email if the form has been submitted and all validation checks pass. This means we need to work inside the if request.method == 'POST': block again. We’ve already added logic inside the if form.validate() == False: block to handle validation failures. If all validation checks pass, form.validate() will be True and the program will enter the else block. Therefore, let’s go ahead and add logic inside the else: block.

app/routes.py

@app.route('/contact', methods=['GET', 'POST'])
def contact():
  form = ContactForm()

  if request.method == 'POST':
    if form.validate() == False:
      flash('All fields are required.')
      return render_template('contact.html', form=form)
    else:
      msg = Message(form.subject.data, sender='contact@example.com', recipients=['your_email@example.com'])
      msg.body = """
      From: %s <%s>
      %s
      """ % (form.name.data, form.email.data, form.message.data)
      mail.send(msg)

      return 'Form posted.'

  elif request.method == 'GET':
    return render_template('contact.html', form=form)

We start by composing a new message (line 10). The Message class takes a subject line, a “from” address, and a “to” address. We then collect the contact form’s subject field data with form.subject.data and set it as the new message’s subject line. The email will be sent from the account you configured in app.config["MAIL_USERNAME"], so that’s what we used here for the from address. The email will be sent to your personal email address so that you can receive and respond to new messages.

Next, we write the email itself (lines 11-14). We include the user’s name, email and message. I use Python’s string formatting operator % to format the email. And finally, we use mail.send(msg) to send the email (line 15).

Let’s see if everything works. Visit http://localhost:5000/contact, fill out each field, and click “Send.” If all goes well, you’ll receive a new email from your Flask app.

Checkpoint: 10_send_email

Tidying Up

Our penultimate step is to remove the temporary placeholder string 'Form posted.' with a message thanking the user for his feedback. This message should only appear if our application sends the email. Once again, this logic can be expressed in an if...else statement.

When the contact form has been successfully submitted, we’ll send a success flag from routes.py to contact.html.

We’ll place the if...else logic inside contact.html. If the success flag is set to True, we’ll display the thank you message. Otherwise, we’ll display the contact form.

Let’s start in routes.py inside the contact() function. Replace the temporary placeholder line return 'Form posted.' with return render_template('contact.html', success=True) in order to send a success flag to contact.html. The contact() function now looks like this:

app/routes.py

@app.route('/contact', methods=['GET', 'POST'])
def contact():
  form = ContactForm()

  if request.method == 'POST':
    if form.validate() == False:
      flash('All fields are required.')
      return render_template('contact.html', form=form)
    else:
      msg = Message(form.subject.data, sender='contact@example.com', recipients=['your_email@example.com'])
      msg.body = """
      From: %s &lt;%s&gt;
      %s
      """ % (form.name.data, form.email.data, form.message.data)
      mail.send(msg)

      return render_template('contact.html', success=True)

  elif request.method == 'GET':
    return render_template('contact.html', form=form)

Next open contact.html and add the if...else logic. We’ll use Jinja2′s if...else syntax to make this happen.

app/templates/contact.html

{% extends "layout.html" %}

{% block content %}
  <h2>Contact</h2>

  {% if success %}
    <p>Thank you for your message. We'll get back to you shortly.</p>

  {% else %}

    {% for message in form.name.errors %}
      <div class="flash">{{ message }}</div>
    {% endfor %}

    {% for message in form.email.errors %}
      <div class="flash">{{ message }}</div>
    {% endfor %}

    {% for message in form.subject.errors %}
      <div class="flash">{{ message }}</div>
    {% endfor %}

    {% for message in form.message.errors %}
      <div class="flash">{{ message }}</div>
    {% endfor %}

    <form action="{{ url_for('contact') }}" method=post>
      {{ form.hidden_tag() }}

      {{ form.name.label }}
      {{ form.name }}

      {{ form.email.label }}
      {{ form.email }}

      {{ form.subject.label }}
      {{ form.subject }}

      {{ form.message.label }}
      {{ form.message }}

      {{ form.submit }}
    </form>

  {% endif %}
{% endblock %} 

Starting in line six, {% if success %} means that if the success flag we sent from routes.py is set to True, then display <p>Thank you for your message. We'll get back to you shortly.</p>. Otherwise, follow the {% else %} branch and display the contact form. Jinja2 syntax asks that we close the if...else statement with {% endif %}, so we include that at the end (line 45).

Checkpoint: 11_success_message

Finally, let’s visit http://localhost:5000/contact one more time. Fill in each field and click “Send”.

Our last step is to add a navigation link to the contact page. In the previous article, we added these links to layout.html inside the <header> element. Let’s also do that for the contact page (line eight).

app/templates/layout.html

<header>
  <div class="container">
    <h1 class="logo">Flask App</h1>
    <nav>
      <ul class="menu">
        <li><a href="{{ url_for('home') }}">Home</a></li>
        <li><a href="{{ url_for('about') }}">About</a></li>
        <li><a href="{{ url_for('contact') }}">Contact</a></li>
      </ul>
    </nav>
  </div>
</header>

Checkpoint: 12_contact_nav_link

Open up the browser and refresh http://localhost:5000/ to see the newly added navigation link.


Conclusion

In article, we added a contact page that contains a form to our Flask app. Forms appear in several places in web applications, most notably during sign up and login. This workflow can be adapted to meet those needs. In creating a contact page, we learned how to use Flask extensions.

Flask extensions are simple, powerful tools that extend the functionality of your Flask-based app.

Check out the Flask Extension Registry to explore many more extensions that you can integrate into your app.

Tags: Python flask

August 24 2012

18:08

Behavior-Driven Development in Python

behavior Driven Development is an excellent process to follow in software development. With testing often a practice that is pushed aside to the last minute (or ignored, entirely), baking the process into your daily workflow can prove to be hugely beneficial to the quality of your code. The structure and design of the tests, coupled with the Gherkin syntax makes tests easy to read – even for team members with non-technical backgrounds.

All code should be tested thoroughly, meaning that defects should ideally never reach production. If they do, then a thorough test suite, focused on the behavior of your application as a whole, ensure that they are easy to both detect and fix. This speed, clarity, focus and quality in your code is why you need to be adopting this process…now.


What is Behavior Driven Development?

Behavior Driven Development (which we will now refer to as “BDD”) follows on from the ideas and principles introduced in Test Driven Development. The key points of writing tests before code really apply to BDD as well. The idea is to not only test your code at the granular level with unit tests, but also test your application end to end, using acceptance tests. We will introduce this style of testing with the use of the Freshen testing framework.

Behavior Driven Development (BDD) is a subset of Test Driven Development (TDD).

The process can be simply defined as:

  • Write a failing acceptance test
  • Write a failing unit test
  • Make the unit test pass
  • Refactor
  • Make the acceptance test pass

Rinse and repeat for every feature, as is necessary.


BDD in Agile Development

BDD really comes into its own, when used with agile development.

Tip: Refer to The Principles of Agile Development for more information on agile development methods.

With new features and requirements coming in every 1, 2 or 4 weeks, depending on your team, you need to be able to test and write code for these demands quickly. Acceptance and unit testing in Python allows you to meet these goals.

Acceptance tests famously make use of an English (or possibly alternative) language format “feature” file, describing what the test is covering and the individual tests themselves. This can engage everyone in your team – not just the developers, but also management and business analysts who otherwise would play no part in the testing process. This can help to breed confidence across the whole team in what they are striving to achieve.


Gherkin Syntax

Acceptance tests usually make use of the Gherkin Syntax, introduced by the Cucumber Framework, written for Ruby. The syntax is quite easy to understand, and, in the Freshen Python package, makes use of the following eight keywords to define your features and tests:

  • Given
  • When
  • Then
  • And
  • Feature:
  • Background:
  • Scenario:
  • Scenario Outline:

Below, you can review these keywords in action, and how they can be used to structure your acceptance tests.


Feature Files

Feature files are written in plain English, and specify the area of the application that the tests cover. They also provide some setup tasks for the tests. This means that you are not only writing your tests, but are actually forcing yourself to write good documentation for all aspects of your application. So, you can clearly define what each piece of code is doing and what it is handling.

An example feature file would take the form of (my examples are written using the ‘freshen’ python package. More details can be found below.):

From this simple example, you can see how straightforward it is to describe your tests and share them across the various people involved in your team.

There are four key areas of note in the feature file:

  • Importing the step definitions – Informs Freshen where to look for your steps (see the Steps section below.) These are the mappings to each line to execute Python code underneath.
  • Feature block – Here is where you write documentation for what this group of tests is going to cover. No code is executed here, but allows the reader to understand exactly what this Feature is testing.
  • Background block – Executed prior to every Scenario within the Feature file. This is similar to the SetUp() method and allows you to perform necessary setup code, such as making sure you are on some page, or have certain conditions in place.
  • Scenario block – Here, you define the test. The first line serves as the documentation again, and then you drop into your Scenario to execute the test. It should be fairly easy to see how you can write any test in this style.

Steps File

Following on from the Feature file, we have the steps file underneath. This is where the ‘magic’ happens. Obviously, the Feature file, itself, will not do anything; it requires the steps to actually map each line to execute Python code underneath. This is achieved through the use of regular expressions.

“Regular Expressions? Too complex to bother with in testing” can often be a response to RegEx’s in these tests. However, in the BDD world, they are used to capture the whole string or use very simple RegEx’s to pick out variables from a line. Therefore you shouldn’t be put off by their use here.

Regular Expressions? Too complex to bother with in testing? Not in Freshen. Simple and Easy!

If we review an example. you’ll see how easy the Steps file follows on from the Feature.

The first thing worth noting is the standard imports at the top of the file. So, we need access to our ‘calculator’ program and, of course, the tools provided by Freshen. Then, we can begin to define the steps for each line in the Feature file. We can see that, as explained earlier, the regular expressions are mostly just picking up the whole string, except where we want access to the variable within the line.

If we use the “Given I have pressed ‘xxx’” line as our example, you can see that the line is first picked up using the @Given decorator. Then, you use the 'r' character at the start to indicate the regular expression. Following that, it’s just the line itself and a very simple RegEx to match anything within the quotes – the button in this case. You should then see that the Python method follows directly after this, with the variable passed into the method with whatever name you wish. Here, I have called it, “button”.

Another item of note here is the use of “scc”. This is the Scenario Context Container, and allows variables to be used across steps within a scenario. If we didn’t, all variables would be local to their method, but, here, we create an instance of Calculator() once, and then access that in each step.


Freshen Python Package

As you can hopefully see, the Freshen package is quite simple to pick up and begin using. The installation is straight forward, following the usual pip install pattern that most Python developers will be familiar with.

Perform the following steps to begin using Freshen:

  • $ pip install freshen
  • $ nosetests --with-freshen /path/to/example.feature – to run your tests. You can either run just one feature file, or, if you pass a directory of feature files, you can run all of them.

Alternative Tools

There are plenty of alternative options within Python to do this form of testing. We have examples, such as Behave, Lettuce and also Cucumber, which, as mentioned, defined this structure. The other tools are essentially clones/ports of Cucumber. Cucumber can be used with Python code, via the use of a Ruby-Python interpreter, but that is beyond the scope of this tutorial.

  • Behave – a near exact port of Cucumber into Python. Has a good level of documentation, and is updated constantly by the developers. They also offer a comparison with other tools here, which is worth a read.
  • Lettuce – another direct port of Cucumber, featuring tutorials and examples on their website, and simple installation tools, such as ‘pip’.

The key point, with all of these tools, is that they are all more or less the same. Once you have mastered one, you’ll quickly pick up on the others, should you choose to switch. A quick review of the documentation should be enough for most developers, who are proficient in Python.


Advantages

You can dive into refactoring confidently.

There are significant advantages to be made using a thorough test suite. One of the major ones revolves around the refactoring of code. With a robust test suite in place, you can dive into refactoring confidently, knowing that you have not broken any previous behavior in your application.


Final Thoughts

Having worked in a team using the process and tools outlined above, I have personally experienced the huge advantages of working in this manner. BDD provides your team with clarity, focus, and the confidence to deliver great code, while keeping any potential bug to a minimum.


Tags: Python bdd TDD

August 10 2012

15:00

The Best Way to Learn Python

Python is more popular than ever, and is being used everywhere from back-end web servers, to front-end game development, and everything in between. Python is a true general purpose language and is quickly becoming a must-have tool in the arsenal of any self-respecting programmer.

But Python isn’t popular just because it’s popular. It is easy to learn, reads like pseudo-code, and is wickedly agile. However, learning any new language can be a daunting task, and finding the right places and people to learn from is half the battle. That’s where this guide can help. This is your blueprint for making Python easy, fun and rewarding to learn.


Assignment 1: Start With the Basics

At the time of this writing, there are two main versions of Python in circulation: Python 2.7 and Python 3.2. Which you choose to learn really doesn’t matter too much, as the differences will be minimal—especially to a beginner. But you should know that, while Python 2 has far, far more 3rd party support, Python 3 is the primary focus of the developers designing the language. The choice is yours, but if your code acts differently to any given tutorial, make sure you use the same Python version used in the tutorial.

Wikibooks’ Non-Programmers Tutorial for Python

Wikibooks is always a great source to learn something new, and Python is no exception. Here you will find a solid and to the point series of guides that will teach you the ropes of Python. It doesn’t get too technical, and you can jump into coding something somewhat useful and rewarding pretty quickly. Because of this, I recommend this site as the best place to start on your Python journey.

The Official Python Tutorial

You won’t find a better source of information than the official python.org documentation. However, if you want to jump right in, as I’m sure many of you will, this might not be the best place to start.

The content tends to be more technical than wikibooks, which will be helpful later on as you progress through the language. But, for a beginner, it may just get in the way of what really is a very simple and beautiful language.

For beginners, the biggest difference between Python 2 and Python 3 will most likely be that Python 2 can use print without parentheses. Python 3 requires parentheses, but that’s all.


Assignment 2: Tutorials and Screencasts

TheNewBoston’s  Python Programming Tutorials

TheNewBoston’s playlists are always great, and you can learn a whole host of languages. ‘Bucky’ is a great instructor because he strikes a really great balance between being funny while also being easy to listen. I highly recommend any of his playlists—especially his Python playlist. He assumes zero prior knowledge of programming and will leave you with a firm grasp of the language.

Nettuts+’s Python from Scratch

A little closer to home is Giles Lavelle’s intro to Python. Like TheNewBoston’s series, Lavelle also assumes zero prior programming experience.

If you want to see some real-world applications of your applications, or would like to aim towards web development with Python, this series might be the one for you.

The screencast takes you from nothing to building a dynamic website using a Python web framework called Django.

ShowMeDo’s Python Screencasts

StackOverflow isn’t just full of ‘newbie’ errors and problems.

ShowMeDo has a huge catalogue of Python related videos. While it may not be the most user friendly experience around, the videos range in level from absolute beginner to the most advanced Python techniques. It is well worth checking out.

Build a Python Bot That Can Play Web Games

This tutorial is pretty specific in terms of what you’re going to learn, and I do not recommend it for a complete beginner. I feel, however, that it’s worth a mention. In this guide, Chris Kiehl shows you how to build a very interesting Python bot that will play a simple game for you. The tutorial really goes to show the power of Python; it can be applied to do everyday repetitive tasks that you may have on your computer.


Assignment 3: Free e-books!

Its hard to beat a good book when your trying to learn something new, and with the great community that has developed around the Python language, there is a plethora of free high quality e-books to choose from. Below is a quick list of some of the best. You can download a free e-book version for each of them, or you can choose to buy the physical book (or donate) if you want to support the author, which I’m sure they would greatly appreciate.

Learn Python the Hard Way

Despite the name, Learn Python The Hard Way makes learning Python incredibly easy—the way it’s meant to be! In this book, Zed A. Shaw works from the ground up giving you a detailed and comprehensive guide to Python without getting in your way of the actual coding. Shaw is informal but thorough, making the book an easy but rewarding read.

Think Python: How to Think Like a Computer Scientist

You won’t find a better source of information than the official python.org documentation.

As the sub title might suggest, “How to think like a computer scientist”, Think Python stays a little more towards the theoretical side of things. This may prove a little frustrating for a total beginner, but the book is well worth the read in regards to algorithm theory and high level concepts.

Invent With Python

If ‘learning by doing’ is your thing, then building your own game will prove to be a rewarding experience! In this book, Al Sweigart assumes no prior knowledge of Python and takes you all the way to building your own game. Given that the book is geared towards game development, the book may move a little quickly for a total beginner. Later on in this article, I mention a similar book of his where he assumes a prior knowledge of Python. If you feel you are have a good grasp of the language, then his other book might be a better resource for you.

The Django Book

If you want to learn Python for web development, then you’re probably going to be using the Django framework. This book assumes fluency in Python, but it teaches Django as if you are a beginner to the framework. The Django Book is as good as they come and will be invaluable for any budding web developer.

Python Books

If you don’t have enough to read yet, or want a book on a specific topic, then you should follow this link. The folks over at python.org have complied an extensive list of books, sorted by difficulty and topic.


Assignment 4: Get Familiar With StackOverflow

Thousands of developers have experienced every problem that you are bound to face. StackOverflow is a great resource where developers find solutions to their problems. When you happen upon an error that you’re not sure how to fix, search StackOverflow. You will more than likely find a solution and how other people solved their problem.

But StackOverflow isn’t just full of ‘newbie’ errors and problems; there are some really clever and helpful people that use the site – learn from them!

Take a look at the Hidden features of Python thread, for example.

Many of the tips and tricks you see here may not be covered in many formal tutorials, but they will be extremely helpful for intermediate to advanced Python users.


Assignment 5: Project Euler

Project Euler (pronounced ‘Oil-er’, to save you some embarrassment later on) is one of my favorite websites. After making an account, you can work through the ~400 problems on the website. Each problem is about 50% mathematics and 50% programming and, in my opinion, the most rewarding way to learn more of either subject.

The problems start off easy to test your knowledge of the language, but grow in difficulty to challenge even the most seasoned programmers. Eventually, the difficulty of the problems will force you to find the most efficient algorithm – that is if you don’t want to wait hours to compute the answer.

Nothing will turn you into a programming wizard more quickly than pushing yourself for the fastest and most efficient solutions to the Project Euler problems.

When you crack a new problem, you gain access to that problem’s forum thread where many people discuss their solutions and ideas with one another. Many of the solutions in later pages of the thread will be in Python. This is really the key to growing your programming prowess. If there is someone with a solution that is faster than yours, take the time to analyze it to see where you could improve your own solution. Over time, you will pick up all the tricks of the trade and grow your Python knowledge in a meaningful and rewarding way.

Additionally, there are also some really great blogs that work through the Project Euler problems in Python. If you are stuck on a problem, there is no shame in having a sneak peak at other developers’ work, as long as you intend on learning from it. Here are two of my favorite:


Assignment 6: Build a Game

Few things are more satisfying than building your own game.

Few things are more satisfying than building your own game. It can be a steep learning curve, but well worth it and very rewarding. PyGame is the best known game library for Python, and you will be able to find many free tutorials on it. Here are some of the best PyGame tutorials. 

Official PyGame Documentation

As with the case of the original Python tutorials, the developers of PyGame also have their own intro documentation. But these may be overly technical if you want to jump straight in and start building your game. Developer documentation, however, will always be your best source of information; so, I still recommend you get familiar with the site.

Invent With Python (With PyGame)

This free e-book by Al Sweigart runs through the PyGame library, taking you from zero knowledge to building a couple of games for yourself. The simple games will provide you with a perfect platform to start your own projects, if you are so inclined. Sweigart provides extensive and detailed comments throughout all of his code to help you learn as you go.

TheNewBoston’s Computer Game Development Tutorial

This is another Playlist from TheNewBoston. It serves well as an intro to PyGame. It assumes zero knowledge and gives you a good feel for the PyGame library, but unlike the InventWithPython guide, it is more to the point and won’t bring you to making your own full game.


Assignment 7: Get to Know Some Common Libraries and Tools

Python is a general purpose language that can do almost anything; so, of course, there is a seemingly endless supply of libraries and tools out there. Here are some of the most popular.

PyPy

If you ever want to scrape a HTML for some information… BeautifulSoup will do all this for you and add years to your life.

If you are doing some CPU intensive work, and you find that Python is proving itself to be a bottleneck, then maybe you need PyPy. PyPy is an alternative compiler for Python that can really speed up your processing.

NumPy + SciPy

These two usually go hand in hand (SciPy is dependent NumPy). If you are doing some serious number crunching for mathematical or scientific research, then these two libraries will be your best friends. NumPy and SciPy extend the mathematical functions and capabilities of Python and can greatly speed up some of your tasks.

BeautifulSoup

BeautifulSoup really is beautiful. If you need to scrape a HTML page for some information, you will know all too well the frustration and hair loss that it can bring. BeautifulSoup will do all this for you and add years to your life. Highly recommended and fun to play around with.

Python Image Library

The Python Image Library (PIL) is an extensive library that is great for anything to do with images. If you need to manipulate an image, chances are PIL can do it for you.

Django

As mentioned previously in the article, the Django framework is what you will probably use if your aim is web development. It is the most common web framework for Python and also has the most learning resources available.


Assignment 8: Get Involved in Open Source Projects

After you have a decent grasp of the language, being able to read and understand other people’s code is always an important skill to have – not to mention that it is a really great way to learn too.

For this reason, open source projects are great. Github or Bitbucket are the to go-to places for this. Don’t worry about people judging your code, you don’t have to contribute right away. You are always free to fork a project, tinker with it yourself, and see how things work. If you do happen to see something that you think could be improved, great! Go for it and submit your improvement. That’s what open source is for.


Conclusion

I hope I have been able to provide a solid base of Python knowledge for you. If there are other places of interest that you think should have been included in this syllabus, let me know in the comments below to help others out!


November 19 2011

21:32

Python from Scratch – Create a Dynamic Website

We’ve covered quite a bit of Python in the previous tutorials in this Session. Today, we’re going to combine everything we’ve learned so far to build a dynamic website with Python.


Prefer a Video Tutorial?

So, how do you get started creating websites with Python? Well, you could do it all yourself, and write a program that runs on a web server, accepting page requests and serving up responses in the form of HTML and other resources. However, that’s a lot of work, so why go to all the trouble when there are plenty of existing tools out there to do the job for you? These tools are called frameworks, and they’re what we’ll use today to create our website.

Python Frameworks

There are quite a few Python web frameworks, but here are some of the best:

  • Django – We’re going to use this today. It has a huge set of features, but remains simple to use. The documentation is also excellent, so if you get stuck, you’ll have the easiest time solving your problem with Django.
  • Grok – Another framework with a feature set that comes close to Django. If you decide you don’t prefer Django, this is a good alternative.
  • WebPy – A much more lightweight framework. It doesn’t have as many features, though it did power Reddit for a period of time!
  • TurboGears – Though previously having a reputation for poor documentation, TurboGears has improved substantially in the last year.

A more comprehensive list can be found on the Python website if you’re in need of additional options. Today we’re going to set Django up for development on a local machine, and then build a simple blog. We’re also going to review the process of installing it on a remote web server.


Installing Django

We’ll be performing most of our work today in the Terminal. This should all work on Mac and Linux; however, if you’re running Windows, the process is somewhat different. A familiarity with the command line isn’t necessary if you’re only writing Python, though, if you’re planning on using Django, or running a dynamic website in general, it’s worth learning.

Terminal Tutorials

Consider reviewing these tutorials to get yourself up and running with the Terminal.

Here are the commands you need to install Django. It’s not compatible with Python 3, so you’ll need to install version 2.7 or earlier to get it running.

    wget http://www.djangoproject.com/download/1.3.1/tarball/
    tar xzvf Django-1.3.1.tar.gz
    cd Django-1.3.1
    python setup.py install

Next, we can optionally remove the install files.

    cd ..
    rm Django-1.3.1.tar.gz

That should do it! Let’s test it out.

    python
    from django import get_version
    get_version()

You should see ’1.3.1′. If you do, everything worked and Django is installed on your system. Congratulations! We’re ready to begin creating our site!


Building our Blog

We’re going to build a blog system today, because it’s an excellent way to learn the basics. First, we need to create a Django project.

cd ~/Documents/Projects
django-admin.py startproject FirstBlog
cd FirstBlog
ls

What do each of these files do?

  • __init__.py tells Python that this folder is a Python package. We learned about these in the third lesson; it allows Python to import all of the scripts in the folder as modules.
  • manage.py isn’t actually part of your website; it’s a utility script that you run from the command line. It contains an array of functions for managing your site.
  • settings.py contains your website’s settings. Django doesn’t use XML files for configuration; everything is Python. This file is simply a number of variables that define the setting for your site.
  • urls.py is the file that maps URLs to pages. For example, it could map yourwebsite.com/about to an About Us page.

Django refers to itself an MTV framework, which stands for Model Template View.

Apps

However none of these files on their own make a functional website. For that, we need Apps. Apps are where you write the code that makes your website function, but before we take a look at them, we need to understand a bit about Django’s design principles.

First, Django is an MVC framework, which stands for Model View Controller. Django refers to itself an MTV framework, which stands for Model Template View. It’s a slightly different approach than MVC, but fundamentally, they’re quite similar. Anyhow, MVC is an architectural pattern that provides a method for structuring your projects. It separates the code that’s used to process data from the code that manages the user interface.

Django subscribes to the DRY, or “Don’t Repeat Yourself” philosophy.

Secondly, Django subscribes to the DRY, or Don’t Repeat Yourself philosophy, which means that you should never be writing code that performs a certain task more than once. For example, in our blog, if we wrote a feature that picked a random article from the archive, and implemented this feature on multiple pages, we wouldn’t code it again each time it was needed. We’d code it once and then use it on each page.

So how does this relate to apps? Well, apps allow you to write your website in a DRY style. Each project, like the one we have here, can contain multiple apps. Conversely, each app can be part of multiple projects. Using the example from earlier, this means that if we made another site in the future that also needed a random page feature, we wouldn’t have to write it all over again. We could simply import the app from this project. Because of this, it’s important that each app serves one distinct purpose. If you write all the functionality of your site within one app, and then need to use part of it again later, you have to import it all. If you were making an eCommerce website, for example, you wouldn’t want to import all the blog features. However, if you make one app for the random feature and one app for the blog publishing system, you could pick and choose the bits that you require.

This also means that within the site, the code is well organized. If you want to alter a feature, you don’t have to search through one massive file; you can instead browse to the relevant app and change it without worrying about interfering with anything else.

python mangage.py startapp blog
cd blog
ls

Again, we’ve got an __init__.py file to make it a package, and three other files: models, tests and views. We don’t need to worry about tests for now, but the other two are important. Models and Views are the M and V parts of MVC.

In models, we define our data structures.

If you’ve ever worked with PHP before, you might have used PhpMyAdmin to create your MySQL tables, and then written out your SQL queries manually in your PHP scripts. In Django, it’s much easier. We define all the data structures we need in this models file, then run a command and all the necessary databases are made for us.

When you wish to access that data, you go via these models by calling method on them, instead of running raw queries. This is very helpful, because Django can use several database programs. We’re going to use MySQL today, because it’s the most powerful, and is what most hosts provide, but if we needed to switch to a different database in the future, all of the code will still be valid! In other languages, if you wanted to switch to SQLite or something similar, you would need to rewrite the code that accesses your database.

In the views file, we write the code that actually generates the web pages. This ties all the other parts together. When a user types in a URL, it is sent by the urls script we saw earlier to the views script, which then gets relevant data from the models, processes it and passes it into a template, which finally gets served up as the page the user sees. We’ll take a look at those templates shortly. They’re the easiest part – mostly HTML.

For a blog, we’ll need a table of posts, with several fields for the title, body text, author, the time it was written, and so on. A real blog would have comments, but that’s beyond the scope of today’s demo.

from django.db import models

class posts(models.Model):
    author = models.CharField(max_length = 30)
    title = models.CharField(max_length = 100)
    bodytext = models.TextField()
    timestamp = models.DateTimeField()

MySQL

These models are just a description. We need to make an actual database from them. First, however, we need MySQL running on our system. On an actual web server, this wouldn’t be a problem, because they usually have it preinstalled. Luckily, with a package manager, it’s easy to install. First, you need to install Homebrew and Easy Install

brew install mysql
easy_install mysql-python

mysqld_safe --skip-grant-tables #let anyone have full permissions
mysql -u root
UPDATE mysql.user SET Password=PASSWORD('nettuts') WHERE User='root'; #give the user 'root' a password
FLUSH PRIVILEGES;

mysql -u root -p #log in with our password 'nettuts'
CREATE DATABASE firstblog;
quit

python2.6 manage.py runserver

When you reboot, MySQL won’t be running, so every time you need to do this in the future, run mysqld to start the server. You can then run python2.6 manange.py runserver in a new tab to start the development server.

This command won’t run the server yet, it will just return an error. That’s because we need to configure our settings. Let’s take a look at settings.py.

You need to change the database settings first. These begin on line twelve.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'firstblog',                      # Or path to database file if using sqlite3.
        'USER': 'root',                      # Not used with sqlite3.
        'PASSWORD': 'nettuts',                  # Not used with sqlite3.
        'HOST': '',                      # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '',                      # Set to empty string for default. Not used with sqlite3.
    }
}

If you try to run the server again, it should work, provided that you successfully installed MySQL. If you visit 127.0.0.1:8000 in your web browser, you should see the default Django page.

Now let’s turn our Django site into a blog. First, we need to use our Models to create tables in the database by running the following command:

python2.6 manage.py syncdb

Every time you change your models, you should run this command to update the database. Note that this can’t alter existing fields; it may only add new ones. So if you want to remove fields, you’ll have to do that manually with something like PhpMyAdmin. Because this is the first time we’ve run the command, Django will set up all the default built in tables for things like the administration system. Just type ‘yes’ and then fill in your details.

Now let’s set up the urls.py file. Uncomment the first line in the examples section, and change it to say url(r'^$', 'FirstBlog.blog.views.home', name='home') .

Now, let’s create the views file to respond to these requests.

from django.shortcuts import render_to_response

from blog.models import posts

def home(request):
    return render_to_response('index.html')

Templates

This index.html file doesn’t exist yet, so let’s make it. Create a folder, called templates in the blog app and save a file in it called index.html, which can simply contain “Hello World” for now. Then, we need to edit the settings file so Django knows where this template is located.

Line 105 is where the section for declaring template folders starts; so adjust it, like so:

TEMPLATE_DIRS = (
    "blog/templates",
    # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
    # Always use forward slashes, even on Windows.
    # Don't forget to use absolute paths, not relative paths.
)

If you run the server again and refresh the page in your browser, you should see the “Hello World” message. We can now begin laying out our blog. We’ll add some boilerplate HTML for the home page.

<!DOCTYPE html>

<html lang="en">

<head>
    <meta charset="utf-8" />
    <link rel="stylesheet" href="css/style.css">
    <link href="images/favicon.ico" rel="shortcut icon">
    <title>First Blog</title>
</head>

<body>

<div class="container">
    <h1>First Blog</h1>
    <h2>Title</h2>
    <h3>Posted on date by author</h3>
    <p>Body Text</p>

</div>

</body>

</html>

If you save and refresh the page, you should see that the page has been updated with this new content. The next step is to add dynamic content from the database. To accomplish this, Django has a templating language that allows you to embed variables with curly braces. Change the middle section of your page to look like this:

<div class="container">
    <h1>First Blog</h1>
    <h2>{{ title }}</h2>
    <h3>Posted on {{ date }} by {{ author }}</h3>
    <p>{{ body }}</p>

</div>

We can then pass in values to these variable placeholders from the views.py file by creating a dictionary of values.

from django.shortcuts import render_to_response

from blog.models import posts

def home(request):
    content = {
        'title' : 'My First Post',
        'author' : 'Giles',
        'date' : '18th September 2011',
        'body' : 'Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam cursus tempus dui, ut vulputate nisl eleifend eget. Aenean justo felis, dapibus quis vulputate at, porta et dolor. Praesent enim libero, malesuada nec vestibulum vitae, fermentum nec ligula. Etiam eget convallis turpis. Donec non sem justo.',
    }
    return render_to_response('index.html', content)

Save and refresh, and you should see that you’re now passing in content to a template from your views file. The final step is to get data from our database and pass that in instead. Luckily, we can do this all without SQL queries, using Django’s models. We need to add our blog app to our FirstBlog project by changing another setting. Go to INSTALLED_APPS on line 112, and add the

'FirstBlog.blog',

to the list.

Then change views.py so it adds data from the database.

from django.shortcuts import render_to_response

from blog.models import posts

def home(request):
    entries = posts.objects.all()[:10]
    return render_to_response('index.html', {'posts' : entries})

Next, update the template to access this data.

<div class="container">
    <h1>First Blog</h1>
    <hr />
    {% for post in posts %}
        <div class="post">
        <h2>{{ post.title }}</h2>
        <h3>Posted on {{ post.timestamp }} by {{ post.author }}</h3>
        <p>{{ post.bodytext }}</p>
        </div>
        <hr />
    {% endfor %}
</div>

Here, we can access all the data in our table in the views.py file, then select only the first ten entries. We pass this data into the template, loop through the entries and display the data with the HTML of our site. This won’t work just yet, because there’s nothing in the database. Stop the server and run:

python2.6 manage.py syncdb

This will add the new table for our posts to the database. Then, open a new tab and type:

mysql -u root -p

…type your password, hit enter, and execute:

INSERT INTO blog_posts (author, title, bodytext) values ('Bob', 'Hello World', 'Lorem Ipsum');

Return to the previous tab and run the server again. Refresh the page and you should see a blog post with the dummy content you just added. If you run the MySQL command a few more times, you should see more posts appear on the page when you refresh.

Django’s Admin System

The last thing we need to do today is review Django’s administration system. This is a really powerful feature of Django that lets you manage your site without writing any more code, as you would have to if you were creating a site from scratch. To enable it, we need to change a few settings. First, uncomment lines 4, 5, 13 and 16 within urls.py, so that you can actually access the admin page. Next, go to the INSTALLED_APPS section of settings.py and uncomment 'django.contrib.admin', and 'django.contrib.admindocs',. To let the admin control your posts table, create a new file called admin.py in the blog folder, and add the following lines:

from django.contrib import admin
from blog.models import posts

admin.site.register(posts)

Run python2.6 manage.py syncdb again to add the tables for the admin section, and restart the server.

If you visit 127.0.0.1:8000/admin now in your browser, you should see a login page. Use the details you chose earlier when you first ran the syncdb command to log in. You should see a section, called Blog, with a subtitle for the posts table. You can use this to create, edit and remove blog posts with a simple interface.

That’s all there is to do. You’ve just created a fully functioning, albeit simple, blog. To finish this lesson, we’re going to look at installing Django on a web server.


Installing on a Web Server

There are two types of web hosting, and which one you have will affect whether you can use Django. If you have shared hosting, you’re entirely at the mercy of your host.

Many cheap web hosts don’t support Python. While PHP is nearly guaranteed, support for other languages often isn’t. You’ll have to check the control panel to determine if Python (and Django) are available. Obviously the process is slightly different with every host. Almost all hosting runs on Apache, and we can use it to host Django, using the mod_wsgi or mod_python Apache modules.

Most web hosts run scripts in several languages using CGI. Django can run on FastCGI, and also, theoretically, on CGI, but this is not officially supported and would be far too slow for an actual production website. You’ll need to check if these are installed. They’re usually found under a heading, like “CGI and Scripting Language Support”.

If you have VPS hosting, or are lucky enough to have a dedicated server, your life is much easier. Usually these come with Python preinstalled, and from there, you only need to follow the same steps we went through to get a local copy of Django running. If you don’t have Python, you can install it with a package manager. Your system may even come with Django.

ssh root@example.com

wget http://www.djangoproject.com/download/1.3.1/tarball/
tar xzvf Django-1.3.1.tar.gz
cd Django-1.3.1
python setup.py install

Once you’ve installed Django on your server, upload the site you just made using any file transfer client. You can put the files anywhere, but keep them out of the public folder, or anyone will be able to see the source code of your site. I use /home for all my projects.

Next, create a MySQL database, called ‘firstblog’ on your server and run syncdb again. You’ll have to create your account for the admin control panel again, but this is a one-time thing.

If you try and run this, you might receive an error, and that’s because the settings for the server are different those on your local computer. You may need to change the database password within settings.py, but depending on your server configuration, you may also encounter other issues. Google is your friend in these situations!

To run the server this time, the command is slightly different. You have to specify an IP address and port so that you can access the site over the internet.

python manage.py runserver 0.0.0.0:8000

If you visit your site in a web browser, on port 8000, you should see your site!


Conclusion

That’s it for this lesson…and our series. I hope you’ve learned a number of useful skills over these past five lessons, and that you’re ready to go on and learn even more Python in the future. If you like the look of Django, and wish to continue increasing your knowledge of the framework, here’s some additional tutorials on the subject.

As always, I’m happy to discuss any questions about this tutorial or Python in general within the comments. Thanks for reading.


Tags: Python Videos

November 14 2011

16:20

How to Create a Sublime Text 2 Plugin

Sublime Text 2 is a highly customizable text editor that has been increasingly capturing the attention of coders looking for a tool that is powerful, fast and modern. Today, we’re going to recreate my popular Sublime plugin that sends CSS through the Nettuts+ Prefixr API for easy cross-browser CSS.

When finished, you’ll have a solid understanding of how the Sublime Prefixr plugin is written, and be equipped to start writing your own plugins for the editor!


Preface: Terminology and Reference Material

The extension model for Sublime Text 2 is fairly full-featured.

The extension model for Sublime Text 2 is fairly full-featured. There are ways to change the syntax highlighting, the actual chrome of the editor and all of the menus. Additionally, it is possible to create new build systems, auto-completions, language definitions, snippets, macros, key bindings, mouse bindings and plugins. All of these different types of modifications are implemented via files which are organized into packages.

A package is a folder that is stored in your Packages directory. You can access your Packages directory by clicking on the Preferences > Browse Packages… menu entry. It is also possible to bundle a package into a single file by creating a zip file and changing the extension to .sublime-package. We’ll discuss packaging a bit more further on in this tutorial.

Sublime comes bundled with quite a number of different packages. Most of the bundled packages are language specific. These contain language definitions, auto-completions and build systems. In addition to the language packages, there are two other packages: Default and User. The
Default package contains all of the standard key bindings, menu definitions, file settings and a whole bunch of plugins written in Python. The User package is special in that it is always loaded last. This allows users to override defaults by customizing files in their User package.

During the process of writing a plugin, the Sublime Text 2 API referencewill be essential.

During the process of writing a plugin, the Sublime Text 2 API referencewill be essential. In addition, the Default package acts as a good reference for figuring out how to do things and what is possible. Much of the functionality of the editor is exposed via commands. Any operation other than typing characters is accomplished via commands. By viewing the Preferences > Key Bindings – Defaultmenu entry, it is possible to find a treasure trove of built-in functionality.

Now that the distinction between a plugin and package is clear, let’s begin writing our plugin.


Step 1 - Starting a Plugin

Sublime comes with functionality that generates a skeleton of Python code needed to write a simple plugin. Select the Tools > New Plugin…menu entry, and a new buffer will be opened with this boilerplate.

import sublime, sublime_plugin

class ExampleCommand(sublime_plugin.TextCommand):
    def run(self, edit):
        self.view.insert(edit, 0, "Hello, World!")

Here you can see the two Sublime Python modules are imported to allow for use of the API and a new command class is created. Before editing this and starting to create our own plugin, let’s save the file and trigger the built in functionality.

When we save the file we are going to create a new package to store it in. Press ctrl+s (Windows/Linux) or cmd+s (OS X) to save the file. The save dialog will open to the User package. Don’t save the file there, but instead browse up a folder and create a new folder named Prefixr.

Packages/
…
- OCaml/
- Perl/
- PHP/
- Prefixr/
- Python/
- R/
- Rails/
…

Now save the file inside of the Prefixr folder as Prefixr.py. It doesn’t actually matter what the filename is, just that it ends in .py. However, by convention we will use the name of the plugin for the filename.

Now that the plugin is saved, let’s try it out. Open the Sublime console by pressing ctrl+`. This is a Python console that has access to theAPI. Enter the following Python to test out the new plugin:

view.run_command('example')

You should see Hello World inserted into the beginning of the plugin file. Be sure to undo this change before we continue.


Step 2 - Command Types and Naming

For plugins, Sublime provides three different types of commands.

  • Text commands provide access to the contents of the selected file/buffer via a View object
  • Window commands provide references to the current window via a Window object
  • Application commands do not have a reference to any specific window or file/buffer and are more rarely used

Since we will be manipulating the content of a CSS file/buffer with this plugin, we are going to use the sublime_plugin.TextCommand class as the basis of our custom Prefixr command. This brings us to the topic of naming command classes.

In the plugin skeleton provided by Sublime, you’ll notice the class:

class ExampleCommand(sublime_plugin.TextCommand):

When we wanted to run the command, we executed the following code in the console:

view.run_command('example')

Sublime will take any class that extends one of the sublime_plugin classes
(TextCommand, WindowCommand or ApplicationCommand), remove the suffix Command and then convert the CamelCaseinto underscore_notation for the command name.

Thus, to create a command with the name prefixr, the class needs to be PrefixrCommand.

class PrefixrCommand(sublime_plugin.TextCommand):

Step 3 - Selecting Text

One of the most useful features of Sublime is the ability to have multiple selections.

Now that we have our plugin named properly, we can begin the process of grabbing CSS from the current buffer and sending it to the Prefixr API. One of the most useful features of Sublime is the ability to have multiple selections. As we are grabbing the selected text, we need to write our plug into handle not just the first selection, but all of them.

Since we are writing a text command, we have access to the current view via self.view. The sel() method of the View object returns an iterable RegionSet of the current selections. We start by scanning through these for curly braces. If curly braces are not present we can expand the selection to the surrounding braces to ensure the whole block is prefixed. Whether or not our selection included curly braces will also be useful later to know if we can tweak the whitespace and formatting on the result we getback from the Prefixr API.

braces = False
sels = self.view.sel()
for sel in sels:
    if self.view.substr(sels[0]).find('{') != -1:
        braces = True

This code replaces the content of the skeleton run() method.

If we did not find any curly braces we loop through each selection and adjust the selections to the closest closing curly brace. Next, we use the built-in command expand_selection with the to arg set to brackets to ensure we have the complete contents of each CSS block selected.

if not braces:
    new_sels = []
    for sel in sels:
        new_sels.append(self.view.find('}', sel.end()))
    sels.clear()
    for sel in new_sels:
        sels.add(sel)
    self.view.run_command("expand_selection", {"to": "brackets"})

If you would like to double check your work so far, please compare the source to the file Prefixr-1.py in the source code zip file.


Step 4 - Threading

To prevent a poor connection from interrupting other work, we need to make sure that the Prefixr API calls are happening in the background.

At this point, the selections have been expanded to grab the full contents of each CSS block. Now, we need to send them to the Prefixr API. This is a simple HTTP request, which we are going to use the urllib and urllib2 modules for. However, before we start firing off web requests, we need to think about how a potentially laggy web request could affect the performance of the editor. If, for some reason, the user is on a high-latency, or slow connection, the requests to the Prefixr API could easily take a couple of seconds or more.

To prevent a poor connection from interrupting other work, we need to make sure that the Prefixr API calls are happening in the background. If you don’t know anything about threading, a very basic explanation is that threads are a way for a program to schedule multiple sets of code to run seemingly at the same time. It is essential in our case because it lets the code that is sending data to, and waiting for a response from, the Prefixr API from preventing the rest of the Sublime user interface from freezing.


Step 5 - Creating Threads

We will be using the Python threading module to create threads. To use the threading module, we create a new class that extends threading.Thread called PrefixrApiCall. Classes that extend threading.Thread include a run() method that contains all code to be executed in the thread.

class PrefixrApiCall(threading.Thread):
    def __init__(self, sel, string, timeout):
        self.sel = sel
        self.original = string
        self.timeout = timeout
        self.result = None
        threading.Thread.__init__(self)

    def run(self):
        try:
            data = urllib.urlencode({'css': self.original})
            request = urllib2.Request('http://prefixr.com/api/index.php', data,
                headers={"User-Agent": "Sublime Prefixr"})
            http_file = urllib2.urlopen(request, timeout=self.timeout)
            self.result = http_file.read()
            return

        except (urllib2.HTTPError) as (e):
            err = '%s: HTTP error %s contacting API' % (__name__, str(e.code))
        except (urllib2.URLError) as (e):
            err = '%s: URL error %s contacting API' % (__name__, str(e.reason))

        sublime.error_message(err)
        self.result = False

Here we use the thread __init__() method to set all of the values that will be needed during the web request. The run() method contains the code toset up and execute the HTTP request for the Prefixr API. Since threads operate concurrently with other code, it is not possible to directly return values. Instead we set self.result to the result of the call.

Since we just started using some more modules in our plugin, we must add them to the import statements at the top of the script.

import urllib
import urllib2
import threading

Now that we have a threaded class to perform the HTTP calls, we need to create a thread for each selection. To do this we jump back into the run() method of our PrefixrCommand class and use the following loop:

threads = []
for sel in sels:
    string = self.view.substr(sel)
    thread = PrefixrApiCall(sel, string, 5)
    threads.append(thread)
    thread.start()

We keep track of each thread we create and then call the start() method to start each.

If you would like to double check your work so far, please compare the source to the file Prefixr-2.py in the source code zip file.


Step 6 - Preparing for Results

Now that we’ve begun the actual Prefixr API requests we need toset up a few last details before handling the responses.

First, we clear all of the selections because we modified them earlier. Later we will set them back to a reasonable state.

self.view.sel().clear()

In addition we start a new Edit object. This groups operations for undo and redo. We specify that we are creating a group for the prefixr command.

edit = self.view.begin_edit('prefixr')

As the final step, we call a method we will write next that will handle the result of the API requests.

self.handle_threads(edit, threads, braces)

Step 7 - Handling Threads

At this point our threads are running, or possibly even completed. Next, we need to implement the handle_threads() method we just referenced. This method is going to loop through the list of threads and look for threads that are no longer running.

def handle_threads(self, edit, threads, braces, offset=0, i=0, dir=1):
    next_threads = []
    for thread in threads:
        if thread.is_alive():
            next_threads.append(thread)
            continue
        if thread.result == False:
            continue
        offset = self.replace(edit, thread, braces, offset)
    threads = next_threads

If a thread is still alive, we add it to the list of threads to check again later. If the result was a failure, we ignore it, however for good results we call a new replace() method that we’ll be writing soon.

If there are any threads that are still alive, we need to check those again shortly. In addition, it is a nice user interface enhancement to provide an activity indicator to show that our plugin is still running.

if len(threads):
    # This animates a little activity indicator in the status area
    before = i % 8
    after = (7) - before
    if not after:
        dir = -1
    if not before:
        dir = 1
    i += dir
    self.view.set_status('prefixr', 'Prefixr [%s=%s]' % 
        (' ' * before, ' ' * after))

    sublime.set_timeout(lambda: self.handle_threads(edit, threads,
        braces, offset, i, dir), 100)
    return

The first section of code uses a simple integer value stored in the variable i to move an = back and forth between two brackets. The last part is the most important though. This tells Sublime to run the handle_threads()method again, with new values, in another 100 milliseconds. This is just like the setTimeout() function in JavaScript.

The lambda keyword is a feature of Python that allows us to create a new unnamed, or anonymous, function.

The sublime.set_timeout() method requires a function or method and the number of milliseconds until it should be executed. Without lambda we could tell it we wanted to run handle_threads(), but we would not be able to specify the parameters.

If all of the threads have completed, we don’t need to set another timeout, but instead we finish our undo group and update the user interface to let the user know everything is done.

self.view.end_edit(edit)

self.view.erase_status('prefixr')
selections = len(self.view.sel())
sublime.status_message('Prefixr successfully run on %s selection%s' %
    (selections, '' if selections == 1 else 's'))

If you would like to double check your work so far, please compare the source to the file Prefixr-3.py in the source code zip file.


Step 8 - Performing Replacements

With our threads handled, we now just need to write the code that replaces the original CSS with the result from the Prefixr API. As we referenced earlier, we are going to write a method called replace().

This method accepts a number of parameters, including the Edit object for undo, the thread that grabbed the result from the Prefixr API, if the original selection included braces, and finally the selection offset.

def replace(self, edit, thread, braces, offset):
    sel = thread.sel
    original = thread.original
    result = thread.result

    # Here we adjust each selection for any text we have already inserted
    if offset:
        sel = sublime.Region(sel.begin() + offset,
            sel.end() + offset)

The offset is necessary when dealing with multiple selections. When we replace a block of CSS with the prefixed CSS, the length of that block will increase. The offset ensures we are replacing the correct content for subsequent selections since the text positions all shift upon each replacement.

The next step is to prepare the result from the Prefixr API to be dropped in as replacement CSS. This includes converting line endings and indentation to match the current document and original selection.

result = self.normalize_line_endings(result)
(prefix, main, suffix) = self.fix_whitespace(original, result, sel,
    braces)
self.view.replace(edit, sel, prefix + main + suffix)

As a final step we set the user’s selection to include the end of the last line of the new CSS we inserted, and then return the adjusted offset to use for any further selections.

end_point = sel.begin() + len(prefix) + len(main)
self.view.sel().add(sublime.Region(end_point, end_point))

return offset + len(prefix + main + suffix) - len(original)

If you would like to double check your work so far, please compare the source to the file Prefixr-4.py in the source code zip file.


Step 9 - Whitespace Manipulation

We used two custom methods during the replacement process to prepare the new CSS for the document. These methods take the result of Prefixr and modify it to match the current document.

normalize_line_endings() takes the string and makes sure it matches the line endings of the current file. We use the Settings class from the Sublime API to get the proper line endings.

def normalize_line_endings(self, string):
    string = string.replace('
', '
').replace('
', '
')
    line_endings = self.view.settings().get('default_line_ending')
    if line_endings == 'windows':
        string = string.replace('
', '
')
    elif line_endings == 'mac':
        string = string.replace('
', '
')
    return string

The fix_whitespace() method is a little more complicated, but does the same kind of manipulation, just for the indentation and whitespace in the CSS block. This manipulation only really works with a single block of CSS, so we exit if one or more braces was included in the original selection.

def fix_whitespace(self, original, prefixed, sel, braces):
    # If braces are present we can do all of the whitespace magic
    if braces:
        return ('', prefixed, '')

Otherwise, we start by determining the indent level of the original CSS. This is done by searching for whitespace at the beginning of the selection.

(row, col) = self.view.rowcol(sel.begin())
indent_region = self.view.find('^s+', self.view.text_point(row, 0))
if self.view.rowcol(indent_region.begin())[0] == row:
    indent = self.view.substr(indent_region)
else:
    indent = ''

Next we trim the whitespace from the prefixed CSS and use the current view settings to indent the trimmed CSS to the original level using either tabs or spaces depending on the current editor settings.

prefixed = prefixed.strip()
prefixed = re.sub(re.compile('^s+', re.M), '', prefixed)

settings = self.view.settings()
use_spaces = settings.get('translate_tabs_to_spaces')
tab_size = int(settings.get('tab_size', 8))
indent_characters = '	'
if use_spaces:
    indent_characters = ' ' * tab_size
prefixed = prefixed.replace('
', '
' + indent + indent_characters)

We finish the method up by using the original beginning and trailing white space to ensure the new prefixed CSS fits exactly in place of the original.

match = re.search('^(s*)', original)
prefix = match.groups()[0]
match = re.search('(s*)Z', original)
suffix = match.groups()[0]

return (prefix, prefixed, suffix)

With the fix_whitespace() method we used the Python regular expression (re)module, so we need to add it to the list of imports at the top of the script.

import re

And with this, we’ve completed the process of writing the prefixr command.The next step it to make the command easy to run by providing a keyboard shortcut and a menu entry.


Step 10 - Key Bindings

Most of the settings and modifications that can be made to Sublime are done via JSON files, and this is true for key bindings. Key bindings are usually OS-specific, which means that three key bindings files will need to be created for your plugin. The files should be named Default (Windows).sublime-keymap, Default (Linux).sublime-keymap and Default (OSX).sublime-keymap.

Prefixr/
...
- Default (Linux).sublime-keymap
- Default (OSX).sublime-keymap
- Default (Windows).sublime-keymap
- Prefixr.py

The .sublime-keymap files contain a JSON array that contains JSON objects to specify the key bindings. The JSON objects must contain a keys and command key, and may also contain a args key if the command requires arguments. The hardest part about picking a key binding is to ensure the key binding is not already used. This can be done by going to the Preferences > Key Bindings – Default menu entry and searching for the keybinding you wish to use. Once you’ve found a suitably unused binding, add it to your .sublime-keymap files.

[
	{
		"keys": ["ctrl+alt+x"], "command": "prefixr"
	}
]

Normally the Linux and Windows key bindings are the same. The cmd key on OS Xis specified by the string super in the .sublime-keymap files. When porting a key binding across OSes, it is common for the ctrl key onWindows and Linux to be swapped out for super on OS X. This may not, however, always be the most natural hand movement, so if possible try and test your keybindings out on a real keyboard.


Step 11 - Menu Entries

One of the cooler things about extending Sublime is that it is possible to add items to the menu structure by creating .sublime-menu files. Menufiles must be named specific names to indicate what menu they affect:

  • Main.sublime-menu controls the main program menu
  • Side Bar.sublime-menu controls the right-click menu on a file or folder in the sidebar
  • Context.sublime-menu controls the right-click menu on a file being edited

There are a whole handful of other menu files that affect various other menus throughout the interface. Browsing through the Default package is the easiest way to learn about all of these.

For Prefixr we want to add a menu item to the Edit menu and some entries to the Preferences menu for settings. The following example is the JSON structure for the Edit menu entry. I’ve omitted the entries for the Preferences menu since they are fairly verbose being nested a few levels deep.

[
{
	"id": "edit",
	"children":
	[
	    {"id": "wrap"},
	    { "command": "prefixr" }
	]
}
]

The one piece to pay attention to is the id keys. By specifying the idof an existing menu entry, it is possible to append an entry without redefining the existing structure. If you open the Main.sublime-menu file from the Default package and browse around, you can determine what idyou want to add your entry to.

At this point your Prefixr package should look almost identical to the official version on GitHub.


Step 12 - Distributing Your Package

Now that you’ve taken the time to write a useful Sublime plugin, it is time to get into the hand of other users.

Sublime supports distributing a zip file of a package directory as a simple way to share packages. Simply zip your package folder and change the extension to .sublime-package. Other users may now place this into their Installed Packages directory and restart Sublime to install the package.

Along with easy availability to lots of users, having your package available via Package Control ensures users get upgraded automatically to your latest updates.

While this can certainly work, there is also a package manager forSublime called Package Controlthat supports a master list of packages and automatic upgrades. To get your package added to the default channel, simply host it on GitHubor BitBucket and then fork the channel file (on GitHub, or BitBucket), add your repository and send a pull request. Once the pull request is accepted, your package will be available to thousands of users using Sublime. Along with easy availability to lots of users, having your package available via Package Control ensures users get upgraded automatically to your latest updates.

If you don’t want to host on GitHub or BitBucket, there is a customJSON channel/repository system that can be used to host anywhere, while still providing the package to all users. It also provides advanced functionality like specifying the availability of packages by OS. See the PackageControl page for more details.


Go Write Some Plugins!

Now that we’ve covered the steps to write a Sublime plugin, it is time for you to dive in! The Sublime plugin community is creating and publishing new functionality almost every day. With each release, Sublime becomes more and more powerful and versatile. The Sublime Text Forum is a great place to get help and talk with others about what you are building.


August 24 2011

17:00

Python from Scratch: Object Oriented Programming

Welcome back to lesson four in our Python from Scratch series. This tutorial will assume some prior knowledge of variables, data types, functions and print output. If you’re not up to date, check out the previous three articles in the series to catch up.

Today, we’re going to be delving into the subject of Object Oriented Programming (OOP). OOP is a very powerful way of organizing your code, and a solid understanding of the concepts behind it can really help you get the most out of your coding.


Prefer a Screencast?

Subscribe to our YouTube and Blip.tv channels to watch more screencasts.

Transcription


What is Object Oriented Programming?

Python is primarily designed as an object-oriented programming language – but what does ‘object oriented’ actually mean?

There are a variety of definitions for the term, and you could talk for literally hours trying to explain the complicated ins and outs, nuances and differences in implementations, but I’ll try to give a quick overview.

Broadly, object oriented programming is the concept that, in programming, the objects that we’re manipulating are more important than the logic needed to manipulate those objects. Traditionally, a program has been seen as a recipe – a set of instructions that you follow from start to finish in order to complete a task. That can still be true, and for many simple programs, that’s all which is required. That approach is sometimes known as procedural programming.

OOP puts objects at the center of the process.

On the other hand, as programs get more and more complex and convoluted, the logic needed to write them in a purely procedural way gets more and more twisted and hard to understand. Often object oriented approaches can help with that.

When we talk about object oriented approaches, what we do is put the objects at the center of the process, instead of simply using them as necessary containers for information as part of our procedural instructions. First, we define the objects we want to manipulate and how they relate to each other, and then we start to flesh it out with logic to make the program actually work.

When I talk about ‘objects’, I can be talking about all sorts of things. An ‘object’ can represent a person (as defined by properties such as name, age, address etc.), or a company (as defined by things like number of employees and so on), or even something much more abstract, like a button in a computer interface.

In this introduction, we’re not going to be covering all of the concepts in this topic because we’d be here all night, but by the end of the tutorial, I hope you’ll have a solid understanding of the principles you need to start straight away using some simple object-oriented techniques in your Python programs. Even better, these concepts are fairly similar in a lot of programming environments. The knowledge transfers over from language to language quite nicely.


Getting Started

I mentioned earlier that the first thing we should do when we’re going for an OOP approach is to define the objects we’re going to be using. The way we do this is to first define the properties that it possesses using a class. You can think of a class as a sort of template; a guide for the way an object should be structured. Each object belongs to a class and inherits the properties of that class, but acts individually to the other objects of that class.

An object is sometimes referred to as an ‘instance’ of a class.

As a simple example, you might have a class named ‘person’ with, say, an age and a name property, and an instance of that class (an object) would be a single person. That person might have a name of “Andy” and an age of 23, but you could simultaneously have another person belonging to the same class with the name of “Lucy” and an age of 18.

It’s hard to understand this without seeing it in practice, so let’s get some actual code going.

Defining a class

To define a class, in typical simple Python fashion, we use the word ‘class,’ followed by the name of your new class. I’m going to make a new class here, called ‘pet’. We use a colon after the name, and then anything contained within the class definition is indented. However, with a class, there are no parentheses:

	class pet:

So now we’ve got a class, but it’s rather useless without anything in it. To start, let’s give it a couple of properties. To do this, you simply define some variables inside the class – I’m going to go with the number of legs to start with. As usual, you should always name your variables so that it’s easy to tell what they are. Let’s be original and call it ‘number_of_legs’. We need to define a value or we’ll get an error. I’ll use 0 here (it doesn’t matter too much in this case since the number of legs will be specific to each instance of the class – a fish doesn’t have the same amount of legs as a dog or a duck, etc. – so we’ll have to change that value for each object anyway).

	class pet:
		number_of_legs = 0

Instances and member variables

A class on its own isn’t something you can directly manipulate; first, we have to create an instance of the class to play with. We can store that instance in a variable. Outside of the class (without any indentation), let’s make an instance of the class and store it in the variable, ‘doug’. To make a new instance of a class, you simply type the name of the class, and then a pair of parentheses. At this point, there’s no need to worry about the parentheses, but later on you’ll see that they’re there because, like a function, there’s a way of passing in a variable for use by the class when you first create the instance.

A class on its own isn’t something that you can directly manipulate.

	class pet:
		number_of_legs = 0

	doug = pet()

Now that we have an instance of a class, how do we access and manipulate its properties? To reference a property of an object, first we have to tell Python which object (or which instance of a class) we’re talking about, so we’re going to start with ‘doug’. Then, we’re going to write a period to indicate that we’re referencing something that’s contained within our doug instance. After the period, we add the name of our variable. If we’re accessing the number_of_legs variable, it’s going to look like this:

	doug.number_of_legs

We can treat that now exactly as we would treat any other variable – here I’m going to assume doug is a dog, and will give that variable the value of 4.

To access this variable, we’re going to use it again exactly as we would treat any other variable, but using that doug.number_of_legs property instead of the normal variable name. Let’s put in a line to print out how many legs doug has so that we can show that it’s working as it should:

	class pet:
		number_of_legs = 0

	doug = pet()
	doug.number_of_legs = 4
	print "Doug has %s legs." % doug.number_of_legs 

If you run the code above, you’ll see that it’s printed out for us. It defined our ‘pet’ class, created a new instance of that class and stored it in the variable ‘doug’, and then, inside that instance, it’s assigned the value of 4 to the number_of_legs variable that it inherited from its class.

So you can see from that very simplified example how you can begin to build nice, modular data structures that are clear and easy to use, and can start to scale quite nicely.


Introducing Logic

Okay, so that’s the very basics of classes and objects, but at the moment we can only really use classes as data structures – or, containers for variables. That’s all well and good, but if we want to start performing more complex tasks with the data we’re manipulating, we need a way of introducing some logic into these objects. The way we do that is with methods.

Methods, essentially, are functions contained within a class. You define one in exactly the same way as you would a function, but the difference is that you put it inside a class, and it belongs to that class. If you ever want to call that method, you have to reference an object of that class first, just like the variables we were looking at previously.

Methods, essentially, are functions contained within a class.

I’m going to write a quick example here into our pet class to demonstrate; let’s create a method, called ‘sleep’, which is going to print out a message when it’s first called. Just like a function, I’m going to put ‘def’ for ‘define’, and then I’m going to write the name of the method I want to create. Then we’re going to put our parentheses and semicolon, and then start a new line. As usual, anything included in this method is going to be indented an extra level.

Now, there is another difference between a method and a function: a method always, always, always has to have an argument, called ‘self’ between the parentheses. When Python calls a method, what it does is passes the current object to that method as the first argument. In other words, when we call doug.sleep(), Python is actually going to pass the object ‘doug’ as an argument to the sleep method.

We’ll see why that is later, but for now you need to know that, with a method, you always have to include an argument called ‘self’ first in the list (if you want to add more arguments, you can add them afterwards, exactly like if you were passing multiple arguments to a function). If you don’t include that argument, when you run the code, you’re going to get an error thrown because Python is passing in an argument (this ‘self’ object), and the method is saying, ‘Hey, man, I don’t take any arguments, what are you talking about?’. It’s the same as if you tried to pass an argument into a function that doesn’t accept any arguments.

So here’s what we have so far:

	class pet:
		number_of_legs = 0

		def sleep(self):

	doug = pet()

Inside this method, we’re going to write a print statement like so:

	class pet:
		number_of_legs = 0

		def sleep(self):
			print "zzz"

	doug = pet()

Now, if we want to use this method, we simply use an instance of the pet class to reference it. Just like the number_of_legs variable, we write the name of the instance (we’ve got one called doug), then a period, then the name of the method including parentheses. Note that we’re calling sleep using no arguments, but Python is going to add in that argument by itself, so we’re going to end up with the right amount of arguments in total.

	class pet:
		number_of_legs = 0

		def sleep(self):
			print "zzz"

	doug = pet()
	doug.sleep()

If you run this code, you should see that it prints out the message we wrote.

Data

Great, so now how about we write a new method to print out how many legs the pet has, to demonstrate how you can use methods to start manipulating the data within the class, and to demonstrate why we need to include this confusing ‘self’ argument. Let’s make a new method, called ‘count_legs’.

This is where the ‘self’ argument comes in. Remember when we were accessing number_of_legs from outside the class and we had to use ‘doug.number_of_legs’ instead of just ‘number_of_legs’? The same principle applies; if we want to know what is contained in that variable, we have to reference it by first specifying the instance containing that variable.

However, we don’t know what the instance is going to be called when we write the class, so we get around that using the ‘self’ variable. ‘self’ is just a reference to the object that is currently being manipulated. So to access a variable in the current class, you simply need to preface it with ‘self’ and then a period, like so:

	class pet:
		number_of_legs = 0

		def sleep(self):
			print "zzz"

		def count_legs(self):
			print "I have %s legs" % self.number_of_legs

	doug = pet()
	doug.number_of_legs = 4
	doug.count_legs()

In practice, what this means is that wherever you write ‘self’ in your method, when you run the method that self is replaced by the name of the object, so when we call ‘doug.count_legs()’ the ‘self’ is replaced by ‘doug’. To demonstrate how this works with multiple instances, let’s add a second instance, representing another pet, called ‘nemo’:

	class pet:
		number_of_legs = 0

		def sleep(self):
			print "zzz"

		def count_legs(self):
			print "I have %s legs" % self.number_of_legs

	doug = pet()
	doug.number_of_legs = 4
	doug.count_legs()

	nemo = pet()
	nemo.number_of_legs = 0
	nemo.count_legs()

This will print out a message for 4 and then 0 legs, just as we wanted, because when we call ‘nemo.count_legs(),’ the ‘self’ is replaced by ‘nemo’ instead of ‘doug’.

In this way, our method will run exactly as intended because the ‘self’ reference will dynamically change depending on the context and allow us to manipulate the data only within the current object.

The main things you need to remember about methods is that they’re exactly like functions, except that the first argument has to be ‘self’ and that to reference an internal variable you have to preface the variable name with ‘self’.

Just as a note: You can actually use any name instead of ‘self’ for your methods. –The methods here would work just as well if we renamed the variable ‘self’ to any word. Using the name ‘self’ is simply a convention which is useful to Python programmers because it makes the code much more standard and easy to understand, even if it’s written by someone else. My advice would be to stick to the conventions.


Some More Advanced Features

Now that we’ve gone over the basics, let’s have a look at some more advanced features of classes, and how they can help make your programming easier to structure.

The next thing we’re going to talk about is inheritance. As its name might hint, inheritance is the process of making a new class based around a parent class, and allowing the new class to inherit the features of the parent class. The new class can take all of the methods and variables from the parent class (often called the ‘base’ class).

Inheritance is the process of making a new class based around a parent class.

Let’s extend our pet example to see how this might be useful. If we use ‘pet’ as our parent class, we could create a child class which inherited from the pet class. The child class might be something like ‘dog’, or ‘fish’ – something that is still a ‘pet’, but is more specific than that. A dog is a pet, and does the same things that all pets do – for example it eats and sleeps and has an age and a number of legs – but it does other things that are specific to being a dog, or at least more specific than to being a pet: dogs have fur, but not all pets do. A dog might bark, or fetch a stick, but not all pets would.

Getting back to the point, say we wanted to make a class in our program to represent a dog. We could use inheritance to inherit the methods and variables contained in ‘pets’ so that our dog could have a ‘numberOf Legs’ and the ability to ‘sleep’, in addition to all the dog specific things we might store or do.

Now, you might be wondering why we don’t put those methods and variables into the dog class and get rid of the pet class entirely? Well, inheritance gives us two distinct advantages over that approach: One, if we want an object that is a pet, but isn’t a dog – a generic pet, if you will – we can still do that. Two, perhaps later we want to add a second type of pet – maybe a fish. We can make that second class also inherit from pet, and so both classes can share everything in pet, but at the same time have their own more specific methods and variables that apply only to that type of object.

We’re getting a little bogged down in the theory here, so let’s write something to make it a little clearer. First, we’re going to write a new class, called ‘dog’, but this time, between the class name and the colon, we’re going to put some parentheses, and in them, we’re going to write the name of the class that we want to inherit from, sort of as if we’re passing this new class an argument, like we would a function.

Next, let’s give this class a simple method to demonstrate how it works. I’m going to add a ‘bark’ method which will print ‘woof’:

	class pet:
		number_of_legs = 0

		def sleep(self):
			print "zzz"

		def count_legs(self):
			print "I have %s legs" % self.number_of_legs

	class dog(pet):
		def bark(self):
			print "Woof"

So, now let’s see what happens if we make an instance of this class. I’’m going to call our new dog ‘doug’ again. Now, if we call doug.bark():

	class pet:
		number_of_legs = 0

		def sleep(self):
			print "zzz"

		def count_legs(self):
			print "I have %s legs" % self.number_of_legs

	class dog(pet):
		def bark(self):
			print "Woof"

	doug = dog()
	doug.bark()

As expected, doug barks. That’s great, but we haven’t seen anything new yet – just a class with a method. What inheritance has done for us, though, is to make all of the pet functions and variables available to us through our ‘doug’ object, so if I do something like this:

	class pet:
		number_of_legs = 0

		def sleep(self):
			print "zzz"

		def count_legs(self):
			print "I have %s legs" % self.number_of_legs

	class dog(pet):
		def bark(self):
			print "Woof"

	doug = dog()
	doug.sleep()

Then the sleep method will also execute correctly. In effect, our doug object belongs to both the ‘pet’ AND the ‘dog’ class. To ensure that the variables do the same as the methods, let’s try this:

	class pet:
		number_of_legs = 0

		def sleep(self):
			print "zzz"

		def count_legs(self):
			print "I have %s legs" % self.number_of_legs

	class dog(pet):
		def bark(self):
			print "Woof"

	doug = dog()
	doug.number_of_legs = 4
	doug.count_legs()

You can see that doug acts exactly as before, showing that our variables are being inherited. Our new child class is simply a specialized version of the parent one, with some extra functionality but retaining all of the previous functionality.


So there you have it, a quick introduction to object oriented programming. Stay tuned for the next installment in this series, where we’re going to be working with Python on the web!


July 22 2011

02:55

Python from Scratch – Functions and Modules

Welcome back to the Python from Scratch series. In the previous lesson, we learned how to use variables and control structures to store and manipulate data. Be sure to review it if you need a refresher!


Video Tutorial

Press the HD button for the clearest picture.
Subscribe to our YouTube and Blip.tv channels to watch more screencasts.

Transcript

In today’s tutorial, we’re going to be looking at functions – what they are, how they work, and how to make your own. We’re also going to be reviewing some of the built-in functions, and how to use them effectively. Finally, we’re going to have a look at creating and importing modules.


Functions – Writing your Own

Functions are an important step to cover, when introducing additional complexity into your programming. If a variable is a named container for data, then a function is a named container for a block of code that can be executed on demand. This is useful if you have a program that executes a certain operation lots of times. Instead of copying and pasting the code to do that operation each section where it’s needed, you can simply write one single function to do it for you.

A function is a named container for a block of code.

There are two types of functions: the ones that you write yourself and include in your code, and the ones that are included in Python natively, which carry out common procedures, such as converting an integer to a string, or finding the length of a string.

We’re going to look at writing a simple function now, and demonstrate how it can be useful in the real world code. Then, we’ll have a look at some of the most important built-in functions.

A simple Example

Let’s imagine that we want to make a script for a shopping cart that takes the cost of all the items in your cart, and then adds them together to return a total cost. To write a function that would take two costs, add them up, and then print out the output, we might do something like:

	#our two costs to add up
	cost1 = 15
	cost2 = 20

	def sumCart():
		totalCost = cost1 + cost2
		print totalCost

	sumCart()

To define the function, we need to use the ‘def‘ keyword, and then the name of the function. Next, we type two parentheses (we’ll come back to those later), and then a colon. After that, all of the code that we want to be encased within the function should be indented, just as with if, while and for loops.

To make the code in the function run, we type the name of the function, followed by the two parentheses again. If you run this code, you’ll see it print out ’35′, which is the correct output.

Arguments

That’s great, but at the moment, our functions are a bit rigid – everything about the function is hard-coded into them. For example, let’ say we want to use the sumCart function somewhere else, in a different part of the program, but instead of adding cost1 and cost2 together, we want to add two other costs together that were held in different variables. We’d have to write a whole new function, even though that new function would be exactly the same as the current one, with the names of the variables swapped around – that hardly seems like an efficient solution.

To solve this issue, we use ‘arguments’, and that’s what the parentheses are for. An argument is a way of passing data into a function when we don’t know which variable or variables that data is going to be in. If that’s confusing, let’s take the example that I just referenced. We’ll add two more costs: cost3 and cost4.

Now, we’re going to add two arguments for the two items that we want to add up. Arguments are defined inside the parentheses, and each argument is given a name, with a comma separating each. An argument acts as a temporary reference to the data that you passed in while the function is running.

	cost1 = 15
	cost2 = 20
	cost3 = 5
	cost4 = 10

	def sumCart(item1, item2):
		totalCost = item1 + item2
		print totalCost

	sumCart(cost1, cost2)

When we call the sumCart function, the two variables that we’ve passed in (cost1 and cost2) are put into the arguments item1 and item2. This always happens in the same order you define the arguments – in other words, the first variable you pass in is assigned to the first argument, the second to the second argument, and so on for as many arguments as your function takes.

What actually happens when the function is run is that those two arguments are converted into local variables and are assigned the values of the variables that you pass into the function when you call it – in this case the value of cost1 is put into the local variable item1, and the value of cost2 is put into the local variable item2. This means you can easily use them inside the function.

Another way you could look at it is, when you pass a variable as an argument, everywhere that that argument’s name appears in the function, it is replaced with the variable you passed in. So, in this example, everywhere that item1 is written inside the function, it’s replaced with cost1, and the same with item2 and cost2.

To prove that you can pass any numbers you want, if you run this code, you should now receive the sum of cost3 and cost4:

	cost1 = 15
	cost2 = 20
	cost3 = 5
	cost4 = 10

	def sumCart(item1, item2):
		totalCost = item1 + item2
		print totalCost

	sumCart(cost3, cost4)

Argument Defaults

Note that a function can accept any number of arguments, but remember: when you call the function, you have to pass it the same number of arguments as you defined, otherwise you will receive an error. There is one way around this: you can define a default value for any argument, which is used in the cases when you don’t supply a value.

Considering our current code, let’s imagine that we want the second item to cost 5 if we don’t give the function a different value. It’s not a likely scenario, but it’ll demonstrate the concept.

	cost1 = 15
	cost2 = 20
	cost3 = 5
	cost4 = 10

	def sumCart(item1, item2 = 5):
		totalCost = item1 + item2
		print totalCost

	sumCart(cost1)
	sumCart(cost3, cost4)

If we run this code, you’ll see that, in the first call, the output is 20, which is indeed (cost1 + 5). This doesn’t, however, affect the behavior when both arguments are supplied.

Returning values

There’s one final feature of functions that we’re going to review today: returning values. We’ve learned how to make a function that doesn’t mind what inputs it takes, but what if we wanted to store the answer to ‘totalCost’ in a variable rather than print it out? Perhaps what we want to do is work out the cost of the first two items, and store it in one variable, then do the same with the second two items and store that answer in a second variable. With the current layout of our function, we couldn’t do that in a readable and easy to understand way.

Just like the way we can pass arguments into a function so that we don’t have to hard-code where the data is coming from, we can do the same with the output. I’ll show you with an example – in our previous function, I’m going to replace the word ‘print’ with the word ‘return’:

	cost1 = 15
	cost2 = 20
	cost3 = 5
	cost4 = 10

	def sumCart(item1, item2):
		totalCost = item1 + item2
		return totalCost

	cart1 = sumCart(cost1, cost2)
	cart2 = sumCart(cost3, cost4)

	print cart1
	print cart2

Obviously, the function is no longer going to print out the answer for us – what it’s going to do instead is pass back the value of the variable totalCost to wherever we called the function from. That way, we can use the answer in whatever context we need to. Now, in our previous program, that wouldn’t be particularly useful, because we didn’t specify anywhere to store it, so I’ve also changed the way we call the function.

We need to provide Python with a variable in which to store the answer, and then we’re simply going to set that variable equal to our statement calling the function. When the function returns a value, that value will now be stored in the variable we specified.


Functions – Built-in

We’ve learned how to make our own functions, and how to use them, but there are some operations that are carried out often enough that Python has included them for use in any program. The way you call them is exactly the same as calling your own, but you don’t have to define them first. We’re going to have a brief look at the most important ones now, but be sure be have a look at the Python documentation because there are loads of useful functions that are available for use, which we don’t have time to cover right now.

Refer to the Python documentation for a list of all built-in functions.

str()

First, let’s look at one of the most useful functions in Python: the string conversion function. There are many times in scripting when you have a variable that contains a number, or some other type of data, but you need to convert that value to a string in order to do something with it – normally to print it out onto the screen.

Let’s say we have a number in a variable, and that we want to print it out, with the message ‘The number is ‘ and then the number. If we write:

	number = 10
	print 'The number is ' + number

…then we’re going to encounter an error, because what you’re asking Python to do is to add a number and a string together – which doesn’t really make a whole lot of sense. What we need to do is to convert the number 10 into the string 10. If we do that, Python will understand that what we’re trying to do is not find a sum, but instead concatenate the two strings into one.

This is where the str function comes in. It accepts a value, and returns a string to represent it. In the case you pass it a number, it will simply return the number but in string format.

	number = 10
	print 'The number is ' + str(number)

Now that we’ve encased the number in a string conversion function, the code runs perfectly. The str function doesn’t have to take a number though – it can also take other values, such as a Boolean:

	bool = True
	print 'The value is ' + str(bool)

The str() function finds the closest string match it can find for the value of ‘true’ and returns it, meaning that we can output it nicely.

len()

Another common task for strings is to be able to find the length of them, and, again, Python has a built in function for this. Let’s take the string ‘Hello World’ and try to find its length. To do this, we’re going to need the built-in len() function, which stands for length. All we need to do is pass it the string we want to use, and it will return the number of characters in it:

	string = 'Hello World'
	print len(string)

The len() function is capable of more than this, though. What it actually does is count the number of items in the object you give it – if you give it a string then that’s the number of characters, but you can also give it a list or a tuple, for example, and it will return the amount of items in that particular object.

int()

Moving on, often you are given a number like 10.6 or 3.896, which is not an integer, but you need an integer from them. The built-in function to convert to an integer is called int(), and it works fairly predictably. The function will return the integer conversion of that number. Note that this function does NOT round the input to the nearest integer – it very simply throws away anything after the decimal point and returns the number to you. So the input of 10.6 will return 10, NOT 11. Similarly, 3.25 would return 3.

	number = 10.6
	print int(number)

The int function can also convert a string to an int data type. For example, given this, the output would still be 10:

	number = '10'
	print int(number)

However, be careful, because, when it is converting from a string to an integer, int() will NOT deal with decimal places. If you provide it:

	number = '10.6'
	print int(number)

Then it will throw an error, because the input wasn’t an integer, so it didn’t know how to handle it.

range()

Finally, we’re going to quickly look at the range() function; it comes up surprisingly often when you start doing more complex tasks in Python, and, while it might not seem massively useful at the moment, it’s worth having a look at and getting to know how it works. Range is a function that you use to create a list of numbers following a certain pattern. The simplest example is that perhaps you need a list containing all of the numbers 0 to 10. Instead of typing them out manually, you can use the range function to do it in one line.

Range accepts an integer as the parameter, and, in this case, we’ll pass it 11. The function thens starts at zero, and makes a list counting upwards until it hits one less than the number you put in. So here, if we put in 11, it will store the numbers 0 – 10 in the list numbers:

	numbers = range(11)
	print(numbers)

What if we wanted to print the numbers five to ten instead? Well, optionally, we can include another argument for the starting number, so this will print the numbers five to ten:

	numbers = range(5, 11)
	print(numbers)

Lastly, what if we wanted to print only the odd numbers under 10? Well, range accepts a third value, which is the amount to go up each step. So if we start at 1, set the step to two and the upper limit to 11, we should end up with all the odd numbers under ten:

	numbers = range(1, 11, 2)
	print(numbers)

As a final note, any of these numbers can be negative as well, so if we wanted to count DOWN from 10 to 1 instead of up, we could set the step to be negative 1, the starting value to 10 and the limit to 0, like so:

	numbers = range(10, 0, -1)
	print(numbers)

Note that range will only deal with integers, so make sure that you don’t give it any values that aren’t integers or you’ll throw an error.


Modules – Creating your Own

Alright, we’ve now covered functions; let’s move on to the second topic of today’s lesson: modules. If functions are groups of code, then modules are groups of functions. There’s more to them, but that’s a good way of thinking about them for now.

As we discussed earlier, you can’t make a complex program without sorting code into functions, and as your program continues to grow further, even they become unwieldy. You have to sort them into another hierarchical level. These are modules. So we can summarize them as a structural and organizational tool for our code.

Modules are quite easy to create. They are simply Python files, like your regular scripts. To create a module, write one or more functions in a text file, then save it with a .py extension. Let’s do that now with an example. Open up a new file your text editor or IDE, and make a function. I’m going to continue with the example of a shopping cart from earlier and make a function to calculate tax on the products.

Create a new file in your IDE or text editor and let’s create a function for calculating tax on a product.

	def addTax(price, tax):
		newPrice = price / 100 * (100 + tax)
		return newPrice

If we save this file with a .py extension in the same directory as our other script, we can use it as a module! It’s important to give it a descriptive name so you know what it does when you come back later, so call this one finance.py


Importing modules

To use modules, we can use either the import or the from keyword. import is the simplest and most common, so let’s try that first. You then specify the name of the module, which is simply the filename, without the .py extension. For example, with our finance module in the same folder as our script, we could write:

import finance

…and we would have access to all the functions in the script. We can then call them just as we did before. You can also use the from keyword, which allows you to select specific functions from the library, if you know which ones you require beforehand. For a module that has hundreds of functions, this is recommended – it saves loading time on functions that aren’t even being used.

from finance import addTax

You can import several by separating their names with commas.

from finance import addTax, calculateDiscount

And you can even use the asterisk wildcard to import everything.

from finance import *

After importing the module, to use the functions contained within it, you use the module name, followed by a dot, and then the function name.

	import finance
	print finance.addTax(100, 5)

This should result in 105 when the script is run. A simple result, but it means you’ve got a working function!


Built-in Modules

There are plenty of built in modules, just as there are with built in functions. This is where Python really excels! It takes what’s called the “batteries included” approach.

Because Python has such an extensive range of built-in modules, there’s no way to cover them all in one lesson. Some are quite obscure, and there’s no point teaching a load of information you might never have a use for. Instead, let’s cover several of the most useful ones.

  • random
  • math
  • os
  • datetime
  • urllib2

random

A good one to start with is random , because it’s easy to understand and useful in most scripts you’ll write. As you’d expect, this module lets you generate random numbers. For example, when using Python to create a website, they can be used in making your password database more secure or powering a random page feature. If we wanted a random integer, we can use the randint function, like so:

	import random
	print random.randint(0, 5)

This will output either 1, 2, 3, 4 or 5. Randint accepts exactly two parameters: a lowest and a highest number. As you can see, it includes the highest number but goes from 1 above the lowest number. If you want a random floating point number, you can use the random function:

	import random
	random.random()

…which outputs a random number between 0 and 1, to a load of decimal places. If you want a larger number, you can multiply it. For example, a random number between 0 and 100:

	import random
	random.random() * 100

The random module also has a function for choosing a random element from a set such as a list, called choice.

	import random
	myList = [1, 8, True, 77, "Lorem", 482, "Ipsum"]
	random.choice(myList)

math

The math module provides access to mathematical constants and functions.

	import math

	math.pi #Pi, 3.14...
	math.e  #Euler's number, 2.71...

	math.degrees(2)  #2 radians = 114.59 degreees
	math.radians(60) #60 degrees = 1.04 radians

	math.sin(2)    #Sine of 2 radians
	math.cos(0.5)  #Cosine of 0.5 radians
	math.tan(0.23) #Tangent of 0.23 radians

	math.factorial(5) #1 * 2 * 3 * 4 * 5 = 120
	math.sqrt(49)   #Square root of 49 = 7

There are lots more functions in the module, but these are some of the most useful. You can find a full list here.

datetime

If you need to work with dates and times, then the datetime module is your friend. This one is a very useful one to know about for web development. If you’re making an website with comments, or a blog, then you’ll probably be displaying their age, or the time they were created.

Let’s import the module as usual:

	import datetime

For convenience, you can also import the date and the time components separately, so you don’t have to type as much later.

	import datetime
	from datetime import date
	import time

Let’s have a look at some of the functions it provides:

	time.time() #Returns the number of seconds since the Unix Epoch, January 1st 1970

	date.fromtimestamp(123456789)   #Converts a number of seconds to a date object
	date.fromtimestamp(time.time()) #We can combine the two to get an object representing the current time

	date.fromordinal(10000) #Returns the date a given number of days since January 1st in the Year 1 - 28th of May 0018 in this case

There are some useful functions for converting dates into useable strings. strftime allows you to format a string based on special characters preceded by a percent symbol. %d represents day, for example.

	currentDate = date.fromtimestamp(time.time()) #Create a variable representing the time now
	currentDate.strftime("%d/%m/%y") #Format the date like DD/MM/YY - in this case 09/07/11

	currentDate.isoformat() #Format as an ISO standard date - in this case 2011-07-09

os

The next module we’re looking at is os, which provides functions that allow you to interface with the underlying operating system that Python is running on – be that Windows, Mac or Linux. We’ll focus on the path submodule. It allows you to manipulate and find the properties of files and folders on the system, so it’s the most useful for web development. You’ll often need to manage files, for example user uploads.

	from os import path

	path.exists("/Users/Giles") #Checks if the directory exists, in my case it's True
	path.exists("/Users/Bob")   #False this time
	from os import path
	path.getatime("/Users") #Get the last time the given directory was accessed as a timestamp
	path.getmtime("/Users") #Get the last time the given directory was modified as a timestamp

These timestamps can be converted to useable strings using the datetime module – you can see how you can combine modules.

	from os import path
	path.getsize("/Users/Giles/Desktop/boot") #Returns the size of a file in bytes - for this file it was 314400 bytes, or 314kb

The last function in the os module we’re going to look at is join. It combines two paths into one. You might be thinking: “Why can’t I just concatenate the strings?”, but it’s not as simple as that. On Windows, the path delimiter is a backslash, on Mac and Linux it’s a forward slash. This module alleviates problems like that and makes sure your Pythons scripts work on any system.

	path.join("C:", "Users") #Returns "C:/Users"

urllib2

To finish our tour of Python’s standard library, we’re going to have a brief look at urllib2. This module lets you interface with the web, so it’s obviously quite relevant to us. The most useful function it provides is urlopen, which downloads a page.

You use it as follows:

	import urllib2
	urllib2.urlopen("http://net.tutsplus.com")

You can obviously swap the URL string for any site. This will download the HTML content of the page. This won’t return a string though, so we need to read the data to get that out:

	import urllib2
	urllib2.urlopen("http://net.tutsplus.com").read(100)

This will return the first 100 characters of the HTML source of Nettuts+. Once you have this data, you can sort through it and extract the necessary bits that you require.


Final Tips

If your module is in a different directory than your script, it’s a bit trickier to import them. You have to create a file that tells Python the folder is a package. For example, if you have a folder, called subdirectory, in the same folder as your script, and then your module within that subdirectory, you would have to create a file called __init__.py in the same folder as the module. This file can be empty. Then, from your script, you’d import it:

%review it if you need a refresher!


Video Tutorial

Press the HD button for the clearest picture.
Subscribe to our YouTube and Blip.tv channels to watch more screencasts.

Transcript

In today’s tutorial, we’re going to be looking at functions – what they are, how they work, and how to make your own. We’re also going to be reviewing some of the built-in functions, and how to use them effectively. Finally, we’re going to have a look at creating and importing modules.


Functions – Writing your Own

Functions are an important step to cover, when introducing additional complexity into your programming. If a variable is a named container for data, then a function is a named container for a block of code that can be executed on demand. This is useful if you have a program that executes a certain operation lots of times. Instead of copying and pasting the code to do that operation each section where it’s needed, you can simply write one single function to do it for you.

A function is a named container for a block of code.

There are two types of functions: the ones that you write yourself and include in your code, and the ones that are included in Python natively, which carry out common procedures, such as converting an integer to a string, or finding the length of a string.

We’re going to look at writing a simple function now, and demonstrate how it can be useful in the real world code. Then, we’ll have a look at some of the most important built-in functions.

A simple Example

Let’s imagine that we want to make a script for a shopping cart that takes the cost of all the items in your cart, and then adds them together to return a total cost. To write a function that would take two costs, add them up, and then print out the output, we might do something like:

	#our two costs to add up
	cost1 = 15
	cost2 = 20

	def sumCart():
		totalCost = cost1 + cost2
		print totalCost

	sumCart()

To define the function, we need to use the ‘def‘ keyword, and then the name of the function. Next, we type two parentheses (we’ll come back to those later), and then a colon. After that, all of the code that we w


Tags: Python

July 21 2011

21:07

Python from Scratch – Functions and Modules

Welcome back to the Python from Scratch series. In the previous lesson, we learned how to use variables and control structures to store and manipulate data. Be sure to review it if you need a refresher!


Video Tutorial

Press the HD button for the clearest picture.
Subscribe to our YouTube and Blip.tv channels to watch more screencasts.

Transcript

In today’s tutorial, we’re going to be looking at functions – what they are, how they work, and how to make your own. We’re also going to be reviewing some of the built-in functions, and how to use them effectively. Finally, we’re going to have a look at creating and importing modules.


Functions – Writing your Own

Functions are an important step to cover, when introducing additional complexity into your programming. If a variable is a named container for data, then a function is a named container for a block of code that can be executed on demand. This is useful if you have a program that executes a certain operation lots of times. Instead of copying and pasting the code to do that operation each section where it’s needed, you can simply write one single function to do it for you.

A function is a named container for a block of code.

There are two types of functions: the ones that you write yourself and include in your code, and the ones that are included in Python natively, which carry out common procedures, such as converting an integer to a string, or finding the length of a string.

We’re going to look at writing a simple function now, and demonstrate how it can be useful in the real world code. Then, we’ll have a look at some of the most important built-in functions.

A simple Example

Let’s imagine that we want to make a script for a shopping cart that takes the cost of all the items in your cart, and then adds them together to return a total cost. To write a function that would take two costs, add them up, and then print out the output, we might do something like:

	#our two costs to add up
	cost1 = 15
	cost2 = 20

	def sumCart():
		totalCost = cost1 + cost2
		print totalCost

	sumCart()

To define the function, we need to use the ‘def‘ keyword, and then the name of the function. Next, we type two parentheses (we’ll come back to those later), and then a colon. After that, all of the code that we want to be encased within the function should be indented, just as with if, while and for loops.

To make the code in the function run, we type the name of the function, followed by the two parentheses again. If you run this code, you’ll see it print out ’35′, which is the correct output.

Arguments

That’s great, but at the moment, our functions are a bit rigid – everything about the function is hard-coded into them. For example, let’ say we want to use the sumCart function somewhere else, in a different part of the program, but instead of adding cost1 and cost2 together, we want to add two other costs together that were held in different variables. We’d have to write a whole new function, even though that new function would be exactly the same as the current one, with the names of the variables swapped around – that hardly seems like an efficient solution.

To solve this issue, we use ‘arguments’, and that’s what the parentheses are for. An argument is a way of passing data into a function when we don’t know which variable or variables that data is going to be in. If that’s confusing, let’s take the example that I just referenced. We’ll add two more costs: cost3 and cost4.

Now, we’re going to add two arguments for the two items that we want to add up. Arguments are defined inside the parentheses, and each argument is given a name, with a comma separating each. An argument acts as a temporary reference to the data that you passed in while the function is running.

	cost1 = 15
	cost2 = 20
	cost3 = 5
	cost4 = 10

	def sumCart(item1, item2):
		totalCost = item1 + item2
		print totalCost

	sumCart(cost1, cost2)

When we call the sumCart function, the two variables that we’ve passed in (cost1 and cost2) are put into the arguments item1 and item2. This always happens in the same order you define the arguments – in other words, the first variable you pass in is assigned to the first argument, the second to the second argument, and so on for as many arguments as your function takes.

What actually happens when the function is run is that those two arguments are converted into local variables and are assigned the values of the variables that you pass into the function when you call it – in this case the value of cost1 is put into the local variable item1, and the value of cost2 is put into the local variable item2. This means you can easily use them inside the function.

Another way you could look at it is, when you pass a variable as an argument, everywhere that that argument’s name appears in the function, it is replaced with the variable you passed in. So, in this example, everywhere that item1 is written inside the function, it’s replaced with cost1, and the same with item2 and cost2.

To prove that you can pass any numbers you want, if you run this code, you should now receive the sum of cost3 and cost4:

	cost1 = 15
	cost2 = 20
	cost3 = 5
	cost4 = 10

	def sumCart(item1, item2):
		totalCost = item1 + item2
		print totalCost

	sumCart(cost3, cost4)

Argument Defaults

Note that a function can accept any number of arguments, but remember: when you call the function, you have to pass it the same number of arguments as you defined, otherwise you will receive an error. There is one way around this: you can define a default value for any argument, which is used in the cases when you don’t supply a value.

Considering our current code, let’s imagine that we want the second item to cost 5 if we don’t give the function a different value. It’s not a likely scenario, but it’ll demonstrate the concept.

	cost1 = 15
	cost2 = 20
	cost3 = 5
	cost4 = 10

	def sumCart(item1, item2 = 5):
		totalCost = item1 + item2
		print totalCost

	sumCart(cost1)
	sumCart(cost3, cost4)

If we run this code, you’ll see that, in the first call, the output is 20, which is indeed (cost1 + 5). This doesn’t, however, affect the behavior when both arguments are supplied.

Returning values

There’s one final feature of functions that we’re going to review today: returning values. We’ve learned how to make a function that doesn’t mind what inputs it takes, but what if we wanted to store the answer to ‘totalCost’ in a variable rather than print it out? Perhaps what we want to do is work out the cost of the first two items, and store it in one variable, then do the same with the second two items and store that answer in a second variable. With the current layout of our function, we couldn’t do that in a readable and easy to understand way.

Just like the way we can pass arguments into a function so that we don’t have to hard-code where the data is coming from, we can do the same with the output. I’ll show you with an example – in our previous function, I’m going to replace the word ‘print’ with the word ‘return’:

	cost1 = 15
	cost2 = 20
	cost3 = 5
	cost4 = 10

	def sumCart(item1, item2):
		totalCost = item1 + item2
		return totalCost

	cart1 = sumCart(cost1, cost2)
	cart2 = sumCart(cost3, cost4)

	print cart1
	print cart2

Obviously, the function is no longer going to print out the answer for us – what it’s going to do instead is pass back the value of the variable totalCost to wherever we called the function from. That way, we can use the answer in whatever context we need to. Now, in our previous program, that wouldn’t be particularly useful, because we didn’t specify anywhere to store it, so I’ve also changed the way we call the function.

We need to provide Python with a variable in which to store the answer, and then we’re simply going to set that variable equal to our statement calling the function. When the function returns a value, that value will now be stored in the variable we specified.


Functions – Built-in

We’ve learned how to make our own functions, and how to use them, but there are some operations that are carried out often enough that Python has included them for use in any program. The way you call them is exactly the same as calling your own, but you don’t have to define them first. We’re going to have a brief look at the most important ones now, but be sure be have a look at the Python documentation because there are loads of useful functions that are available for use, which we don’t have time to cover right now.

Refer to the Python documentation for a list of all built-in functions.

str()

First, let’s look at one of the most useful functions in Python: the string conversion function. There are many times in scripting when you have a variable that contains a number, or some other type of data, but you need to convert that value to a string in order to do something with it – normally to print it out onto the screen.

Let’s say we have a number in a variable, and that we want to print it out, with the message ‘The number is ‘ and then the number. If we write:

	number = 10
	print 'The number is ' + number

…then we’re going to encounter an error, because what you’re asking Python to do is to add a number and a string together – which doesn’t really make a whole lot of sense. What we need to do is to convert the number 10 into the string 10. If we do that, Python will understand that what we’re trying to do is not find a sum, but instead concatenate the two strings into one.

This is where the str function comes in. It accepts a value, and returns a string to represent it. In the case you pass it a number, it will simply return the number but in string format.

	number = 10
	print 'The number is ' + str(number)

Now that we’ve encased the number in a string conversion function, the code runs perfectly. The str function doesn’t have to take a number though – it can also take other values, such as a Boolean:

	bool = True
	print 'The value is ' + str(bool)

The str() function finds the closest string match it can find for the value of ‘true’ and returns it, meaning that we can output it nicely.

len()

Another common task for strings is to be able to find the length of them, and, again, Python has a built in function for this. Let’s take the string ‘Hello World’ and try to find its length. To do this, we’re going to need the built-in len() function, which stands for length. All we need to do is pass it the string we want to use, and it will return the number of characters in it:

	string = 'Hello World'
	print len(string)

The len() function is capable of more than this, though. What it actually does is count the number of items in the object you give it – if you give it a string then that’s the number of characters, but you can also give it a list or a tuple, for example, and it will return the amount of items in that particular object.

int()

Moving on, often you are given a number like 10.6 or 3.896, which is not an integer, but you need an integer from them. The built-in function to convert to an integer is called int(), and it works fairly predictably. The function will return the integer conversion of that number. Note that this function does NOT round the input to the nearest integer – it very simply throws away anything after the decimal point and returns the number to you. So the input of 10.6 will return 10, NOT 11. Similarly, 3.25 would return 3.

	number = 10.6
	print int(number)

The int function can also convert a string to an int data type. For example, given this, the output would still be 10:

	number = '10'
	print int(number)

However, be careful, because, when it is converting from a string to an integer, int() will NOT deal with decimal places. If you provide it:

	number = '10.6'
	print int(number)

Then it will throw an error, because the input wasn’t an integer, so it didn’t know how to handle it.

range()

Finally, we’re going to quickly look at the range() function; it comes up surprisingly often when you start doing more complex tasks in Python, and, while it might not seem massively useful at the moment, it’s worth having a look at and getting to know how it works. Range is a function that you use to create a list of numbers following a certain pattern. The simplest example is that perhaps you need a list containing all of the numbers 0 to 10. Instead of typing them out manually, you can use the range function to do it in one line.

Range accepts an integer as the parameter, and, in this case, we’ll pass it 11. The function thens starts at zero, and makes a list counting upwards until it hits one less than the number you put in. So here, if we put in 11, it will store the numbers 0 – 10 in the list numbers:

	numbers = range(11)
	print(numbers)

What if we wanted to print the numbers five to ten instead? Well, optionally, we can include another argument for the starting number, so this will print the numbers five to ten:

	numbers = range(5, 11)
	print(numbers)

Lastly, what if we wanted to print only the odd numbers under 10? Well, range accepts a third value, which is the amount to go up each step. So if we start at 1, set the step to two and the upper limit to 11, we should end up with all the odd numbers under ten:

	numbers = range(1, 11, 2)
	print(numbers)

As a final note, any of these numbers can be negative as well, so if we wanted to count DOWN from 10 to 1 instead of up, we could set the step to be negative 1, the starting value to 10 and the limit to 0, like so:

	numbers = range(10, 0, -1)
	print(numbers)

Note that range will only deal with integers, so make sure that you don’t give it any values that aren’t integers or you’ll throw an error.


Modules – Creating your Own

Alright, we’ve now covered functions; let’s move on to the second topic of today’s lesson: modules. If functions are groups of code, then modules are groups of functions. There’s more to them, but that’s a good way of thinking about them for now.

As we discussed earlier, you can’t make a complex program without sorting code into functions, and as your program continues to grow further, even they become unwieldy. You have to sort them into another hierarchical level. These are modules. So we can summarize them as a structural and organizational tool for our code.

Modules are quite easy to create. They are simply Python files, like your regular scripts. To create a module, write one or more functions in a text file, then save it with a .py extension. Let’s do that now with an example. Open up a new file your text editor or IDE, and make a function. I’m going to continue with the example of a shopping cart from earlier and make a function to calculate tax on the products.

Create a new file in your IDE or text editor and let’s create a function for calculating tax on a product.

	def addTax(price, tax):
		newPrice = price / 100 * (100 + tax)
		return newPrice

If we save this file with a .py extension in the same directory as our other script, we can use it as a module! It’s important to give it a descriptive name so you know what it does when you come back later, so call this one finance.py


Importing modules

To use modules, we can use either the import or the from keyword. import is the simplest and most common, so let’s try that first. You then specify the name of the module, which is simply the filename, without the .py extension. For example, with our finance module in the same folder as our script, we could write:

import finance

…and we would have access to all the functions in the script. We can then call them just as we did before. You can also use the from keyword, which allows you to select specific functions from the library, if you know which ones you require beforehand. For a module that has hundreds of functions, this is recommended – it saves loading time on functions that aren’t even being used.

from finance import addTax

You can import several by separating their names with commas.

from finance import addTax, calculateDiscount

And you can even use the asterisk wildcard to import everything.

from finance import *

After importing the module, to use the functions contained within it, you use the module name, followed by a dot, and then the function name.

	import finance
	print finance.addTax(100, 5)

This should result in 105 when the script is run. A simple result, but it means you’ve got a working function!


Built-in Modules

There are plenty of built in modules, just as there are with built in functions. This is where Python really excels! It takes what’s called the “batteries included” approach.

Because Python has such an extensive range of built-in modules, there’s no way to cover them all in one lesson. Some are quite obscure, and there’s no point teaching a load of information you might never have a use for. Instead, let’s cover several of the most useful ones.

  • random
  • math
  • os
  • datetime
  • urllib2

random

A good one to start with is random , because it’s easy to understand and useful in most scripts you’ll write. As you’d expect, this module lets you generate random numbers. For example, when using Python to create a website, they can be used in making your password database more secure or powering a random page feature. If we wanted a random integer, we can use the randint function, like so:

	import random
	print random.randint(0, 5)

This will output either 1, 2, 3, 4 or 5. Randint accepts exactly two parameters: a lowest and a highest number. As you can see, it includes the highest number but goes from 1 above the lowest number. If you want a random floating point number, you can use the random function:

	import random
	random.random()

…which outputs a random number between 0 and 1, to a load of decimal places. If you want a larger number, you can multiply it. For example, a random number between 0 and 100:

	import random
	random.random() * 100

The random module also has a function for choosing a random element from a set such as a list, called choice.

	import random
	myList = [1, 8, True, 77, "Lorem", 482, "Ipsum"]
	random.choice(myList)

math

The math module provides access to mathematical constants and functions.

	import math

	math.pi #Pi, 3.14...
	math.e  #Euler's number, 2.71...

	math.degrees(2)  #2 radians = 114.59 degreees
	math.radians(60) #60 degrees = 1.04 radians

	math.sin(2)    #Sine of 2 radians
	math.cos(0.5)  #Cosine of 0.5 radians
	math.tan(0.23) #Tangent of 0.23 radians

	math.factorial(5) #1 * 2 * 3 * 4 * 5 = 120
	math.sqrt(49)   #Square root of 49 = 7

There are lots more functions in the module, but these are some of the most useful. You can find a full list here.

datetime

If you need to work with dates and times, then the datetime module is your friend. This one is a very useful one to know about for web development. If you’re making an website with comments, or a blog, then you’ll probably be displaying their age, or the time they were created.

Let’s import the module as usual:

	import datetime

For convenience, you can also import the date and the time components separately, so you don’t have to type as much later.

	import datetime
	from datetime import date
	import time

Let’s have a look at some of the functions it provides:

	time.time() #Returns the number of seconds since the Unix Epoch, January 1st 1970

	date.fromtimestamp(123456789)   #Converts a number of seconds to a date object
	date.fromtimestamp(time.time()) #We can combine the two to get an object representing the current time

	date.fromordinal(10000) #Returns the date a given number of days since January 1st in the Year 1 - 28th of May 0018 in this case

There are some useful functions for converting dates into useable strings. strftime allows you to format a string based on special characters preceded by a percent symbol. %d represents day, for example.

	currentDate = date.fromtimestamp(time.time()) #Create a variable representing the time now
	currentDate.strftime("%d/%m/%y") #Format the date like DD/MM/YY - in this case 09/07/11

	currentDate.isoformat() #Format as an ISO standard date - in this case 2011-07-09

os

The next module we’re looking at is os, which provides functions that allow you to interface with the underlying operating system that Python is running on – be that Windows, Mac or Linux. We’ll focus on the path submodule. It allows you to manipulate and find the properties of files and folders on the system, so it’s the most useful for web development. You’ll often need to manage files, for example user uploads.

	from os import path

	path.exists("/Users/Giles") #Checks if the directory exists, in my case it's True
	path.exists("/Users/Bob")   #False this time
	from os import path
	path.getatime("/Users") #Get the last time the given directory was accessed as a timestamp
	path.getmtime("/Users") #Get the last time the given directory was modified as a timestamp

These timestamps can be converted to useable strings using the datetime module – you can see how you can combine modules.

	from os import path
	path.getsize("/Users/Giles/Desktop/boot") #Returns the size of a file in bytes - for this file it was 314400 bytes, or 314kb

The last function in the os module we’re going to look at is join. It combines two paths into one. You might be thinking: “Why can’t I just concatenate the strings?”, but it’s not as simple as that. On Windows, the path delimiter is a backslash, on Mac and Linux it’s a forward slash. This module alleviates problems like that and makes sure your Pythons scripts work on any system.

	path.join("C:", "Users") #Returns "C:/Users"

urllib2

To finish our tour of Python’s standard library, we’re going to have a brief look at urllib2. This module lets you interface with the web, so it’s obviously quite relevant to us. The most useful function it provides is urlopen, which downloads a page.

You use it as follows:

	import urllib2
	urllib2.urlopen("http://net.tutsplus.com")

You can obviously swap the URL string for any site. This will download the HTML content of the page. This won’t return a string though, so we need to read the data to get that out:

	import urllib2
	urllib2.urlopen("http://net.tutsplus.com").read(100)

This will return the first 100 characters of the HTML source of Nettuts+. Once you have this data, you can sort through it and extract the necessary bits that you require.


Final Tips

If your module is in a different directory than your script, it’s a bit trickier to import them. You have to create a file that tells Python the folder is a package. For example, if you have a folder, called subdirectory, in the same folder as your script, and then your module within that subdirectory, you would have to create a file called __init__.py in the same folder as the module. This file can be empty. Then, from your script, you’d import it:

%review it if you need a refresher!


Video Tutorial

Press the HD button for the clearest picture.
Subscribe to our YouTube and Blip.tv channels to watch more screencasts.

Transcript

In today’s tutorial, we’re going to be looking at functions – what they are, how they work, and how to make your own. We’re also going to be reviewing some of the built-in functions, and how to use them effectively. Finally, we’re going to have a look at creating and importing modules.


Functions – Writing your Own

Functions are an important step to cover, when introducing additional complexity into your programming. If a variable is a named container for data, then a function is a named container for a block of code that can be executed on demand. This is useful if you have a program that executes a certain operation lots of times. Instead of copying and pasting the code to do that operation each section where it’s needed, you can simply write one single function to do it for you.

A function is a named container for a block of code.

There are two types of functions: the ones that you write yourself and include in your code, and the ones that are included in Python natively, which carry out common procedures, such as converting an integer to a string, or finding the length of a string.

We’re going to look at writing a simple function now, and demonstrate how it can be useful in the real world code. Then, we’ll have a look at some of the most important built-in functions.

A simple Example

Let’s imagine that we want to make a script for a shopping cart that takes the cost of all the items in your cart, and then adds them together to return a total cost. To write a function that would take two costs, add them up, and then print out the output, we might do something like:

	#our two costs to add up
	cost1 = 15
	cost2 = 20

	def sumCart():
		totalCost = cost1 + cost2
		print totalCost

	sumCart()

To define the function, we need to use the ‘def‘ keyword, and then the name of the function. Next, we type two parentheses (we’ll come back to those later), and then a colon. After that, all of the code that we w


June 16 2011

14:17

Python from Scratch: Variables, Data Types and Control Structure

Advertise here

Welcome back to Python from Scratch, where we’re learning Python…from scratch! In the last lesson, we installed Python and got set up. Today, we’re going to cover quite a bit, as we learn the essentials. We’ll review variables, operators, and then finish up by learning about control structures to manage the flow of your data.


Video Tutorial

Alternate Source
Subscribe to our YouTube and Blip.tv channels to watch more screencasts.

Variables

Variables are the first thing you should learn in any new language. You can think of them as named containers for any kind of data. The syntax to declare them is: name = value You can name anything you like (except for a handful of keywords), and their values can be any type of data.


Data Types

There are many data types, but the following four are the most important:

Numbers

Numbers can be either integers or floating point numbers.

  • Integers are whole numbers
  • Floats have a decimal point

Strings

String are lines of text that can contain any characters. They can be declared with single or double quotes.

	empty = ""
	escaped = "Can\'t"
	greeting  = "Hello World"
	multiLine = "This is a long \n\
	string of text"

You have to escape single and double quotes within the string with a backslash. Otherwise, Python will assume that you’re using them to end the string. Insert line breaks with \n. Python also supports string interpolation using the percent symbol as follows:

name = "John Doe"
greeting = "My name is %s" % name

You can access sets of characters in strings with slices, which use the square bracket notation:

"Hello"[2] #outputs "l"

Booleans

Booleans represent either a True or False value. It’s important to note that you have to make the first letter capital. They represent data that can only be one thing or the other. For example:

	isMale = True #Could be used in software with a database of users
	isAlive = False #Could be used in a game, set when the character dies

Lists

Lists are used to group other data. They are called Arrays in nearly all other languages. You can create a list with square brackets.

	emptyList = []
	numbersList = [1, 2, 3]
	stringsList = ["spam", "eggs"]
	mixedList = ["Hello", [1, 2, 3], False]

As you can see above, lists may contain any datatypes, including other lists or nothing at all.

You can access parts of lists just like strings with list indexes. The syntax is the same:

numbersList[1] #outputs 2
stringList[0] #outputs spam
mixedList[1][2] #outputs 3

If you nest a list within another list, you can access them with multiple indexes.


Comments

Comments are used to describe your code, in the case that you want to come back to it later, or work in a project with someone else.

#This a comment on it's own line
#You create them with the hash symbol
var = "Hello" #They can be on the same line as code

Operators

You’ve seen operators before. They’re those things like plus and minus, and you use them in the same way that you learned in school.

	2 + 3 #Addition, returns 5
	8 - 5 #Subtraction, returns 3
	2 * 6 #Multiplication, returns 12
	12 / 3 #Division, returns 4
	7 % 3 #Modulo, returns the remainder from a division, 1 in this case.
	3**2 #Raise to the power, returns 9

You can also assign the result of an operation on a variable back to the same variable by combining the operator with an equals sign. For example, a += b is a more concise version of a = a + b

	x = 2
	x += 4 #Adds 4 to x, it now equals 6
	x /= 2 #Divides x by 2, it now equals 3

Control Structures

Once you’ve created and manipulated variables, control structures allow you to control the flow of data. The two types we’re learning today are conditionals and loops.

Conditionals

Conditionals allow you to run different blocks of code based on the value of data.

a = 2
b = 3

if a < b:
    print "Success"

Loops

The two types of loops we’re discussing here are for loops and while loops. for loops work using lists, and while loops work using conditions.

while loops

a, b = 0, 5

while a < b:
	print a
	a += 1

for Loops

myList = [1, 2, 3, 4, 5]

for a in myList:
	print a

Conclusion

That's it for today, but we've covered a bunch of techniques. Feel free to run though everything a few times until it makes sense. I'll try and answer any more questions in the comments, and I hope you'll join me for the rest of the series!

May 05 2011

22:19

Python from Scratch: Getting Started

Advertise here

Welcome to Python from Scratch, where I’m going to teach you the ins and outs of Python development… from scratch.

In this first lesson, we’re going choose a version, install Python, and then create the obligatory “Hello world” script. If you’re already familiar with Python, feel free to skip ahead to a later lesson in the series.


Video Tutorial

Subscribe to our YouTube and Blip.tv channels to watch more screencasts.

Companion Article


Choosing a Version

“It’s important to choose the right version.”

There are two versions of Python that are currently being developed: 2.x and 3.x. It’s important to make the right choice ,because there are several differences between the two. Once you’ve learned one, it can be a bit annoying to have to transition to the other. In this series, we’ll be working through version 2.7.1. You may want to go this route in order to follow along with the videos and articles in this series. That said, most things should work with either version. Version two has much more support from third party libraries, whereas version three has more features, and plenty of bug fixes and refinements.

To make things easier, a lot of features that are being added to version three have also being added to version two, so there’s less need to worry about the differences.


Installing the Interpreter

Once you’ve chosen a version, it’s time to install. Download the version of Python for your OS, and run the installer which will get it set up on your machine. There are three ways you can now use Python:

  • Python Shell- lets you run commands line by line.
  • IDLE GUI – lets you write more complex scripts, and run them in one go.
  • Text Editor – any text editor that runs on you system. You can then save it with a .py extension, and run it from the shell.

For now, launch the shell to test if it works correctly. On Windows, navigate to the directory you installed Python. By default, it should be C:\Python27 Once there, launch python.exe. If you’re on Mac or Linux, launch the Terminal, then type python.

I personally find IDLE to be unpleasant to use; so for most of this series, we’re going to be using a standard code editor. If you would rather use an IDE, there are plenty of good ones:


Hello World!

No we’re all set up; let’s write your first bit of Python! Go to the shell, type print "Hello World!", and hit enter. The phrase Hello World! should appear.

And that’s it: it really is as simple as that. You’ve written your first Python program! Now is a good opportunity to highlight one of the differences beween version two and three: they changed print from a statement to a function. Don’t worry too much about what those words mean for now. All you need to know is that, if you chose to use version three, you need to write print("Hello World!") — the only difference is that you don’t use brackets. We’ll learn about the technical reasons behind this in a future lesson.


Conclusion

So in this lesson, we set ourselves up with a fresh Python installation, discussed the best tools to write code, and wrote our first program. If you have any questions, I’m happy to answer them in the comments, and I hope you’ll join me for the rest of this series.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.
Get rid of the ads (sfw)

Don't be the product, buy the product!

Schweinderl