Python For Programmers - A Project-Based Tutorial
Python For Programmers - A Project-Based Tutorial
PyCon 2013
Sandy Strong & Katharine Jarmul
Introduction to Workshop
Alexandra (Sandy) Strong (@sandymahalo): Systems Engineer at Dreamhost, Python Education, PyLadies
http://bit.ly/YXKWic
Why Python?
Simple to get started, easy for beginners, powerful enough for professionals
Code is clean, modular, and elegant, making it easy to read and edit Whitespace enforcement
Extensive standard library, many modules to import from Cross platform - Windows, Mac, Linux
Google Reddit Twitter Pinterest/Instragram DreamHost YouTube BitTorrent DropBox ...and countless more!
Why a project?
Getting your hands dirty is the best way to develop a new skill! Your efforts in this tutorial will produce a meaningful project, and give you something to show for your work.
Questions so far?
CherryPy:
o o o
Lightweight and easy to use microframework Has direct interface with WSGI It's fun to learn new things and we know you've all used django or flask ;)
Jinja2
o
o o
Commonly used python templating language Easy to grok syntax Integrates with many python libraries
brew link libevent brew install libmemcached sudo pip install pylibmc
Raise your hand if you were not able to install memcached and/or pylibmc successfully!
IPython
? operator %who and %whos %hist %pdb
More details: http://pages.physics.cornell.edu/~myers/teaching/Comp utationalMethods/python/ipython.html
In comparison, compiled languages (like C) must be explicitly translated into a lower-level machine language executable.
Strings
In [1]: my_name = "Sandy Strong" In [2]: print my_name Sandy Strong Now type, "my_name." and hit the tab key: In [3]: my_name. my_name.capitalize my_name.center my_name.count my_name.decode my_name.encode <snip>
strip
o
upper
o
lower
o
split
o
find
o
String formatting
You can pass variables into strings to format them in a specific way, for example:
In [14]: age = 28 In [15]: name = 'Sandy' In [16]: print "Hello, my name is %s." % name Hello, my name is Sandy.
In [17]: print "Hello, my name is %s, and I'm %s years old." % (name, age) Hello, my name is Sandy, and I'm 28 years old.
Lists
[2]: items = ['bacon', 3.14, ['bread', 'milk'] ] In [3]: print items ['bacon', 3.14, ['bread', 'milk']]
You can put lists (and other Python data types) inside of lists.
insert
o
provide index position and item to be inserted into the list append an item to the end of your list
provide index position to "pop" an item out of your list
append
o
pop
o
Tuples
In [12]: colors = ('red', 'blue', 'green') In [13]: print colors ('red', 'blue', 'green')
Tuples are immutable. Once they're created, they cannot be changed. Because they are immutable, they are hashable.
index
o
provide an item in your tuple, and it returns the index position for that item
Dictionaries
In [1]: favorite_sports = {'John': 'Football', 'Sally': 'Soccer'} In [2]: print favorite_sports {'John': 'Football', 'Sally': 'Soccer'}
The first parameter is the "key" and the second parameter is the "value". A dictionary can have as many key/value pairs as you wish, and keys are immutable.
get
o
retrieves the value of the given key or returns None if the key doesn't exist in the dictionary all values in your dictionary
all keys in your dictionary list of 2-element tuples that correspond to the key/value pairs in your dictionary
values
o
keys
o
items
o
update
for while
What defines a code block? Let's try out some loops and find out...
...: print x
...: IndentationError: expected an indented block
Out[6]: True
In [7]: not True == False Out[7]: True For more practice with booleans in Python, I highly recommend the Boolean Practice chapter from Zed Shaw's Learn Python the Hard Way.
if else elif
In Python we use the above keywords to control flows within our program. Depending on which condition is met (evaluates to True) within a block of control flow code, we
Comparison operators
Equal to: == Less than: < Greater than: > Less than or equal to: <= Greater than or equal to: >=
if and else
In [70]: age = 20 In [71]: if age <= 20: ....: print "You're old enough!" ....: You're old enough!
In [72]: if age > 20: ....: print "You're too old!" ....: else: ....: print "You're the right age." ....: You're the right age.
Using elif
The elif keyword allows you to expand your control flow block, and perform additional comparisons within an if-else statement.
In [74]: if age < 20: ....: print "You're too young." ....: elif age >= 20: ....: print "You're the right age!" ....: You're the right age!
We will only add the pet to pet_list if the user is not Bob.
return True
def tire_type(self, type="standard"): self.tires = cars.tire(type)
return True
Other kinds of members, especially methods, enable the behavior of class instances.
http://en.wikipedia.org/wiki/Class_(computer_programming)
user is represented by a model specific user model data is presented by a view on the profile page for that user functionality allowing a user to change his password-- interact with the model-- is handled by a controller
CherryPy Setup
You should already have all libraries installed and working on your computer. We will be creating a Poll web application. You will be able to create and edit polls, add and edit choices, and vote on the poll. The first step is to create a directory on your computer named my-cherrypy-poll: mkdir my-cherrypy-poll
If you do not have git installed, please review the instructions from our tutorial wiki:
https://us.pycon.org/2013/community/tutorials/5/
db_tools.py (continued)
def set_value(key,value): ''' Set a given key/value pair in mc ''' mc = get_mc() mc.set(key,value) del mc return True def del_value(key): ''' Delete a key/value pair from mc ''' mc = get_mc() mc.delete(key) del mc return True
connect to our locally-running memcache instance retrieve values from memcache based on their key set key/value pairs in memcache delete key/value pairs from memcache
We leveraged methods from pylibmc to build our ORM. This library allows Python to easily talk to memcache.
Questions so far?
What is "slugify"?
Suppose a user created a poll named:
"My favorite food is oranges, what is yours? "
Why? Prettier URLs, more SEO-friendly, handy unique identifier for the record in the datastore.
Normalizing unicode
Unicode is great-- it represents a much larger character set than ASCII. However, it can be challenging to work with.
For the purpose of storing slugs in our datastore, it's simpler if we normalize our slug string to only ASCII characters.
This code normalizes the string, replacing unicode characters with their ASCII equivalent, and then encodes the string in ASCII format, ignoring any errors raised during the encoding process:
normalize('NFKD', word).encode('ascii', 'ignore')
For a production web application, you would want to use a more robust method for slugifying your Poll names.
'id':1,
'text':'yep', 'value':0, }, { 'id':2,
'text':'nerp',
'value':0, }, 'publish': True,
models.py
The models.py file in a Python application is often used to define class types and create and inherit model instances. In our application, we are essentially using the theory of this to help manipulate data within our memcache datastore.
We will use our models to edit and create "poll" instances which we update in our memcache storage.
count = 1
poll_dict = {} poll_dict['question'] = kwargs.get('question') for k,v in kwargs.items():
We grab the poll from kwargs, and then if choices are present, we build out a choice_dict of the choices associated with that poll. The poll question gets slugified, and added to the poll_dict, and then a list containing the choices dictionary is added. to poll_dict. We call set_value and pass in the slug (unique key for the poll), and the contents of poll_dict. Finally, if kwargs contains a key named "publish" we publish the poll.
poll_dict['slug'] = slug
poll_dict['choices'] = choices_arr set_value(slug,poll_dict) if kwargs.get('publish'):
publish_poll(slug)
poll_dict['question'] = kwargs.get('question')
for k,v in kwargs.items(): if 'choice' not in k: continue this_choice = [ c for c in poll.get('choices') if int(k.strip('choice')) == c.get('id') ] if not len(this_choice): return False else: this_choice = this_choice[0] choice_dict = { 'id': this_choice.get('id'), 'text': v,
'value': this_choice.get('value'),
} choices_arr.append(choice_dict)
edit_poll (continued)
slug = str(kwargs.get('slug')) poll_dict['slug'] = slug poll_dict['choices'] = choices_arr set_value(slug,poll_dict) if kwargs.get('publish'): publish_poll(slug) else: unpublish_poll(slug)
return poll_dict
The edit_poll function and add_poll functions have some similarities. They start to differ with the list comprehension used to assign this_choice. We build out this_choice using a conditional to validate that the id for the choice passed in is equivalent to the choice id on the poll object from memcache. From there, we build out the choice_dict , get slug from **kwargs , add
data
from db_tools import get_value, set_value, del_value from utils import slugify def cast_vote(poll_key,choice):
poll = get_value(poll_key)
for c in poll['choices']: if c['id'] == int(choice): c['value'] += 1
set_value(poll_key,poll)
return poll
This is the view that allows visitors to vote on a poll. When executed, it accepts the poll_key (its unique identifier), and the choice that the user selected. The poll is retrieved from memcached, validated, and it's
views.py (continued)
def get_global_links(): return [{
'name': 'Home',
'url': '/' }, { 'name': 'Poll list', 'url': '/polls', }]
This is essentially our site navigation. We return a list of dictionaries, each one containing a pretty name for the view, along with the url path where the view is located within our application.
The view to "Add a poll" is mapped to the "/polls/add" url.
views.py (continued)
def get_poll(key): poll = get_value(key) return poll
Retrieve a specific poll from memcache, based on the key-- the slug-- for the poll.
views.py (continued)
def get_polls(): poll_list = [] published_polls = get_value('published_polls') if not published_polls: set_value('published_polls',[]) for p in published_polls: poll = get_value(p) poll_list.append(poll) return poll_list
Return the value from memcache with the key 'published_polls', loop through each poll id (slug), and retrieve the poll object for that id. Build a list of poll objects, and return them.
views.py (continued)
def publish_poll(key): published_polls = get_value('published_polls') if not published_polls: set_value('published_polls',[]) if key not in published_polls: published_polls.append(key) set_value('published_polls',published_polls)
For a given poll id (key), add it to the list of published_polls in memcache if it is not already there. Notice how we're using if-statements to perform validation on published_polls and the individual key we're attempting to add to the published_polls list.
views.py (continued)
def unpublish_polls(key): published_polls = get_value('published_polls') if not published_polls: set_value('published_polls',[]) if key in published_polls: published_polls.remove(key) set_value('published_polls',published_polls)
For a given poll id (key), remove it from list of published_polls in memcache, if it is currently published. Note again how we perform validation.
So, who has questions? ...I know someone must have a question!
... c'mon...
env tells our app where to find templates at, and conf tells our app where to find the app.conf file.
templ = env.get_template('polls.html')
return templ.render(data_dict) First, we define a class named PollViews. Then we write our index function, the default page for our app. There are some interesting things here, @cherrypy.expose, env.get_template, and templ.render. Everything else is pretty
We pass our data dictionary-- the attributes of our poll model that will be exposed in our view-- into the render method of our template object, templ.
You will observe these same patterns in other methods within the PollViews class.
data_dict['poll'] = cast_vote(key,choice)
data_dict['success'] = True else: data_dict['poll'] = get_poll(key)
templ = env.get_template('poll.html')
return templ.render(data_dict) A little more CherryPy internal magic. Then, if the request is POST, we submit a vote to cast_vote. otherwise, we retrieve the poll to display for the user, get_poll. We then render the data_dict to our poll.html template.
}
if method == 'POST': add_poll(**kwargs) data_dict['success'] = True
templ = env.get_template('add_poll.html')
return templ.render(data_dict)
This lets us add a poll to our datastore. If method is POST, we pass in **kwargs to add_poll, then we render the contents of our data_dict out to the add_poll.html
method
@cherrypy.expose
def edit(self,key,**kwargs):
method = cherrypy.request.method.upper() poll = False data_dict = { If method is POST, we pass **kwargs into edit_poll, then we add poll to our data_dict. Otherwise, we call get_poll with the key passed in to edit, and add that to our data_dict.
poll = edit_poll(**kwargs)
data_dict['poll'] = poll data_dict['success'] = True else:
Finally, we render our data_dict to data_dict['poll'] = get_poll(key) templ = env.get_template('edit_poll.html')edit_poll.html. return templ.render(data_dict)
def index(self):
data_dict = { 'title': 'Welcome to the poll application!', 'links': get_global_links(),
}
templ = env.get_template('index.html') return templ.render(data_dict) polls is a member variable of the PollApp class, and it is an instance of the PollViews class. The PollApp class has one method, index. This method has a simple data_dict, which is rendered out to index.html.
Congratulations! You've completed writing your Python code for this app!
App Homework!
Raise an error if a user enters bad data into the form Use the unpublish_polls function in the application to properly unpublish polls Use CherryPy Sessions to validate that each user can only vote once
http://docs.cherrypy.org/dev/refman/lib/sessions.html
Allow users to add more than 4 choices to a poll using a more dynamically generated form
add_poll.html
<html> <head> </head> <body>
<h1>{{title}}</h1>
<h2>Add a poll</h2> <form action="/polls/add" method='POST'> Question: <input type="text" name="question" value=""><br>
{% endfor %}
edit_poll.html
<html> <head></head> <body> <h1>{{title}}</h1> <h2>Edit your poll</h2> <form action="/polls/edit/{{poll.slug}}" method='POST'> Question: <input type="text" name="question" value="{{poll.question}}"><br>
{% for c in poll.choices %}
Choice {{c.id}}: <input type="text" name="choice{{c.id}}" value="{{c.text}}"><br> {% endfor %} Publish? <input type="checkbox" name="publish" checked="{% if poll.published %}checked{% endif %}"><br> <input type="hidden" name="slug" value="{{poll.slug}}"> <input type="submit" value="Edit poll"> </form> {% for l in links %} <span><a href="{{l.url}}">{{l.name}}</a></span> {% endfor %} </body> </html> |
index.html
<html> <head> </head> <body> <h1>{{title}}</h1> <p>Welcome to the polls app. Feel free to add a poll or answer some questions! :)</p> {% for l in links %} <span><a href="{{l.url}}">{{l.name}}</a></span> {% endfor %} </body> </html> |
poll.html
<html> <head></head> <body> <h1>{{title}}</h1> <h2>Poll: {{poll.question}}</h2> {% if success %} <p>Thanks for voting!</p> <h3>Results</h3> {% for c in poll.choices %} <p>{{c.text}}: {{c.value}}</p> {% endfor %} {% else %} <form action="/polls/poll/{{poll.slug}}" method='POST'> {% for c in poll.choices %} <input type="radio" name="choice" value="{{c.id}}">{{c.text}}<br> {% endfor %} <input type="submit" value="Vote"> </form> {% endif %} {% for l in links %} <span><a href="{{l.url}}">{{l.name}}</a></span> {% endfor %} </body></html> |
polls.html
<html> <head></head> <body> <h1>{{title}}</h1> <h2>Polls</h2> <ul> {% for p in polls %}
<li><a href="/polls/poll/{{p.slug}}">{{p.question}}</a></li>
{% endfor %} </ul> {% for l in links %} <span><a href="{{l.url}}">{{l.name}}</a></span> {% endfor %} </body> |
Reference variables passed into our template with double-braces: Ex. {{ title }}
Reference attributes on variables passed in to the template: Ex. poll.choices Use for-loops: for ... endfor Use control flow structures: if ... endif
python app.py
NOTE: We are running a development server. This method is not suitable for a production environment.
Testing
(/me facepalms)
Writing Tests
Unfortunately...
... we don't have time to go into testing in-depth during this tutorial. We want you guys to come away with:
Understanding of why writing tests is so important A basic understanding of how tests might be written Resources that inspire you to write your own tests!
class BlogLogin(object): def __init__(self, login, password): self._blogin = Blog(login, password) def my_subscriptions(self): return self._subs(self._blogin.subs(self._blogin.login) def _subs(self, call): crs = -1 results = [] while crs != 0: page = call(crs) subscriptions = page['subscriptions']
crs = page['next_cursor']
results.extend(subscriptions) return results
bl = BlogLogin('username', 'password')
bl._blogin = Mock() bl._subs.blogin.subs.return_value = \ {subs:[{'a':1}], 'next_cursor':0}
result = bl.my_subscriptions()
bl._blogin.subs.my_subscriptions.assert_called_with( cursor = -1) self.assertEquals([{'a':1}], result)
Testing homework:
Your app should be well-tested! After PyCon, go home and write a test suite using unittest and mock, you can find great documentation here:
Python unittest Documentation o Mock Library Documentation
o
Errors
(FML)
Syntax errors
o
Exceptions:
o o
Catching exceptions
try: # Try to get "my-first-poll" from memcache return get_value('my-first-poll') except KeyError: # If the key is not found in memcache, # return an empty dictionary instead return {} Catching exceptions is useful when you can anticipate what exception might occur, and have the application do something else instead of raising the exception. In this case, if our key is not found, we want our app to return an empty dictionary.
Raising exceptions
If you do not attempt to catch an exception, then Python will raise it automatically. There are many cases where you can catch an exception and mitigate the negative impact for users. However, there are some exceptions that you can never programmatically predict-- and those bubble up into your application error
Exception homework:
Go through your completed my-cherrypy-poll app and find places that are missing exceptions. Put in your own try/catch blocks, and then try writing your own custom exception classes:
o Python Built-In Exceptions o Creating User-Defined Exception Classes
...:
...: ...: ...:
You can find the full pdb documentation here. Have fun, and happy bug hunting!
Use them to handle exceptions or proper situation handling (sorry, I can't find what you're looking for: 404. sorry, you can't see that: 403)
Conclusions
(what did we learn?)
Whew!
Today we gave you a whirlwind tour of basic Python, and we're hoping it's enough to get you guys hooked!
We covered data types, loops, control flow statements, conditionals, functions, classes, testing, error handling, debugging, and more!
IRC:
o
Cool stuff:
o
Github
Questions?
Thank you so much for attending our tutorial, we hope you enjoyed it!