Conference Notes and Best Practices Release 1.3 ... - Read the Docs

Conference Notes and Best Practices 

Release 1.3 

Derek Stegelman 

January 04, 2014

Contents 

1 Meetups 3 

1.1 Django Deployment with Salt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 

1.2 Django Class Based Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 

1.3 Custom User Model/Django Auth Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 

2 Notes 9 

2.1 Django Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 

2.2 Django Snippets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 

2.3 Django Class Based Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 

2.4 Python Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 

2.5 Selenium Testing with Django . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 

3 Conferences 13 

3.1 Django Con US 2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 

3.2 PyCon 2012 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 

3.3 Django Con US 2012 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 

i

Conference Notes and Best Practices, Release 1.3 

Below is my collection of notes on general Django/Python development as well as conference and meet up notes. 

Contents 1


2 Contents

CHAPTER 1 

Meetups 

1.1 Django Deployment with Salt 

Presenter: Nick Lang 

Description: 

Django deployment with Salt 

1.1.1 Background 

• Developer at Lab 305 

• Worked at Journal World 

1.1.2 What is Salt 

• Config management in Python similar to Chef/Puppet 

• Remote code execution 

• Awesome! 

• salt.readthedocs.org 

• Good install docs 

Master & Minion 

• Master server 

• Minions ping masters for updates or can be pushed to by Master 

• Has states 

• Masters can also be minions 

• Minion can be db server, app server, mail server, etc. 

3


Master Configuration 

• Specify backends 

• Point it at a Git Repo! FTW. 

• Quick updates 

Minion Config 

• Just connect to master 

• Salt key to connect 

1.1.3 Quick Dive into Config 

Salt State 

• Packages to install and versions 

• Files 

• Symlinks 

• Databases 

• Web Servers 

• Fine grain control 

• Create users/groups 

Remote Code Execution 

• Deploy scripts 

1.1.4 Vagrant Integration 

• Salty vagrant 

• Use salt to configure Vagrant 

• Pre-seed salt master with the key of your vagrant vm 

• Demo 

1.2 Django Class Based Views 

Presenter: Nick Lang 

Description: 

Class Based Views 

4 Chapter 1. Meetups


1.2.1 Background 

• Developer at Lab 305 

• Worked at Journal World 

1.2.2 Overview 

• Class version of generic function views. 

• Base, Dates, Detail, Edit, List 

• CRUD Views 

• Subclassed (Mixins) 

1.2.3 Mixins 

Object with methods that can be overridden to customize behavior. 

• ContentMixin 

• TemplateResponseMixin (Adds context variables) 

• Date Views, Many. 

1.2.4 Detail Views 

• Single object 

• Primary key or slug 

1.2.5 Mixins 

• SingleObjectMixin 

• SingleObjectTemplateResponsemixin 

1.2.6 Crud Views 

• Create/Read/Update/Delete 

• Form View/CreateView/UpdateView/DeleteView 

Mixins 

• FormMixin 

• MomdelFormMixin 

1.2.7 List view 

• Mixins - MultipleObjectMixin 

1.2. Django Class Based Views 5


1.2.8 Base View 

• ContentMixin and TemplateView 

• Nav item names 

1.2.9 Why 

• Custom mixins 

• Elegance 

• can push constants to the template by declaring class properties 

• Lots of possibilities for mixins. 

• Django Braces 

1.3 Custom User Model/Django Auth Tools 

Presenter: Aaron Merriam, Rocky Meza 

Description: 

Django Auth Tools 

FusionBox 

1.3.1 Custom User Models 

• Supported in Django 1.5 

• MOst apps don’t use a username (use Email instead) 

• Not the correct pattern 

• More control 

• Adding custom methods and properties 

1.3.2 Adding A Custom User Model 

• Must implement methods and custom manager 

• Custom user admin won’t work 

• Lots of custom code needs to be written 

• Lots of boilerplate 

• Custom login logic 

• Custom password reset logic 



1.3.3 Django Auth Tools 

Custom user app 

• Add to installed apps. 

• Already implemented the required methods 

• AbstractBaseClass 

• Class based views so methods can be drop in replaced. 

• Password reset confirm and login in one view. 

• Generic forms 

• Generic model admin classes. 

• Email as username, but others.. 

Migrating 

• Take over the table. Migrations still needed. 

• Django’s email field on user does not enforce a unique constraint 

• 3rd party applications aren’t all ready. 

Storing Additional Information for a User 

• Don’t store user information in the new user model. 

• Mixes profile code with authorization and authentication. 

User Profiles 

• Just make a relationship between the profile and the user object. 

• Profiles for users in each app/project. 

• Idea is to package user profile info into an app so it can be re-used. 

Dos Don’ts 

• use get_user_model instead of django.contrib.auth.models.user 

• Don’t point them directly at the user model 

• models.ForeignKey(settings.AUTH_USER_MODEL) 

1.3. Custom User Model/Django Auth Tools 7



CHAPTER 2 

Notes 

2.1 Django Resources 

2.1.1 Common Libraries 

• south 

• sorl-thumbnail 

• tastypie 

• fabric 

• boto 

• MySQL extensions 

• django-taggit 

• factory_boy 

• django-crispy-forms 

2.1.2 Talks/Learning 

• http://www.youtube.com/watch?v=A-S0tqpPga4 

• http://djangoproject.com 

2.2 Django Snippets 

2.2.1 Pagination 

Inside the views: 

Templates: 

9


{% if is_paginated %} 

 

 

{% if page_obj.has_previous %} 

Prev 

{{ page_obj.previous_page_number }}


urlpatterns = patterns(’’, 

) 

url(r’^posts/$’, PostListView.as_view(), name="post_list_view"), 

url(r’^posts/(?P[-\w]+)/$’, PostDetailView.as_view(), name="post_detail"), 

2.3.3 Mixins 

Mixins can easily be added to class based views to extend and inherit common functionality: 

from django.utils.decorators import method_decorator 

from django.contrib.auth.decorators import login_required 

from django.core.urlresolvers import reverse_lazy 

from kstate.common.auth.decorators import group_required 

STAFF_LOGIN = reverse_lazy("login") 

class StaffRequiredMixin(object): 

""" 

View mixin for the staff application. 

Requires someone to be a staff member. 

""" 

@method_decorator(login_required(login_url=STAFF_LOGIN)) 

@method_decorator(group_required("Staff")) 

def dispatch(self, *args, **kwargs): 

return super(StaffRequiredMixin, self).dispatch(*args, **kwargs) 

class AdminRequiredMixin(object): 

""" 

View mixin for staff app. Required admin group. 

""" 

@method_decorator(login_required(login_url=STAFF_LOGIN)) 

@method_decorator(group_required("Admin")) 

def dispatch(self, *args, **kwargs): 

return super(AdminRequiredMixin, self).dispatch(*args, **kwargs) 

2.4 Python Resources 

2.4.1 Libraries 

2.4.2 Learning 

2.5 Selenium Testing with Django 

2.5.1 Quick Start 

Install python selenium: 

2.4. Python Resources 11


pip install selenium 

Create a new Test case: 

from django.test import LiveServerTestCase 

from selenium import webdriver 

from selenium.webdriver.support.ui import Select 

import time 

class SomethingAppTest(LiveServerTestCase): 

fixtures = [’groups.json’] 

def setUp(self): 

self.browser = webdriver.Firefox() 

# Use factories to create objects 

def tearDown(self): 

self.browser.quit() 

def test_something(self): 

self.browser.get(self.live_server_url) 

self.browser.find_element_by_id(’clickme’).click() 

2.5.2 Testing Tid Bits 

• It’s a good idea to prime your test cases with data. Either prime the test case with a fixture or better yet, an 

object factory. 

12 Chapter 2. Notes

CHAPTER 3 

Conferences 

3.1 Django Con US 2013 

Location: Hyatt Regency Chicago, IL 

When: September 3rd–6th 

URL: http://www.djangocon.us 

Schedule: http://www.djangocon.us/schedule/ 

Description: 

DjangoCon US is the main opportunity for djangonauts to come together in the United States. It will 

consist of two tracks of talks over three days, and will also provide for open sessions, lightning talks, and 

a development sprint after the conference. 

3.1.1 Tuesday 

Keynote 

Presenter: Russell Keith-MaGee 

Track: N/A 

Description: 

State of the Pony 

Malcolm Tredinnick Memorial Award 

• Monetary award. 

• DSF asking for donations 

• Email foundation@djangoproject.com 

13


State of the Pony 

DSF Mission 

• Support Development 

• Promote Django 

• Protect IP 

• Advance state of the art 

Support Development 

• Sponsored Events 

• Conferences (DjangoCon AU, Kiwi PYcon, PyCon YU, RuPy, PyArkansas) 

• Sprints (Stockholm, LA, Toulouse), DjangoCon US. 

• Grants (Getting started with Django, Travel Grants, Startup Capital) 

• Grants committee 

• Show corporate members (lots) 

Promote Use of Django 

• djangoproject.com -> not finished 

• Platform to show Django success 

• Code of Conduct now required for sponsorship 

• Community code of conduct for Django (djangoproject.com/conduct/) 

Protect IP and Long Term Viability 

• Copyrights (BSD) 

• Need to sign contributor license agreement to contribute to Django 

• Trademarks 

• Trademark licensing agreement (Describes how and when you can use it in a community.) 

** Merchandise ** Groups Events ** Domain Names * Djangoproject.com/trademarks/ * Exemptions 

Advance the State of the Art 

• Django 1.5 (Initial python 3 transition) 

• Django 1.6 in beta 

• Persistent DB Connections 

• Transaction improvements 

• Django 1.7 (Schema migration, 2013 Google summer of Code, validate refactor, composite foreign keys) 

14 Chapter 3. Conferences


Technical Challenges 

• “Real Time” web 

• Nothing in the box that will help you achieve this. 

• Rich client interfaces 

• Attract new users 

• Positive first experiences 

• Limit wasted effort 

• Improve development tools and processes 

• Funding open source 

• Hire a “django fellow” to work full time on Django. Manage the community. Funded by the DSF. 

https://speakerdeck.com/freakboy3742/djangocon-2013-state-of-the-pony 

Django Docker 

Presenter: Ken Cochrane 

Track: I 

Description: 

Docker is a new tool that makes it easy to bundle and deploy your applications to any host running Docker. 

In this talk I’ll show you how to get started with Docker, and how to use it to deploy your Django apps to 

your different server environments. 

Django Docker 

Who 

• Corporate Sponsor of Docker (Dot cloud) 

Survey 

• 80% heard of docker 

Where did Docker Come From 

• Docker is a rewrite of similar code that powers DotCloud 

• Original version in Python, new version in GO 

• Very Young project 

Timeline 

• Jan 2013 internal project 

• March PyCon talk 

• Released March 27 

• June add OpenStack comptatibility 

3.1. Django Con US 2013 15


• 5300 Github stars 

• 125 Contributers 

What Is Docker 

• Docker is an OSS engine that automates deployment of containers 

• Linux containers 

• Control groups and Namespaces 

• AUFS (file system) 

Linux Containers 

• Mini VMs 

• Stacked on top of another LInux system 

• Like a vm but very light weight 

Why Containers 

• boot in seconds 

• 1000s of containers on single machine 

• Containers all use same host OS 

• Share bin/libs 

• No guest OS. 

• Layered approach for file system 

• Build on top (diffs) 

Install Docker 

• Linux kernel 3.8 or above 

• AUFS 

• LXC 

• 64 Bit 

• (Ubuntu 13.04) 

• Vagrant 

• Docker APT repo 

Vagrant 

• Clone and vagrant up 



Binary Install 

• Manage upgrades yourself 

• Need to install system startup script. 

• 5 easy steps for Digital ocean 

Digital Ocean Install 

• Docker install in one step for Digital Ocean (Official docker image) 

• 10$ free credit DJANGOCON2013 tinyurl.com/docker10 

Use Cases 

• Local Dev environment 

• Deployment 

• Unit Testing 

• Parallelize tests 

• one DB per test 

Unit Testing 

• Containers to isolate tests 

• No more worrying about tests not cleaning up 

• Parallelize the tests across multiple machines 

System Tests 

• Easily create all the diff system configs to test against 

• No need to worry about breaking or rebuilding a test server 

• Test fabric scripts 

• Agileq.com/blog/ 

Continuous Integration 

• run tests after each commit 

• StriderCD.com open source CI server 

• Travis CI also playing with Docker 

Deployment 

• Dokku 

• flynn.io 

• deis.io 

• chef, puppet, salt, ansible, etc. 



Dokku 

• Open source 

• Docker powered mini-heroku 

• less than 100 lines of bash 

• Heroku build packs 

• git push deployment 

Flynn.io 

• Open source PASS written in Go 

Deis.io 

• Python 

• Git push 

• Docker images, chef recipes 

• Scaling 

Other Projects 

• Chef-docker 

• chef-cookbook 

• Salt stack 

• Ansible 

Local Dev 

• VMS heavy, containers not so much 

• RUn 100s of containers on laptop 

• Easy to duplicate prod environment if you have a complex setup 

Projects using Docker 

• Node.js module testing 

• Plone/jiffylab - web based enviroment for instruction 

• Kitchen-docker - Run unit tests in isolated environment 

• npmt.abru.pt - Auto testing all NPM modules, one container per module and destroyed when finished. 

• memcached SAAS - Memcached SASS built on Docker 

• Try out Rethink DB, containers killed in 24 hrs. 1000s of containers on one host 

• Open-stack-docker, deploy to linux containers instead of VMs 



Use Docker 

• Container: linux container 

• Image: snapshot 

• index: public docker image directory 

• Dockerfile: auotmated script used to create an image 

• push/pull : commands to get images and push them 

• Run: start a docker image to run 

• Docker run (start image with commands) 

DockerFile 

• Simple scripting language 

• Automate creation of images 

• built in cache 

• Add them to any project repo to dockerize the project 

• Online tut - docker.io/learn/dockerfile 

Docker Index 

• Similar to Pypi but for docker images 

• Written in Django 

• Public directory to store and download re-useable images 

• Docker image meta data 

• Account required to publish images 

• index.docker.io 

Docker Registry 

• Open source python flask app 

• Manages the storages of the images 

• Install private registry for private images 

Docker API 

• rest API 

• Docker CLI uses the same API 

• Clients for most languages 

• Docker clients (docker-py) 

• Docker UI (Shipyard) Docker UI (angular.js) Dockland Ruby 

Demo (https://github.com/kencochrane/django-docker) 



Scaling Your Write-Heavy Django App 

Presenter: Tobias McNulty 

Track: II 

Description: 

Notes: 

Content management systems and other read-heavy Django apps are relatively easy to scale. Scaling 

write-heavy apps is another matter. In this talk I’ll walk through our experience scaling the University of 

Chicago’s custom school survey application to handle over 75,000 requests per minute and upwards of 

9,500 PostgreSQL transactions per second. 

1. Project Overview 

(a) 5Essential survey module for UChicago 

(b) Not about making app fast but scalable, i.e. multiple users at once 

2. Scaling Phase 1: Chicago Public Schools 

(a) Main issue is bottle-neck entering data to DB 

(b) Need an easy way to generate load ... JMeter 

(c) Record yourself testing a complex process 

(d) HTTP Cookie Manager built into JMeter 

(e) pgfouine –> log postgres performance 

(f) django-cache-machine for specific caching 

(g) Choose what you cache via ‘cached =’ property on a model 

(h) use read database to avoid load on write DB 

(i) Streaming replication hit PG in 9.1 – try django-balancer 

3. Scaling Phase 2: The State of Illinois 

(a) gevent worker terrible for CPU-bound applications 

(b) NewRelic makes I/O looks expensive, but each worker is processing too many reqs at once 

(c) Use a sync worker in gevent and it will open up the CPU bottleneck 

(d) Database was the bottleneck, still overloaded 

(e) Increased size of EC2 instance, still slow 

(f) Figuring out max_connections: not web server count ... Machine resources 

(g) Use pgbouncer to share a small number of presistent connections 

(h) Run pgbouncer on your web servers using supervisord 

(i) Don’t need max_conn to be so low, but know what you’re changing when you change that 

4. Slides: http://cakt.us/djangocon-scaling 



Performance Optimization 

Presenter: Joseph Jasinski 

Track: II 

Description: 

Your site is slow. But why is your site slow? There are a myriad of different problems that can cause 

this slowness. Drawn from real world experiences, this talk will help identify different problem areas and 

techniques for increasing performance. This involves both evaluating your performance on the backend 

and understand bottlenecks on the frontend. 

Performance Optimization 

Front End 

• Large and unoptimized payloads 

• Static medai/images 

• slow load of CSS 

• Slow third party resources loading 

• Blocking IO 

• Chrome Tools 

• External Analyizers (Google pagespeed insights) 

• Browser Plugin 

• Pingdom speed tools (DNS checker) 

• Yslow browser plugin 

Backend 

• SQL Quantities 

• Blocking code 

• Django Debug Toolbar 

• Extending Django Debug toolbar (Cache panel & template timings) 

• See what blocks take the longest 

• Profiling middleware 

• Get information on calls. 

Improve Front End 

• Remove comments/whitespace 

• HTML Minification 

• Django=htmlmin 

• Combine and compress css/js 

• django compressor 



• Compress images/Cache 

• Image sprites/logos/icons 

Resource Order 

• Load first styles in critical path 

• Place JS after other resources, ideally at the end of the file. 

• Inline some CSS at the top. 

• Lazy loading (Load images/assets only if the user gets there) 

Assets CDNS 

• Geographically serve assets 

• Improve load time 

Improve Backend 

• SQL Queries 

• Use “values_list” 

• Verify same queries aren’t running multiple times 

• Select_related and prefetch_related 

Select Related and Prefetch Related 

• Get foreign keys 

• If getting foreign keys use select_related 

• Prefetch related can work for many to many queries 

Cache 

• How should I cache? 

• Memcache 

• Use low level cache? 

Low Level 

• Flexibility 

• Reduce lookups 

• get() set() delete() methods 

• Flexibilty to develop your own cache scheme 

• Template fragment cache 

• Can accept context variables 

• example (Get cache template for different users) 



Per Site Cache 

• Great for heavy read sites 

• Not really approprate for immediate change sites 

• Difficult to delete cache backend 

• Google analytics can have issues 

Cache Frameworks 

• Johnny Cache 

• Cache Machine 

• ORM Model cacheing 

CDNs 

• Browsers support 6 connections per hostname 

• More maximum connections 

• Hashring with CDN domains 

• Each domain is a seperate DNS lookup. 

• Limit what you need to do inside of a request. 

• Defer 

Job Queue 

• Process queues off line 

• Requires more configuration 

Server Level 

• Set expires header 

• Cache control headers 

• Gzip responses 

More Requests Less Money 

Presenter: Nick Catalano 

Track: I 

Description: 

Hear how in under 2 weeks Ain’t it Cool News, a movie news and review website with over 15 years of 

raw HTML content and hundreds of thousands of daily visits, was moved from an expensive custom Ruby 

on Rails App on an EC2 cluster to a dedicated server running Django. All with zero downtime. 



More Requests Less Money 

History 

• Old, Old, HTML 

• Perl, Drupal, Rails 

Problem 

• Ads were gradually going down. 

• AWS bills were growing 

• Active scaling horizontally 

• Had to pay licensing fees for using software 

Realizations 

• Can’t offer commenting as well as Disqus 

• OK if articles dont’ appear right way for entire world as long as it eventually shows up 

• A little down time is manageable 

Solution 

• Managed dedicated server behind Amazon cloudfront 

• Use disqus 

Arch 

• Old arch, multiple high=cpu medium instances, extra large RDS instance 

• Over 3000/month 

New Arch 

• Dedicated server, 16gb RAM, mysql, SSD, Apache Modwsgi 

• Cloud instances for forums and domains redirects 

• Disqus 

• Additional cloud instances 

• Managed operations services 

• 1325/month 

Migrating Content 

• 60000 raw html articles 

• 4.5 million non spam comments 

• SQL dump and public facing templates only way to get content 



Inspect DB 

• Based on SQL dump 

• set production DB as secondary DB 

• Rebuilt the site using view source files and simple django views 

Migration 

• Left talkbacks alone 

• Built a management command to migrate articles from secondary database first 

• Get or create runs without duplicate data on both machines 

• Management command also stripped out html to generate meta tags 

• Find and replace for image URLs and handling weird unicode in DB 

• Simple cleanup with beautiful soup 

Cloud Front 

• HTTP POST is not supported 

• Do pay for requests 

• Bring your own analytics 

• No full site purge 

• Do not assume every cloudfront request will have a cloudfront user agent. 

• Cloudfront will not continue to serve your site indefinitely if your origin goes down. 

Scaling Up 

• Scaled from 10,000 visits/hour to 90,000 visits/hour. No downtime 

http://bit.ly/aicndjangocon 

Getting Started with Salt 

Presenter: Peter Baumgartner 

Track: II 

Description: 

Salt is the new kid on the block in the configuration management space. Unlike the Ruby=based Chef and 

Puppet, Salt is written in Python, making it easy to debug and extend for Django developers. This talk 

will introduce Salt as well as explore some of the things that make it unique. 

Getting Started with Salt 

What is Salt 

• Configuration Management 

• Remote execution 



Configuration Management 

• Similar to Chef/Puppet 

• Ansible 

• Configuration through code 

• Version control your servers 

• Self documenting 

• Repeatbale 

• Reuseable 

Remote Execution 

• Run commands against remote servers 

• Similar to Fabric 

• Deploying 

• Run one=off scripts 

• Package updates 

• System monitoring/alerting 

Familiar Tool 

• Python 

• YAML 

• Jinja2 

Community 

• Great docs 

• Responsive to IRC and Github 

• Backed by for=profit org 

Why Not 

• Young project 

• Moves Fast 

• Not SSH (ssh support soon) 

Terminology 

• Chef, knife, cookbook 

• Ansible, playbook, inventory 

• Salt = Master 



Salt Terminology 

• Master = Server that manages the whole stack 

• Minion = A server controlled by master 

• State = A declaritive Representation of the system state 

• Grain = Static information about a minion (RAM, CPU cores, OS, etc) 

• Pillar = Variables for one or more minions 

• Top File = Matches states or pillars to minions 

• High state = All the state data for a minion 

Install 

• pip install for bleeding edge 

• bootstrap.saltstack.org 

• apt=get install salt=master 

• apt=get install salt=minion 

• Accept minion key on the master 

Install a package 

In /srv/salt/mystate.sls 

ngingx: 

pkg.installed 

High State 

• Push from master 

• Pull from minion 

• Masterless 

States 

• 50 Built in states 

• Build your own 

• pip, virtualenv, mysql, postgres, files, cron 

Using Pillars 

• Code examples 

• Can use templating language to configure pillar. 



Advanced 

• Salt cloud 

• Custom modules 

• Scheduler 

• Renderers 

• Returners 

Tips and Tricks 

output_mode: mixed 

• Show me full traceback if error, only respond verbose with errors 

• Jinja2 is powerful, don’t go nuts. 

• Update often and review the change log 

• Test before you deploy 

https://speakerdeck.com/ipmb/getting=started=with=salt 

Finding The Needle: Search and Django 

Presenter: Ben Lopatin/Wellfire Interactive 

Track: II 

Description: 

Most websites share at least two things in common: content and users who want to find content. This 

talk will cover the basics of site search using search engines (the kind you run, not Google), setting up 

a Django project with Haystack, highlight some of the gotchas you might encounter, and touch on some 

more advanced functionality. 

Finding The Needle 

Overview 

• Understand Search 

• role of search engine 

• Nifty search features 

• Adding search with Haystack 

• Implementation Strategies 

• Limitations and Options 

Search Problem 

• Trying to search text content 

• Distinction between searching for and looking for. 

• Trying to find information 



Search Engines 

• Stop words (remove common words) 

• Indexing tokens 

• Document data store based on filtered tokens. 

• ElasticSearch/Solr/Whoosh/Xapian/Sphinx 

• What about SQL full=text indexing? = Can do it, won’t get as many features as a search engine. 

Data in/Data out 

• Analyzers = Tokenizer + Filters 

• Tokenizers 

– Whitespace/N=grams/word delimters 

• Filters 

– ASCII/Stemming/Lowercase/stop words/synonyms 

• Language specific filters. 

• Querying (Data Out) 

• Match tokens against tokens 

• Faceting = Characteristics of a set. 

• Spell checking 

• Geospatial search 

• Autocomplete 

Django/Haystack 

• Haystack is a pythonic abstraction 

• ORM Oriented 

• SearchQuerySet 

• SearchForm 

• Search View 

Index Strategies 

• One Time 

• Real Time 

• Real time=ish (queued) 

• Periodic 



Building Search 

• Model attribute 

• Templates 

• Field method (Method that refers to a field) 

• Queryset to define a search index (Specify what gets added to the index) 

Help Users 

• Improve quality of search 

• Adjust relevance 

• Boot fields, documents, terms 

• Log searches, results, and their success 

• Use search engine as cache 

Doing More with Search 

• ElasticSearch can configure index analysis. 

• Can configure tokenizers and filters 

• Write a custom backend/New Default analyzer/Update search mapping 

Some Gotchas 

• Don’t index fields used for sorting 

• Debug search issues (Is this plugged in, is anything indexed?) 

• Haystack debug pannel for django debug toolbar 

SearchIndex = data mapping 

Writing Fast and Efficient Unit Tests For Django 

Presenter: Casey Kinsey 

Track: I 

Description: 

Many developers have difficulty finding clear guidelines and best practices for how to test efficiently, 

leading to a flimsy, slow, and ineffective test suite. This talk will cover some basic (but oft overlooked) 

principles of unit and integration testing, and dive into more advanced topics such as testing with read 

only data and using Mock ultra-focused and fast testing. 

Writing Faster Tests 

A real need for test speed 

• Made an initial production release of a real product for a national media company 

• Test Coverage not great 



• Started seeing regressions 

• Aggressively pursue greater test coverage 

• Results were successful, but needed faster tests. 

Should I be concerned? 

• You probably have lots of tests 

• You probably run them frequently 

• Slow tests will cause developers to stop running them 

• Preparing code for integration becomes painful 

• Deployment speed is directly affected 

How to write better tests 

• Many project suites are comprised of integration tests 

• Write unit tests 

• Unit test calls a small “unit” of code 

• Integration tests, test the contracts between the units 

• Using the django test client is a giveaway that its an integration test. 

• Write unit tests that are very limited in functionality/scope 

Unit Vs Integration 

• For each function that contains business logic, there should be a unit test 

• for each page/view/user path of your project there should exist an integration test. 

• Setup tests 

Set up tests cautiously 

• Be judicious about how you use setUp/tearDown 

• Think like middleware. Do I really need this for every test in this case? 

• One inefficient computation can cripple the whole test case. 

• Add @classmethod. very effective for read only data. Data will persist between tests! 

The Database is Hot Lava 

• If you touch it, you will die 

• Not really, but its one of the slowest things your application will do in a unit test. 

• Work with read only, non persisted data. 

• Use in memory model instances 

model = ModelName(attribute=x) 

• Avoid fixtures 



• Fixtures don’t adapt 

• Schema changes will result in failures 

Fake it Till You Make It With Mock 

• Library that lets you create stub objects 

• Configure behavior for testing 

• Use mock to emulate model instances 

• No model/ORM overhead 

• Use mock.patch to focus your tests 

• Patch sys.modules with your own module 

• Use mock in more complex situations 

• Track the way objects are used - test assertions, know which attributes that have been called and can report on 

it. 

It’s OK to engineer when testing 

• Don’t be afraid to invest engineering effort into the test suite. 

• Your tests are Python code, take advantage of it. 

• Write tools to help you 

• Mock, Django Nose 

• Custom test runners 

• If you can’t test the code efficiently, refactor the code. 

3.1.2 Wednesday 

Agile Not Vulnerable 

Presenter: Jacob Kaplan-Moss 

Track: I 

Description: 

Startups like to “move fast and break things”. . . but how do you ensure that what breaks isn’t security? 

How do you strike a balance and make sure you’re able to ship quickly while still ensuring that what goes 

out the door doesn’t have vulnerabilities? The answer begins by recognizing that security is a process, not 

a feature, and this has ramifications throughout the organization. It means that security engineering needs 

to be everyone’s responsibility (instead of a select few), and it means that when security vulnerabilities 

crop up — and they will — the organization needs to be prepared and aligned to act quickly. 

Security 

Shipping Often With Security/Agile 

• Ability to ship is incredibly valuable but also dangerous. 



Overview 

• Security is a process not a product. 

• Security is a part of everything you do, every day. 

• Similar to testing, documentation, daily activity. 

• Heroku has two man security team. 

• Security is everyone’s responsibility. 

• Every developer should have some understanding of what can go wrong 

The OWASP Top 10 (Paste in later) 

Example 

• Building a REST API 

• Should you support all major (common) formats? 

• Does this decision have security ramifications? 

Ex 2: 

• You need to store data. One format is common, one is less common, harder to read and write and isn’t used as 

often 

• Which do you choose? 

• Does this decision have security ramifications? 

A security vulnerability has been created. 

2013 Ruby/Rails YAML Vulnerabilities 

• Real world security issues are multifaceted. 

• This can happen to anyone. Django/Python is not exempt. 

More on Security 

• Secure by default matters. Defaults matter!! 

• YAML “load” and “load_safe” 

• You can’t really prove that software is secure. You can only prove that it’s insecure. 

• Unknown Unknowns 

• If an issue of this magnitude was discovered in your stack would you be prepared to respond? 

• Need to define terminology to refer to severity of security issues 

A good security policy 

• Lays out standard terminology used when talking about security issues 

• Explains the expectations and commitments around vulnerability handling. 

• Creates a transparent repeatable assessment mechanism. 



Terminology 

• Advisory 

• Low 

• Medium 

• High 

• Critical 

Advisory 

• Issues that the security team wishes to communicate but carry no specific required action. 

• May contain recommended actions, but no specific response is required. 

Low 

• Issues that are expected to be resolved, but have low risk, or low consequences. Should not interrupt 

day to day operations. 

Medium 

• Carry some risk, but have low impact. May have someone work on. 

High 

• Carry substantial risk, publicly disclosed issues. Will probably interrupt several developers from multiple teams. 

Critical 

• Threaten the integrity of the company. Great financial risk or otherwise “sky is falling” level issues. 

• “All hands on deck” 

Assessment 

• Start with OWASP for risk rating 

• Risk = Likelihood x Impact 

• How likely is that this issue will be discovered and exploited? 

Examples of Vulnerabilities 

• http://bit.ly/13ds9X0 (PostgreSQL) 

• Likelyhood: threat agent 

• Calculate Threat 

• Calculate your impact 

• Matrix of likelyhood and impact 

• This case came out to High level. 



Transactions for Web Developers 

Presenter: Aymeric Augustin 

Track: I 

Description: 

Django has since long outgrown its roots in publishing. It’s used in enterprise systems, under write-heavy 

workloads and strong data integrity requirements. But transaction management barely changed since the 

0.9x days. Even veterans pull their hair out fighting DatabaseErrors when they use it. At last transaction 

handling will be overhauled in Django 1.6. How does the new system work, and why? 

Transaction Management 

• SQL-92 standard 

• A transaction is a sequence of SqL statements that is atomic with respect to recovery. 

• All or nothing. 

• Lifecycle (Transaction initiating statement -> commit, explicit rollback) 

SQL-1999 

• Save points (Like sub-transactions) 

• Statements always run in a transactions 

• Transactions are opened automatically 

• Transactions are advanced technology. 

Dreaded Error 

• Current Transaction is aborted, commands ignored until end of transaction block 

• Means - A previous statement failed, the application must perform a rollback 

• Recovery must be done in the application (Any auto-recovery scheme breaks transactional integrity) 

Auto Commit 

• Commit implicitly after each statement. 

• Wrap each statement in ints own transaction 

• Just execute my query! 

• Most databases default to auto commit. 

Auto Commit in PostgreSQL 

• Server always auto commit 

• Client libs can emulate standard behavior 

• in psql: set autocommit off 



Auto Commit in SQLite 

• Transaction semantics are tightly related to the implementation of atomic commit 

• Sqlite automatically starts a transaction before all statements except select 

• It automatically commits such transactions as soon as all statements finish executing 

• Transactions are always serializable 

Python client libraries 

• PEP 249 

• Connection, performs commits and rollbacks 

• Cursor, Executes queries, fetches results. 

• Auto-commit should be initially off 

• Interface should be provided to turn it back on 

• Closing a connection without committing the changes first will cause an implicit rollback to be performed. 

Transactions in psycopg2 

• Tracks transaction state 

• Inserts a BEGIN before each statement unless there is already a transaction in progress 

• Even before select statements 

• Idle in transactions 

• cnx.autocommit = True disables this behavior. 

Transactions in SQLite 

• Track state 

• Parses statements to insert BEGIN or COMMIT 

• SELECT:COMMIT, INSERT, UPDATE, DELETE, REPLACE: Begin 

• Any other statement uses COMMIT 

• Broken by design 

Key Learnings 

• DB API requires the same transactional behavior as the SQL standard 

• Client libraries for Databases that always auto commit have to emulate this behavior 

• You can turn it off 

Django


Transaction middleware 

• One HTTP request = one transaction. Commit on success, roll back on exception. 

• High level apis transaction.autocommit() transaction.commit_on_success() 

Behind the Scenes 

• Django maintains a stack of transaction management states 

• Auto: the ORM commits every change 

• Managed: Django doesn’t commit 

• Django maintains a “dirty” flag: set automatically by the ORM after writes, must be set manually after raw SQL 

queries. 

• Nesting doesn’t work well 

Default Django 1.6 

• Database-level auto commit 

• ATOMIC_REQUESTS 

• ATOMIC can be used as decorator or as context manager 

• commit on success, roll back on exceptions 

• Guarantees atomicity. 

• Low level API to implement your own transaction management 

Key Learnings 

• If you don’t understand transactions read the docs in django 1.6 

• ATOMIC_REQUESTS is still a reasonable idea 

• Use the atomic decorator when you need aotmicity. 

Django Auth User 

Presenter: Russell Keith-MaGee 

Track: I 

Description: 

An exploration of one of the banner features of Django 1.5 – Custom User models. Includes worked 

examples, a discussion of design decisions that must be made, and a look at the internal architecture that 

makes it all possible. 

Auth User Model 

Why Should We Care? 

• Login with email address 

• Associating profile data with the user model 



• Not clearly understood 

Whats in a name? 

• Names are different (not just first and last) 

• Non western names 

• Some last names do not have family names 

• Some do not have last names at all. 

• Django assumes that you have a distinct first and last name, is wrong. 

Names are Hard 

• Do you need separate fields 

• Just use a Full name? 

• If you need to seperate them, use “Family name” and “Other/given name” 

• Ask “How would you like to be addressed”? 

Tips 

• Don’t assume a single letter is an initial 

• Be wary of name-part algorigthms 

• Spaces, Apostrpohes, and Hypens are all level characters in names 

• Don’t require a “Family Name” 

• “previous name”, not “maiden name” 

• Honorifics are even more complex. Can’t just add “mr” in front of a name 

On the subject of “do you need to ask”? 

• Why do you ask for certain things? (Gender?) 

• Kuzdu and the California Marriage amendment 

When it comes to identity you need to think: 

• Do I need to ask at all? 

I. Define User Model 

• 2 possible base classes (Abstract base user, abstract user) 

• Define username field 

• Define required-fields 

• Define get_full_name and get_short_name 



II. Define Manager 

• Need to describe how to create users. 

• Describe how to create superusers. 

III. Define Forms 

• usercreationform 

• userchangeform 

• passwordresetform 

VI. Register with Admin 

• Only need to do this if your using the admin 

• subclass contrib.admin.UserAdmin 

V. Register the model 

• AUTH_USER_MODEL = ‘myapp.MyUser’ 

IV. Update Foreign Keys 

• NOT ForeignKey(User) 

• ForeignKey(settings.AUTH_USER_MODEL) 

• USERNAME_FIELD must be unique and not in REQUIRED_FIELDS 

Signal Registration 

• Register signals with the actual model that is being used, not hte setting. 

What Isn’t in the Docs 

• Reverse lookup naming. 

• The “User Contract” - You must be explicit about what a user object has. 

Email Based Login 

• Define a user model with email 

• Username_field = ‘email’ 

• Define forms, admin. 

Don’t reinvent the wheel 

• Ticket: #20824 

• API-based login 

• Kerberos single sign-on 

• Authentication backends, can have multiple auth backends. 



• roguelynn.com/words/ 

Profile Data 

• Option 1, put everything in the user model 

• Option 2, keep user separate, link to it with a foreignkey 

• In option 2, you can provide your own profile model to hook in to the user model. 

• Which should you use? 

It Depends 

• Profiles are better architecture. Makes no assumptions about user model 

• Long term user objects. 

• Cost of getting a foreign key 

• Where do draw the line. 

One More Thing 

• How does it all work. 

• No references to auth.User 

• Meta property: Swappable = ‘auth_user_model’ 

• Inspected at run time for the real model class 

• The rest is validation 

• No new features in ForeignKey() or M2M 

• Validation that ForeignKey doesn’t point at a swapped models. 

• You can make your own models swappable. 

https://speakerdeck.com/freakboy3742/red-user-blue-user-myuser-auth-dot-user 

Building Rich Applications with Django and Ember 

Presenter: Gabriel Grant 

Track: I 

Description: 

Ember.js is the most advanced client=side JavaScript framework available: it provides optimistic server 

updates, automatically=updating templates and sensible defaults to dramatically increase developer productivity. 

Building Rich Apps with Django and Ember 

Ember and Django Dot Cloud Example 

• used jquery to return async views 

• Django stack 



• JSON Bypassing template layer 

• Ember! 

Ember 

• Client side mvc framework 

• Data that would go to template views, now goes to Ember to handle on the client side. 

Demo 

• Expose an API 

• Django Rest Framework 

Ember 

• Advanced Client side framework 

• MVC and App structure 

• Standards library 

• Bindings 

Standards Library 

• Object system 

• Eliminate busy work 

• Ember Bindings == @property + dependencies 

Components and Controllers 

• Web components 

• Ember components 

• Controllers store transient application state 

• Controllers can be contrlled by components 

Everybody Loves Migrations 

Presenter: Andrew Godwin 

Track: II 

Description: 

Times are changing - schema migrations are finding their way into core Django and becoming quite 

different in the process. Come and learn what’s happening, why it’s being done this way, and how you 

can best start using them. 



Migrations 

Why is it so hard 

• Versioning not easy in Dbs 

• hard to roll back 

• every change has a side effect 

• Difference databases (Mysql, Postgres, etc.) 

• Not necessarily up front. 

South 

• Released 2008 

• Most popular solution 

• Not without issues 

• 6 or 7 year old design 

Basic Layout 

• schemamigration (have to always add –auto) 

• datamigration 

• migrate apply only migrations 

• syncdb 

• complex and evolved design 

Issues 

• Migrations build up over time 

• VCS merges suck OUCH 

• That file format 

django.db.migrations 

• 5 years in the making. 

Design Goals 

• Clean migrations - Readable diffs are really important. 

• Squashable migrations - No need for those hundreds of old ones. 

• Better merge protection 

• Better commands –auto is pointless 

• Automatic dependencies - stops silent errors 

• Reuseable schema API - There are valid reasons to change tables 



• Third-party compatability 

Migrations 

• Make migrations 

• migrate - Applies migrations and legacy collection. Syncdb removed 

• Autodetector - Makes new migrations 

• Executor - Plans and runs migrations 

Better Format 

• Compound history 

• Stores history as statements in a list 

• Series of Operations 

• Nicer way of serializing fields 

• Fields need a deconstruct() method. 

Multiple AppCaches 

• Basically, you can make multiple versions of the same model in memory at once. 

Don’t Panic 

• Will be upgrade path to new migrations 

• South 2 to backport new format. 

• South 2 out a little after django 1.7 

Future Ideas 

• Better autodetection 

• Renames now work 

• Percona support 

• Nonrelational support 

Overview 

• New migration Format 

• Even easier to use 

• Django models through and through 

• Use a proper database 

• Schemas are your friend 



3.1.3 Thursday 

Guerilla APIs 

Presenter: Russell Keith-Magee 

Track: I 

Description: 

In an ideal world, every web system would provide a well designed REST API with oAuth authentication. 

But what do you do when those things don’t exist? 

Guerilla APIs 

• Big companies do not embrace API centric development 

• In order to get this data you have to use unconventional tactics. 

• Can build user facing API that works how we want to 

• Not productized (but could be) 

RSS handling 

• FeedParser 

• Time tasks/cron to retrieve content 

• Pythonic interface for RSS handling 

Email Handling 

• Get pure text 

• HTML 

• Attachments 

• Emails may be recursive 

• Mimetype 

• DOCX 

• DOC 

PDF Processing 

• PDF is a printing format 

• Internally, vector based drawing instructions 

• May contain attachments 

• PDFMiner 

• Some PDFs are locked down 

Use Selenium to open a browser session to actually submit forms programatically. 



Making Django Play Nice with Third Party Services 

Presenter: Matt Makai 

Track: II 

Description: 

Modern Django projects combine custom apps with third party services, such as Twilio and Stripe, to create 

a complete product. Choosing the right services for your application can make or break its usefulness 

to users as well as your sanity during maintenance. This talk will show you how to properly evaluate, 

integrate, and maximize what you get out of SaaS products in your Django projects. 

Making Django Play Nice with Third Party Services 

@mattmakai 

Why 

• Your app, your responsibility 

• Service usage is increasing 

• Little effort to prevent big headaches 

What will you learn 

• Discover new services 

Third Party Service Usage 

• Discover 

• Evaluate 

• Integrate 

• Maintain 

Discover 

• Research when not under pressure to immediately integrate 

• What problem can this service potentially solve. 

Mindset 

• Background and philosophy 

• Openness about their solution approach 

• Bookmark services and arguements for and against 



Push Sources 

• Leanstack 

• Python Weekly 

• PyCoders’s Weekly 

• Django Round-up 

• Heroku Addons 

• Geckoboard Widgets 

Evaluate 

• Platform ecosystem 

• Is there a python resource at the ruby shop? 

• Developer evangelists? 

• Data gravity? - How much of your critical data is going into the service? Can you get your data out easily? 

• Stack exchange 

• Long term viability 

• Project Readme/Docs 

Example 

Twillio 

• Tutorials in Python 

• Platform is on Python 

• Open source Django Examples 

• Developer evangelists active 

First Party Binding 

• Ideal 

• Update Frequency? 

• Outstanding issues 

Creating Your Own Third Party Binding 

• Learning 

• python-requests 



Tools 

• Pypi-notifier (Notify when new packages come out) 

• Full stack Python 

• Discovering Third Party Services for Django 

• mattmakai.com 

• Geckoboard widget directory 

State of the real-time web with Django 

Presenter: Aymeric Augustin 

Track: I 

Description: 

In 2013, “real-time” is more than a buzzword: it’s a reality on the Web. Unfortunately, for users of Django, 

it’s still a foreign world that involves new concepts and new components, and it doesn’t integrate well with 

traditional infrastructure. What is the real-time web? Why is it hard to support in Django? What are our 

options today? What can we expect in the future? 

State of the real-time web with Django 

Real Time 

• Systems responding within deadlines 

• Simulations running at wall clock time 

• Processing events without perceivable delay 

• Set of technologies and practices taht enable users to recive info as soon as its published, rather than refreshing. 

Use Cases 

• chat 

• games 

• VOIP 

• Notifications 

• Live Data 

• Social feeds 

• PUSH information 

Request - Response model doesn’t allow for this. 

Early Solutions 

• Java Applets 

• Pushlets - call back from java apps into DHTML 

• Comet - Long lived HTTP connections to reduce latency. 



HTTP Long Polling 

• Server keeps request onhold and only send respond when an event to deliver 

• As soon as client gets the response it sends another request 

HTTP Streaming 

• Server sends a series of events ina single HTTP response 

• Chunked 

• Client processes each incoming event. 

Server Sent Events 

• Built on top of HTTP Streaming 

• Format text/event-stream 

• Javascript API 

Web Socket 

• Provides bidirectional communication in the context of the existing HTTP infrasturcutre 

• RFC 6455 

• Binary 

• Socket.IO 

• SockJS 

Long Polling 

• Locks up a gunicorn worker 

Web Sockets 

Execution Model 

• Based on an event loop 

• Handle multiple socket connections in a single thread 

• More efficient than one thread per connection 

• Suitable for network prgoramming 

Programming Model 

• Callbacks 

• coroutines 

• Based on explicit cooperative multi-threading 

• In python: yeild (from) 

• Suitable for concurrent applications 



Pep 3156 

• Pluggable event loop API 

• Callbacks, transports, protocols 

• High level scheduler based on coroutines 

• REference implementation code-named Tulip 

• Effort led by Guido 

Django C10k Demo 

https://github.com/aaugustin/django-c10k-demo 

Django isn’t async 

• @websocket 

HTTP != real-time 

• Execution - threads vs events 

• Programming preemptive vs cooperative 

• Stateless vs stateful 

• CPU vs I/O Bond 

• Request-response vs message streaming 

Key 

• Django isn’t designed for explicitly cooperative multi threading and its unlikely to change 

• Robust client and server stacks are emerging 

• Better best practices 

• Simplified development setups 

• Getting more useable. 

https://speakerdeck.com/nduthoit/denormalize-all-the-things https://speakerdeck.com/nduthoit/the-path-to-smootherdatabase-migrations 

3.2 PyCon 2012 

3.2.1 Tuts 

Python Epiphanies 

March 8 2012 

• Instructor - Stuart Williams 

3.2. PyCon 2012 49


Introduction 

• Static vs Dynamically typed language. 

• Compiler must now ahead of time what type an object is 

• In python you do not have to do that, because is not a static type language 

• Type is checked at run time, not typed at compile time. It is a dynamic language 

• Compiler allocates memory. 

• In python creates types and assigns names to them (not variables) 

• These are references to other objects not actual objects themselves. 

• Names are implemented like ditionaries 

Dictionaries and Namespaces 

Python Objects and Vars 

• Things like a = 17 are essentially dictionaries. These are added to object land. 

• Python object has 

• Single value 

• Single Type 

• Some number of attributes 

• Single ID 

• Zero or one or more names in one or more namespaces 

• One or more base classes 

• When lists are appened IDs do not change, strings do. 

• Can’t set attributes of built in types. 

• IDs match because there only needs to be one actual object in memory 

Namespaces 

• _namespace[’s] - Directly accessble namespace 

• indirect, using dot notation dict.__doc__ or sys.version.major 

Namespace Search Order 

• Local names 

• Namespaces of encolsing function, search starting with the nearst closing scope 

Exercise: 

locals().keys() 

globals().keys() 

locals() == globals() 

locals() is globals() 

In this case locals() is globals() 



Namespace Changes 

• assignment 

• del 

• (globals() and locals()) 

• import 

• def 

• class 

When you make an assignment you are just assigning a second name (alias) to the original function: 

from pprint import pprint as pprint_function 

• __init__.py required for modules 

Functions 

• You can add arbitrary attributes to functions 

• Passing in a dictionary as an argument 

>>> def f(a1, a2, kw1=’k1’, kw2=’k2’): 

... print(repr((a1, a2, kw1, kw2))) 

... 

>>> f(1) 

Traceback (most recent call last): 

File "", line 1, in 

TypeError: f() takes at least 2 arguments (1 given) 

>>> f(1, 2) 

(1, 2, ’k1’, ’k2’) 

>>> f(1, 2, 3) 

(1, 2, 3, ’k2’) 

>>> t = 1, 2 

>>> t 

(1, 2) 

>>> d = dict(kw1=3, kw2=4) 

>>> d 

{’kw1’: 3, ’kw2’: 4} 

>>> f(*t) 

(1, 2, ’k1’, ’k2’) 

>>> f(**d) 


File "", line 1, in 

TypeError: f() takes at least 2 non-keyword arguments (0 given) 

>>> f(1, 2, **d) 

(1, 2, 3, 4) 

Another example: 

>>> name = ’Dad’ 

>>> ’Hi {name}’.format(**locals()) 

’Hi Dad’ 

Lists are Mutable, Strings are Not 

• Lists can be changed. Strings create new objects. 

3.2. PyCon 2012 51


Class Statement 

• single ID 

• Single value 

• Number of attributes 

• Single Type 

• one or more namespaces 

• One or more base classes 

ClassName() 

ClassName().__init__() 

Class Num(object): 

def __init__(self, amount): 

self.amount = amount 

def add(self, value): 

return self.amount + value 

• You can add a method as an attribute of a class by simply assigning it. 

• is comparison has been helpful 

Class Prefixer(object): 

def __init__(self, prefix): 

self.prefix = prefix 

def prepend(self, listing): 

for l in listing: 

l = l + self.prefix 

• Need to brush up on some basic python iterators, loops, etc. 

• Metaclasses 

Iterators 

• A for loop evaluates and expressiosn to get an iterable and then calls iter() to get an iterator. 

• The iterators next() method is called until StopITeration is raised. 

• Iterable items get the iter() method called. 

m = [1, 2, 3] 

it = iter(m) 

it.next() 

it.next() 

it.next() 

Generators 

def list123(): 

yield 1 

yield 2 

yield 3 



it = list123() 

it.next() 

it.next() 

it.next() 

• Look at next (David Beazley talk on Generators) 

import operator 

ops = { 

’+’: operator.add, 

’-’: operator.sub, 

} 

ops[op] (lhs, rhs) 

def calc(expr): 

lhs, op, rhs = expr 

lhs, rhs = int(lhs), int(rhs) 

return ops[op] (lhs, rhs) 

3.2.2 Friday 

Stop Writing Classes 

Classes are overused 

• Obfuscated function calls 

• Classes with an __init__ and one other method. 

• SHouldn’t instantiate use them once, and then throw them away. 

• Namespaces are there to help and prevent naming collisions. 

• Containers are great use case for classes. 

Exceptions 

• Don’t make new exceptions, you don’t need to. 

Advanced Security Topics 

Presenter: Paul McMillan - Django Security Developer 

Track: II 

Description: 

If your Python application has users, you should be worried about security. This talk will cover advanced 

material, highlighting common mistakes. Topics will include hashing and salts, timing attacks, serialization, 

and much more. Expect eye opening demos, and an urge to go fix your code right away. 

https://us.pycon.org/2012/schedule/presentation/467/ 

3.2. PyCon 2012 53


Hasing and Encryption 

• MD5, SHA1, SHA256 

• If you are typing md5 you are doing it wrong 

• Did this file get corrupted? 

• Use has for Message signing. 

• It is hard to generate a file that duplicates your stored hash. 

• Use HMAC for message signing. 

hash(secret + hash) 

• Salt your secret key 

salt = ’session_cookie_signing’ 

hmac.new(salt + secret_key, msg) 

• When using has algorithims do not use MD5. 

• SHA1 is better, but use SHA256 

• Web could use SHA512 as its not 32 bit. 

Why Need Encryption 

• Do not implement yourself. 

• Use SSL/TLS 

Random Numbers 

• Default random number is predictable. 

• Use SystemRandom() instead 

from random import SystemRandom() 

Timing attacks 

• String comparison does not compare the entire string at once. 

• Not safe 

• Compare the length first 

• Compare different sets of characters, even if the previous ones worked. 

Pickle 

• Do not put data straight into pickle. 

• Use JSON for untrusted data 



Always verify your assumptions 

• PIP installing. 

• I trust django developers 

• I truse the people who wrote pip 

• PIP verifies MD5 hash 

Am I Safe? 

• All these things require you to trust everyone on the internet 

• You must verify 

• Python doesn’t make it easy to check SSL certs. 

• Consider using Crate.io 

Interfaces and Python 

Presenter: Eric Snow 

Track: IV 

Description: 

In 2.6, Python introduced the Abstract Base Classes. Before that we had “protocols” (and we still do). 

In this talk we’ll look at how the general concept of interfaces fits into today’s Python. We’ll also look 

at some of the alternate proposals of the past, some of the controversies around ABCs, and the direction 

interfaces might go in the future. 


http://goo.gl/hwkTy 

Object Interfaces 

• Code as documentation 

• Adaption 

• Static analysis 

What is it 

• Communication 

• Documentation 

• Doc strings 

• Comments 

• Progromatic Interfaces 

3.2. PyCon 2012 55


What is it in Python 

• Abstract base classes 

• Protocols 

• Build your own 

Protocols 

• iterator 

• context manager 

• sequence 

• descriptor 

• No Validation is done to make sure they are following the protocol 

EAFP vs LBYL 

It’s easier to ask forgiveness than permission 

Call it and worry about the failing later. 

# EAFP 

def is_dead(obj): 

try: 

obj("only a flesh wound") 

except TypeError: 

raise MyError("expected a callable object") 

Look before you leap 

Check a variable/method for something before you call it. 

# LBYL 

def is_dead(obj): 

if not hasattr(obj, "__call__"): 

raise MyError("expected a callable object") 

obj("only a flesh wound") 

Duck-typing 

polymorphism by capability vs polymorphism by type 

Requiring a specific interface instead of a specific type. 

Abstract Base Classes 

Write Your Own 

class Entree(metaclass=ABCMeta): 

@abstractmethod 

def with_spam(self): 

"""spam spam spam spam""" 

@abstractproperty 

def spamcontent(self): 

"""Lovely spam! Wonderful spam!""" 

• Decorators make your methods Abstract. 



• If a class has abstract methods then that class will be abstract. 

• ABCMeta marks class as abstract 

Use Existing 

• Register an existing class as an implementation 

>>> class MyDict(Mapping): pass 

... 

>>> issubclass(MyDict, Mapping) 

True 

>>> isinstance(MyDict(), Mapping) 

True 

>>> MyDict.__bases__ 

(,) 

Decorators and Context Managers 

Presenter: Dave Brondsema 

Track: IV 

Description: 

Learn how decorators and context managers work, see several popular examples, and get a brief intro to 

writing your own. Decorators wrap your functions to easily add more functionality. Context managers 

use the ‘with’ statement to make indented blocks magical. Both are very powerful parts of the python 

language; come learn how to use them in your code. 


Decorators 

• Pass functions to other functions 

• Example 

@property 

Instead of 

var = property(method) 

def expose(func): 

func.exposed = True 

return func 

• Do this: 

@expose("template.html") 

def expose(template): 

def mark_exposed(func): 

return func 

return mark_exposed 

3.2. PyCon 2012 57


• Decorators replace the original function in memory. 

• Decorator for decorators 

• Allows you to not nest methods 

• Classes can be decorators 

• Decorators can be placed on classes 

• @classmethod decorator - 

• Restrict.post or something 

pip install decorator 

Context Managers 

• __enter__ and __exit__ methods 

• @contextlib.contextmanager 

• contextlib 

http://speakerdeck.com/u/brondsem 

Certainty in an Uncertain World: Gaining Confidence through Security Testing 

Presenter: Geremy Condra 

Track: I 

Description: 

Nobody thinks you have to be a performance expert to write performance tests- why assume that you have 

to be a security expert to write security tests? During this presentation I’ll show you how to use fuzzers, 

attack tools, and other simple techniques to help protect your users, improve the strength of your existing 

tests, and gain confidence in the security of your code. There will be demos! 


Introduction 

• User behavior 

• Normal tests + adversary = security tests 

Common Attacks 

• Common Weakness Enumeration cwe.mitre.org/top25 

• Protect against these 

• OS Command Injection 

• Cross Site Scripting 

• Path Traversal 



OS Command Injection 

from commands import getoutput 

def list_directory(attacker_input): 

return getoutput("ls -la " + attacker_input) 

• Avoid shell 

• Sanitize 

• FuzzDB http://code.google.com/p/FuzzDB 

Cross Site Scripting 

• Inputing script straight into a page. 

• Can’t put client data straight into an html page 

• Removing tags does not work. 

• Sanitizing library, Bleach 

• Problems with CMS’s, blogs, etc. 

• Django has built in escaping. 

Directory Traversal 

A directory traversal (or path traversal) consists in exploiting insufficient security validation / sanitization of usersupplied 

input file names, so that characters representing “traverse to parent directory” are passed through to the file 

APIs. 

The goal of this attack is to order an application to access a computer file that is not intended to be accessible. 

Fixing 

• Find attack tool 

• Testing 

• Integrate 

• Fix 

• Repeat 

What Doesn’t Work 

• Finding really app specific bugs 

• Large search spaces slow it down 

• Hard to define problems 

• Heavily analytic problems. 

3.2. PyCon 2012 59


Wrap Up 

• Security testing isn’t hard 

• Can be very effective 

• You don’t have to be an expert 

Practicing Continuous Deployment 

Presenter: David Cramer of Disqus. Wrote Sentry. 

Track: I 

Description: 

Practice iterative development like the pros. Release sooner, faster, and more often. 


Workflow 

• Continuous integration server runs test suites 

• Fast rollback (Why not rollforward and fix) 

Good 

• Develop features incrementally 

• Release frequently 

• Smaller doses of QA 

• Because manual tests are awful 

Bad 

• Culture shock 

• Statbility depends on test coverage 

• Time investment 

Keep Development Simple 

• Automated testing is required. 

• Simple can be better than complete 

• Puppet, Chef, Buildout, Fabric 

• Packaging your App as tag 1.0? 

CI Server 

• Can be stripped down, don’t need to test apache, nginx 



Bootsrapping Local 

• Git clone it 

• Simple command to make everything work 

• Next step would be python manage.py runserver 

• Need to test dependencies? - Virtualbox + vagrant 

Progressive Rollout 

• Actively release to smaller group before public 

• Feature flipping (Gargoyle) 

• Bump up features 

• Early adopters are free QA 

• Sign up to get new stuff 

Review All Commmits 

• Phabricator from facebook 

Integration 

• Jenkins 

• Painless setup 

• Suppoer proper reporting - Coverage.py 

Bad 

• False positives 

• Bad tests 

• Services fail 

• Feedback delay 

• Integration tests vs Unit Tests 

http://jenkins-ci.org/ 

Fixing False Positives 

• Rerun tets several times on a failure 

• Report continually failing tests to dev 

3.2. PyCon 2012 61


Maintain Coverage 

• Commit tests with code 

• Utilize code review 

• Coverage against a single diff. 

Speed up Tests 

• Write true unit tests 

• Mock external services 

• Distruted and parallel testing 

Reporting 

• Rate of traffic (not just hits) 

• Response time (database, web) 

• Exceptions 

• Social Media/Twitter 

Sentry! 

Getting Started 

• Package your app 

• Value code review 

• Ease deployment/fast rollbacks 

• Setup automated Tests 

• Gather some easy metrics 

• Automate deploys 

• Continueous deployment doesn’t mean deploy all the time, it means deploy any time. 

• Consider the ramifications of schema changes. DO NOT DROP columns 

3.2.3 Saturday 

Django Templating: More Than Just Blocks 

Presenter: Christine Cheung 

Track: V 

Description: 

Django’s template language is designed to strike a balance between power and ease of use; learn how to 

use this balance to create awesome looking websites. This talk will cover the basics and best practices 

of Django templating, from custom tag and filter creation, to the finer points of template rendering and 

loading, and even to replacing the default templating engine itself. 




http://www.xtine.net @plaidxtine 

Intro 

• Front End Developer 

• PyLadies! 

• Introduction to Templating 

• Effective use of Built In Tags 

• Extending Templates 

• Template Loading 

• Extending Current Django Templating 

Basics 

• This is the end user experience 

• Balance between power and ease 

• By Design it is seperated so that backend/front end can be developed seperately 

• Start with a base template, and have pages inherit from it. 

Tools 

• Syntax highlighting, autocompletion 

• django-debug-toolbar 

• Print out tag/filter reference guide. 

Structure 

• Root template folder, argues against seperate apps/templats 

Standards 

• Consistent Spacing 

• Put all loads at the top 

• Use {% comment %} instead of 

Common Blocks 

• Title 

• Meta tags 

• Extra_head 

3.2. PyCon 2012 63


• Content 

• Extra_JS 

Block Best Practices 

• End {% block title %} with {% endblock title %} 

• Blocks cannot be repeated 

• DOn’t over block, only write one if you need one. 

• Use include {% include “snippet.html” %} 

• Do not over use includes 

Variables 

• Modify objects with filters {{ var|filter }} 

• loop through them useing tags 

Security 

• Make sure you sanitize data if you are using {% autoescape %} 

General Best Practices 

• Name URLs 

• Do not hard code static 

• Django-floppyforms (html5) 

• django-crispy-forms - More semantic forms 

• {% include form.html %} 

• Use {{ form.as_ul }} 

• Firstof tag, removing un-necssary if statements 

• Custom tags live in application module 

• Write simple, basic filters 

Example Custom Tag 

from django import template 

register = template.Library() 

@register.filter(name="remove") 

def cut(value, argument): 

return value.replace(argument, ’’) 



Other Types of Tags 

@register.simple_tag 

@register.tag(name="current_time") 

• django-templatag-sugar 

• django-classy-tags - Class based template tags 

Do Not 

• Application logic 

• Dangerous, Difficult to support 

• Do not replicate python logic in the templates 

Loading and Changing Templates/Loaders 

• Template Loaders 

Replacing the Template Engine 

• Jinga2, Mako, Cheetah 

• Django is a bit slower 

• Different logic control and handling 

• Lots of risk switching becuase you are separting django from its template engine. 

Jinja 

• Functions callable from templates 

• Loop controls 

• Multiple filter arguments 

• Performace Increase 

• More dependencies 

• Lose built-in support 

• Too much logic in your templates 

• Speed increase is minimal 

Speeding Up Templates 

• Cache -Django-template-preprocessor 

• django-pancake - Flatten template files 

• Remember other bottlenecks (Cache, load balancing) 

3.2. PyCon 2012 65


New in 1.4 

• Custom project and App templates 

• Startapp/start project 

• Combine with your favorite boilerplate 

• {% elif %} 

Django Form Processing Deep Dive 

Presenter: Nathan Yergler 

Track: V 

Description: 

Django Form processing often takes a back seat to flashier, more visible parts of the framework. But 

Django forms, fully leveraged, can help developers be more productive and write more cohesive code. 

This talk will dive deep into the stock Django forms package, as well as discuss a strategy for abstracting 

validation for forms, and the use of unit and integration tests with forms. 


http://yergler.net/2012/pycon-forms 

Basics 

Forms in Context 

business logic 

VIews - Convert request to response forms = Conert input to python objects models - Data and 

Forms 

• Forms are composed of fields, which have a widget 

• Ubound forms dont have data associated with them but can be rendered 

form= ContactForm() 

• bound forms have specifid data assoicated which can be validated, can be any dict of data 

form = ContactForm(request.POST) 

Two ways to access fields 

form.fields[’name’] 

#Field Object 

form[’name’] 

# outputs html widget 

Forms can be given initial data 



Validation 

• Only bound forms can be valided 

• Fields are validated, then the form itself 

• Validation, then cleaning 

• Validation Error 

Field Cleaning 

• To PYthon, Validation, Cleaning 

• .clean_fieldname() method is called after validators 

• already converted to python 

• methods must return a clean value 

def clean_email(self): 

if (self.cleaned)data.get(’email’, ’’).endswith(’hotmail.com’)):: 

raise ValidationError() 

return self.cleaned_data.get(’email’, ’’) 

• Using get is safer if not a required field. 

Form Validation 

• .clean 

• Called even if errors raised by fields 

• Must return the cleaned data dictionary 

• Initial data != default data 

• Defaults for non-required fields should be specified when accessing the dict. 

Changes 

• form.has_changed() 

• form.changed_fields 

Testing 

• Remember whats forms are for 

• Testing intial states, field validation, final state of cleaned data 

class FormTests(TestCase): 

def test_validation(self): 

form_data = { 

’name’: ’x" * 300, 

} 

form = ContactForm(data=form_data) 

self.assertFalse(form.is_valid()) 

3.2. PyCon 2012 67


• Rebar, open source. Couldn’t find the link. 

from rebar import flatten_to_dict 

form_data = flatten_to_dict(ContactForm()) 

form_data.update({ 

’name’: ’x’ * 300, 

}) 

Rendering Forms 

• Class based views with forms is excellent 

Form Output 

• as_p() 

• as_ul() 

• as_table() 

• field.label 

• field.label_tag 

• field.html_id 

• field.help_text 

• field.errors 

• required_css_class 

• error_css_class 

name = forms.CharField(error_messages={’required’: ’something’}) 

Error Class 

• Error list is used as 

• Specify the error_class kwarg 

from django.forms.util import ErrorList 

form = ContactForm(data, error_class=ErrorListClass) 

• Avoid name collisions with prefix 

contact_form = ContactForm(prefix="contact") 

Other Notes 

• Model forms have an additional method, _post_clean() 

• FormSets to put many of the same form on the same page 

• Same validation structure 

• http://yergler.net/2012/pycon-forms 



Testing and Django 

Presenter: Carl Meyer 

Track: V 

Description: 

A deep dive into writing tests with Django, covering Django’s custom test-suite-runner and the testing 

utilities in Django, what all they actually do, how you should and shouldn’t use them (and some you 

shouldn’t use at all!). Also, guidelines for writing good tests (with or without Django), and my least 

favorite things about testing in Django (and how I’d like to fix them). 


github.com/carljm/django-testing-slides 

• Tests are slow 

• Not necessary to run some third party tests 

• Django test discovery is broken 

• unittest2 discovery 

• Test_RUNNER setting (code example) 

""" 

An alternative Django ‘‘TEST_RUNNER‘‘ which uses unittest2 test discovery from 

a base path specified in settings, rather than requiring all tests to be in 

‘‘tests‘‘ module of an app. 

""" 

from django.conf import settings 

from django.test import TestCase 

from django.test.simple import DjangoTestSuiteRunner, reorder_suite 

from django.utils.importlib import import_module 

from django.utils.unittest.loader import defaultTestLoader 

class DiscoveryDjangoTestSuiteRunner(DjangoTestSuiteRunner): 

"""A test suite runner that uses unittest2 test discovery.""" 

def build_suite(self, test_labels, extra_tests=None, **kwargs): 

suite = None 

discovery_root = settings.TEST_DISCOVERY_ROOT 

if test_labels: 

suite = defaultTestLoader.loadTestsFromNames(test_labels) 

# if single named module has no tests, do discovery within it 

if not suite.countTestCases() and len(test_labels) == 1: 

suite = None 

discovery_root = import_module(test_labels[0]).__path__[0] 

if suite is None: 

suite = defaultTestLoader.discover( 

discovery_root, 

top_level_dir=settings.BASE_PATH, 

) 

if extra_tests: 

for test in extra_tests: 

3.2. PyCon 2012 69


suite.addTest(test) 

return reorder_suite(suite, (TestCase,)) 

• unit test 

• system/integration/functional tests 

Unit Tests 

• Test one single piece of code 

• Should be fast 

Integeration Tests 

• Integration is slow, less usefull failures 

• Database makes your tests slow 

• Write tests that don’t hit the database 

Don’t hit the DB 

• Slow 

• self.save() 

def frobnicate_thing(thing): 

# ... do something complicated 

return thing 

class Thing(models.Model): 

def frobnicate(self): 

"""Frobnicate and save the thing.""" 

frobnicate_thing(self) 

self.save() 

No to Fixtures 

• Hard to maintain 

• Increase test interdependence 

• Slow 

Model Factories 

def create_profile(**kwargs): 

defaults = { 

"likes_cheese": True, 

"age": 32, 

"address": "3815 Brookside Dr", 

} 

defaults.update(kwargs) 



if "user" not in defaults: 

defaults["user"] = create_user() 

return Profile.objects.create( 

**defaults) 

• Using 

def test_can_vote(self): 

"""A user age 18+ can vote in the US.""" 

profile = create_profile(age=18) 

self.assertTrue(profile.can_vote) 

Factory Boy 

class ProfileFactory(factory.Factory): 

FACTORY_FOR = Profile 

likes_cheese = True 

age = 32 

address = "3815 Brookside Dr" 

user = factory.SubFactory(UserFactory) 

profile = ProfileFactory.create( 

age=18, user__username="carljm") 

Why Use Factories 

• Test data local to test code (explicit). 

• Easy to maintain. 

• Don’t create any data you don’t need for that test. 

• Works great even for large/complex test data sets (helper functions). 

• Mock Library 

from django.utils.unittest import TestCase 

import mock 

cursor_wrapper = mock.Mock() 

cursor_wrapper.side_effect = \ 

RuntimeError("No touching the database!") 

@mock.patch( 

"django.db.backends.util.CursorWrapper", 

cursor_wrapper) 

class NoDBTestCase(TestCase): 

"""Will blow up if you database.""" 

Views 

• Write less view code 

• Use RequestFactory() 

3.2. PyCon 2012 71


• Call the view callable directly 

Web Test 

url = "/case/edit/{0}".format(case.pk) 

form = self.app.get(url).forms["case-form"] 

form["steps-1-step"] = "Click link." 

form["steps-1-expected"] = "Account active." 

response = form.submit() 

• Markup matters 

• If it can break, it should be tested 

self.assertEqual( 

response.json, ["one", "two", "three"]) 

self.assertEqual( 

resp.html.find("a", title="Login").href, 

"/login/" 

) 

Selenium 

• pip install selenium 

• Django 1.4 

from django.test import LiveServerTestCase 

from selenium.webdriver.firefox.webdriver import WebDriver 

class MySeleniumTests(LiveServerTestCase): 

@classmethod 

def setUpClass(cls): 

cls.selenium = WebDriver() 

super(MySeleniumTests, cls).setUpClass() 

@classmethod 

def tearDownClass(cls): 

super(MySeleniumTests, cls).tearDownClass() 

cls.selenium.quit() 

def test_login(self): 

self.selenium.get( 

"%s%s" % (self.live_server_url, "/login/")) 

username_input = self.selenium.find_element_by_name( 

"username") 

username_input.send_keys("myuser") 

password_input = self.selenium.find_element_by_name( 

"password") 

password_input.send_keys("secret") 

self.selenium.find_element_by_xpath( 

’//input[@value="Log in"]’).click() 

Other 



• Write system tests for your viwes 

• Write selenium tests for ajax other js 

• Write unit tests for everything else. 

• Avoid multiple step tests 

@override_settings(ALLOW_COMMENTS=True) 

def test_comments_allowed(self): 

# ... 

Web Server Bottlenecks And Performance Tuning 

Presenter: Graham Dumpleton 

Track: V 

Description: 

New Python web developers seem to love running benchmarks on WSGI servers. Reality is that they 

often have no idea what they are doing or what to look at. This talk will look at a range of factors which 

can influence the performance of your Python web application. This includes the impact of using threads 

vs processes, number of processors, memory available, the GIL and slow HTTP clients. 


Big Picture 

• Many moving parts in the whole system 

• Main User grief is on teh front end. (Static, Network isues) 

• 80% - 90% of the end user reponse time is spent on the front end. Start There. 

• Database/Application Server 

Are Benchmarks Stupid 

• Little value 

• People reference them, and they are often wrong 

• Test only a single narrow use case. 

• Hitting a site with extram load will only show you whtat i till likely fail under a denial of service attack. 

• Should test corner cases, typical use cases 

Environment 

• Memory 

• Processors 

• Threads 

• Golbal Interpreter Lock 

3.2. PyCon 2012 73


Client Impacts 

• Slow HTTP Browsers/clients 

• Need to handle static assets ( Don’t use same server ) 

Use Cases 

• Memory USage 

• Threads 

• Long Running Requests 

• HTTP Clients 

Memory Usage 

• Web server base memory usage 

• Web server per thrad memory usage 

• Application base memory usage 

• Loaded before forking? 

• Adding more processes increases memory quicker. 

• Issue is mainly about configuration 

• Change and evaluate what your config is, don’t just use the defaults. 

• Number of overall threads, dictated by overall concurrent users. 

Threads 

• More processes, more memory 

• Find a balance 

• If CPU is slow, more processes 

• IO Wait 

Long Running Request 

• Uploads 

• Slow HTTP 

• Large Response 

Slow HTTP Clients 

• Proxy NGINX with Gunicorn 

• Offload static 

• Put Nginx in front of apache workers -> mod_wsgi dameons 



Restarts 

• Restarts to fix issues 

• Reloading of the application 

• Requests continue to backload 

• Should do a full shut down if backlog cannot be cleared. 

Pre load Everything 

• Load maximum processes into server at the begninng 

• No auto scaling 

Horizontal Scaling 

• Use more servers 

If you treat your server like a black box you will never know what is going 

on. 

Tools 

• New Relic 

• Sentry - Captures errors, but not performance problems 

Conclusion 

• Don’t trust server defaults 

• Monitor live production systems 

• Use benchmarks to explore a specific system, not to compare different systems. 

RESTful APIs With Tastypie 

Presenter: Daniel Lindsley 

Track: I 

Description: 

Providing full-featured REST APIs is an increasingly popular request. Tastypie allows you to easily 

implement a customizable REST API for your Python or Django applications. 


What is Tastypie 

• REST framework for Django 

• Designed for Extension 

• Supports both Model and non Model Data 

3.2. PyCon 2012 75


• tastypieapi.org 

Philosophy 

• Make good use of HTTP 

• Use REST methods/status codes properly 

• Graceful Degradation (Backwards Compatable) 

• Flexible everything 

• Data can round trip (Anything you can GET you should be able to PUT and POST) 

• Reasonable defaults - but easy to extend 

HATEOAS 

• Hit API at the highest level and you should be able to explore it without documentation. 

• Users shouldn’t have to know anything in advance 

Tastypie 

• Builds on top of Django and plays nicely 

• Full GET/POST/DELETE/PATCH 

• Any data source (not just models) 

• Designed to be extended 

• Supports JSON, XML, etc. 

• Well tested 

• Lots of hooks 

• Decent documentation 

‘‘References to the Install Docs http://django-tastypie.readthedocs.org/en/latest/index.html ‘‘ 

• Going over resources, and API structure 

• api, api/__init__.py, api/resources.py 

Automatic 

• /api/v1/ 

• /api/v1/user/ 

• /api/v1/user/2/ 

• /api/v1/user/schema 

• /api/v1/user/multiple/1;4;5/ 



Next 

• Filter 

• Leaking senstive info 

• Auth 

More 

• Exclude 

• Authentication 

• Authorization 

• Filtering 

• Cache 

• Throttling 

Extensibility 

• Why classes - It makes it easy to extend 

• Composition > inheritance 

• Why so many methods - Hooks, Hooks, Hooks. 

• Reasonable defaults, probably want JSON. 

• Serialization 

• Override or extend as you need. 

• Resource has lots of methods, many of which are pretty granular 

• Remove some formats 

serializer = Searizalier(formats=[’json’, ’xml’]) 

Fields 

• Control how data gets prepared for presenetation 

• Full control over the API/Schema 

• Hydrate and dehydrate 

Advanced Celery 

Presenter: Ask Solem Hoel 

Track: V 

Description: 

This talk will delve deep into advanced aspects of the Celery task queue and ecosystem. Previous experience 

with task queues and message oriented middleware is beneficial. 


3.2. PyCon 2012 77


Introduction 

• Lead Celery developer 

• Task Queue 

• Tasks are just dictionaries 

• Subtasking 

Task Granularity 

• Coarse-grained - More Computation 

• Chunking can make very granular tasks good 

• Chunks can use threads, reduce latency 

Chords 

• Synchronization Primitive 

• Barrier 

• Header is a taskset 

• Body is applied with the results of the headers 

• Native support for Redis and Memcached 

Consider Subtasking 

:: 

def smothing(): subtask.delay() 

Blocking 

• Bad 

• Use timeouts and retry if possible to stalled tasks. 

• Be smarter about routing. 

socket.settimeout() 

socket.setdefaulttimeout() 

• Reroute tasks to machines/workers with free CPU 

Cyme 

• Cyme node is a branch 

• No Master (Decentralized) 

• Branches know neighbors 

• API 



API 

• Create and manage 

• Create worker instances 

• Queues 

• Configure workers 

• Autoscaling 

pip install cyme 

cyme-branc -D 

What Python can learn from Java 

Presenter: Jonathan Ellis 

Track: V 

Description: 

Java is in some ways a bogeyman to the Python community – the language that parents scare their children 

with, the Cobol of the 21st century. But if we look past the cesspool of JEE it turns out that Java has quietly 

become an excellent systems environment, one that is still in many ways ahead of its time. 


3.2.4 Sunday 

Sketching a Better Product 

Presenter: Idan Gazit 

Track: I 

Description: 

If writing is a means for organizing your thoughts, then sketching is a means for organizing your thoughts 

visually. Just as good writing requires drafts, good design requires sketches: low-investment, lowresolution 

braindumps. Learn how to use ugly sketching to iterate your way to a better product. 


Intro 

• Django’s BDFL Designer 

• Skillsapp.com 

Sketching is not drawing 

• Skethcing is a tool taht serves a purpose 

• Drawing is art, serves itself 

• Sketching is for ideas as drafts are to writing 

3.2. PyCon 2012 79


• Different parts of brain used to process designs, look, feel. 

• Act of getting ideas out and bak in. 

Sketches Should Be 

• Cheap, Try different Ideas, No guilt to throwing away 

• Fast, quick to create, low resolution 

• Less detail 

• Ugly, communicate unfinished product 

Design Funnel 

• Start out wide in scope and ideas 

• As project progresses, iterate, and get more narrow 

• Sketches lead to prototype 

• Waste less time 

Tools 

• Paper 

• Pen/Markers 

• Wall to display them, visible 

• Iteration 

• iPad 

• Penultimate 

• Cosmonaut 

• iPad resolution isn’t as good as real paper 

37 Signals 

• Use two fat markers, black and red. 

• Epicenter Design, sketch the core of the layout or design 

How to Sketch 

• Draw lines 

• Boxes 

• And that is it. 

• Greeked Text 



Images 

• Represent a box with an X 

• Fill in 

Technique 

• Drawing a straight line is hard 

• Try rotating page if you can’t draw in one direction 

• Ruler for more professional sketching 

Notes 

• Stencils 

• Sketch the Iconic Thing 

Improving Documentation with “Beginner’s Mind” (or: Fixing the Django Tutorial) 

Presenter: Karen Rustad 

Track: III 

Description: 

This talk evaluates a well-known free software tutorial (the official Django tutorial) from the perspective 

of a web development novice in order to point out omissions and common sticking points and suggest 

improvements. More generally, this talk is useful to anyone looking to improve their project’s tutorials 

and other newcomer-targeted documentation by approaching them with “beginner’s mind”. 


Documentation Six Audiences and Purposes 

• First contact ( new users ) 

• Education ( New ) 

• Support ( Experienced ) 

• Troubleshooting 

• Internals 

• Reference 

Documentation for New Users 

Most Important docs 

• Install/setup 

• Tuts 

3.2. PyCon 2012 81


What Makes a Good Tut 

• Advertises what is cool or unique 

• Enjoyable - not too long 

• Consistently likely to succeed, (platform agnostic, testing) 

• Prepares the reader for using it on their own 

• These Goals can conflict 

New users use tutorials as a scaffold for building whatever they wanted to 

build using your project. 

Hypothetical Tutorial Using Persona 

Kira 

• CS Undergrad 

• Familiar with python 

• New to web dev 

Kevin 

• Designer 

• Knows basic 

Django Tutorial Runthrough 

• Installation should come first. 

• Bad installation hygiene 

• Django doesn’t lead users to isolate using virtualenv. 

• How do I debug? 

• Where to find help? 

• Projects vs Apps , what is the difference? 

• Idea of code reuse. 

• No real discussion of test-driven development 

• What is tests.py used for? 

• Schema migrations - no talk of South 

• How to style forms 

• Static and Media files - not explained in the tutorial 

• Template inheritance 

• Deployment 



Target Audience Chnage 

• Build a community or a library 

• If you want a long running project and a community you’ll need longer, more complete tutorial 

• Example: Railsbridge, PyStar.org 

• Comprehensive tutorials are longer, takes more time. 

• Modular structure can make this easier 

• Blind searching is hard, put linke in your docs instead 

• Better doc, “Intro to web programming using Django” 

Issues 

• Assuming familiarity with base 

• Unstated assumptions 

• List of directions withou why 

• Code samples 

• No obvious place for help 

Better Docs 

• Remember you were a novice once 

• Realize your own expertise 

• Have a user test your docs 

• Announce audience 

• Explicit Dependencies 

• Different tutorials for different audiences 

What’s new and interesting in standard library 

Presenter: Senthil Kumaran 

Track: II 

Description: 

This talk distills some intereting stuff from What’s new document from 2.7, 3.2 and upcoming 3.3 release. 

Look out for those new arguments to your favorite methods, functions add the wow! factor to your code. 

Heard of @lru_cache? 


Slides are Available: 

http://uthcode.googlecode.com/svn/trunk/presentations/pycon2012/index.html 

3.2. PyCon 2012 83


faulthandler 

• New faulthandler module. 

• This module contains functions to dump Python tracebacks explicitly, on a fault, after a timeout, or on a user 

signal. Call faulthandler.enable() to install fault handlers for the SIGSEGV, SIGFPE, SIGABRT, SIGBUS, and 

SIGILL signals. 

• It is version 3.3 and was contributed by Victor Stinner. 

lzma 

• The newly-added lzma module provides data compression and decompression using the LZMA algorithm, including 

support for the .xz and .lzma file formats. 

• lzma compression is usually better than bz2. 

• Python 3.3 

• Per Øyvind Karlsen, Nadeem Vawda and others. 

bz2 module 

• bz2.BZ2File can now read from and write to arbitrary file-like objects, by means of its constructor’s fileobj 

argument. (Nadeem Vawda) 

• bz2.BZ2File and bz2.decompress() can now decompress multi-stream inputs. bz2.BZ2File can now also be used 

to create this type of file, using the ‘a’ (append) mode. (Nir Aides) 

os module 

• sendfile() function which provides an efficent “zero-copy” way for copying data from one file (or socket) descriptor 

to another. ( Ross Lagerwall and Giampaolo Rodola’) 

• Use of sendfile instead of send provides 1.5x speed up! 

fwalk() function similar to walk() except that it also yields file descriptors referring to the directories visited. (Interesting!) 

* Since 3.2 - The os module provides two new functions, fsencode() and fsdecode(), for encoding and decoding 

filenames based on file-system encoding. 

packaging 

• distutils module is called packaging, helper functions for building, packaging, distributing and installing additional 

projects into a Python installation. 

• distutils is still provided in the standard library, but users are encouraged to transition to packaging. 

• New features from packaging will be available under distutils2 in PyPI. 

signal module 

• signal.signal() and signal.siginterrupt() raise an OSError, instead of a RuntimeError: OSError has an errno 

attribute. 

• signal module has functions such as pthread_sigmask , pthread_kill, sigpending, sigwait, sigwaitinfo. 



• Jean-Paul Calderone, Antoine Pitrou and others. 

socket module 

• The socket class now supports the PF_CAN protocol family. (Matthias Fuchs, Tiago Gonçalves) - Control Area 

Network Bus Drivers. 

• The socket class now supports the PF_RDS protocol family - Reliable High performance, low latency reliable 

connectioness protocol for delivering datagrams. 

ssl module 

RAND_bytes(): generate cryptographically strong pseudo-random bytes. RAND_pseudo_bytes(): generate 

pseudo-random bytes. (Both by Victor Stinner) Query the SSL compression algorithm used by an SSL 

socket, thanks to its new compression() method. You can also supress Compression. ( Antoine Pitrou) 

sys module 

The sys module has a new thread_info struct sequence holding informations about the thread implementation. 

>>> sys.thread_info 

sys.thread_info(name=’pthread’, lock=’semaphore’, version=’NPTL 2.13’) 

urllib package 

The Request class, now accepts a method argument used by get_method() to determine what HTTP 

method should be used. For example, this will send a ‘HEAD’ request. 

>>> urlopen(Request(’http://www.python.org’, method=’HEAD’)) 

urllib package 

• The parse.urlparse() function now supports IPv6 addresses as described in RFC 2732 

request.urlopen can take POST which can be an iterable. * http.client.HTTPSConnection, urllib.request.HTTPSHandler 

and urllib.request.urlopen() now take optional arguments to allow for server certificate 

checking against a set of Certificate Authorities, as recommended in public uses of HTTPS 

argparse - 3.2 

• argparse will be the future and optparse will slowly be deprecated. 

• Support for positional args, sub-commands, ‘required options’, pattern for specifying and validating options. 

• argparse has the ability to define subparsers, each with their own argument patterns and help displays: 

3.2. PyCon 2012 85


logging module - 3.2 

• The logging documentation has been augmented by a basic tutorial, an advanced tutorial, and a cookbook of 

logging recipes. 

• logging.config.dictConfig() - logging configuration with plain Python dictionaries. 

with open(’conf.json’, ’r’) as f: 

conf = json.load(f) 

logging.config.dictConfig(conf) 

from concurrent import futures - 3.2 

• Code for creating and managing concurrency is being collected in a new top-level namespace, concurrent 

first package is futures high level interface for managing threads and processes. * Inspired by java.utils.concurrent 

and Future Object. status checks (running or done), timeouts, cancellations, adding callbacks, and access to results or 

exceptions 

functools - 3.2 

• The functools module includes a new decorator for caching function calls. functools.lru_cache() can save repeated 

queries to an external resource whenever the results are expected to be the same. 

>>> import functools 

>>> @functools.lru_cache(maxsize=300) 

>>> def get_phone_number(name): 

c = conn.cursor() 

c.execute(’SELECT phonenumber FROM phonelist WHERE name=?’, (name,)) 

return c.fetchone()[0] 

... 

>>> get_phone_number(name) # cached lookup 

functools - 3.2 

• We have cache stats 

>>> get_phone_number.cache_info() 

CacheInfo(hits=4805, misses=980, maxsize=300, currsize=300) 

• OMG! Way to get unwrapped function. 

>>> get_phone_number = get_phone_number.__wrapped__ # uncached function 

• functools.total_ordering - rich comparison methods, a new decorator functools.total_ordering() will use a existing 

equality and inequality methods to fill in the remaining methods. 

@total_ordering 

class Student: 

def __eq__(self, other): 

return ((self.lastname.lower(), self.firstname.lower()) == 

(other.lastname.lower(), other.firstname.lower())) 

def __lt__(self, other): 

return ((self.lastname.lower(), self.firstname.lower()) < 



Magic happens. 

(other.lastname.lower(), other.firstname.lower())) 

itertools - 3.2 

>>> from itertools import accumulate 

>>> list(accumulate([8, 2, 50])) 

[8, 10, 60] 

collections 

• The collections.Counter class now has two forms of in-place subtraction, the existing -= operator for saturating 

subtraction and the new subtract() method for regular subtraction 

• http://en.wikipedia.org/wiki/Saturation_arithmetic If the result of an operation is greater than the maximum it is 

set (“clamped”) to the maximum, while if it is below the minimum it is clamped to the minimum. 

• All these features were added by Raymond Hettinger 

collections 

>>> tally = Counter(dogs=5, cat=3) 

>>> tally -= Counter(dogs=2, cats=8) # saturating subtraction 

>>> tally 

Counter({’dogs’: 3}) 

>>> tally = Counter(dogs=5, cats=3) 

>>> tally.subtract(dogs=2, cats=8) # regular subtraction 

>>> tally 

Counter({’dogs’: 3, ’cats’: -5}) 

unittest - 3.2 

• Improvements supporting test discovery for packages, easier experimentation at the interactive prompt 

python -m unittest discover -s my_proj_dir -p _test.py 

Interactivity! 

>>> TestCase().assertEqual(pow(2, 3), 8) 

pyc directories - 3.2 

• 3.2 onwards 

• Multiple implementations can refer to their own .pyc files. 

• mymodule.cpython-32.pyc, mymodule.cpython-33.pyc, and mymodule.unladen10.pyc 

• pyc files are now collected in a __pycache__ directory stored under the package directory 

3.2. PyCon 2012 87


• Imported modules now have a __cached__ attribute which stores the name of the actual file that was imported 

tag that is unique to each interpreter is accessible from the imp module 

WSGI 1.1.1 

• Well Intentioned Upgrade for WSGI to support Python3. 

• Informational PEP clarifies how bytes/text issues are to be handled by the WGSI protocol 

New string formatting 3.2 

• str.format_map 

• It can take dictionaries from defaultdict, shelve, ConfigParser, dbm. 

>>> import shelve 

>>> d = shelve.open(’tmp.shl’) 

>>> ’The {project_name} status is {status} as of {date}’.format_map(d) 

’The testing project status is green as of February 15, 2011’ 

>>> class PlaceholderDict(dict): 

def __missing__(self, key): 

return ’’.format(key) 

>>> ’Hello {name}, welcome to {location}’.format_map(PlaceholderDict()) 

’Hello , welcome to ’ 

threading 3.2 

• The threading module has a new Barrier synchronization class for making multiple threads wait until all of them 

have reached a common barrier point. 

from threading import Barrier, Thread 

def get_votes(site): 

ballots = conduct_election(site) 

all_polls_closed.wait() # do not count until all polls are closed 

totals = summarize(ballots) 

publish(site, totals) 

all_polls_closed = Barrier(len(sites)) 

for site in sites: 

Thread(target=get_votes, args=(site,)).start() 

ast module 

• The ast.literal_eval() function serves as a secure alternative to the builtin eval() function which is easily abused. 

>>> from ast import literal_eval 

>>> request = "{’req’: 3, ’func’: ’pow’, ’args’: (2, 0.5)}" 

>>> literal_eval(request) 

{’args’: (2, 0.5), ’req’: 3, ’func’: ’pow’} 

>>> request = "os.system(’do something harmful’)" 



>>> literal_eval(request) 


... 

ValueError: malformed node or string: 

array module - 3.3 

• array module takes long long type. 

shutil - 3.3 

• shutil.disk_usage() - total, used and free disk space statistics. 

Deprecation Warnings - 2.7 

• DeprecationWarning and its descendants are now ignored unless otherwise requested, preventing users from 

seeing warnings triggered by an application. 

• Previous Python 2.x releases had DeprecationWarning ON by default. Now, since the path to upgrade is 3.x, 

those have been silenced unless explictly requested. 

• You can re-enable display of DeprecationWarning messages by running Python with the -Wdefault (short form: 

-Wd) switch, or by setting the PYTHONWARNINGS environment variable to “default” (or “d”) before running 

Python. 

3.x Backported Features in 2.7 

• The syntax for set literals ({1,2,3} is a mutable set). 

• Dictionary and set comprehensions ({i: i*2 for i in range(3)}). 

• Multiple context managers in a single with statement. 

• A new version of the io library, rewritten in C for performance. 

• The ordered-dictionary type described in PEP 372 

• The new ”,” format specifier for Thousands Separator PEP 378 

• The memoryview object and this is further improved in 3.3 

• A small subset of the importlib module and full version of importlib will be in place in 3.3 

Dictionary Views 

• viewkeys(), viewvalues(), and viewitems() return an object called views. 

Bug fixes in modules 

• http://docs.python.org/whatsnew/2.7.html#new-and-improved-modules 

• 2.7.x is the maintained bug fix release. All bug reports which have been reported have found it’s way to 2.7.x 

• Only new features do not make it to 2.7. 

3.2. PyCon 2012 89


• It’s a stable release which can you to upgrade to Python 3.x 

There is more 

• http://docs.python.org/dev/whatsnew/3.3.html 



• Misc/NEWS file. 

print(’{0} {1}’.format(’Thank’,’ you!’)) 

• Presentation - http://bit.ly/pycon2012stdlib 

3.3 Django Con US 2012 

Location: Hyatt Regency Crystal City, Washington, D.C. 

When: September 4th–6th 2012.djangocon.us 

Description: 

DjangoCon US is the main opportunity for djangonauts to come together in the United States. It will 

consist of two tracks of talks over three days, and will also provide for open sessions, lightning talks, and 

a development sprint after the conference. 

3.3.1 Tuesday 

Keynote 

Presenter: Eric Sterling 

Track: N/A 

Description: 

Keynote Address 

Local vs National 

• Local involvement is critical 

• Role of Money 

• Influence of people with money and access 

State Rights 

• Citizen power scares elected officials. 

• Knowledge is power 

• Knowledge and passion go together 



Maintaining Your Sanity While Maintaining Your Open Source App 

Presenter: Mark Lavin 

Track: I 

Description: 

Django has a thriving community of open source pluggable applications. Maintaining an external application 

can be a rewarding experience and doesn’t need to take over your life. Learn what it takes to package, 

document and test your Django app so that others can use and contribute to what you’ve built. 

Brewedbyus.com 

Packaging 

• Package to use PIP 

• Direct users to use PIP first 

• Setuptools find_packages 

‘ Include Package Data ‘ 

• Pull description from ReadMe 

• Follow PEP386 

• 3 Numbers 

• Be consistent in version numbers 

• MANIFEST.in 

• Register on Pypi! 

Documentation 

• No Giant READMEs 

• Docs should be available online 

• Use sphinx and Read the Docs 

Things To Document 

• How to install 

• Description of the project 

• How to configure the app 

• Release Notes 

• Be clear 



Hosting Docs 

• Setup post commit hook 

• Link to your repo 

Testing 

• Tests should not fail without an example project 

• Tests should not depend on small settings changes 

• Test only models 

• Run tests.py 

Test with TOX 

• User virtualenv to test 

• Test different versions of python/django 

• Test different DBs 

Goals 

• State your goals 

• Set expectations 

License 

• Always include a License 

• Prepare for the future 

Be Ready for Python 3 

• Be ready to be replaced 

• TOX 

Rejecting Requests 

• You can’t reject every contribution 

• Don’t reject based on tests and docs. Point them in the right direction. 

• Do NOT accept every request. 

• If you accept it, be prepared to maintain it. 



Avoid Burnout 

• Its okay to step away for a while. 

• Should not feel like a burden 

Debugging Live Python Web Applications 

Presenter: Amjith Ramanujam 

Track: II 

Description: 

Monitoring tools record the result of what happened to your web application when a problem arises, but 

for some classes of problems, monitoring systems are only a starting point. Sometimes it is necessary 

to take more intrusive steps to plan for the unexpected by embedding mechanisms that will allow you to 

interact with a live deployed web application and extract even more detailed information. 

Why Debug? 

• Obvious - Python exceptions 

• Subtle - Memory leaks 

• Performance - Slowness 

• Heisen - Only shows up in production. 

• Devops - Says no to debugging in production 

Things to Avoid 

• Do not do more damage 

** Crashing the site ** Loss of customer data 

Manage Risk 

• Use software that restricts what you can do. 

• Script changes 

• Test what you are going to do first 

• Develop contingency plans 

Monitoring 

Passive 

• Collection of log info 

• Collection of Python exceptions 

• Collection of performance data from hosts 



• Logstash 

• graylog2 

• Sentry and New Relic 

Server Monitoring 

• Monit 

• Munin 

• Cacti 

• Nagios 

• New Relic 

Application Performance Monitoring 

• New Relic 

Web Page Performance 

• YSlow 

• GooglePageSpeed 

• WebPageTest 

• Firebug 

Transaction Tracing 

• Newrelic provides tracing at the function/method level. 

• Monkey patching instrumentation 

newrelic.api.function_trace.wrap_function_code 

Profiling 

• Thread Sampling 

** Plop ** statprof * Full Profile ** cprofile ** pytrace 

Browser 

• django-debug-toolbar 

• paste error middleware 



Live Debugging (Dark Art) 

• Building backdoors 

• USing Pythons built in logging 

ispyd 

• Interactive debug console for use in production 

• Application must have backdoor, listens to socket 

• Demo 

• bit.ly/LiveDebugging 

Views Can Be Classy 

Presenter: Kenneth Love 

Track: II 

Description: 

An overview of what class-based views (CBVs) are, which ones are available, and how to use them in 

your projects. I’ll also cover creating mixins, where function-based views still make sense, and how to 

test CBVs. As part of this, I’ll be covering my django-braces (https://crate.io/packages/django-braces/) 

package. 

Who Am I 

• @kennethlove 

Class Based Views 

• Bad right? 

Bad 

• Decorators must be wrapped around dispatch() 

• Inheritance chains. Not very obvious what is going on. 

• Combining mixin and views creates order exceptions 

• So much more going that you can’t see. 

MRO 

• Two classes that inherit from the same base class, but has methods in different orders. 

• Bit.ly/PythonMRO 

• A little enterprise-ish 



Why Use CBV 

• Faster implementation 

• Keep views.py concise 

• Everything is a class (Models, Forms, Templates) 

• Special cases are not special enough to break the rules, Views should be classes too. 

Batteries Included 

Object Based Views 

• Single object 

• Multitple object mixins 

• Detail/List View 

Form Based Views 

• Form View 

• Create View 

• Update View 

Date based Views 

• Year 

• Month 

• Day 

• Date 

Utility Views 

• Template Response 

• Template View 

• Redirect 

• View 

Common Methods 

• Dispatch, (Get, Post, Put, Delete) 

• get_context_data 

• get_object 

• get_queryset 



• get_form_class 

• get_form_kwargs 

• get_success_url 

• form_valid 

• Demo 

• Add base classes for common functionality in views. Convert them into Mixins. 

Customizing 

• context_object_name 

• Base classes and Mixins are the same thing. 

• Mixins have a single purpose 

• Base class can have multiple mixins or whole new functions. 

Design Patterns 

• Using None as default 

• Move mixins to the front 

• Prevent code from running if it doesn’t meet requirements (Users aren’t logged in). 

Decoration 

• You can decorate inside the urls.py 

• You can add a second variable and wrap the .as_view() method. 

Function Based Views 

• Session manipulation 

• Login/Logout 

Django on Gevent 

Presenter: Cody Soyland 

Track: II 

Description: 

This is an introduction to using the Gevent networking library to empower your Django application with 

realtime features and resource-efficient cooperative concurrency. Django’s synchronous APIs make it 

impractical to use in a callback-based networking library, but fast single-threaded concurrency is still 

possible using the mind-blowing capabilities of coroutines. 



Real Time Web 

• Delivery of information as it happens 

• Open connections 

• C10K Problem 0 How do web servers handle thousands of connections at once 

New Challenges 

• Non blocking I/O 

• Low resource overhead 

• Distributed 

Concurrent Systems 

• Processes 

• Threads 

• Callbacks 

• Coroutines 

Threads 

• Memory Overhead 

• Readable, synchronous interface 

• Guaranteed cooperation 

Callbacks 

• Call stack not preserved 

• Simple things are intuitive 

• Complex things become confusing 

Coroutines 

• Call stack preserved 

• Synchronous API 

• Benefits of threads without the non-determinism 



Greenlet 

• True coroutines in Python 

• Expands upon greenlet to provide “green threads” 

• Provides an event loop 

Green Threads 

• POSIX threads are pre-emptive 

• Green threads are cooperative 

• Very light weight 

Django Software Foundation Keynote 

Presenter: Russell Keith-Magee 

Track: N/A 

Description: 

State of the DSF 

• Board Members 

• Developer Members 

Corporate Members 

• Small/$500 

• Medium/$1000 

• Large/$5000 

Responsibilities 

• DSF doesn’t own copyright 

• DSF licenses code 

• Contributor License Agreements 

• Django Trademark 

CLA 

• Submit one if you have contributed code 



Not Okay To 

• Anything that implies endorsement 

• Especially in Commerce 

Trademark 

• Cannot start a company with Django 

Django Con 

• Anyone can run a Django Conference 

• DSF licenses “DjangoCon” to the organizers 

• DSF not involved in DjangoCon operation 

Infrastructure 

• Djangoproject.com Rebuild 

• people.djangoproject.com 

• djangosnippets.com 

• $1000 to read the docs 

• Hosting thanks to Heroku, MediaTemple 

• Training 

• Sprints/PyCon sprint 

• Conference Sponsorships (Pycon/DjangoCons) 

• Travel Grants 

• DSF as a publisher? 

• Merchandise 

• Help us help you 

Designing Your Open Source Project 

Presenter: Bryan Veloso 

Track: II 

Description: 

There is an ever present rift between designers and developers and it is one that has existed since our paths 

started crossing. We’ll talk about how to be mindful of design in your own open source project and how 

to attract designers to help your project become even more awesome. 



Design and Open Source 

• Developers and Designers are quite similar. 

• Respecting clarity 

• Find hybrids 

• Teach each other 

These notes are short, because this talk was very good and I forgot to take a lot of notes.. 

https://speakerdeck.com/u/bryan/p/designing-your-open-source-project 

API Design Tips 

Presenter: Daniel Lindsley 

Track: I 

Description: 

The focus of this talk will be on some pragmatic tips on how to design programmatic (non-web-based) 

APIs for use by other developers. 

What 

• Not HTTP APIs 

• Programmatic APIs 

• Libraries 

Why 

• Other people use your code all the time 

• You might be not happy with past you. 

You cannot make everyone happy 

• You make assumptions about your environment 

• These don’t always apply for other people 

• More people are happy if they can extend libraries 

• No copy-paste should be needed 

• Good docs matter 

• Real world use is the best use. 



Design 

• Bottom up 

• Top down 

• Bottom up sucks 

• Top down feels better 

• Everything fits together 

• Less duplication 

• Test Driven Design 

Things you Should Do 

• Small components 

• Reflection (To and From) Be able to reverse operate 

• Narrow Familiarity - How similar is the code itself. 

• Assume the worst 

• Use it, then step back and ask yourself how to make it better, easier for the user 

• Quick wins 

• Return values should be consistent 

Things you should NOT do 

• Low level API is good enough 

• Wildly different return values 

• If its diffcult to test, its probably wrong 

Django Specific 

• Pluggable backend all the things 

• Declaritive syntax 

• Avoid global state (Use __init__ ) 

• Decrease reliance on self 

• Resist urge to use magic 

https://speakerdeck.com/u/daniellindsley/p/api-design-tips 



3.3.2 Wednesday 

Keynote - Fixing Computer Science 

Presenter: Selena Deckelmann 

Track: N/A 

Description: 


Fixing Computer Science 

• Computer science enrollment has decreased 

• 50% growth expected by 2018 

• 70% of FOSS devs have at least a Bachelors degree 

Open Source Lifestyle 

• Freedom 

• Sharing 

• Licensing 

• Work/Learn/Teach 

Teaching Someone How to Program 

• Mentoring 

• Filesystem navigation 

• Very confusing for new users GUI to Terminal 

• Explain filesystems for BOTH GUI and Terminal 

• New users get frustrated easily 

Open Licensing of Materials 

• Paris Declaration (http://bit.ly/MTvxzl) 

How to Teach 

• Just challenging enough to be interesting, but not hard enough to be frustrating 

• Modeling 

• Guided practice 

• Independent Practice 

• Generalization 



• Have them teach/explain the concept back to you 

Lost 35% of CS courses in High Schools 

• Other academic areas are represented in High Schools 

• Shortage of teachers in areas where CS is desired 

• Minorities are not represented 

• Computer science seen as an elective 

• Not a part of Math or Science 

What we need to Teach 

• Computational Thinking 

• Large gap between teachers and open source developers 

Invite more people into Open Source 

• Speak language of education 

• Open source IT in schools is not the answer, right now. 

• Teachers are nautural alies for FOSS developers 

Access to computer science is a social justice issue. 

The Dungeon Master’s guide to Django’s ORM 

Presenter: MALCOLM TREDINNICK 

Track: II 

Description: 

If you’ve ever been curious about Django’s ORM implementation, you will have noticed the required 

learning curve. Some of the code is fairly complicated. Other bits are worse. Here comes a 30 or 40 

minute guided tour of the uncharted realms: how the pieces fit together, where to look for things, why the 

current design is what it is. A portion of this is my fault; I should probably explain myself. 

Me 

• Python since 1997 

• Django user since Sept 2005 

• Django committer in May 2006 



History 

• 12 July 2005, import from private svn repo 

• Magic removal branch - 1 May 2006 

• 3 Ways of structuring the ORM were attempted 

• 4 July 2007 - Merge new ORM back in (Unicode) 

• 2008 - 1.0 

Worth It 

• Code structure has remained fairly stable 

• Abstraction feels right 

• Minor code duplication in recent times 

• Mostly logical code flow. 

Useful Rule 

• Developers who were here before you where probably not insane. 

Layers 

• django/db/modles/query.py - 1800 lines 

• django/db/models/sql/query.py - 2000 lines 

• django/db/backends/* - base.py, operations.py 

Down the Rabbit Hole 

• Simple filter query Article.objects.filter() 

• db.models.query.QuerySet 

• db.models.sql.query.Query 

• db.models.sql.where.WhereNode 

• db.models.sql.compiler.SQLCompailer 

• Continually filter from previous filters. 

Background 

• Nested queries 

• QuerySets can be merged 

• All aliases in a QuerySet can be changed at once. 



Cryptography for Django Applications 

Presenter: ERIK LABIANCA 

Track: II 

Description: 

A review of encryption in the context of a web application storing sensitive information. Topics covered 

include choosing whether to use crypto, selection of tools, proper usage (including examples), and 

operational considerations with respect to security assessment. 

Who 

• Developer, Not cryptographer 

• Should you trust me? Maybe. 

TLDR 

• Analyze risks 

• Don’t write your own 

• Operate correctly 

• Commit to keeping up 

Hacks happen all the time 

Analyze 

• Data 

• Systems 

• Identify Vulnerabilities (Backups, Laptops, Compromised systems) 

• Analyze controls (Locked safe, Cryptography) 

Hash Properties 

• No Keys 

• Easy to compute the has value 

• Very hard to generate a message for a known hash value, modify without changing the hash. 

• Used for signed cookie and sessions in Django 

• Password verification 



Symmetric Encryption Algorigthms 

• Secret Key 

• Reversible 

• Requires shared secret 

Public Key Cryptography 

• Asymmetric 

• N-way 

• 2 + keys 

Asymmetric Encryption Properties 

• Lots of complex keys 

• Slow 

Really need to be running HTTPS 

Django 

• Django does it right 

• Enable HTTPS 

• Enforce use of HTTPS via redirects 

• Inform django your useing HTTPS 

Tell Django your using HTTPS: 

SESSION_COOKIE_SECURE = True 

CSRF_COOKIE_SECURE = TRue 

SECURE_PROXY_SSL_HEADER 

• Protect data via SKC 

• Support encrypted payloads 

• FIPS / NIST recommendations 

• 1.4 is much better than 1.3 

• Keys can be kept in memory 

Lesson on Testing 

Presenter: David Cramer 

Track: II 

Description: 



Learn from failures (and successes) around testing patterns and culture in a growing company, both in 

amount of code, and number of engineers. We’ll dive into how DISQUS adopted testing, and the many 

challenges we’ve had to overcome. 

Time Consuming to Write 

• Takes long time to write good tests 

• 10 lines of code has 36 lines of tests 

• 50% of time is spent writing tests 

• Legacy code is expensive to test 

• Spend more time running them, hardware is cheap. 

• Lots of Demos 

• If path changes, mock fails 

• Tests break a lot on code changes 

• Mock is useful for testing external services 

• Test the lifecycle of requests 

• Kind of works 

• Brittle 

• Use Phantom JS for js tests 

• Start with a Goal 

• Write testable Code 

• Break up code 

• Create Structure 

• Put tests in the top level directory 

• Document best practices 

• Continuous builds 

• Tests should be a part of your culture 

• Code Review 

• Test throughout the process 

• Nose (nose.readthedocs.org) 

• Tests aren’t enough 

• Deep trace 

• Code Reviews 

• Culture is Key 



Keynote - BDFL 

Presenter: Adrian Holovaty 

Track: N/A 

Description: 


Local Flavor Failure 

The Year Ahead 

• Between Now and the next Django Con 

Django 1.5 

• Alpha Oct 1 

• Beta Nov 

• Final release by Christmas 

• Porting to Python 3 

Coming UP 

• Remove local flavor 

• Comments also being removed 

• Removal of settings 

• New User/Auth design 

• Build your own User model 

• Concept of an APP object, get rid of global state 

PJAX 

• Single piece of middleware 

• Real time pushing 

Django Forms in an API World 

Presenter: TAREQUE HOSSAIN 

Track: I 

Description: 



In a world of django powered web APIs and arbitrary consumers, traditional methods of rendering & 

validating django forms are ineffective. We discuss how to uphold the API provider/ consumer separation, 

yet utilize provider’s django form subsystem to power forms in a pure JS consumer. We achieve this by 

serializing form configurations, rendering metadata, error handlers & exposing them over API. 

Whats wrong with Forms 

New Way 

• Django forms live on API server 

• Validates/saves API 

• Trying to match frontend to the API 

API Clients 

• Website no longer lives on the same server 

• Forms exist on phones/web sites/other devices 

Issue 

• Forms have to be re created on each device 

• Browser considered a device 

• API/Form doesn’t match up 

What is a form 

• Blank document with places to add informations 

Django Forms 

• Binds/Validates data 

• Display 

• Model Forms 

• Easy 

• Widgets 

Distributed Services 

• Build an API 



Deliver Form Definition over API 

• Define form in API 

• Serialize Form 

• Deliver, Recieve, Validate, Show Errors, and Process 

• Render, Submit, Validate 

• django remote forms 

• Encapsulate processing in form.save similar to model form 

Render forms with Handlebars/JS/CSS 

Why Django Sucks 

Presenter: KENNETH REITZ 

Track: I 

Description: 

This talk dives into the specifics of why Django isn’t always the best tool for the job, general frustrations 

with the framework, and potential fixes. It will balance out with many Django praises too, of course. 

Django Benefits 

• Makes modular decisions for you 

• Makes security decisions for you 

• Excellent Docs 

• Installable 3rd party apps 

• Community resources 

Django Apps 

• Tools 

• WEb Process 

• Worker Processes 

Single Codebase is Great 

• Benefits of the whole stack 

• Architecutre 

• DRY 

• Only deploy once 



Single Codebases are evil 

• Tightly coupled 

• Broad knowledge is required 

• Tech debt is high 

• Must deploy everything at once 

Constraints are Good 

• Editors vs IDE 

• Mac OS X vs Desktop Linux 

• Pen and Paper vs Digital Notes 

• Monolithic apps 

Seperated Services 

• API vs Front End 

• Build for Services 

• Decouple front end from backend? 

Django not for API services 

Django as API Consumer 

• Database is handled by API 

• Makes modular decisions for you 

Flask 

• Simple HTTP 

• WSGI App Framework 

• Jinja2 Template 

• Lots of docs 

• Very simple 

• Bring your own batteries 

• No ORM for form validation 

• Greater flexibility 

• Configuration is simple 



3.3.3 Thursday 

Under the Microscope: Evaluating Existing Django Code 

Presenter: JOE JASINSKI 

Track: I 

Description: 

As a Web development firm that specializes in Django, we receive many inquiries from organizations 

looking for assistance with their existing Django websites. This session will describe our process for 

evaluating existing codebases and deployment structures. The goal is to provide a framework for evaluating 

other people’s code and understand the scrutiny your code may someday endure. 

Django Consulting 

• Clients call when they have staff that cannot handle load 

• Clients have developers that leave on them 

When clients call 

• Wide array of scenarios 

• Caller often has no technical knowledge 

• Is developer accessible 

Get Access to the Site 

Code Review 

• Standard questions 

• Document 

• Enables comparison 

• Find code 

• find manage py 

• Lookup urls.py 

• Checklist 

Questions 

• What webserver 

• What django version 

• Check settings 

• Using logging 

• Do a pip freeze to see what packages are being used 



• What type of version control is being used if any. 

• Tests 

• South? 

• Virtualenv? 

• Do they have a README 

• Document things you find. 

• Keep an open mind 

• Research the client 

Additional 

• Assume your code will last forever 

• Code like django core team is reviewing it 

• Resist pull for speed from your boss 

• Excercise your right to say no. 

• Take time to be great. Bad coders hurt good ones. 

• It’s your reputation 

Accelerating and Enhancing Django with Redis 

Presenter: JOSHUA “JAG” GINSBERG 

Track: II 

Description: 

This tutorial introduces Redis, an in-memory key-object NoSQL datastore. We discuss out-of-the-box 

ways Redis can help improve the performance of your Django deployments, ways that using Redis instead 

of SQL for some data management can accelerate your apps, and more advanced and unconventional uses 

for Redis to solve real-time and big-data problems. 

What is Redis 

• Absurdly Fast 

• All in memory 

• Keys and objects 

• lists, strings, sets 

What isn’t redis 

• Not NoSQL 

• data structures limited and not nestable 

• No views 



• Not a simple key value store 

• Not memcached 

Crash Course 

• Get and Set right away 

• Doesn’t care about char encoding 

• Linked lists 

• Sets 

• Scores, floating point values 

• Dictionary keys not ordered 

Boring 

• Basically a cache 

• Clearing house 

Case Study 

• Run hundreds of ads on Facebook and snapshot their performance as fast as possible. 

• Facebook API does not update stats in real time 

• Gather stats and store as a hash map 

Drop In 

• Celery 

• Cache for Django 

• Use Redis for celery instead of RabbitMQ 

Give SQL A break 

• Counting 

Good Ideas 

• Use hierarchy 

• Atomic transactions 

• Different keyspaces for difference apps 

• Do not store large values in Redis 

• Do not count on persistence 

• Don’t forget that operations are Atomic 



LUA - Lightweight Embeddable Scripting 

• Construct and deconstruct JSON 

Redis Hate 

• Non persistence 

• Need to hire a Sys Admin 

• Needs to fit in Memory 

• Redis does not return memory to system 

• Redis on dedicated hardware with lots of RAM 

• Single process, single CPU 

Django Core Team 

Presenter: Djanog Core 

Track: I 

Django Core 

What is a potential plan or way forward for Django to move forward in Real Time 

• Not confident about time (1 or 10 years) 

• Web is going toward “Thick Clients” (Phone Apps) 

• Adapt or Die 

• Django must move forward but not sure about timeline 

What is happening on the Schema change API 

• South is the defacto solution for migrations 

• Adding south abstractions in to 1.5 

• Time to start rolling migrations into Django 

Python 3 - Strategies and Tips for Migration 

• Not hard 

• Django has documented how they approached the change 

• Confident that 3rd party apps will be able to convert it easily. 

New Areas of Interest for new dev 

• Real time 

• Performance 

• 2.0 game plan 



How do you break contrib out and still maintain them (namespaces) 

• Support for better namespacing in Python 3.3 

• In practice cannot use name spaced packages 

What would be the thing that you think you could actually change in 2.0 

• Everything is a view. 

ORM is agnositc, should it be broken to allow performance improvements in Postgres 

• There should be more support for performance improvements in certain DB Backends 

• 1.5 1.6 might have some improvements 

Admin Radical restart? 

• Backend could serve an API and allow clients to provide better admin functionality 

• Very challenging project 

• Very large task 

• A lot of the admin does not use django tools that exist now 

• Admin does not use Class based views 

Will a team approach to design help django? 

• Designers work best not solo 

• Motivate each other 

• Unified vision 

Models and Migrations and Schemas - oh my! 

Presenter: ANDREW GODWIN 

Track: I 

Description: 

A look at the past, current and future of schemas, migrations and Django, and what it means for both 

website developers and ops staff. 

Past 

• Databases hate schema changes 

• Locks whole tables 

• Hammer I/O 

• Inconsistent 

• Django-evolution 

• dmigrations 



• South 0.1 2008 

• 0.2 - MySQL 

• 0.3 - Dependencies 

• 0.4 - Alter columns 

• 0.5 - ORM Freezing 

• 0.6 - Field introspection 

• 0.7 - data/schema split, missing defaults for Not null, custom fields ignored. 

Things to Change 

• No rebase/collapse 

• Opaque migrations (Impossible to peek inside migrations) 

Databases 

• Code/schemas split 

• Database isn’t going to use Git 

• Extra fields are fine 

• Missing fields are not 

• Painful/slow to sync 

The Future 

• django.db.backends.schema 

• Database abstraction layer 

• contrib.migrations 

• Migration creation/running, will replace South for these operations. 

• No frozen ORM 

• Raw SQL support 

• SQL Output support 

South 1.0 

• Python 3 support with Django 1.5 

• Python 2.6 required 



Django Nose 

Presenter: ERIK ROSE 

Track: II 

Description: 

Django’s testrunner gets you started quickly, but you soon hit your head on its limitations as your project 

grows. By trading it for nose, a testing framework popular in the wider Python community, we can reduce 

boilerplate, boost performance, and improve testing UI, with only a few lines of setting changes. Re-use 

your test DBs, integrate with Jenkins, split your suite into pieces, and more. 

Django Tests Pain 

• Crowded 

• Slow 

• Overbroad 

• Rough 

• Extensible but not scalably so 

Installation 

• pip install django-nose 

• django_nose 

• django_nose.NoseTestSuiteRunner 

Discovery 

• Find test by reg ex 

• @istest 

• Subclasses of TestCase 

• No more accidental shadowing 

• No more forgotten imports 

Functions as tests 

• Package level setup and tear down 

Test Generators 

• cannot use in TestCase subclass 



Test Attributes 

• attribute plugin 

• @attr(‘selenium’) 

• manage.py test - a selenium 

Goodies 

• Custom error classes 

• Extensible 

• Plugins 

Speed 

• Switching contexts 

• Encourages running tests 

• No more test fixtures!

Conference Notes and Best Practices Release 1.3 ... - Read the Docs

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?