An introduction to

Web Operations

The running of a popular website

http://changedmy.name/talk---web-operations/

James Pearson

[email protected]

github.com/xiongchiamiov

changedmy.name

reddit.com/r/CsCareerQuestions

Table of Contents

What is Web Operations?

General Web Architecture

Server Provisioning

Monitoring and Alerting

Web Security

Performance

What is Web Operations?

Web Operations Engineer

aka Site Reliability Engineer

aka DevOps Engineer

Ensure shit works.

We're still programmers

General Web Architecture

Server Provisioning

#!/bin/bash -ex

source ../../shared/setup.sh

cd /tmp/build

PACKAGE=httpd
VERSION=2.2.22
DESTDIR=/tmp/fpm/$PACKAGE-$VERSION
mkdir -p $DESTDIR
export DESTDIR

wget --no-verbose "http://archive.apache.org/dist/httpd/httpd-$VERSION.tar.gz"
tar -zxf httpd-$VERSION.tar.gz
cd httpd-$VERSION
# We need to use the included version of apr:
# http://apache-http-server.18135.n6.nabble.com/Bug-53162-New-Cannot-compile-Apache-2-2-22-td4940024.html
./configure --prefix=/etc/httpd \
            --bindir=/usr/bin \

Monitoring and Alerting

Healthcare.gov

http://dpaste.com/1756566/#wrap

"Nobody could even tell us if the system was up as we were sitting there, except by taking out laptops and trying to go on it."

You can't fix what you can't measure.

What do we measure?

What do we use to measure?

1. It should be easy to get data in.
2. It should be easy to get data out.

Graphite

https://graphite.readthedocs.org/

StatsD

Stats::recordEvent('talk.web-ops');

pipe-to-graphite

# Produce a huge number of stats about mysql
mysql -u root -e "show global status" |
# Skip the first line (column headers "Variable_name Value")
tail -n +2 |
# lower-case everything because Capitals_with_underscores_are_annoying
tr '[A-Z]' '[a-z]' |
# Prepend 'mysql.' and turn a few instances of 'name_' into 'name.' mainly
# so they are grouped in the Graphite UI
sed -re "s/^(com|handler|innodb|key|qcache|select|sort|threads)_/./" \
     -e "s/^/mysql./" -e "/\S+\s+[0-9.-]+/!d"

Web Security

Security Categorizations

Confidentiality

What most people think of - preventing people from seeing things they shouldn't.

Data Integrity

Not only should people not be able to see things they shouldn't, but they shouldn't be able to modify them either.

Availability

Users being able to do what they want to.

Consistency

Auditability

Common Vulnerabilities

Cross-Site Scripting (XSS)

Cross-Site Request Forgery (CSRF)

http://itbrokeand.ifixit.com/2013/07/21/csrf-how-we-do-it.html

Performance

Frontend

Questions?

http://changedmy.name/talk---web-operations/

[email protected]