micmath blog

Publish A Developer Blog on GitHub

Posted Monday, 05 April 2010 by Michael Mathews in Code.

If you use GitHub you may not realise you already have access to website hosting, accessed via your user name. You can publish content there, once it is set up, but I want to go a step further and set up a blog. This post documents the steps I followed to do that.

Set up you github user page

First you need a user account on GitHub. If you go to a URL like http://yourUserName.github.com you will see the instructions you need to follow to set up your user page. I'll detail how I created mine:

Create a pages repo on github. Mine will be at http://micmath.github.com/

$ mkdir micmath.github.com
$ cd micmath.github.com
$ git init
$ echo "My GitHub Page" > index.html
$ git add index.html
$ git commit -a -m 'Initial commit.'
$ git remote add origin git@github.com:micmath/micmath.github.com.git
$ git push origin master

It can take about 10 minutes for the initial setup. That will give me time to install a few other things on my local machine...

Install the template system

GitHub supports Tom Preston's Jekyll engine and Pygment natively, so it makes sense to use those for a blog hosted on GitHub. I'm using gem and port to get the software on my Mac, you may need to use a different installation process supported by your own OS.

Install Jekyll for templates and site generation.

$ gem install jekyll

Install Pygments, for source code highlighting.

$ sudo port install python25 py25-pygments

Set up the site file structure.

$ mkdir _posts
$ mkdir _site
$ mkdir _layouts
$ touch _config.yml
$ mkdir css

Add the Pygment stylesheet

Pygment can generate it's own stylesheet, to be used for source code highlighting, using the command shown below. Jekyll uses the CSS selector ".highlight code", so add that in and create the file, like so:

$ pygmentize -S default -f html -a ".highlight code" > css/syntax.css

Configure Jekyll

You can configure Jekyll to your liking. For example I have my own Apache server running locally so I don't need Jekyll to run one.

$ cat _config.yml
auto: false
server: false
pygments: true

Create the default layout and css

$ touch _layout/default.html
$ touch css/default.css

In _layout/default.html, add some template code. Jekyll uses the Liquid template syntax, plus a few Jekyll-specific extensions.

$ cat _layout/default.html
<!DOCTYPE html>
<html>
    <head>
        <title>{{ page.title }}</title>
        <meta http-equiv="Content-Type"
              content="text/html;charset=utf-8">
        <link rel="stylesheet" type="text/css" media="screen"
              href="/css/default.css" />
        <link rel="stylesheet" type="text/css" media="screen"
              href="/css/syntax.css" />
    </head>

    <body>
        <h1>{{ page.title }}</h1>

        {{ content }}
    </body>
</html>

Create some content

Finally, all the infrastructure is now in place and we can start blogging! To create the content for a blog post, simply add a new markdown file to your _posts folder. The name of the file should be formatted like yyyy-mm-dd-your-title.markdown.

$ cat _posts/2010-04-05-hello-world.markdown
---
layout: default
title: Hello World
---

This is an example blog post.


This is a just a _test_. Here's some code, which will get syntax highlighting from Pygment:

{% highlight js %}
    function Foo(x, y) {
        this.x = x;
    }
{% endhighlight %}

Generate the HTML output

If you want to preview your formatted blog post you can run jekyll locally. This step is technically unnecessary, since GitHub will automatically run this same command for you on the server whenever you push new content. However it's handy if you want to proofread and tweak anything before publishing.

$ jekyll
Configuration from ./_config.yml
Building site: . -> ./_site
Successfully generated site: . -> ./_site

There is no need to publish the HTML generated in your own workarea, GitHub will automatically regenerate it whenever you commit anyway. So, if you haven't already done so, add the _site folder to your .gitignore file.

$ cat .gitignore 
.DS_Store
_site

Publish the content

Publishing is easy, just commit and push and new files. If you haven't already done so, add all the new folders and files to git. Finally commit and push:

$ git commit -a -m 'Added source code example.'
$ git push

permalink | Tags: git.

Transparent Content Filtering for Web Developers

Posted Sunday, 04 April 2010 by Michael Mathews in Code.

Imagine you want to play around with some JavaScript resource that is installed on a client's live server. You might want to do this in order to test some changes you've made to that file: it may work on your development server, but you want to see if that will still be true when it goes live.

This can be tricky. You definitely don't want to upload and pray. Maybe some specific configuration that is only on the live server will cause your AJAX features to behave in unexpected ways? How could see those errors for yourself without anyone else seeing them at the same time?

One way is to use a content-filtering proxy. This is basically an application that sits between your web browser and the your internet connection. Any request that browser makes will go through the proxy and any response from the internet will come back through that same proxy. Once you configure this, it's invisible to the browser. So, to accomplish what we need we could configure the proxy to replace any SCRIPT tags in the page on returned from the live server with a different SCRIPT tag that points to the new file on your development server. Everything else will be exactly as it is on the live server, because the page you will see will be coming from the live server. Only you, thanks to your proxy, will see the affects of the new JavaScript.

To illustrate I'm going to show how I could use this technique to insert my own picture into my view of the New York Times Technology Page. Niether the browser nor the server will know that anything has been changed, only I will see the alteration. Naturally it's possible to change anything on the page, I'm changing an image only because it's easy to see.

To accomplish this you'll need two prerequisites: You'll need to be able to install a proxy server, and you'll need to be able to write Perl-style regular expressions.

Let's start by installing a proxy server. I'm on a Mac, but you may need to adjust this for your own operating system. I've decided to use Privoxy because it's free and works on a wide range of computers. The easiest way to install Privoxy on Mac is to use the Darwin Ports package manager.

$ sudo port install privoxy

Once that completes, you will, by default, have a privoxy binary installed in /opt/local/sbin/privoxy. Before you start it, I would suggest a couple tweaks to the configuration file.

$ sudo vi /opt/local/etc/privoxy/config

We want to tell privoxy to use our own personal files as part of the configuration, this will make it easier and safer to tweak the configuration going forward. My personal configuration files will live in ~/.privoxy/, so I'll add the following two lines to /opt/local/etc/privoxy/config:

filterfile /Users/michael/.privoxy/user.filter
actionsfile /Users/michael/.privoxy/user.action

Save and close the main cinfiguration file now, that's the last time we'll need to touch it. Now we can create a folder to hold our personal configurations, and add a couple files to it, like so:

$ mkdir /Users/michael/.privoxy
$ touch /Users/michael/.privoxy/user.filter
$ touch /Users/michael/.privoxy/user.action

Ok, we can now start privoxy up, and it will include our own (temporarily empty) configuration files.

$ sudo /opt/local/sbin/privoxy /opt/local/etc/privoxy/config

Now you need to configure your web browsers to use your new proxy. On Mac, go to System Preferences > Network > Advanced > Proxies and set the Web Proxy (HTTP) and Secure Web Proxy (HTTPS) items to use 127.0.0.1:8118. Save and close the Preferences, and open up a web browser like Safari.

To test that your proxy is running and that your web browser is using it, go to the following URL: http://config.privoxy.org/. You should see some information about your running Privoxy application. By default privoxy will filter out most web ads, though you can adjust this. The important thing is we can add our own filtering. To do that we need to edit those two configuration files we created earlier.

First lets create a filter, one that will replace the URL to an image on the NYTimes domain, with an image on my own development server. The rule for that will go in my /Users/michael/.privoxy/user.filter file and looks like this:

FILTER: justtesting This is a test.
s%http://graphics\.nytimes\.com/example\.jpg%http://192.168.0.3/example.jpg%g

(Those aren't real URL's, I've shortened them for purposes of illustration, the point is you can match any pattern in the page source and replace it with any of your own text.)

If you grok Perl regular expressions, then this substitution syntax should be familiar to you. Here I'm defining a filter named "justtesting" which swaps my own example.jpg URL in wherever it sees a URL matching the one for the example.jpg on the NYTimes server.

Save that, and now you're halfway done. The second part of this process is to add that filter to the user.action file. So I will add the following to my /Users/michael/.privoxy/user.action file:

{+filter{justtesting}}
/

The first line turns on the filter name "justtesting" and the second line is a glob pattern saying which web addresses to apply the filter to. Use a forward slash if you want apply the filter to all web addresses. But, if I'd wanted to limit my filter just to the nytimes server I could have written this instead:

{+filter{justtesting}}
*.nytimes.com

Save that, and you're done. You don't need to restart privoxy for those changes to be applied. Now, if I go to the live web site I should see the affects of my switcheroo. Can you spot my somewhat Simpsonsesque family on my view of the live New York Times Technology Page?

NY Times page with my own image inserted

Note: If you want to always run privoxy and you've used port to install it, run the following command on Mac:

$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.Privoxy.plist
$ sudo launchctl list | grep privoxy

permalink | Tags: tools.