Django snippets: Revisiting Pygments and Markdown

from pygments.lexers import LEXERS, get_lexer_by_name
from pygments import highlight
from pygments.formatters import HtmlFormatter
from BeautifulSoup import BeautifulSoup
from markdown import markdown

# a tuple of known lexer names
_lexer_names = reduce(lambda a,b: a + b[2], LEXERS.itervalues(), ())

# default formatter
_formatter = HtmlFormatter(cssclass='source')

def pygmented_markdown(raw):
    '''
    Accepts raw markdown text for markup processing. Using BeatifuleSoup on the
    results of markdown processing, the following constructs will be replaced
    by with pygmented highlighting. E.g.::
    
        <pre class="???">
            ...
        </pre>
        
    Where ``???`` is the name of a supported pygments lexer, e.g.: ``python``, 
    ``css``, ``html``.
    
    Note: Semantically, it would make more sense to wrap the code in a 
    ``<code>...</code>`` tag; however, my tests using markdown.py - as well as 
    markdown.pl from John Gruber - have shown that the inner HTML of the 
    ``<code>`` tag is not immune to translation.
    '''
    soup = BeautifulSoup(markdown(raw))
    for tag in soup.findAll('pre'):
        lexer_name = tag.get('class')
        if lexer_name and lexer_name in _lexer_names:
            lexer = get_lexer_by_name(lexer_name, stripnl=True, encoding='UTF-8')
            tag.replaceWith(highlight(tag.renderContents(), lexer, _formatter))
    
    return unicode(soup)

More like this

typygmentdown by ubernostrum 4 years, 9 months ago
Multiple querysets by t_rybik 2 years, 3 months ago
YAAS (Yet Another Auto Slug) by carljm 4 years ago
MarkupTextField by myles 3 years, 6 months ago
Markdown and Syntax Highlighting in Django by blinks 5 years, 2 months ago

A variation on a theme, inspired by snippet 39 and snippet 119. The intent is to provide a more generic and simple mechanism for combining Markdown with Pygments. Common scenarios could include blogging or commenting. Snippet 119 seemed too specific and perhaps not as efficient, needing to process the HTML twice to accomplish it's ends. The one snag in the implementation is the need to use a tag other than code as a wrapper. See the comments for details.

You will need the BeautifulSoup module installed.

Sample usage:

from django.db import models

class Blog(models.Model):
    '''Bare bones blogging model'''
    title = models.CharField(maxlength=255)
    slug = models.SlugField(maxlength=255, prepopulate_from=('title',))
    pub_date = models.DateTimeField()

    # the cooked view, cached for quick retrieval 
    blog = models.TextField()

    # the raw markdown-encoded text, saved for subsequent edits
    markdown = models.TextField()

    def save(self):
        from datetime import datetime
        if not self.id and not self.pub_date:
            self.pub_date = datetime.now()

        self.blog = pygmented_markdown(self.markdown)
        super(Blog, self).save()

Author:: djypsy
Posted:: August 8, 2007
Language:: Python
Django Version:: .96
Tags:: pygments markdown beautifulsoup
Score:: 1 (after 1 ratings)

Tools

Comments

waylan (on August 8, 2007):

You do realize that Markdown has the CodeHilite extension which uses Pygments, don't you? That seems a lot better than processing your text twice - once with Markdown and once with BeautifulSoup/Pygments.

There's a suggested template filter on the Markdown site as well as ticket #2910 which builds Markdown extension support into Django itself (currently waiting for a design decision). You could either apply the patch or copy the code into your own custom filter.

Disclaimer: I wrote the extension, filed the ticket and wrote (most of) the patch so I may be a little bias to this solution.

djypsy (on August 8, 2007):

@waylan:

Thanks for the feedback. I was unaware of this extension, and will make use of it for my own personal site. However, in a few other work-related scenarios, I don't have this option open to me, so I'm stuck using the brute force mechanism shown here.

Django snippets

Revisiting Pygments and Markdown

More like this

Tools

Comments

Snippets

About