Syntax Highlighting

July 17, 2008 James Adam

Working through some of the outstanding issues for vanilla-rb, I’ve implemented some simple syntax highlighting (Ticket 16):

class Ruby
  def coloured(with='syntax')
    if with =~ /joy/
      puts "Hello, You Coloured World You"
    end
  end
end

It’s pretty simple behind the scenes, although there are a few conceptual choices which I may revisit. Here’s the scoop.

Blocks of content

I recently fixed code syntax highlighting on the rails-engines site, which is built using the reasonably-excellent radiant-cms system. In Radiant, you markup blocks of content, typically like this:

<r:code lang="ruby">
class Blah
  def something
  end
end
</r:code>

Radiant lets you define tags that wrap around content within the page, making this pretty simple.

However, Vanilla doesn’t work like that, or at least it doesn’t at the moment. The building block of a page is the snip, not a chunk of text wrapped in a tag. There’s no tag processing going on here at all, beyond the single, magically snip inclusion that makes it all work. This presents a problem when we want to treat a certain piece of text differently to the rest of the body of a snip.

The Vanilla Way

Since the building block is the snip, the natural thing to do is to move the code snippet into its own snip, and include that via a code dynasnip (see below for the self-syntax-highlighted source!). And so this call

Unknown part to render 'ruby' for snip syntax-highlighting-demo

works nicely for us. After the call to the code dynasnip, the first parameter is the language, and the second is the snip name to include:

Unknown part to render 'ruby' for snip syntax-highlighting-demo

… but it’s a pain to have to move every code sample out into its own snip (although that’s certainly useful for larger chunks of code.

The solution to this, is to allow rendering of individual parts of snips via syntax highlighting. By adding the snip part to the parameter list

class Test
  def initialize(name=nil)
    puts "Hello, World"
  end
end

we get

class Test
  def initialize(name=nil)
    puts "Hello, World"
  end
end

The code dynasnip is rendering the rubycodesample part of this very blog post!

Next steps

While this certainly works, it’s a pain to have to reference the current snip in order to get to the snip part. I have to do this with the comments dynasnip too, in order to find all the snips that are related to it. We could solve this if:

Each renderer knew which snip it was rendering - currently true :)
Each renderer knew which renderer called it - not currently possible :(

Ideally, the code dynasnip would be able to ask it’s renderer (the Ruby renderer) to ask the renderer that is invoking it (in the case of this post, the Markdown renderer) which snip is doing the including. With me?

Code dyna
    --> rendered by Ruby Renderer
        --> invoked by Markdown Render while rendering this blog post

Yeah, it’s a bit complicated, but it’s probably worth it; it could help avoid circular rendering problems at the same time.

Anyway - syntax highlighting. Woot!

(setq variable-name "string value"
      another-variable 123
      yet-another (with-this :family "that" :size 521))
(setq something-else 'doom)

Syntax Highlighting

July 17, 2008 James Adam

Working through some of the outstanding issues for vanilla-rb, I’ve implemented some simple syntax highlighting (Ticket 16):

class Ruby
  def coloured(with='syntax')
    if with =~ /joy/
      puts "Hello, You Coloured World You"
    end
  end
end

It’s pretty simple behind the scenes, although there are a few conceptual choices which I may revisit. Here’s the scoop.

Blocks of content

I recently fixed code syntax highlighting on the rails-engines site, which is built using the reasonably-excellent radiant-cms system. In Radiant, you markup blocks of content, typically like this:

<r:code lang="ruby">
class Blah
  def something
  end
end
</r:code>

Radiant lets you define tags that wrap around content within the page, making this pretty simple.

However, Vanilla doesn’t work like that, or at least it doesn’t at the moment. The building block of a page is the snip, not a chunk of text wrapped in a tag. There’s no tag processing going on here at all, beyond the single, magically snip inclusion that makes it all work. This presents a problem when we want to treat a certain piece of text differently to the rest of the body of a snip.

The Vanilla Way

Since the building block is the snip, the natural thing to do is to move the code snippet into its own snip, and include that via a code dynasnip (see below for the self-syntax-highlighted source!). And so this call

Unknown part to render 'ruby' for snip syntax-highlighting-demo

works nicely for us. After the call to the code dynasnip, the first parameter is the language, and the second is the snip name to include:

Unknown part to render 'ruby' for snip syntax-highlighting-demo

… but it’s a pain to have to move every code sample out into its own snip (although that’s certainly useful for larger chunks of code.

The solution to this, is to allow rendering of individual parts of snips via syntax highlighting. By adding the snip part to the parameter list

class Test
  def initialize(name=nil)
    puts "Hello, World"
  end
end

we get

class Test
  def initialize(name=nil)
    puts "Hello, World"
  end
end

The code dynasnip is rendering the rubycodesample part of this very blog post!

Next steps

While this certainly works, it’s a pain to have to reference the current snip in order to get to the snip part. I have to do this with the comments dynasnip too, in order to find all the snips that are related to it. We could solve this if:

Each renderer knew which snip it was rendering - currently true :)
Each renderer knew which renderer called it - not currently possible :(

Ideally, the code dynasnip would be able to ask it’s renderer (the Ruby renderer) to ask the renderer that is invoking it (in the case of this post, the Markdown renderer) which snip is doing the including. With me?

Code dyna
    --> rendered by Ruby Renderer
        --> invoked by Markdown Render while rendering this blog post

Yeah, it’s a bit complicated, but it’s probably worth it; it could help avoid circular rendering problems at the same time.

Anyway - syntax highlighting. Woot!

(setq variable-name "string value"
      another-variable 123
      yet-another (with-this :family "that" :size 521))
(setq something-else 'doom)

Syntax Highlighting

July 17, 2008 James Adam

Working through some of the outstanding issues for vanilla-rb, I’ve implemented some simple syntax highlighting (Ticket 16):

class Ruby
  def coloured(with='syntax')
    if with =~ /joy/
      puts "Hello, You Coloured World You"
    end
  end
end

It’s pretty simple behind the scenes, although there are a few conceptual choices which I may revisit. Here’s the scoop.

Blocks of content

I recently fixed code syntax highlighting on the rails-engines site, which is built using the reasonably-excellent radiant-cms system. In Radiant, you markup blocks of content, typically like this:

<r:code lang="ruby">
class Blah
  def something
  end
end
</r:code>

Radiant lets you define tags that wrap around content within the page, making this pretty simple.

However, Vanilla doesn’t work like that, or at least it doesn’t at the moment. The building block of a page is the snip, not a chunk of text wrapped in a tag. There’s no tag processing going on here at all, beyond the single, magically snip inclusion that makes it all work. This presents a problem when we want to treat a certain piece of text differently to the rest of the body of a snip.

The Vanilla Way

Since the building block is the snip, the natural thing to do is to move the code snippet into its own snip, and include that via a code dynasnip (see below for the self-syntax-highlighted source!). And so this call

Unknown part to render 'ruby' for snip syntax-highlighting-demo

works nicely for us. After the call to the code dynasnip, the first parameter is the language, and the second is the snip name to include:

Unknown part to render 'ruby' for snip syntax-highlighting-demo

… but it’s a pain to have to move every code sample out into its own snip (although that’s certainly useful for larger chunks of code.

The solution to this, is to allow rendering of individual parts of snips via syntax highlighting. By adding the snip part to the parameter list

class Test
  def initialize(name=nil)
    puts "Hello, World"
  end
end

we get

class Test
  def initialize(name=nil)
    puts "Hello, World"
  end
end

The code dynasnip is rendering the rubycodesample part of this very blog post!

Next steps

While this certainly works, it’s a pain to have to reference the current snip in order to get to the snip part. I have to do this with the comments dynasnip too, in order to find all the snips that are related to it. We could solve this if:

Each renderer knew which snip it was rendering - currently true :)
Each renderer knew which renderer called it - not currently possible :(

Ideally, the code dynasnip would be able to ask it’s renderer (the Ruby renderer) to ask the renderer that is invoking it (in the case of this post, the Markdown renderer) which snip is doing the including. With me?

Code dyna
    --> rendered by Ruby Renderer
        --> invoked by Markdown Render while rendering this blog post

Yeah, it’s a bit complicated, but it’s probably worth it; it could help avoid circular rendering problems at the same time.

Anyway - syntax highlighting. Woot!

(setq variable-name "string value"
      another-variable 123
      yet-another (with-this :family "that" :size 521))
(setq something-else 'doom)