Composr Tutorial: Advanced Comcode

Written by Chris Graham (ocProducts)
This tutorial will explain some of the more advanced aspects of the Comcode system, for those who already have a handle on the basics.


Comcode Security

A great deal of emphasis is placed on making sure Comcode is secure, as Comcode is available to all members of a Composr website.

There's essentially a 5-tier distinction you should keep in mind:
  1. Totally untrusted random users
  2. Somewhat trusted users who you still don't really know well
  3. Trusted users who you are happy to trust to not explicitly go out of their way to find sneaky ways to subvert access controls
  4. Staff trusted with almost all access who you are happy to trust to not explicitly go out of their way to find sneaky ways to subvert the highest level of access controls (typically super-moderators)
  5. Staff trusted with full access (super-administrators) [on this level you can't limit privileges, all privileges are automatically granted]

The following privileges impact Comcode permissions (referencing the tiers above)…
Privilege Codename Tier Purpose
Use potentially-troublesome Comcode comcode_nuisance 2 This privilege currently doesn't do anything, but we reserve it for the future.
Subject to a more liberal HTML filter allow_html 2 Instead of only whitelisting HTML, it applies a blacklist †.
Avoid broad input filtering security layer unfiltered_input 3 This bypasses the rough supplementary filtering that applies to all Composr requests.
Use unrestricted markup that could be abused for XSS attacks use_very_dangerous_comcode 3 This allows dangerous HTML code to be posted within Comcode ††.
Use dangerous Comcode comcode_dangerous 4 Allow use of things such as [block] tags, allowing arbitrary Admin Zone access (e.g. by embedding an Admin module on the front-end of the website)

† There are two alternative security filters available in Composr:
  1. The whitelist filter. This is the most secure filter, and is used by default. It only allows certain HTML fragments to be used (fragments likely to be produced via usage of the WYSIWYG editor).
  2. The blacklist filter. This is designed for high security, filtering out anything it sees as dangerous. At this level you are trusting the user won't and can't find a security hole in the sophisticated blacklist filter.

†† Very dangerous code includes:
  • JavaScript code (reason: code can be malicious, for example steal cookies)
  • CSS code rules block (style element) (reason: style changes can be used to deceive a visitor)
  • Code to embed external files, such as JavaScript, CSS, applets, or browser plugins (reason: external code can either be malicious or adjust existing code to be malicious)
  • Meta tags (reason: could inject the authorisation needed for taking over the Google Search Console, for example)
  • Code to change the overall page document structure (reason: can significantly change how the browser processes the page)
There are sophisticated mechanisms in Comcode to maintain this security, such as ones to stop JavaScript-powered URLs, and ones to stop people trying to using HTML entities to bypass JavaScript filtering.

Edited content

Comcode is interpreted with particular access credentials. When added, obviously this is the level of the submitter. When editing then the situation is more complex:
  • We set the access credentials to that of the editing user (not the content owner) if the editing user does not have all of "Subject to a more liberal HTML filter" and "Use unrestricted markup that could be abused for XSS attacks" and "Use dangerous Comcode".
  • Otherwise, we set the access credentials that of the content owner.

These rules uphold the following 3 principles:
  1. A non-admin who has permission to edit something an admin posted (e.g. a moderator) should not be able to edit under the credentials of the admin.
  2. An admin should not be tricked into raising the access level of some Comcode when they think they are just correcting typos. A hacker could have left some dangerous (but subtle) "bombs" in their Comcode.
  3. In general we want to maintain editing under the same credentials as the original user to keep things consistent.

In other words: we pick the lowest access credentials with a bias to the editing user if the status is mixed and a bias to the content owner otherwise.

Note that "Avoid broad input filtering security layer" happens on the input layer, not the Comcode layer, so always runs based on the credentials of the editing user. Don't worry about this though because it's an additional layer of security, not the primary layer.

The future

Some time in the future browsers may gain the ability to sandbox parts of web pages, which would allow us to relax these rules, but it seems a long way off still. We would need to wait for all web browsers in use to have adopted the new sandboxing behaviour before we could rely on it.

Custom Comcode

Image

Adding a Custom Comcode tag

Adding a Custom Comcode tag

(Click to enlarge)

Custom Comcode tags are useful if you have developed a sophisticated piece of HTML that you wish to be used anywhere on your website, by referencing it with a simple Comcode tag. They can also be used like 'templates', to display data in a standardized and easy way.

To add a custom tag, do as follows:
  1. Go to Admin Zone > Setup > Custom Comcode > Add Custom Comcode tag
  2. Choose the tag codename you would like to use, a title for the tag, a description, and an example of how the tag would be used.
  3. Tick the various check boxes as necessary.
  4. Place the HTML you wish to use inside the 'Replacement text' field, and put '{content}' at the place where you wish for the tag content to be (the text that sits between the opening of the Comcode tag and the closing of the Comcode tag).
  5. If you have additional parameters for the tag, list your chosen codenames, separated by commas, in the 'Parameters' field. Then place the codenames in the 'Replacement text' field, in a similar way to as you did for '{content}'.
  6. Click the 'Add Custom Comcode tag' button

Your tag will be shown in your local copy of the Comcode tutorial, with your title, description and usage example.

You can edit a tag from Admin Zone > Setup > Custom Comcode > Edit Custom Comcode tag.
You can delete a tag from the bottom of its edit form.

Note that Comcode is cached, and if you edit a tag after adding Comcode using it, it won't be updated until you edit that content (or the Comcode field pre-calculation cache is emptied).

Tip

If you want to let users use some specific "dangerous" Comcode (such as Tempcode symbols, or an iframe), wrapping it up within a Custom Comcode tag is a great solution. As long as a user has permission to the tag itself (i.e. it's not added as dangerous), they can indirectly do what code it contains regardless of the nature of that code.

In the Replacement text you can either put your parameters in lower-case like {example} or upper-case like {EXAMPLE}. If you are comfortable with Tempcode programming, use upper-case, as this is standard Tempcode and thus the conventional way to do it; you can also add escaping, e.g. {EXAMPLE*}. If you use lower-case then a substitution to upper-case will be done for you while also automatically adding HTML escaping (i.e. special HTML characters like brackets with parameters will be shown literally rather than run as code within the web browser).

Sophisticated example

Image

Our tag in the WYSIWYG editor

Our tag in the WYSIWYG editor

(Click to enlarge)

Image

Display our tag

Display our tag

(Click to enlarge)

Image

Adding our tag

Adding our tag

(Click to enlarge)

This is a simple example tag for showing reminders/advice to walkers, based on the month, with a given highlighter colour, and a name for the walking location…

Comcode tag:
walking_reminders

Title:
Walking reminders

Description:
Show a walking reminder, tailored to the month.

Parameters:
color=red

Replacement text:

Code

{+START,IF,{$EQ,{$FROM_TIMESTAMP,%b},Nov,Dec,Jan}}
   Pack your <span style="color: {COLOR}">gloves</span>, a <span style="color: {COLOR}">hat</span>, and a thick <span style="color: {COLOR}">coat</span>, it'll be cold on the {CONTENT}.
{+END}
{+START,IF,{$EQ,{$FROM_TIMESTAMP,%b},Feb,Mar,Apr}}
   Be careful of the <span style="color: {COLOR}">wildlife</span>, it's breeding season on {CONTENT}.
{+END}
{+START,IF,{$EQ,{$FROM_TIMESTAMP,%b},May,Jun,Jul}}
   Bring plenty of <span style="color: {COLOR}">water</span>, it may be very hot on the {CONTENT}.
{+END}
{+START,IF,{$EQ,{$FROM_TIMESTAMP,%b},Aug,Sep,Oct}}
   Wear good <span style="color: {COLOR}">shoes</span>, it may be slippy on the {CONTENT}.
{+END}

Example:

Code

[walking_reminders color="red"]moor[/walking_reminders]
(Leave the checkboxes as the defaults)

Use of Tempcode in this example allows us to do some computation when the tag displays.

Semi-HTML

The [semihtml] tag is a special tag that allows Comcode and HTML to be naturally mixed together. Primarily, the tag is used so that the WYSIWYG editor, which is an HTML editor, may have Comcode placed within it that is not visual, and thus could not be edited in a WYSIWYG fashion.

The [semihtml] tag is also useful if you are writing some content that has very tight formatting, so it involves a lot of HTML, but also includes elements of Comcode, such as blocks. Without the [semihtml] tag, it would be necessary to keep opening and closing [html] tags to achieve the same effect.

Syntax highlighting

Composr supports syntax highlighting via GeSHi. This operates using the normal code and codebox tags, adding support for GeSHi highlighter codes to the param attribute of the tags. We maintain a non-bundled GeSHi addon that provides the necessary files.

If you want to know how it looks, just look at the code as displayed throughout this documentation.

Comcode Menus (advanced)

Image

The syntax sample in action

The syntax sample in action

(Click to enlarge)

Tip

You can put a '~' symbol (tilde, commonly known as "that wiggly/twiddly symbol") before a URL if you want it to open in a new window.

Composr has had a dedicated menu editor, however you may also write menus in Comcode if you prefer. This method is likely to be preferred by users using a screen reader, as the whole menu is represented in a single readable text field.

The menu tag takes a single parameter – the codename assigned to the menu. Each menu should be assigned a different codename, and you may just make one up as you write the Comcode.

It is important to be careful not to break the syntactic rules for the menu formation, the syntax is delicate. The menu is only designed for organising links. You should not attempt to store other content in a Comcode menu tag.

Here is a sample of how the syntax works…

Code

[menu="some_codename"]
-contracted section
 +expanded section
  page = URL
  page = URL
+expanded section
 page = URL
 page = URL
[/menu]

Or, a more practical example…

Code

[menu="my_menu"]
-Sample pages
 Example Page = http://example.com
 Composr CMS = http://compo.sr
+Computer companies
 Google = http://google.com
 Apple = http://apple.com
[/menu]

Like the list syntax, the menu syntax specifies a tree structure, and uses indentation to do it. Things within a section, including the definition of sub-sections, are indented from the section start line itself.

The - means that a section is not expanded by default, while the + means that a section is. Those lines without a + or - are lines that are actually entries on the menu, and contain the caption of the entry, followed by = , followed the the URL of the entry.

Menu entries are typed so that their indentation is equal to the indentation of the start of the section's title.
i.e. one further depth of indentation from the section's + or -.

Relationships with other languages, and design choices

BBCode compatibility

Comcode is largely able to deal with BBCode code. BBCode is a popular (but not formally-specified) system among standalone forum systems. Comcode is not formally designed as an extension of BBCode, but we intentionally aimed to be a super-set of it so that we are compatible with code imported from third-party forum systems, and so that our forum drivers can work well.

Comcode can support all major features of all the major BBCode-like languages, and lax syntax (for example, missing quotation symbols) as is often allowed in BBCode. However, BBCode implementations do not support a lot of the functionality of Comcode, such as the block tag and our list syntax; in addition, the Comcode implementation is that of a fully parsed language, with a well-defined syntax, while BBCode is almost always implemented as an ad hoc language to be turned into HTML via regular expressions.

Wiki-Code compatibility

Wiki-Code is a text-formatting based language, rather than a tag based language.

We support a few other Wiki-Code conventions, in a way that is most similar to that of MediaWiki. We have opted to maintain most similarity to BBCode though, and have not provided Wiki-Code syntax where there was already support via existing Comcode syntax.

Markdown compatibility

Markdown has become a very popular way of representing documents. In many ways it is similar to Comcode, but we have not made any effort to be compatible for these reasons:
  1. it would add inconsistencies (multiple ways of doing things, leading to confusion)
  2. it would add extra parsing overhead and complexity (i.e. likely bugs)
  3. we target Comcode to have a much wider set of functionality than just document formatting (the sole purpose of Markdown)
  4. Markdown does not play well with WYSIWYG editors, while Comcode is designed to work both with and without WYSIWYG

Semantics (advanced)

This section may be of interest to people with particular interest in programming language grammar.

Separation of concerns

Comcode is designed to be a language that is easy to use, versatile, rich, indistinguishable from plain-text in the simple case, and useful for writing small code snippets as well as long documents. It is not designed to have "separation of concerns" with any kind of special separation between visual coding (such as a font tag) and structural coding (such as a block tag, or arguably a ticker tag). While the separation of concerns is appropriate for a website's main layout, which is often worked upon by different people, and where the mark-up is complex and detailed enough to warrant a separation, it is not appropriate for Comcode. This said, Comcode has good support for semantics…

Formatting tags vs semantic tags

Formatting tags define how things look. Semantic tags often do too, but they also define how things interpreted by computers, to give cues to search engines or accessibility devices.

Paragraphs vs line breaks

Comcode has excellent support for semantics, but there is just one small area where it isn't able to support them, which is paragraphs. In Comcode the XHTML5 is created using XHTML br (line break) tags to separate lines. This has few practical ramifications known to us, and it is by necessity to make sure Comcode produces conformant and consistent XHTML5. However if you want to use XHTML p tags, you should use the Comcode semihtml tag so that you have full control over how whitespace will display.

Example:

Code

First paragraph.

Second paragraph.

Composr will produce:

Code

First paragraph.<br />
<br />
Second paragraph.

Purists would expect:

Code

<p>First paragraph.</p>
<p>Second paragraph.</p>

Comcode is not strictly a markup language

It is worth noting that the 'tag contents' of certain tags is of no visual significance what-so-ever (such as the block tag, where it has the codename of a block placed there, which is not itself displayed). This fact could be used to argue that Comcode is not a pure mark-up language.

Accessibility

There are some accessibility issues with certain Comcode tags, and these are described in the Helping improve site accessibility for disabled users tutorial.

Concepts

Wiki-Code
The mark-up language used on most Wiki systems
BBCode
The mark-up language used on many discussion forum systems, which is similar to Comcode

See also


Feedback

Please rate this tutorial:

Have a suggestion? Report an issue on the tracker.