From Eterna Wiki

A pseudoknot is a name applied to any RNA secondary structure (folding) that cannot be expressed in standard dot-bracket notation, i.e. with only '.', '(' and ')'.


A simple pseudoknot

Consider this example taken from Nando's blog post.  To write this structure in dot-bracket notation, we start at the 5' end in the lower left corner. First there are six unpaired bases, so that would be 5 dots.  Next come six paired bases, where this is the first time we have come to this pair.  These are wriiten as open parentheses.  Then come 6 more unpaired bases, written as six more dots.  At this point, we have


The next three bases are paired, and it is the first time we've visited these pairs, so we add three more open parentheses


The next six bases are also paired, but these are the second half of pairs we have already visited, so we would normally write them as closed parentheses, i.e.


But this isn't right, because the first three closing brackets would be interpreted as matching the closest previously unmatched brackets, and that would say we have a hairoin with an empty loop.

A common convention for writing pseudoknots is to use square brackets in addition to parentheses.  With this convention, our example can be written


Here, open parentheses are only paired with closed parentheses and open brackets are only paired with closed brackets.  This notation suffices for most pseudoknots seen in natural RNA.  But from the perspective of mathematical possibilities, even more distinct symbols might be needed to have all the bases match up correctly.