Creating Syntaxes
PRE-RELEASE WARNING
This is documentation for as yet unreleased features and is subject to change. Custom syntax definitions are not currently available in Drafts, but will be coming later this year. This documentation is provided to request feedback from parties interested in creating custom syntax definitions.
Table of Contents
- What Do I Need to Know?
- File Format
- Editing Syntax Definitions
- Workflow for Editing Syntaxes
- Sample Files
- Root Level Elements
- Patterns
- Scopes
- Navigation Patterns
- Indentation Patterns
- Task Mark Definitions
- Link Definitions
What Do I Need to Know?
Syntax definitions rely heavily on regular expressions to identify patterns in text. A strong familiarity with regular expressions is needed to develop custom syntaxes.
File Format
Syntax definition files are stored in JSON format with UTF-8 encoding. Supported keys as described below. Additional keys will be ignored. When distributed, Drafts exports and imports using the .draftsSyntax
file extension, but internally these are JSON files.
Editing Syntax Definitions
Drafts does not have a built-in editor for syntax definitions. Development of syntax definitions is done in JSON files outside of the app. For details, see developer mode information.
Since syntax definitions rely heavily on regular expression, we also recommend use of a regular expression tool/app to work on testing and regular expression patterns. There are many, the one we typically use is Regex 101.
Once a syntax definition is complete, it can be imported into Drafts to be used, and, if desired, shared to the Drafts Directory for other users.
Workflow for Editing Syntaxes
Generally speaking, the recommended workflow for developing a custom syntax is as below. These steps apply on both iOS and macOS.
- Enable Developer Mode
- In Drafts, choose an existing built-in or custom syntax that most closely matches the syntax you wish to develop. Select this syntax in editor preferences, and export is as a file using the available share options. Save this file in
iCloud Drive/Drafts/Library/Syntaxes
(you may have to create this folder). - Open your new syntax file in an external JSON editor, make your modifications, and save the file.
- In Drafts, create a testing draft with sample text for your syntax, and select your new file-based syntax as the syntax assigned to the draft.
- As you iterate and make changes to your syntax, save the file, and reload it in Drafts by either selecting a different draft and returning to your testing draft, or re-assigning the syntax to force it to reload from file.
- When you are happy with your syntax, import it into Drafts as a custom syntax using the “Import” options in editor preferences.
- The imported custom syntax can be used in regular production mode.
- Disable Developer Mode.
If you are making modifications to a syntax you have worked on in the past, when you use the “Import” feature, you will be offered the option to replace the version you have imported in the past, or import as a new custom syntax.
Sample Files
Complete example syntax definitions:
- grammar-markdown.json: Markdown syntax currently used in Drafts.
- grammar-javascript.json: JavaScript syntax currently used in Drafts.
- grammar-simple list.json: Simple List syntax currently used in Drafts.
Other demo syntaxes:
- Demo-Hashtags.json: Example patterns to highlight Twitter-style #hashtags and @mentions.
- Demo-Rainbow.json: A silly example which simple colorizes the word “rainbow” when typed in a draft.
- Demo-Tasks.json: Demostrates a few more advanced possibilities for tappable/clickable task elements in text.
Root Level Elements
The root of the JSON file should be a object which defines the following root element keys:
- name [string, required]: A friendly name for the Syntax definition. Used in user interface for selection.
- author [string]: Source credit for who created this theme.
- description [string, required]: Description of the syntax with more details about what is supported by the definition. May be used in user interface elements to assist user in selecting syntaxes.
- sampleText [string, required]: A brief sample text that demonstrates key features supported in the syntax. This text will be displayed in the syntax manager in Drafts.
- rangeExtensionType [object, required]: Controls how many characters are examined for syntax highlighting changes when text is changed in the document. Most objects will have a single
default
entry. It is also possible to have separateiOS
andmacOS
entries if it makes sense to have different rangeExtensionTypes by platform for performance reasons. Syntax definitions should select the least of these as necessary to ensure that changes to the text are properly reflected. One of the below values:- none: Only for the changed text.
- line: The entire containing line of the changed text.
- lineExtended: Include the line before and line after the changes.
- lookAround500: Extend evaluated range up to 500 characters before and after the changes.
- lookAround1000: Extend evaluated range up to 1000 characters before and after the changes.
- lookAround2000: Extend evaluated range up to 2000 characters before and after the changes.
- fullText: Re-evaluate the entire text when any changes occur.
- patterns [array, required]: Pattern definitions for the syntax. See Patterns for details.
- linkDefinitions [array, required]: Used to create live links within a document. See Link Definitions for details.
- navigationPatterns [array, required]: Definitions used to build in-document navigation markers. See Navigation Patterns for details.
- indentationPatterns [array, required]: Definitions used to define blocks with indentation, such as Markdown lists. See Indentation Patterns for details.
- taskMarkDefinitions [array, required]: Definitions used to define tap/clickable tasks in the text. See Task Mark Definitions for details.
Patterns
The patterns
key is the heart of the syntax definition, and defines the array of regular expression patterns to apply to identify markup in the text. When changes are made to the text in Drafts, each of these pattern definitions is applied in order to tokenize the text, applying scopes
which determine styles to be applied to characters in the text.
Each pattern definition object should contains the following keys:
- match [string]: Regular expression pattern to use evaluating the pattern. This expression will be applied to the range of characters currently being evaluated to determine any matching strings. The regular expression may contain capture groups mapping to each of the scopes to apply to the text. For example, a match expression for Markdown bold (
**bold**
) strings might have separate capture groups for the**
before, the**
after, and the text between to style each with a different scope. - comment [string]: Description of the purpose of this pattern.
- exclusive [boolean]: Is this range exclusive. If true, the text range matched by this pattern will be considered complete and not evaluated for matches by and subsequent patterns defined in the syntax.
- captures [object]: The capture object must contain a dictionary for each capture group in the match expression, named by index. “0” will always be the entire match result. If the match expression contains additional capture group, each should be included by index, like “1”, “2”, etc.
- [capture index] [object]: Key for object should be integer index of the capture group it applies to.
- scope [string]: Comma-separated list of scope names to apply to this capture group. Scopes are sets of font traits and styles which are defined in themes. Scopes will be applied in the order listed, so if attributes from one scope may override attributes from another be careful of ordering.
- exclusive [boolean]: Is this range exclusive. If true, the text range matched by this capture group will be considered complete and not evaluated for matches by and subsequent patterns defined in the syntax. If the exclusive value for the parent pattern is true, this is ignored, but allows a capture group within a pattern override the parent to be considered exclusive.
- [capture index] [object]: Key for object should be integer index of the capture group it applies to.
Patterns are applied in order. It is important to plan correctly using the exclusive
property and or the correct ordering of patterns to prevent subsequent patterns from overriding scope values for previously styled text where undesirable.
Example
Below is a partial patterns
object with two example patterns to perform syntax highlighting on markdown header lines and on horizontal rule markup.
"patterns": [
{
"match": "^(#+) ([^\\n]+?)(\\1?)$",
"exclusive": true,
"comment": "Markdown header like ###",
"captures": {
"1": {
"scope": "markup"
},
"2": {
"scope": "text.header"
},
"3": {
"scope": "markup"
}
}
},
{
"match": "^(\\*{3,}|-{3,})",
"exclusive": true,
"comment": "Markdown horizontal rule like *** or ---",
"captures": {
"1": {
"scope": "markup"
}
}
}
]
Scopes
Scopes are the means by which the tokens identified by the regular expression patterns in the syntax definition are tied into themes, which determine how that piece of text will actually appear in the editor. Scope names are arbitrary strings, but if a scope name is used which does not exist in the current theme in use, no changes are made to the text.
The themes which come with Drafts define a set of recommended default scope names which are associated with common text styles, like text.normal
, text.bold
, text.monospace.bold
. They also contain some semantic scopes for common types identified in syntaxes, like code.comment
, markup
, code.keyword
- as well as some common colors, like color.blue
, color.green
.
When designing syntaxes, it’s important to familiarize yourself with the default scopes in Drafts styles. It is also possible to use your own scope names, but for them to be meaningful, a custom theme may need to be made and used with your syntax.
In most cases, multiple scopes can be applied in a syntax by providing a comma-separate list. For example, to get bold red text, you might have a pattern with a scope text.bold, color.red
. When muliple scopes are applied, the are applied in order, so it’s possible to override values from an early scope in the list. For example, using color.blue, color.red
would result in red text.
Navigation Patterns
The navigationPatterns
array containing objects which define markers within the text which are used by Drafts’ navigation feature to easily jump between locations in a longer draft.
When building the list of navigation markers, all patterns defined in the syntax will be evaluated against the full text of the draft, then the list of all found markers will be sorted in the order they appear in the text and presented to the user for selection.
Each navigation pattern object should contain the following keys:
- match [string]: Regular expression pattern used to locate markers in the text. The expression should contain at least one capture group to define a label to use to identify the marker in the navigation interface.
- comment [string]: Description of the purpose of the match pattern.
- rangeCapture [string]: Integer index of capture group in the
match
pattern to use to identify the location of the marker in the text. When selecting a marker in the navigation interface, the editor will jump the cursor to this location. - labelCapture [string]: Integer index of capture group in the
match
pattern to use as a string label for the marker. For example, when navigating to a Markdown heading, you might want to capture only the text of the header without the leading markup (##
) to be displayed. - prefix [string]: Prefix to display before the label in the navigation interface. This should be used to distinguish the type of marker (For example, “H1”, “H2”).
- level [number]: Integer value from 0 to 9 to indicate the indention level of this marker in the navigation interface.
Example
The below example is the navigation patterns entry from the default Markdown syntax. It creates markers for each Markdown heading level (1-6), each with appropriate indentation and labelling.
"navigationPatterns": [
{
"match": "^# (.*)$",
"comment": "H1 level markdown headers with #",
"rangeCapture": "0",
"labelCapture": "1",
"prefix": "H1",
"level": 0
},
{
"match": "^## (.*)$",
"comment": "H2 level markdown headers with ##",
"rangeCapture": "0",
"labelCapture": "1",
"prefix": "H2",
"level": 1
},
{
"match": "^### (.*)$",
"comment": "H3 level markdown headers with ###",
"rangeCapture": "0",
"labelCapture": "1",
"prefix": "H3",
"level": 2
},
{
"match": "^#### (.*)$",
"comment": "H4 level markdown headers with ####",
"rangeCapture": "0",
"labelCapture": "1",
"prefix": "H4",
"level": 3
},
{
"match": "^##### (.*)$",
"comment": "H5 level markdown headers with #####",
"rangeCapture": "0",
"labelCapture": "1",
"prefix": "H5",
"level": 4
},
{
"match": "^###### (.*)$",
"comment": "H6 level markdown headers with ######",
"rangeCapture": "0",
"labelCapture": "1",
"prefix": "H6",
"level": 5
}
]
Indentation Patterns
Indentation patterns define block level elements which should be indented as a block when lines wrap beyond the editor width. Examples are Markdown list lines and quotations.
Each the indentationPatterns
array should contain objects with the following keys:
- match [string]: Regular expression pattern used to find the indentation string. This pattern should anchor at the beginning of the line.
- comment [string]: Description of the purpose of the match pattern.
- captures +[object]_: The capture object must contain a dictionary for each capture group in the match expression, named by index. “0” will always be the entire match result. If the match expression contains additional capture group, each should be included by index, like “1”, “2”, etc.
- [capture index] [object]: Key for object should be integer index of the capture group it applies to. This object currently has no supported sub-keys, but is reserve for possible future use. For the vast majority of purposes, this will be a single entry named “1”.
Example
The below example indentationPatterns
entry is from the default Markdown syntax and defines a single indentation pattern to locate and intend Markdown list and quote blocks.
"indentationPatterns": [
{
"match": "(^\\h*\\* |^\\h*\\- |^\\h*\\+ |^\\h*\\d+\\. |^\\h*>+ ).*",
"comment": "Indent Markdown lists beginning with -,+,* and quotations beginning with >",
"captures": {
"1": { }
}
}
]
Task Mark Definitions
The objects in the taskMarkDefinitions
array create tap/clickable task marks that change state when tapped. The most common usage of these definitions is to create on-off [ ] / [x]
style checkboxes which can be tapped to toggle state.
Task mark definitions are not limited to two states, however. They can have more states and each state can have a different scope specification to affect styling of the text.
Each object in the taskMarkDefinitions
array should have the following keys:
- enabled [boolean]: Is this definition active.
- match [string]: Regular expression which identifies a task mark string. This expression should find all possible states of the task mark.
- rangeType [string]: Controls handling of task matches. Supported values:
- task: Use if the
match
expression will matches the state values only and no surrounding text. This allows more than one mark per line, but these tasks will only toggle the text used to define the task. - line: Use if the
match
expression matches additional content beyond the interface capture. This type allows state values to be separate from the tappable interface task, but are limited to one match per line.
- task: Use if the
- captures [object]: Dictionary with the following required keys:
- interactive [string]: Group capture index of the
match
expression which represents the tap/clickable task text. - state [string]: Group capture index of the
match
expression which represents thestate
of the task.
- interactive [string]: Group capture index of the
- states [array of strings]: An array, in order, of the possible states of the task mark. When tapping the mark in the text, it will cycle through these states in order changing the value in the
state
capture range of thematch
expression. - scopes [object]: Scopes to apply to the task captures. Currently supports only one key:
- interactive [string]: Scope name to apply to the
interactive
(tap/clickable) capture group from thematch
expression. This allows the task to be styled with themes. Note that foreground colors cannot be applied.
- interactive [string]: Scope name to apply to the
Example
The below example taskMarkDefinitions
entry is the one used in Markdown syntax to create on-off [ ]
and [x]
task marks, which will be rendered using the current theme’s text.monospace.bold
scope.
"taskMarkDefinitions": [
{
"enabled": true,
"match": "(\\[[ xX]\\])",
"rangeType": "task",
"captures": {
"interactive": "1",
"state": "1"
},
"states": [
"[ ]",
"[x]"
],
"scopes": {
"interactive": "text.monospace.bold"
}
}
Link Definitions
The objects in the linkDefinitions
array create tap/clickable links to URLs, which can dynamically use data from key
and value
capture groups within the pattern defined by the definition to general URLs.
Each object in the linkDefinitions
array should have the following keys:
- enabled [boolean]: Is this definition active.
- match [string]: Regular expression which identifies a link text. The match expression should define capture groups which isolate two values for use constructing the link URL. The match pattern should match all possible values for the
key
. - captures [object]: Dictionary with the following required keys:
- key [string]: Group capture index of the
match
expression which represents akey
to use to determine the template to use in creating the link URL. - value [string]: Group capture index of the
match
expression which represents avalue
which can be inserted in the resulting URL dynamically using the tag[[value]]
- link [string]: Group capture index for the range to be highlighted as a link. This could be the entire matching range, or a substring with the range.
- prefix [string]: Group capture index for any markup before the link. Optional, but allows for separate scope to be applied.
- suffix [string]: Group capture index for any markup after the link. Optional, but allows for separate scope to be applied.
- key [string]: Group capture index of the
- templates [object]: An object containing string entries keyed for each possible value of the
key
, and providing a template for a URL which will be generated and attached to the link. These templates can contain a[[value]]
tag, which will insert the URL encoded version of thevalue
capture group in the URL.[[value_unencoded]]
is also available to use the raw value string, primarily for the case where thevalue
is, itself, a valid URL. - scopes [object]: Scopes to apply to the task captures to allow themes to apply styling:
- key [string]: Scope name to apply to the
key
capture group from thematch
expression. - value [string]: Scope name to apply to the
value
capture group from thematch
expression. - prefix [string]: Scope name to apply to the
prefix
capture group from thematch
expression. - suffix [string]: Scope name to apply to the
suffix
capture group from thematch
expression.
- key [string]: Scope name to apply to the
Example
The below example linkDefinitions
entry which enables links in the style [[key:value]]
, including the [[d:Draft Title]]
internal links to other drafts, [[s:Search]]
to link to a drafts search, and [[google:Search Terms]]
links to Google, etc.
"linkDefinitions": [
{
"enabled": true,
"match": "(\\[\\[)((d|u|s|w|google|wikipedia|bear):(.+))(\\]\\])",
"captures": {
"key": "3",
"value": "4",
"link": "2",
"prefix": "1",
"suffix": "5"
},
"templates": {
"": "drafts://open?title=[[value]]&allowCreate=true",
"d": "drafts://open?title=[[value]]&allowCreate=true",
"u": "drafts://open?uuid=[[value]]",
"s": "drafts://quickSearch?query=[[value]]",
"w": "drafts://workspace?name=[[value]]",
"google": "https://www.google.com/search?q=[[value]]",
"wikipedia": "https://en.wikipedia.org/wiki/[[value]]",
"bear": "bear://x-callback-url/open-note?title=[[value]]"
},
"scopes": {
"key": "text.bold",
"value": "text.italic",
"prefix": "markup",
"suffix": "markup"
}
}
]