You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

README.md 12KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286
  1. # Chroma — A general purpose syntax highlighter in pure Go [![Golang Documentation](https://godoc.org/github.com/alecthomas/chroma?status.svg)](https://godoc.org/github.com/alecthomas/chroma) [![CircleCI](https://img.shields.io/circleci/project/github/alecthomas/chroma.svg)](https://circleci.com/gh/alecthomas/chroma) [![Go Report Card](https://goreportcard.com/badge/github.com/alecthomas/chroma)](https://goreportcard.com/report/github.com/alecthomas/chroma) [![Slack chat](https://img.shields.io/static/v1?logo=slack&style=flat&label=slack&color=green&message=gophers)](https://invite.slack.golangbridge.org/)
  2. > **NOTE:** As Chroma has just been released, its API is still in flux. That said, the high-level interface should not change significantly.
  3. Chroma takes source code and other structured text and converts it into syntax
  4. highlighted HTML, ANSI-coloured text, etc.
  5. Chroma is based heavily on [Pygments](http://pygments.org/), and includes
  6. translators for Pygments lexers and styles.
  7. <a id="markdown-table-of-contents" name="table-of-contents"></a>
  8. ## Table of Contents
  9. <!-- TOC -->
  10. 1. [Table of Contents](#table-of-contents)
  11. 2. [Supported languages](#supported-languages)
  12. 3. [Try it](#try-it)
  13. 4. [Using the library](#using-the-library)
  14. 1. [Quick start](#quick-start)
  15. 2. [Identifying the language](#identifying-the-language)
  16. 3. [Formatting the output](#formatting-the-output)
  17. 4. [The HTML formatter](#the-html-formatter)
  18. 5. [More detail](#more-detail)
  19. 1. [Lexers](#lexers)
  20. 2. [Formatters](#formatters)
  21. 3. [Styles](#styles)
  22. 6. [Command-line interface](#command-line-interface)
  23. 7. [What's missing compared to Pygments?](#whats-missing-compared-to-pygments)
  24. <!-- /TOC -->
  25. <a id="markdown-supported-languages" name="supported-languages"></a>
  26. ## Supported languages
  27. Prefix | Language
  28. :----: | --------
  29. A | ABAP, ABNF, ActionScript, ActionScript 3, Ada, Angular2, ANTLR, ApacheConf, APL, AppleScript, Arduino, Awk
  30. B | Ballerina, Base Makefile, Bash, Batchfile, BibTeX, BlitzBasic, BNF, Brainfuck
  31. C | C, C#, C++, Caddyfile, Caddyfile Directives, Cap'n Proto, Cassandra CQL, Ceylon, CFEngine3, cfstatement, ChaiScript, Cheetah, Clojure, CMake, COBOL, CoffeeScript, Common Lisp, Coq, Crystal, CSS, Cython
  32. D | D, Dart, Diff, Django/Jinja, Docker, DTD, Dylan
  33. E | EBNF, Elixir, Elm, EmacsLisp, Erlang
  34. F | Factor, Fish, Forth, Fortran, FSharp
  35. G | GAS, GDScript, Genshi, Genshi HTML, Genshi Text, Gherkin, GLSL, Gnuplot, Go, Go HTML Template, Go Text Template, GraphQL, Groff, Groovy
  36. H | Handlebars, Haskell, Haxe, HCL, Hexdump, HLB, HTML, HTTP, Hy
  37. I | Idris, Igor, INI, Io
  38. J | J, Java, JavaScript, JSON, Julia, Jungle
  39. K | Kotlin
  40. L | Lighttpd configuration file, LLVM, Lua
  41. M | Mako, markdown, Mason, Mathematica, Matlab, MiniZinc, MLIR, Modula-2, MonkeyC, MorrowindScript, Myghty, MySQL
  42. N | NASM, Newspeak, Nginx configuration file, Nim, Nix
  43. O | Objective-C, OCaml, Octave, OpenSCAD, Org Mode
  44. P | PacmanConf, Perl, PHP, PHTML, Pig, PkgConfig, PL/pgSQL, plaintext, Pony, PostgreSQL SQL dialect, PostScript, POVRay, PowerShell, Prolog, PromQL, Protocol Buffer, Puppet, Python 2, Python
  45. Q | QBasic
  46. R | R, Racket, Ragel, Raku, react, ReasonML, reg, reStructuredText, Rexx, Ruby, Rust
  47. S | SAS, Sass, Scala, Scheme, Scilab, SCSS, Smalltalk, Smarty, Snobol, Solidity, SPARQL, SQL, SquidConf, Standard ML, Stylus, Svelte, Swift, SYSTEMD, systemverilog
  48. T | TableGen, TASM, Tcl, Tcsh, Termcap, Terminfo, Terraform, TeX, Thrift, TOML, TradingView, Transact-SQL, Turing, Turtle, Twig, TypeScript, TypoScript, TypoScriptCssData, TypoScriptHtmlData
  49. V | VB.net, verilog, VHDL, VimL, vue
  50. W | WDTE
  51. X | XML, Xorg
  52. Y | YAML, YANG
  53. Z | Zig
  54. _I will attempt to keep this section up to date, but an authoritative list can be
  55. displayed with `chroma --list`._
  56. <a id="markdown-try-it" name="try-it"></a>
  57. ## Try it
  58. Try out various languages and styles on the [Chroma Playground](https://swapoff.org/chroma/playground/).
  59. <a id="markdown-using-the-library" name="using-the-library"></a>
  60. ## Using the library
  61. Chroma, like Pygments, has the concepts of
  62. [lexers](https://github.com/alecthomas/chroma/tree/master/lexers),
  63. [formatters](https://github.com/alecthomas/chroma/tree/master/formatters) and
  64. [styles](https://github.com/alecthomas/chroma/tree/master/styles).
  65. Lexers convert source text into a stream of tokens, styles specify how token
  66. types are mapped to colours, and formatters convert tokens and styles into
  67. formatted output.
  68. A package exists for each of these, containing a global `Registry` variable
  69. with all of the registered implementations. There are also helper functions
  70. for using the registry in each package, such as looking up lexers by name or
  71. matching filenames, etc.
  72. In all cases, if a lexer, formatter or style can not be determined, `nil` will
  73. be returned. In this situation you may want to default to the `Fallback`
  74. value in each respective package, which provides sane defaults.
  75. <a id="markdown-quick-start" name="quick-start"></a>
  76. ### Quick start
  77. A convenience function exists that can be used to simply format some source
  78. text, without any effort:
  79. ```go
  80. err := quick.Highlight(os.Stdout, someSourceCode, "go", "html", "monokai")
  81. ```
  82. <a id="markdown-identifying-the-language" name="identifying-the-language"></a>
  83. ### Identifying the language
  84. To highlight code, you'll first have to identify what language the code is
  85. written in. There are three primary ways to do that:
  86. 1. Detect the language from its filename.
  87. ```go
  88. lexer := lexers.Match("foo.go")
  89. ```
  90. 3. Explicitly specify the language by its Chroma syntax ID (a full list is available from `lexers.Names()`).
  91. ```go
  92. lexer := lexers.Get("go")
  93. ```
  94. 3. Detect the language from its content.
  95. ```go
  96. lexer := lexers.Analyse("package main\n\nfunc main()\n{\n}\n")
  97. ```
  98. In all cases, `nil` will be returned if the language can not be identified.
  99. ```go
  100. if lexer == nil {
  101. lexer = lexers.Fallback
  102. }
  103. ```
  104. At this point, it should be noted that some lexers can be extremely chatty. To
  105. mitigate this, you can use the coalescing lexer to coalesce runs of identical
  106. token types into a single token:
  107. ```go
  108. lexer = chroma.Coalesce(lexer)
  109. ```
  110. <a id="markdown-formatting-the-output" name="formatting-the-output"></a>
  111. ### Formatting the output
  112. Once a language is identified you will need to pick a formatter and a style (theme).
  113. ```go
  114. style := styles.Get("swapoff")
  115. if style == nil {
  116. style = styles.Fallback
  117. }
  118. formatter := formatters.Get("html")
  119. if formatter == nil {
  120. formatter = formatters.Fallback
  121. }
  122. ```
  123. Then obtain an iterator over the tokens:
  124. ```go
  125. contents, err := ioutil.ReadAll(r)
  126. iterator, err := lexer.Tokenise(nil, string(contents))
  127. ```
  128. And finally, format the tokens from the iterator:
  129. ```go
  130. err := formatter.Format(w, style, iterator)
  131. ```
  132. <a id="markdown-the-html-formatter" name="the-html-formatter"></a>
  133. ### The HTML formatter
  134. By default the `html` registered formatter generates standalone HTML with
  135. embedded CSS. More flexibility is available through the `formatters/html` package.
  136. Firstly, the output generated by the formatter can be customised with the
  137. following constructor options:
  138. - `Standalone()` - generate standalone HTML with embedded CSS.
  139. - `WithClasses()` - use classes rather than inlined style attributes.
  140. - `ClassPrefix(prefix)` - prefix each generated CSS class.
  141. - `TabWidth(width)` - Set the rendered tab width, in characters.
  142. - `WithLineNumbers()` - Render line numbers (style with `LineNumbers`).
  143. - `LinkableLineNumbers()` - Make the line numbers linkable and be a link to themselves.
  144. - `HighlightLines(ranges)` - Highlight lines in these ranges (style with `LineHighlight`).
  145. - `LineNumbersInTable()` - Use a table for formatting line numbers and code, rather than spans.
  146. If `WithClasses()` is used, the corresponding CSS can be obtained from the formatter with:
  147. ```go
  148. formatter := html.New(html.WithClasses())
  149. err := formatter.WriteCSS(w, style)
  150. ```
  151. <a id="markdown-more-detail" name="more-detail"></a>
  152. ## More detail
  153. <a id="markdown-lexers" name="lexers"></a>
  154. ### Lexers
  155. See the [Pygments documentation](http://pygments.org/docs/lexerdevelopment/)
  156. for details on implementing lexers. Most concepts apply directly to Chroma,
  157. but see existing lexer implementations for real examples.
  158. In many cases lexers can be automatically converted directly from Pygments by
  159. using the included Python 3 script `pygments2chroma.py`. I use something like
  160. the following:
  161. ```sh
  162. python3 _tools/pygments2chroma.py \
  163. pygments.lexers.jvm.KotlinLexer \
  164. > lexers/k/kotlin.go \
  165. && gofmt -s -w lexers/k/kotlin.go
  166. ```
  167. See notes in [pygments-lexers.txt](https://github.com/alecthomas/chroma/blob/master/pygments-lexers.txt)
  168. for a list of lexers, and notes on some of the issues importing them.
  169. <a id="markdown-formatters" name="formatters"></a>
  170. ### Formatters
  171. Chroma supports HTML output, as well as terminal output in 8 colour, 256 colour, and true-colour.
  172. A `noop` formatter is included that outputs the token text only, and a `tokens`
  173. formatter outputs raw tokens. The latter is useful for debugging lexers.
  174. <a id="markdown-styles" name="styles"></a>
  175. ### Styles
  176. Chroma styles use the [same syntax](http://pygments.org/docs/styles/) as Pygments.
  177. All Pygments styles have been converted to Chroma using the `_tools/style.py` script.
  178. When you work with one of [Chroma's styles](https://github.com/alecthomas/chroma/tree/master/styles), know that the `chroma.Background` token type provides the default style for tokens. It does so by defining a foreground color and background color.
  179. For example, this gives each token name not defined in the style a default color of `#f8f8f8` and uses `#000000` for the highlighted code block's background:
  180. ~~~go
  181. chroma.Background: "#f8f8f2 bg:#000000",
  182. ~~~
  183. Also, token types in a style file are hierarchical. For instance, when `CommentSpecial` is not defined, Chroma uses the token style from `Comment`. So when several comment tokens use the same color, you'll only need to define `Comment` and override the one that has a different color.
  184. For a quick overview of the available styles and how they look, check out the [Chroma Style Gallery](https://xyproto.github.io/splash/docs/).
  185. <a id="markdown-command-line-interface" name="command-line-interface"></a>
  186. ## Command-line interface
  187. A command-line interface to Chroma is included. It can be installed with:
  188. ```sh
  189. go get -u github.com/alecthomas/chroma/cmd/chroma
  190. ```
  191. The CLI can be used as a preprocessor to colorise output of `less(1)`,
  192. see documentation for the `LESSOPEN` environment variable.
  193. The `--fail` flag can be used to suppress output and return with exit status
  194. 1 to facilitate falling back to some other preprocessor in case chroma
  195. does not resolve a specific lexer to use for the given file. For example:
  196. ```shell
  197. export LESSOPEN='| p() { chroma --fail "$1" || cat "$1"; }; p "%s"'
  198. ```
  199. Replace `cat` with your favourite fallback preprocessor.
  200. When invoked as `.lessfilter`, the `--fail` flag is automatically turned
  201. on under the hood for easy integration with [lesspipe shipping with
  202. Debian and derivatives](https://manpages.debian.org/lesspipe#USER_DEFINED_FILTERS);
  203. for that setup the `chroma` executable can be just symlinked to `~/.lessfilter`.
  204. <a id="markdown-whats-missing-compared-to-pygments" name="whats-missing-compared-to-pygments"></a>
  205. ## What's missing compared to Pygments?
  206. - Quite a few lexers, for various reasons (pull-requests welcome):
  207. - Pygments lexers for complex languages often include custom code to
  208. handle certain aspects, such as Raku's ability to nest code inside
  209. regular expressions. These require time and effort to convert.
  210. - I mostly only converted languages I had heard of, to reduce the porting cost.
  211. - Some more esoteric features of Pygments are omitted for simplicity.
  212. - Though the Chroma API supports content detection, very few languages support them.
  213. I have plans to implement a statistical analyser at some point, but not enough time.