depish/cppdraft_translate
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Init

Browse Source
This commit is contained in:
Leonov Artur (Depish)
2025-10-25 03:02:53 +03:00
commit 043225d523
3416 changed files with 681196 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
cppdraft/lex/token.md Normal file
View File

@@ -0,0 +1,32 @@
[lex.token]
# 5 Lexical conventions [[lex]](./#lex)
## 5.10 Tokens [lex.token]
[token:](#nt:token "5.10 Tokens [lex.token]")
[*identifier*](lex.name#nt:identifier "5.11 Identifiers [lex.name]")
[*keyword*](lex.key#nt:keyword "5.12 Keywords [lex.key]")
[*literal*](lex.literal.kinds#nt:literal "5.13.1 Kinds of literals [lex.literal.kinds]")
[*operator-or-punctuator*](lex.operators#nt:operator-or-punctuator "5.8 Operators and punctuators [lex.operators]")
[1](#1)
[#](http://github.com/Eelis/draft/tree/9adde4bc1c62ec234483e63ea3b70a59724c745a/source/lex.tex#L873)
There are five kinds of tokens: identifiers, keywords, literals,[12](#footnote-12 "Literals include strings and character and numeric literals.") operators, and other separators[.](#1.sentence-1)
Comments and the characters U+0020 space, U+0009 character tabulation,U+000b line tabulation, U+000c form feed, and new-line
(collectively, “whitespace”), as described below, are ignored except
as they serve to separate tokens[.](#1.sentence-2)
[*Note [1](#note-1)*:
Whitespace can separate otherwise adjacent identifiers, keywords, numeric
literals, and alternative tokens containing alphabetic characters[.](#1.sentence-3)
— *end note*]
[12)](#footnote-12)[12)](#footnoteref-12)
Literals include strings and character and numeric literals[.](#footnote-12.sentence-1)