Luce packages
LuceLuce BaseLuciaOS

dymokomi/luce-browser-engine

Source of release 0.6.1

0.6.1 / docs / regions / r30b.md

# Region r30b: the CSS parser core — what other regions need to know

Region r30b ports `CSS/Parser/Parser.cpp` and the inline members of `Parser.h`,
`CSS/Parser/Helpers.cpp`, `CSS/Parser/ErrorReporter.cpp/.h`, `CSS/Parser/RuleContext.cpp/.h` and
the engine half of `CSS/Serialize.cpp/.h` (the other half is css_syntax's, r30a). The stub
fragment `stubs/stub_r30b_css_parser_core.lucb` is gone; every one of its functions is ported,
except `Parser::parse_font_variant_emoji`, which Parser.h declares and the donor never defines
(dropped, namemap row removed).

## Fragments

| Fragment | Donor |
| --- | --- |
| `css/parser/error_reporter` | `ErrorReporter.h` (the errors' `hash()`, the ParsingError variant's `==` and Traits) and `ErrorReporter.cpp` |
| `css/parser/helpers` | `Helpers.cpp`: `internal_css_realm`, `parse_css_stylesheet` and the other `parse_*` entry points, `css_decode_bytes`, `is_valid_custom_ident` |
| `css/parser/parser_1` | `Parser.cpp` 1-556: ParsingParams, `create`, the constructor, stylesheets, `convert_rules`, @supports, `<boolean-expr>`, container query conditions, `<general-enclosed>` |
| `css/parser/parser_2` | 557-1171: the consumers (stylesheet contents, at-rule, qualified rule, block, block contents, component value, list of component values, simple block, function) and the per-T steps (below) |
| `css/parser/parser_3` | 1173-1628: declarations, the remnants of a bad one, `parse_a_*`, parse a CSS declaration block (properties, descriptors) |
| `css/parser/parser_4` | 1630-1998 and `Parser.h`'s `push_temporary_value_parsing_context`: validity in the current context, `extract_property`, `convert_to_style_*`, `parse_source_size_value`, value contexts, `parse_as_css_value/descriptor_value/type` |
| `css/parser/parser_5` | 2000-2253: `parse_as_sizes_attribute`, substitution-function presence, `has_ignored_vendor_prefix`, the accessors |
| `css/parser/rule_context` | `RuleContext.cpp` |
| `css/serialize` | `Serialize.cpp` 125-131, 180-354 and `Serialize.h`'s template |
| `css/parser/generated_value_types_parsing` | the reference build's `GeneratedValueTypesParsing.cpp` (two parsers, written by hand; r36 owns that generator) |
| `generated/math_functions` | the engine part of the reference build's `MathFunctions.cpp` (`parse_math_function`, `parse_rounding_strategy`), written by `tools/gen_css_values` |
| `css/parser/tests_css_parser_cases`, `tests_css_parser`, `tests_css_parser_2` | tests (Ladybird has none for these files) |

## What other regions must know

### Generic parser functions and the per-T steps (r31, r32, r33)

The `TokenStream<T>` templates are generics `[T: css_syntax.TokenStreamToken[T]]`
(`css_parser_consume_a_declaration`, `css_parser_parse_a_list_of_component_values`, ...). C++
specializes `consume_a_component_value` (and its "do nothing" variant) for Token and
ComponentValue; a luce-base generic cannot be specialized, so parser_2 has four small per-T steps
that tell the two token types apart by size (a module assert checks the sizes differ):
`css_parser_consume_a_component_value[T]`, `css_parser_consume_a_component_value_and_do_nothing[T]`,
`css_parser_component_value_from[T]` (C++'s implicit `ComponentValue(Token)`) and
`css_parser_token_from[T]` (`(Token)x`). Call the generic names; the `_token` /
`_component_value` specializations are what they dispatch to.

C++-private members and internal-linkage functions are module-private (`parse_a_stylesheet`,
`consume_*`, `parse_a_*`, `log_parse_error`, ...); Parser members of other regions (RuleParsing,
SelectorParsing, ...) are in the same module and call them directly.

### Signature and type changes

- `ParseErrorOr<T>` is `T!`; the error codes are `css_parser_parse_error_includes_ignored_vendor_prefix`
  (`ErrorCode.package(3)`) and `css_parser_parse_error_syntax_error` (`ErrorCode.package(4)`).
  `convert_to_style_property` tells them apart by `failure.code`; r33's `parse_css_value` must fail
  with these codes.
- `css_parser_create(context, input, encoding: str = "utf-8")` (the StringView default has no
  constant form; every caller passes a literal or nothing).
- `Parser::push_temporary_value_parsing_context` returns a ScopeGuard in C++: here it only pushes,
  and the guard is `css_parser_pop_temporary_value_parsing_context(this)` (new), which callers
  `defer` right after pushing.
- `const&` parameters are const pointers: `css_parser_is_valid_in_the_current_context_*`,
  `css_parser_convert_to_style_property`, `css_parser_extract_property`,
  `css_parser_convert_to_style_declaration`, `css_parser_convert_rules`,
  `css_serialize_parsing_error`, `css_serialize_unicode_ranges`,
  `css_parser_collect_arbitrary_substitution_function_presence_vector` /
  `_component_value`, `css_parser_parse_math_function(this, function: const Function*, context:
  const CssCalculationContext*)`, `css_decode_bytes(..., encoded_string: const ByteBuffer*)`.
- `css_serialize_a_comma_separated_list[T](builder, items: const Vector[T]*, serialize_item:
  ak.Function2[StringBuilder*, const T*, unit])` (the template's callable is an ak.Function).
- `css_parser_parse_supports_declaration -> CssSupportsDeclaration*?` (as before); the other
  Supports expressions are cast to `CssBooleanExpression*`.
- Types filled: `CssParserPropertiesAndCustomProperties` (`properties`, `custom_properties`),
  `CssParserParsedStyleSheet` (`location`, `rules`), `CssErrorReporterMetadata` (`occurrences`),
  the `"style"` / `"@media"` default member initializers of the error structs, and the
  `CssParsingErrorTraits` bodies (`css_parsing_error_hash`, `css_parsing_error_eq`).

### Changes in other regions' files

- **r42**: the skeleton's closure missed Supports' nested classes. Added opaque
  `CssSupportsSelector`, `CssSupportsFontTech`, `CssSupportsFontFormat` (types_supports, next to
  the existing opaque `CssSupportsDeclaration`) and stubs `css_supports_declaration_create`,
  `_selector_create`, `_font_tech_create`, `_font_format_create`. r42 gives them the embedded
  BooleanExpression base first (the parser casts them to `CssBooleanExpression*`) and class ids.
- **r37**: `css_rule_list_create(realm, rules: const (CssRule*)[])` (was a span of pointers to
  const rules; DESIGN §2.2 drops const on cells).
- **r36**: the `css_is_a_custom_property_name_string` / `css_is_invalid_custom_property_name_string`
  stubs are gone (css_data's, HANDOFF §5 item 4). `GeneratedValueTypesParsing` is r36's generator;
  its two functions are written by hand here until then.
- The ten CSS/Serialize stubs are gone; call `css_syntax.serialize_*` / `escape_a_character*`.

### The ErrorReporter (everyone reporting parse errors)

`css_error_reporter_report(css_error_reporter_the(), CssParsingError.<kind>(...))`. Write the C++
default `"style"` / `"@media"` names out (or call the `_init_fields` helpers). The reporter is a
module variable; its table and the Strings of the errors it keeps are made in the C heap
(`with memory.heap`), because the collector neither scans nor sweeps module data. The parser's
constructor installs `css_syntax.invalid_rule_location_reporter` (the r30a hook): every rule
css_syntax's walkers see was made by a Parser.

### Generated code: tools/gen_css_values

The value-dependent half of Ladybird's CSS generators belongs to the engine (DESIGN §4.1.3, §5).
`tools/gen_css_values` (a Luce program; JSON reader and text helpers copied from
luce-browser-css's tools/gen_css) reads `data/css/*.json` and writes `web/generated/*.lucb`;
`test.sh` regenerates and compares. It writes `math_functions.lucb` from MathFunctions.json
(each function's branch is a function of its own, called by `parse_math_function` in order).
Regions porting `property_initial_value`, `property_accepts_*`, `descriptor_initial_value`,
`environment_variable_type` or GeneratedValueTypesParsing add emitters there.

## Deviations and notes

- Donor quirks kept: `consume_a_list_of_component_values` falls through after appending a `}`
  token when not nested, so it consumes the next value too (`)]}` ends with an EOF token);
  `parse_a_rule` consumes an at-rule from the parser's own token stream, not from its input;
  `<supports-env-fn>` makes a `Supports::FontFormat`; `context_allows_random_functions` reads
  `m_value_context.first()` and VERIFYs on an empty context.
- `internal_css_realm`'s statics are module variables (`gc.Root` for the realm and the window).
- Phase-1 traps in other regions' stubs on the way: every top-level at-rule (`is_margin_rule_name`,
  r37), at-rule names after "function" in `rule_context_type_for_at_rule`
  (`is_font_feature_value_type_at_keyword`, r37), every declaration converted to a style property
  (`PropertyNameAndID::from_name`, r36), rule conversion (`convert_to_rule`, r31), selectors and
  media queries (r31), values (r32/r33), sheets and lists (r37), Supports and boolean expressions
  (r42), calculation nodes (r35). `parse_css_stylesheet` of an empty string (Document's
  constructor) reaches r37's CSSRuleList/MediaList/CSSStyleSheet stubs and nothing of r30b traps.

## Tests

- `tests_css_parser_cases` (131 cases) and `tests_css_parser`: the cases run through the port and
  a local oracle (`luce-browser-tools/oracles/luce-browser-engine/css_parser`: `oracle.cpp` links
  the reference build's LibWeb objects, `cases.txt`, `gen_luce_cases.py` writes the table) and
  their dumps compared byte for byte: component values (debug strings and
  `serialize_a_series_of_component_values`), the raw rules and declarations of stylesheets and
  blocks (with and without a style context: nesting, `!important`, custom properties' original
  text, `{}`-block rules, bad declarations, unclosed blocks), single rules, declarations and
  component values, comma-separated lists, a declaration's original full text, ignored vendor
  prefixes, `is_valid_custom_ident`, `serialize_a_css_declaration`, `css_decode_bytes`,
  `rule_context_type_for_at_rule`, every ParsingError's message and hash, and
  `parse_math_function`'s argument-count errors.
- `tests_css_parser_2` (8 tests): the ErrorReporter's counting and copies, the css_syntax hook,
  `rule_context_type_for_rule`, the ParsingParams constructors and copied stacks, value contexts
  (tree counting, random functions, the auto sharing name, the guard's index reset),
  substitution-function presence and `<any-value>`, unicode-range serialization, empty inputs.

## Compiler issues

`luce-browser-tools/compiler-issues/const_span_of_pointer_diagnostic.lucb`: a mismatch between
`const Rule*[]` and `const (Rule*)[]` is reported with both types printed as `const Rule*[]`
(diagnostic only). Language limit worked with, not a bug: generics cannot be specialized (the
per-T steps above).