[bartveneman]

Summary

Lexer had two ways to advance: next_token_fast() (returns the token type, no allocation) and next_token() (wraps the former and allocates a Token object "for backwards compatibility"). This removes the wrapper and gives the fast method the plain name.

• Lexer.next_token(skip_whitespace?) now returns a TokenType number.
• Lexer.next_token_fast is removed.
• tokenize() is unchanged for consumers: it still yields Token objects, now built directly from the lexer's position fields.
• Internal parsers now call next_token.
• tokenize.test.ts uses a small local next(lexer) helper to keep its Token-shaped assertions.

Breaking change / migration

Only affects code that uses Lexer directly (the @projectwallace/css-parser/tokenizer subpath). tokenize() users need no changes.

1. Old next_token() (returned a Token | null)

// before
const token = lexer.next_token()
if (token?.type === TOKEN_IDENT) {
	const text = source.slice(token.start, token.end)
}

// after
const type = lexer.next_token()
if (type === TOKEN_IDENT) {
	const text = source.slice(lexer.token_start, lexer.token_end)
}

Position fields on the lexer after each call: token_start, token_end, token_line, token_column. If you need a standalone object, build it yourself:

const token = {
	type: lexer.next_token(),
	start: lexer.token_start,
	end: lexer.token_end,
	line: lexer.token_line,
	column: lexer.token_column,
}

Note the old method's null return never actually occurred; end of input is signalled by TOKEN_EOF, as before.

2. next_token_fast()

// before
const type = lexer.next_token_fast(skip_whitespace)

// after
const type = lexer.next_token(skip_whitespace)

Just rename the call.

Verification

vitest run (1496 tests), tsc --noEmit, oxlint / oxfmt --check and knip all pass.

---

Generated by Claude Code

[github-actions]

🎉 Package Size Decrease

📦 Package	📏 Base Size	📏 Source Size	📈 Size Change
@projectwallace/css-parser	47.5 kB	47.5 kB	-27 B

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/@projectwallace/css-parser@ff0e989

commit: ff0e989