Weave Code
Code Weaver
Helps Laravel developers discover, compare, and choose open-source packages. See popularity, security, maintainers, and scores at a glance to make better decisions.
Feedback
Share your thoughts, report bugs, or suggest improvements.
Subject
Message
Log in Register
Home
Packages
Tokenizer
Changelog

Tokenizer Laravel Package

theseer/tokenizer

A PHP tokenizer library for parsing PHP source code into tokens, built on top of ext/tokenizer. Used by tools like PHPUnit to analyze code, inspect syntax, and support static analysis, refactoring, and code quality workflows.

View on GitHub
Deep Wiki
Context7
2.0.1

Fixed

  • Removed the custom token T_AMPERSAND as PHP 8.1+ provides tokens for it already and our 2.0.0 release overwrote it. See #44 for details.
2.0.0

This release bumps the minimum version required to PHP 8.1.

Added

  • New API XMLSerializer::appendToWriter to allow easier integration and better performance when writing fragments (Thanks @staabm)

Changed

  • Internal change: Now uses PHPToken::tokenize in favor of token_get_all (requires PHP 8+) (Thanks @staabm)
1.3.1

Fixed

  • #37: v1.3.0 introduced a breaking change on the token collection (ArrayAccess interface removed)
1.3.0

Changed

  • Require at least PHP 7.3 for building, code should still be PHP 7.2 compliant
  • Merge various performance improvements provided by @staabm
  • Merge some code cleanups provided by @staabm
1.2.3

Changed

  • Do not use implicitly nullable parameters
1.2.2

Fixed

  • #18: Tokenizer fails on protobuf metadata files
1.2.1

Fixed

  • #13: Fatal error when tokenizing files that contain only a single empty line
1.2.0

This release is now PHP 8.0 compliant.

Fixed

  • Whitespace handling in general (only noticeable in the intermediate TokenCollection) is now consistent

Changed

  • Updated Tokenizer to deal with changed whitespace handling in PHP 8.0 The XMLSerializer was unaffected.
1.1.3

Changed

  • Ensure XMLSerializer can deal with empty token collections

Fixed

  • #2: Fatal error in infection / phpunit
1.1.2

Changed

  • Reverted PHPUnit 8 test update to stay PHP 7.0 compliant
1.1.1

Fixed

  • #1: Empty file causes invalid array read

Changed

  • Tests should now be PHPUnit 8 compliant
1.1.0

Added

  • Allow use of custom namespace for XML serialization
1.0.0

Initial Release

Weaver

How can I help you explore Laravel packages today?

Conversation history is not saved when not logged in.
Prompt
Add packages to context
No packages found.
hexters/coinpayment
rjcodes/rjcms
act-training/laravel-permissions-manager
alimarchal/laravel-chart-of-accounts
babenkoivan/elastic-scout-driver
mkwebdesign/filament-watchdog-v5
renatomarinho/laravel-page-speed
zedmagdy/filament-business-hours
renatovdemoura/blade-elements-ui
devgeek/beacon-admin
benjamin-rqt/data-watcher-bundle
atriumphp/atrium
sandermuller/package-boost-laravel
sandermuller/boost-skills
redaxo/core
yusufgenc/filament-api-forge
l3aro/rating-star-for-filament
leek/filament-subtenant-scope
anil/file-picker
broqit/fields-ai