Code Weaver

Helps Laravel developers discover, compare, and choose open-source packages. See popularity, security, maintainers, and scores at a glance to make better decisions.

Tokenizer Laravel Package

theseer/tokenizer

A PHP tokenizer library for parsing PHP source code into tokens, built on top of ext/tokenizer. Used by tools like PHPUnit to analyze code, inspect syntax, and support static analysis, refactoring, and code quality workflows.

View on GitHub

Deep Wiki

Context7

2.0.1

Fixed

Removed the custom token T_AMPERSAND as PHP 8.1+ provides tokens for it already and our 2.0.0 release overwrote it. See #44 for details.

2.0.0

This release bumps the minimum version required to PHP 8.1.

Added

New API XMLSerializer::appendToWriter to allow easier integration and better performance when writing fragments (Thanks @staabm)

Changed

Internal change: Now uses PHPToken::tokenize in favor of token_get_all (requires PHP 8+) (Thanks @staabm)

1.3.1

Fixed

#37: v1.3.0 introduced a breaking change on the token collection (ArrayAccess interface removed)

1.3.0

Changed

Require at least PHP 7.3 for building, code should still be PHP 7.2 compliant
Merge various performance improvements provided by @staabm
Merge some code cleanups provided by @staabm

1.2.3

Changed

Do not use implicitly nullable parameters

1.2.2

Fixed

#18: Tokenizer fails on protobuf metadata files

1.2.1

Fixed

#13: Fatal error when tokenizing files that contain only a single empty line

1.2.0

This release is now PHP 8.0 compliant.

Fixed

Whitespace handling in general (only noticeable in the intermediate TokenCollection) is now consistent

Changed

Updated Tokenizer to deal with changed whitespace handling in PHP 8.0 The XMLSerializer was unaffected.

1.1.3

Changed

Ensure XMLSerializer can deal with empty token collections

Fixed

#2: Fatal error in infection / phpunit

1.1.2

Changed

Reverted PHPUnit 8 test update to stay PHP 7.0 compliant

1.1.1

Fixed

#1: Empty file causes invalid array read

Changed

Tests should now be PHPUnit 8 compliant

1.1.0

Added

Allow use of custom namespace for XML serialization

1.0.0

Initial Release

Weaver

How can I help you explore Laravel packages today?

Conversation history is not saved when not logged in.

Add packages to context

No packages found.

davejamesmiller/laravel-breadcrumbs

artisanry/parsedown

christhompsontldr/phpsdk

enqueue/dsn

bunny/bunny

enqueue/test

enqueue/null

enqueue/amqp-tools

milesj/emojibase

bower-asset/punycode

bower-asset/inputmask

bower-asset/jquery

bower-asset/yii2-pjax

laravel/nova

spatie/laravel-mailcoach

spatie/laravel-superseeder

laravel/liferaft

nst/json-test-suite

danielmiessler/sec-lists

jackalope/jackalope-transport