Code Weaver

Helps Laravel developers discover, compare, and choose open-source packages. See popularity, security, maintainers, and scores at a glance to make better decisions.

Tokenizer Laravel Package

theseer/tokenizer

A PHP tokenizer library for parsing PHP source code into tokens, built on top of ext/tokenizer. Used by tools like PHPUnit to analyze code, inspect syntax, and support static analysis, refactoring, and code quality workflows.

View on GitHub

Deep Wiki

Context7

2.0.1

Fixed

Removed the custom token T_AMPERSAND as PHP 8.1+ provides tokens for it already and our 2.0.0 release overwrote it. See #44 for details.

2.0.0

This release bumps the minimum version required to PHP 8.1.

Added

New API XMLSerializer::appendToWriter to allow easier integration and better performance when writing fragments (Thanks @staabm)

Changed

Internal change: Now uses PHPToken::tokenize in favor of token_get_all (requires PHP 8+) (Thanks @staabm)

1.3.1

Fixed

#37: v1.3.0 introduced a breaking change on the token collection (ArrayAccess interface removed)

1.3.0

Changed

Require at least PHP 7.3 for building, code should still be PHP 7.2 compliant
Merge various performance improvements provided by @staabm
Merge some code cleanups provided by @staabm

1.2.3

Changed

Do not use implicitly nullable parameters

1.2.2

Fixed

#18: Tokenizer fails on protobuf metadata files

1.2.1

Fixed

#13: Fatal error when tokenizing files that contain only a single empty line

1.2.0

This release is now PHP 8.0 compliant.

Fixed

Whitespace handling in general (only noticeable in the intermediate TokenCollection) is now consistent

Changed

Updated Tokenizer to deal with changed whitespace handling in PHP 8.0 The XMLSerializer was unaffected.

1.1.3

Changed

Ensure XMLSerializer can deal with empty token collections

Fixed

#2: Fatal error in infection / phpunit

1.1.2

Changed

Reverted PHPUnit 8 test update to stay PHP 7.0 compliant

1.1.1

Fixed

#1: Empty file causes invalid array read

Changed

Tests should now be PHPUnit 8 compliant

1.1.0

Added

Allow use of custom namespace for XML serialization

1.0.0

Initial Release

Weaver

How can I help you explore Laravel packages today?

Conversation history is not saved when not logged in.

Add packages to context

No packages found.

codraw/entity-migrator

codraw/doctrine-extra

codraw/aws-tool-kit

codraw/validator

codraw/workflow

codraw/open-api

codraw/cron-job

codraw/process

codraw/log

nexmo/api-specification

capell-app/block-library

axium/identity

cetria/laravel-dummy-models

cetria/reflection-helper

agropredict/sso-auth-bundle

evolvestudio/spam-protection

datacore/hub-sdk

develia/commons

cuci/prototurk-sdk

cuci/prototurk-sdk-symfony