yethee/tiktoken
This PHP port of OpenAI's tiktoken lets you encode text into tokens for models like gpt-3.5-turbo and any BPE encoding such as p50k_base. Includes vocabulary caching by default and an experimental FFI lib mode for faster encoding of larger texts.
The first stable release. There are no changes in the public API since version 0.12.0.
The lib mode is still in an experimental state. And method Encoder::encodeInChunks() is not supported.
How can I help you explore Laravel packages today?