pdphilip/elasticsearch
Laravel Eloquent-style ORM for Elasticsearch. Use familiar models and query builder methods to create, update, paginate, delete, and run ES searches: term/phrase, match, fuzzy, geo distance, highlighting, and more—designed to feel native in Laravel.
This release is compatible with Laravel 11, 12 & 13
composer test:l11, composer test:l12, composer test:l13, composer test:allpdphilip/omniterm bumped to ^3.0Connection::select() signature updated with $fetchUsing parameter (Laravel 13 compatibility)class_alias hacks that were needed for the old pattern.Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.5.3...v5.6.0
distinct(), bulkDistinct(), and groupBy() now work on nested fields (e.g., distinct('tags.key', true)). Previously these returned empty results because the compiled DSL lacked the required nested aggregation wrapper. The package now auto-detects nested mappings and wraps aggregations accordingly; no changes are needed in userland code.whereNestedObject() is combined with distinct() on the same nested path, the nested filter is injected inside the aggregation context so that only matching sub-documents are aggregated.Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.5.2...v5.5.3
elastic:re-index mapping analysis now includes settings for analyzers, tokenizers, char filters, filters, and normalizers.Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.5.1...v5.5.2
Schema::compileMapping() — compile a Blueprint callback into its resulting ES mapping structure without creating the index. Useful for debugging and previewing what mappingDefinition() will produce.Grammar::compileMapping() — public access to the Blueprint-to-properties compilation pipeline.elastic:re-index now correctly detects nested field mappings (e.g., nested('tags')->properties(...)), including their sub-fields and keyword sub-field changes.Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.5.0...v5.5.1
This release is compatible with Laravel 10, 11 & 12
New elastic:re-index command that automates the entire re-indexing process when your field mappings change. Pass a model name and the command handles the rest — creating a temp index, copying data, verifying counts, swapping, and cleaning up across 9 interactive phases with confirmation prompts between each step. - Docs
php artisan elastic:re-index UserLog
php artisan elastic:re-index "App\Models\ES\UserLog"
Features include:
hasKeyword: true)--force flag to skip all confirmation promptsNew elastic:make command to scaffold Elasticsearch models with the correct base class, connection, and a starter mappingDefinition(). - Docs
php artisan elastic:make UserLog
php artisan elastic:make ES/UserLog
Define your index field mappings directly on the model by overriding mappingDefinition(). Uses the same Blueprint syntax as migrations. Powers the elastic:re-index command's mapping analysis. - Docs
public static function mappingDefinition(Blueprint $index): void
{
$index->keyword('status');
$index->text('title', hasKeyword: true);
$index->geoPoint('location');
}
All five Artisan commands (elastic:status, elastic:indices, elastic:show, elastic:make, elastic:re-index) are now documented on a dedicated page. - Docs
whereNestedObject() and filterNested() no longer leak parentField into the parent query builder. The nested path was being set on $this instead of the sub-query, causing it to persist across subsequent queries on the same builder.Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.4.1...v5.5.0
_id no longer leaks into serialized output (toArray(), toJson()). The internal _id metadata field was being exposed alongside id, resulting in duplicate ID fields in model serialization.Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.4.0...v5.4.1
This release is compatible with Laravel 10, 11 & 12
When a model queries an index that doesn't exist yet, the index is created automatically instead of throwing index_not_found_exception. Matches Elasticsearch's own auto-create behavior for writes, extended to reads.
// No migration needed - index is created on first query
$products = Product::where('status', 'active')->get(); // returns empty collection
Controlled via options.auto_create_index config (default: true). - Docs
Why: New models shouldn't crash before the first write. Elasticsearch already auto-creates on insert; this extends the same behavior to reads.
First-class CLI tools for managing your Elasticsearch connection and indices:
php artisan elastic:status - Connection health check with cluster info and license detailsphp artisan elastic:indices - List all indices with health, doc count, and store sizephp artisan elastic:show {index} - Inspect an index: overview, mappings, settings, and analysis configphp artisan elastic:status
php artisan elastic:indices --all
php artisan elastic:show products
All commands support --connection= for non-default connections.
Why: Until now, inspecting your Elasticsearch setup meant leaving Laravel for curl or Kibana. These commands bring that visibility into Artisan where it belongs.
New upsert() method matching Laravel's native signature. Insert or update records by unique key in a single bulk operation. - Docs
Product::upsert(
[
['sku' => 'ABC', 'name' => 'Widget', 'price' => 10],
['sku' => 'DEF', 'name' => 'Gadget', 'price' => 20],
],
['sku'], // unique key
['name', 'price'] // columns to update if exists
);
Supports single documents, batch operations, and composite unique keys.
Why: Elasticsearch has no native upsert-by-field. This queries for existing documents first, then issues a single bulk request mixing index and update actions.
New GeneratesTimeOrderedIds trait for sortable, chronologically-ordered IDs. 20 characters, URL-safe, lexicographic sort matches creation order across processes. - Docs
use PDPhilip\Elasticsearch\Eloquent\GeneratesTimeOrderedIds;
class TrackingEvent extends Model
{
use GeneratesTimeOrderedIds;
}
$event->id; // "0B3kF5XRABCDE_fghijk"
$event->getRecordTimestamp(); // 1771160093773 (ms)
$event->getRecordDate(); // Carbon instance
Safe for mixed datasets; returns null for pre-existing IDs not generated by this trait.
Why: When you need IDs that sort chronologically across multiple processes/workers, ideal for high-volume event tracking and time-sequenced analytics.
BuildsAggregations, BuildsSearchQueries, BuildsFieldQueries, BuildsGeoQueries, BuildsNestedQueries, HandlesScripts, ManagesPitCompilesAggregations, CompilesOrders, CompilesWheres, FieldUtilitiesElasticsearchModel trait into focused traits for clarityElasticClient wrapperMetaDTOManagesOptions parameter inferenceaddFieldQuery() dispatcher in BuildsFieldQueries to deduplicate field query methodsgetIndexes(), getForeignKeys(), getViews() for Laravel compatibilityid is now always present in serialized model outputFull Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.3.0...v5.4.0
This release is compatible with Laravel 10, 11 & 12
distinct() queries now return ElasticCollections;
If a model relation exists and the aggregation is done on the foreign key, you can load the related model
UserLog::where('created_at', '>=', Carbon::now()->subDays(30))
->with('user')
->orderByDesc('_count')
->select('user_id')
->distinct(true);
Why: You can now treat distinct aggregations like real Eloquent results, including relationships.
New query method bulkDistinct(array $fields, $includeDocCount = false) - Docs
Run multiple distinct aggregations in parallel within a single Elasticsearch query.
$top3 = UserSession::where('created_at', '>=', Carbon::now()->subDays(30))
->limit(3)
->bulkDistinct(['country', 'device', 'browser_name'], true);
Why: Massive performance gains vs running sequential distinct queries.
groupByRanges() performs a range aggregation on the specified field. - Docs
groupByRanges()->get() — return bucketed results - Docs
groupByRanges()->agg() - apply metric aggregations per bucket -Docs
groupByDateRanges() performs a date range aggregation on the specified field. - Docs
groupByDateRanges()->get() — bucketed date ranges
groupByDateRanges()->agg() — metrics per date bucket
New model method getMetaValue($key) - Docs
Convenience method to get a specific meta value from the model instance.
$product = Product::where('color', 'green')->first();
$score = $product->getMetaValue('score');
When a bucketed query is executed, the raw bucket data is now stored in model meta. -Docs
$products = Product::distinct('price');
$buckets = $products->map(function ($product) {
return $product->getMetaValue('bucket');
});
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.2.0...v5.3.0
This release is compatible with Laravel 10, 11 & 12
This release introduces Query String Queries, bringing full Elasticsearch query_string syntax support directly into your Eloquent-style queries.
searchQueryString(query, $fields = null, $options = []) and related methods (orSearchQueryString, searchNotQueryString, etc.)query_string features — logical operators, wildcards, fuzziness, ranges, regex, boosting, field scoping, and moreQueryStringOptions class for fluent option configuration or array-based parametersExample:
Product::searchQueryString('status:(active OR pending) name:(full text search)^2')->get();
Product::searchQueryString('price:[5 TO 19}')->get();
// vanilla optional, +pizza required, -ice forbidden
Product::searchQueryString('vanilla +pizza -ice', function (QueryStringOptions $options) {
$options->type('cross_fields')->fuzziness(2);
})->get();
//etc
unmapped_type flag to your ordering query #88Product::query()->orderBy('name', 'desc', ['unmapped_type' => 'keyword'])->get();
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.1.0...v5.2.0
This release is compatible with Laravel 10, 11 & 12
withTrackTotalHits(bool|int|null $val = true)Appends the track_total_hits parameter to the DSL query, setting value to true will count all the hits embedded in the query meta not capping to Elasticsearch default of 10k hits
$products = Product::limit(5)->withTrackTotalHits(true)->get();
$totalHits = $products->getQueryMeta()->getTotalHits();
This can be set by default for all queries by updating the connection config in database.php:
'elasticsearch' => [
'driver' => 'elasticsearch',
.....
'options' => [
'track_total_hits' => env('ES_TRACK_TOTAL_HITS', null),
....
],
],
createOrFail(array $attributes)By default, when using create($attributes) where $attributes has an id that exists, the operation will upsert. createOrFail will throw a BulkInsertQueryException with status code 409 if the id exists
Product::createOrFail([
'id' => 'some-existing-id',
'name' => 'Blender',
'price' => 30,
]);
withRefresh(bool|string $refresh)By default, inserting documents will wait for the shards to refresh, ie: withRefresh(true), you can set the refresh flag with the following (as per ES docs):
true (default)
Refresh the relevant primary and replica shards (not the whole index) immediately after the operation occurs, so that the updated document appears in search results immediately.wait_for
Wait for the changes made by the request to be made visible by a refresh before replying. This doesn’t force an immediate refresh, rather, it waits for a refresh to happen.false
Take no refresh-related actions. The changes made by this request will be made visible at some point after the request returns.Product::withRefresh('wait_for')->create([
'name' => 'Blender',
'price' => 30,
]);
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.0.7...v5.1.0
This release is compatible with Laravel 10, 11 & 12
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.0.6...v5.0.7
This release is compatible with Laravel 10, 11 & 12
$count value fixed for setting query limit correctly, via #68Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.0.5...v5.0.6
This release is compatible with Laravel 10, 11 & 12
has() methodFull Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.0.4...v5.0.5
This release is compatible with Laravel 10, 11 & 12
disconnect() resets connection - removing connection is unnecessary in the context of Elasticsearch. Issue #64getTotalHits() helper method from query metasearchFuzzy() parses options as a closureFull Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.0.3...v5.0.4
This release is compatible with Laravel 10, 11 & 12
meta renamed to _meta to avoid the issue where a model could have a field called metahighlight() passed without fields did not highlight all hitsBelongsTo in some SQL cases used ES connectionorderBy('_score') was not parsing correctlyFull Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.0.2...v5.0.3
This release is compatible with Laravel 10, 11 & 12
bulkInsert()bulkInsert() is identical to insert() but will continue on errors and return an array of the results.
People::bulkInsert([
[
'id' => '_edo3ZUBnJmuNNwymfhJ', // Will update (if id exists)
'name' => 'Jane Doe',
'status' => 1,
],
[
'name' => 'John Doe', // Will Create
'status' => 2,
],
[
'name' => 'John Dope',
'status' => 3,
'created_at' => 'xxxxxx', // Will fail
],
]);
Returns:
{
"hasErrors": true,
"total": 3,
"took": 0,
"success": 2,
"created": 1,
"modified": 1,
"failed": 1,
"errors": [
{
"id": "Y-dp3ZUBnJmuNNwy7vkF",
"type": "document_parsing_exception",
"reason": "[1:45] failed to parse field [created_at] of type [date] in document with id 'Y-dp3ZUBnJmuNNwy7vkF'. Preview of field's value: 'xxxxxx'"
}
]
}
distinct() aggregation now appends searchAfter key in metaFull Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.0.1...v5.0.2
This release is compatible with Laravel 10, 11 & 12
orderByNestedDesc()class_alias to set the correct traits for the given Laravel versionFull Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.0.0...v5.0.1
We’re excited to announce v5 of the laravel-elasticsearch package - compatible with Laravel 10, 11, and 12.
V5 is the brainchild of [@use-the-fork](https://github.com/use-the-fork) and is a near-complete rewrite of the package; packed with powerful new features, deep integration with Elasticsearch’s full capabilities, and a much tighter alignment with Laravel’s Eloquent. It lays a solid, future-proof foundation for everything that comes next.
Please take a look at the upgrade guide carefully, as there are several significant breaking changes.
"pdphilip/elasticsearch": "^5",
ES_INDEX_PREFIX no longer auto-appends an underscore (_).ES_INDEX_PREFIX=my_prefix → my_prefix_ES_INDEX_PREFIX=my_prefix_Model ID Field
$model->_id is deprecated. Use $model->id instead.
If your model had a separate id field, you must rename it.
Default Limit Constant
MAX_SIZE constant is removed. Use $defaultLimit property:
use PDPhilip\Elasticsearch\Eloquent\Model;
class Product extends Model
{
protected $defaultLimit = 10000;
protected $connection = 'elasticsearch';
}
where() Behavior Changed
Now uses term query instead of match.
// Old:
Product::where('name', 'John')->get(); // match query
// New:
Product::whereMatch('name', 'John')->get(); // match query
Product::where('name', 'John')->get(); // term query
orderByRandom() Removed
Replace with functionScore() Docs
Full-text Search Options Updated
Methods like asFuzzy(), setMinShouldMatch(), setBoost() removed.
Use callback-based SearchOptions instead:
Product::searchTerm('espresso time', function (SearchOptions $options) {
$options->searchFuzzy();
$options->boost(2);
$options->minimumShouldMatch(2);
})->get();
Legacy Search Methods Removed
All {xx}->search() methods been removed. Use {multi_match}->get() instead.
distinct() and groupBy() behavior updated. Docs
Review queries using them and refactor accordingly.
IndexBlueprint and AnalyzerBlueprint has been removed and replaced with a single Blueprint class
- use PDPhilip\Elasticsearch\Schema\IndexBlueprint;
- use PDPhilip\Elasticsearch\Schema\AnalyzerBlueprint;
use PDPhilip\Elasticsearch\Schema\Blueprint;
Schema::hasIndex has been removed. Use Schema::hasTable or Schema::indexExists instead.
geo($field) field property has been replaced with geoPoint($field)
{field}->index($bool) field property has been replaced with {field}->indexField($bool);
alias() field type has been removed. Use aliasField() instead.
settings() method has been replaced with withSetting()
map() method has been replaced with withMapping()
analyzer() method has been replaced with addAnalyzer()
tokenizer() method has been replaced with addTokenizer()
charFilter() method has been replaced with addCharFilter()
filter() method has been replaced with addFilter()
DynamicIndex trait. upgrade guideConnection::on('elasticsearch')->elastic()->{clientMethod}();
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.5.3...v5.0.0
This release is compatible with Laravel 10 and 11
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.5.1...v4.5.3
This second release candidate of V5 brings in Laravel 12 compatibility along with a few bug fixes
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v5.0.0-RC1...v5.0.0-RC2
This is the first release candidate of V5, a near-complete rewrite of the Laravel-Elasticsearch package. This update delivers a more seamless integration with Laravel’s Models, Eloquent, and Query Builders while providing greater access to Elasticsearch’s advanced features.
V5 introduces significant breaking changes, major upgrades, and new functionality. A full breakdown will be available in the updated documentation upon the final release.
The current RC is compatible with Laravel 10 and 11, while the final 5.0.0 release will support Laravel 12 as well.
Special thanks to @use-the-fork for initiating this rebuild and laying the groundwork (and then some) for this major release.
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.5.1...v5.0.0-RC1
This release is compatible with Laravel 10 and 11
updateOrCreate and updateOrCreateWithoutRefresh methods by @IonutRusen in https://github.com/pdphilip/laravel-elasticsearch/pull/57Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.5.0...v4.5.1
This release is for compatibility with both Laravel 10 and 11, and marks a version bump to 4.5.0.
Keyword type queries are checked by default and will select the keyword sub-mapping if it is found; however, this invokes an extra query to check the mapping first.
You can now disable this by setting options.bypass_map_validation = true
'elasticsearch' => [
......
'options' => [
'bypass_map_validation' => env('ES_OPT_BYPASS_MAP_VALIDATION', false),
.....
],
.....
],
When performing bulk inserts, the default is 1000 at a time.
You can now adjust this by setting options.insert_chunk_size to your desired amount.
'elasticsearch' => [
......
'options' => [
'insert_chunk_size' => env('ES_OPT_INSERT_CHUNK_SIZE', 1000),
.....
],
.....
],
'elasticsearch' => [
'driver' => 'elasticsearch',
'auth_type' => env('ES_AUTH_TYPE', 'http'), //http or cloud
'hosts' => explode(',', env('ES_HOSTS', 'http://localhost:9200')),
'username' => env('ES_USERNAME', ''),
'password' => env('ES_PASSWORD', ''),
'cloud_id' => env('ES_CLOUD_ID', ''),
'api_id' => env('ES_API_ID', ''),
'api_key' => env('ES_API_KEY', ''),
'ssl_cert' => env('ES_SSL_CA', ''),
'ssl' => [
'cert' => env('ES_SSL_CERT', ''),
'cert_password' => env('ES_SSL_CERT_PASSWORD', ''),
'key' => env('ES_SSL_KEY', ''),
'key_password' => env('ES_SSL_KEY_PASSWORD', ''),
],
'index_prefix' => env('ES_INDEX_PREFIX', false),
'options' => [
'bypass_map_validation' => env('ES_OPT_BYPASS_MAP_VALIDATION', false),
'insert_chunk_size' => env('ES_OPT_INSERT_CHUNK_SIZE', 1000),
'logging' => env('ES_OPT_LOGGING', false),
'allow_id_sort' => env('ES_OPT_ID_SORTABLE', false),
'ssl_verification' => env('ES_OPT_VERIFY_SSL', true),
'retires' => env('ES_OPT_RETRIES', null),
'meta_header' => env('ES_OPT_META_HEADERS', true),
],
'error_log_index' => env('ES_ERROR_INDEX', false),
],
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.4.1...v4.5.0
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.4.0...v4.4.1
This release introduces compatibility with both Laravel 10 and 11, and marks a version bump to 4.4.0.
While there are no breaking changes, the connection class has been fully rebuilt by @use-the-fork. This foundational improvement justifies the version increment and sets the stage for further enhancements.
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.3.0...v4.4.0
This release marks a version bump within the 4.x branch due to one breaking change in rawSearch(). This branch is committed to Laravel 10 and 11 compatibility.
rawSearch($bodyParams, $returnRaw = false) has now been split into rawSearch($bodyParams) and rawDsl($bodyParams) where:
rawSearch($bodyParams) returns an ElasticCollection of resultsrawDsl($bodyParams) returns the result body as is from Elasticsearch. Equivalent to $returnRaw = true previously.getFieldMapping(string|array $field = '*', $raw = false)Schema method that can be called from your model:
Product::getFieldMapping('color'); //Returns a key/value array of field/types for color
Product::getFieldMapping('color',true); //Returns the mapping for color field as is from Elasticsearch
Product::getFieldMapping(['color','name']); //Returns maapings for color and name
Product::getFieldMapping(); //returns all field mappings, same as getFieldMapping('*')
or via Schema: Schema::getFieldMapping($index, $field, $raw)
Schema::getFieldMapping('products','color',true);
orderByRandom(string $column, int $seed = 1)Uses ES's random_score to randomise ordering
Product::where('color', 'silver')->orderByRandom('created_at', rand(1, 1000))->limit(5)->get();
The value of $seed will return the same results if unchanged. This is required for consistencey, ex: pagination
whereExact() keyword field validator has been fixed
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.2.0...v4.3.0
This release marks a version bump within the 4.x branch committed to Laravel 10 & 11 compatibility. There are no breaking changes from 4.1.x
This version introduces query-building methods to allow matching across multiple fields: ES docs
The current search builder works in isolation for full-text searching; this upgrade brings those features into the standard query builder that will run like a normal query. Meaning you can:
get(), first(), aggregate(), paginate() etc on full text search results. In time, this will replace the current search methods like: Book::term('Eric')->search();
searchTerm($term, $fields = ['*'], $options = []) - type: best_fields
searchTermMost($term, $fields = ['*'], $options = []) type: most_fields
searchTermCross($term, $fields = ['*'], $options = []) type: cross_fields
searchPhrase($phrase, $fields = ['*'], $options = []) type: phrase
searchPhrasePrefix($phrase, $fields = ['*'], $options = []) type: phrase_prefix
searchBoolPrefix($phrase, $fields = ['*'], $options = []) type: bool_prefix
Each method has a corresponding OR version, ex orSearchPhrase('Laravel Elasticsearch')
These methods will introduce an Elasticsearch score and will be ordered by score by default.
$fields: By default, all fields will be searched through; you can specify which as well as boost certain fields, ex:
People:searchTerm('John',['name^3','description^2','friends.name'])->get();
$options: Allows you to set any options for the multi_match clause to use, ex:
analyzer, boost, operator, minimum_should_match, fuzziness, lenient, prefix_length, max_expansions, fuzzy_rewrite, zero_terms_query.
searchFor($value) is a convenience method that will either use term or phrase depending on the word count of $value
withHighlights($fields = [], $preTag = '<em>', $postTag = '</em>', $globalOptions = [])
Option helpers
asFuzzy()
setMinShouldMatch(int $value)
{"minimum_should_match": $value} to the previous clausesetBoost(int $value)
{"boost": $value} to the previous clauseExamples
Product::searchTerm('remarkble')->asFuzzy()->withHighlights()->get();
Product::searchPhrasePrefix('coffee bea')->where('is_active',true)->paginate();
Product::searchPhrase('United States')->orSearchPhrase('United Kingdom')->sum('orders');
find($id), findOrFail($id) and findOrNew($id) now uses the ES get call directly ie: /my_index/_doc/my_id
findOrNew($id): If the $id was not found, then it will return a new model instance with the $id value as providedFull Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.1.1...v4.2.0
_id in the error payloadFull Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.1.0...v4.1.1
This release marks a version bump within the 4.x branch committed to Laravel 10 & 11 compatibility.
This release represents a significant advancement in the package's development, primarily driven by PR #32. Key improvements include:
A sincere thank you to @use-the-fork for their substantial contributions to this release. Their efforts have been instrumental in elevating the quality, reliability and maturity of this package.
There's only one breaking change:
$query->orderBy(string $column, string $direction = 'asc', string $mode = null, array $missing = '_last');
// Has been changed back to:
$query->orderBy(string $column, string $direction = 'asc');
// With a new method to add any ES sort parameter to a given column:
$query->withSort(string $col, string $key, mixed $value);
orderByDescExamples
$products = Product::orderBy('color.keyword', 'desc', null, '_first')->get();
// Is now:
$products = Product::orderBy('color.keyword', 'desc')->withSort('color.keyword', 'missing', '_first')->get();
And:
$products = Product::where('is_active', true)->orderBy('order_values', 'desc', 'avg')->get();
// Is now:
$products = Product::where('is_active', true)->orderBy('order_values', 'desc')->withSort('order_values', 'mode', 'avg')->get();
This change is to future-proof for the outlier sorting options that ES offers, ex:
$products = Product::withSort('price', 'unmapped_type', 'long')->get();
$products = Product::withSort('price', 'ignore_unmapped', true')->get();
Inspired by #36 via @ildar-developer
Given the default limitations of Elasticsearch to paginate beyond 10k records, cursor pagination allows you to paginate indefinitely using search_after given a sort map.
As is the case with core Laravel's cursorPaginate , you can only page next and prev
cursorPaginate($perPage, $columns, $cursorName, $cursor)
Example:
Product::orderByDesc('orders')->orderByDesc('price')->cursorPaginate(50)->withQueryString();
The accuracy of this method depends on the sort, as would be the case using ES directly. If no sort is provided, it will try to sort by created_at and updated_at if those mappings are found; otherwise, it will throw a MissingOrderException.
insert($values, $returnData = null);
insertWithoutRefresh($values, $returnData = null);
These methods use Elasticsearch's Bulk API under the hood, providing efficient ways to insert multiple documents at once.
If the $returnData parameter is null or false, it will return a summary as an array:
{
"success": true,
"timed_out": false,
"took": 7725,
"total": 100000,
"created": 100000,
"modified": 0,
"failed": 0
}
Otherwise, it will return an ElasticCollection of all the inserted records.
insert($values, $returnData = null)
insertWithoutRefresh but provides immediate consistency.insertWithoutRefresh($values, $returnData = null)
insert().insert() when:
insertWithoutRefresh() when:
Queries that return collections (get(), search(), insert()) now return them as an ElasticCollection. ElasticCollection is the same as Laravel's Eloquent Collection but with the executed query's metadata embedded in it.
$fetch = Product::where('orders', '>', 100)->limit(50)->orderByDesc('price')->get();
$shards = $fetch->getShards(); // Shards info
$dsl = $fetch->getDsl(); // The full query DSL that was used
$total = $fetch->getTotal(); // Total records in the index if there were hits
$maxScore = $fetch->getMaxScore(); // Max score of the search
$took = $fetch->getTook(); // Time taken for the search in milliseconds
$meta = $fetch->getQueryMetaAsArray(); // All the metadata in an array
return $fetch; //gives your collection of results as always
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.0.4...v4.1.0
This unified release includes updates for multiple Laravel versions:
sum(), avg(), min() and max() can now process multiple fields in one callProduct::where('color','blue')->avg(['orders', 'price']);
//Previously required two separate calls:
Product::where('color','blue')->avg('orders');
Product::where('color','blue')->avg('price');
rawSearch(array $bodyParams, bool $returnRaw = false)trueProduct::rawSearch($body, true);
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.0.3...v4.0.4
This unified release includes updates for multiple Laravel versions:
wherePhrasePrefix($field, $phraseWithPrefix)Method for looking up a specific sequence of words where the last word starts with a particular prefix
Person::wherePhrasePrefix('description', 'loves es')->get();
// returns: loves espresso, loves essays, loves eskimos, etc
Docs: https://elasticsearch.pdphilip.com/es-specific#where-phrase-prefix
phrase($field)Method for searching across multiple fields for a specific phrase (sequence of words in order)
Book::phrase('United States')->orPhrase('United Kingdom')->search();
// Search for books that contain either 'United States' or 'United Kingdom', phrases like 'United Emirates' will not be included.
Docs: https://elasticsearch.pdphilip.com/full-text-search#phrase-search-phrase
agg(array $functions,$field)Optimization method that allows you to call multiple aggregation functions on a single field in one call.
Available aggregation functions: count, avg, min, max, sum, matrix.
Product::where('is_active',true)->agg(['count','avg','min','max','sum'],'sales');
https://elasticsearch.pdphilip.com/aggregation#grouped-aggregations
toDsl() (or toSql())Returns the parsed DSL query from the query builder
Product::whereIn('color', ['red', 'green'])->orderByDesc('sales')->toDsl();
Returns
{
"index": "products",
"body": {
"query": {
"terms": {
"color.keyword": [
"red",
"green"
]
}
},
"_source": [
"*"
],
"sort": [
{
"sales": {
"order": "desc"
}
}
]
}
}
Docs: https://elasticsearch.pdphilip.com/es-specific#to-dsl
Tests for new features: https://github.com/pdphilip/laravel-elasticsearch-tests/blob/main/tests/EloquentTests/Update24-01Test.php
BelongsToMany() which is not supported (but can be worked around easily)// database.php
'elasticsearch' => [
'driver' => 'elasticsearch',
//......
//......
//......
'error_log_index' => env('ES_ERROR_INDEX', false),
],
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.0.2...v4.0.3
New numeric type mappings for IndexBlueprint
double($field) - A double-precision 64-bit IEEE 754 floating point number, restricted to finite values.byte($field) - A signed 8-bit integer with a minimum value of -128 and a maximum value of 127.halfFloat($field) - A half-precision 16-bit IEEE 754 floating point number, restricted to finite values.scaledFloat($field, $scalingFactor = 100) - A floating point number that is backed by a long, scaled by a fixed double scaling factor.unsignedLong($field) - An unsigned 64-bit integer with a minimum value of 0 and a maximum value of 264-1.Example:
Schema::create('my_index', function (IndexBlueprint $index) {
$index->double('some_field_a');
$index->byte('some_field_b');
$index->halfFloat('some_field_c');
$index->scaledFloat('some_field_d', 100);
$index->unsignedLong('some_field_e');
});
elasticsearchExample with multiple connections (database.php):
'elasticsearch' => [
'driver' => 'elasticsearch',
'auth_type' => env('ES_AUTH_TYPE', 'http'), //http, cloud or api
'hosts' => explode(',', env('ES_HOSTS', 'http://localhost:9200')),
'username' => env('ES_USERNAME', ''),
'password' => env('ES_PASSWORD', ''),
'cloud_id' => env('ES_CLOUD_ID', ''),
'api_id' => env('ES_API_ID', ''),
'api_key' => env('ES_API_KEY', ''),
'ssl_cert' => env('ES_SSL_CA', ''),
'ssl' => [
'cert' => env('ES_SSL_CERT', ''),
'cert_password' => env('ES_SSL_CERT_PASSWORD', ''),
'key' => env('ES_SSL_KEY', ''),
'key_password' => env('ES_SSL_KEY_PASSWORD', ''),
],
'index_prefix' => env('ES_INDEX_PREFIX', false),
'options' => [
'allow_id_sort' => env('ES_OPT_ID_SORTABLE', false),
'ssl_verification' => env('ES_OPT_VERIFY_SSL', true),
'retires' => env('ES_OPT_RETRIES', null),
'meta_header' => env('ES_OPT_META_HEADERS', true),
],
'query_log' => [
'index' => false,
'error_only' => true,
],
],
'elasticsearch-cloud' => [
'driver' => 'elasticsearch',
'auth_type' => env('ES_CLOUD_AUTH_TYPE', 'http'), //http or cloud
'hosts' => explode(',', env('ES_CLOUD_HOSTS', 'http://localhost:9200')),
'username' => env('ES_CLOUD_USERNAME', ''),
'password' => env('ES_CLOUD_PASSWORD', ''),
'cloud_id' => env('ES_CLOUD_CLOUD_ID', ''),
'api_id' => env('ES_CLOUD_API_ID', ''),
'api_key' => env('ES_CLOUD_API_KEY', ''),
'ssl_cert' => env('ES_CLOUD_SSL_CA', ''),
'ssl' => [
'cert' => env('ES_CLOUD_SSL_CERT', ''),
'cert_password' => env('ES_CLOUD_SSL_CERT_PASSWORD', ''),
'key' => env('ES_CLOUD_SSL_KEY', ''),
'key_password' => env('ES_CLOUD_SSL_KEY_PASSWORD', ''),
],
'index_prefix' => env('ES_CLOUD_INDEX_PREFIX', false),
'options' => [
'allow_id_sort' => env('ES_CLOUD_OPT_ID_SORTABLE', false),
'ssl_verification' => env('ES_CLOUD_OPT_VERIFY_SSL', true),
'retires' => env('ES_CLOUD_OPT_RETRIES', null),
'meta_header' => env('ES_CLOUD_OPT_META_HEADERS', true),
],
'query_log' => [
'index' => false,
'error_only' => true,
],
],
Examples of selecting connection:
Schema::on('elasticsearch-cloud')->create('my_index', ...... );
Product::on('elasticsearch-cloud')->get() //If $connection in Product model is not 'elasticsearch-cloud';
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v4.0.1...v4.0.2
New numeric type mappings for IndexBlueprint
double($field) - A double-precision 64-bit IEEE 754 floating point number, restricted to finite values.byte($field) - A signed 8-bit integer with a minimum value of -128 and a maximum value of 127.halfFloat($field) - A half-precision 16-bit IEEE 754 floating point number, restricted to finite values.scaledFloat($field, $scalingFactor = 100) - A floating point number that is backed by a long, scaled by a fixed double scaling factor.unsignedLong($field) - An unsigned 64-bit integer with a minimum value of 0 and a maximum value of 264-1.Example:
Schema::create('my_index', function (IndexBlueprint $index) {
$index->double('some_field_a');
$index->byte('some_field_b');
$index->halfFloat('some_field_c');
$index->scaledFloat('some_field_d', 100);
$index->unsignedLong('some_field_e');
});
elasticsearchExample with multiple connections (database.php):
'elasticsearch' => [
'driver' => 'elasticsearch',
'auth_type' => env('ES_AUTH_TYPE', 'http'), //http, cloud or api
'hosts' => explode(',', env('ES_HOSTS', 'http://localhost:9200')),
'username' => env('ES_USERNAME', ''),
'password' => env('ES_PASSWORD', ''),
'cloud_id' => env('ES_CLOUD_ID', ''),
'api_id' => env('ES_API_ID', ''),
'api_key' => env('ES_API_KEY', ''),
'ssl_cert' => env('ES_SSL_CA', ''),
'ssl' => [
'cert' => env('ES_SSL_CERT', ''),
'cert_password' => env('ES_SSL_CERT_PASSWORD', ''),
'key' => env('ES_SSL_KEY', ''),
'key_password' => env('ES_SSL_KEY_PASSWORD', ''),
],
'index_prefix' => env('ES_INDEX_PREFIX', false),
'options' => [
'allow_id_sort' => env('ES_OPT_ID_SORTABLE', false),
'ssl_verification' => env('ES_OPT_VERIFY_SSL', true),
'retires' => env('ES_OPT_RETRIES', null),
'meta_header' => env('ES_OPT_META_HEADERS', true),
],
'query_log' => [
'index' => false,
'error_only' => true,
],
],
'elasticsearch-cloud' => [
'driver' => 'elasticsearch',
'auth_type' => env('ES_CLOUD_AUTH_TYPE', 'http'), //http or cloud
'hosts' => explode(',', env('ES_CLOUD_HOSTS', 'http://localhost:9200')),
'username' => env('ES_CLOUD_USERNAME', ''),
'password' => env('ES_CLOUD_PASSWORD', ''),
'cloud_id' => env('ES_CLOUD_CLOUD_ID', ''),
'api_id' => env('ES_CLOUD_API_ID', ''),
'api_key' => env('ES_CLOUD_API_KEY', ''),
'ssl_cert' => env('ES_CLOUD_SSL_CA', ''),
'ssl' => [
'cert' => env('ES_CLOUD_SSL_CERT', ''),
'cert_password' => env('ES_CLOUD_SSL_CERT_PASSWORD', ''),
'key' => env('ES_CLOUD_SSL_KEY', ''),
'key_password' => env('ES_CLOUD_SSL_KEY_PASSWORD', ''),
],
'index_prefix' => env('ES_CLOUD_INDEX_PREFIX', false),
'options' => [
'allow_id_sort' => env('ES_CLOUD_OPT_ID_SORTABLE', false),
'ssl_verification' => env('ES_CLOUD_OPT_VERIFY_SSL', true),
'retires' => env('ES_CLOUD_OPT_RETRIES', null),
'meta_header' => env('ES_CLOUD_OPT_META_HEADERS', true),
],
'query_log' => [
'index' => false,
'error_only' => true,
],
],
Examples of selecting connection:
Schema::on('elasticsearch-cloud')->create('my_index', ...... );
Product::on('elasticsearch-cloud')->get() //If $connection in Product model is not 'elasticsearch-cloud';
New numeric type mappings for IndexBlueprint
double($field) - A double-precision 64-bit IEEE 754 floating point number, restricted to finite values.byte($field) - A signed 8-bit integer with a minimum value of -128 and a maximum value of 127.halfFloat($field) - A half-precision 16-bit IEEE 754 floating point number, restricted to finite values.scaledFloat($field, $scalingFactor = 100) - A floating point number that is backed by a long, scaled by a fixed double scaling factor.unsignedLong($field) - An unsigned 64-bit integer with a minimum value of 0 and a maximum value of 264-1.Example:
Schema::create('my_index', function (IndexBlueprint $index) {
$index->double('some_field_a');
$index->byte('some_field_b');
$index->halfFloat('some_field_c');
$index->scaledFloat('some_field_d', 100);
$index->unsignedLong('some_field_e');
});
elasticsearchExample with multiple connections (database.php):
'elasticsearch' => [
'driver' => 'elasticsearch',
'auth_type' => env('ES_AUTH_TYPE', 'http'), //http, cloud or api
'hosts' => explode(',', env('ES_HOSTS', 'http://localhost:9200')),
'username' => env('ES_USERNAME', ''),
'password' => env('ES_PASSWORD', ''),
'cloud_id' => env('ES_CLOUD_ID', ''),
'api_id' => env('ES_API_ID', ''),
'api_key' => env('ES_API_KEY', ''),
'ssl_cert' => env('ES_SSL_CA', ''),
'ssl' => [
'cert' => env('ES_SSL_CERT', ''),
'cert_password' => env('ES_SSL_CERT_PASSWORD', ''),
'key' => env('ES_SSL_KEY', ''),
'key_password' => env('ES_SSL_KEY_PASSWORD', ''),
],
'index_prefix' => env('ES_INDEX_PREFIX', false),
'options' => [
'allow_id_sort' => env('ES_OPT_ID_SORTABLE', false),
'ssl_verification' => env('ES_OPT_VERIFY_SSL', true),
'retires' => env('ES_OPT_RETRIES', null),
'meta_header' => env('ES_OPT_META_HEADERS', true),
],
'query_log' => [
'index' => false,
'error_only' => true,
],
],
'elasticsearch-cloud' => [
'driver' => 'elasticsearch',
'auth_type' => env('ES_CLOUD_AUTH_TYPE', 'http'), //http or cloud
'hosts' => explode(',', env('ES_CLOUD_HOSTS', 'http://localhost:9200')),
'username' => env('ES_CLOUD_USERNAME', ''),
'password' => env('ES_CLOUD_PASSWORD', ''),
'cloud_id' => env('ES_CLOUD_CLOUD_ID', ''),
'api_id' => env('ES_CLOUD_API_ID', ''),
'api_key' => env('ES_CLOUD_API_KEY', ''),
'ssl_cert' => env('ES_CLOUD_SSL_CA', ''),
'ssl' => [
'cert' => env('ES_CLOUD_SSL_CERT', ''),
'cert_password' => env('ES_CLOUD_SSL_CERT_PASSWORD', ''),
'key' => env('ES_CLOUD_SSL_KEY', ''),
'key_password' => env('ES_CLOUD_SSL_KEY_PASSWORD', ''),
],
'index_prefix' => env('ES_CLOUD_INDEX_PREFIX', false),
'options' => [
'allow_id_sort' => env('ES_CLOUD_OPT_ID_SORTABLE', false),
'ssl_verification' => env('ES_CLOUD_OPT_VERIFY_SSL', true),
'retires' => env('ES_CLOUD_OPT_RETRIES', null),
'meta_header' => env('ES_CLOUD_OPT_META_HEADERS', true),
],
'query_log' => [
'index' => false,
'error_only' => true,
],
],
Examples of selecting connection:
Schema::on('elasticsearch-cloud')->create('my_index', ...... );
Product::on('elasticsearch-cloud')->get() //If $connection in Product model is not 'elasticsearch-cloud';
This Release is for both Laravel 10 and 11 users, require:
"pdphilip/elasticsearch": "^4.0"
highlight() - This package now dials into Elaticsearch's highlight feature - docswhereTimestamp() - convenience clause to help sanitize timestamp values - docs - issue #22rawAggregation() - similar to rawSearch(), passing in aggregation DSL queries will be processed and formatted- docs - issue #24_id disabled by default. The package will now remove any clauses that include _id sorting.Update your databse.php to fit the new config features:
'elasticsearch' => [
'driver' => 'elasticsearch',
'auth_type' => env('ES_AUTH_TYPE', 'http'), //http or cloud
'hosts' => explode(',', env('ES_HOSTS', 'http://localhost:9200')),
'username' => env('ES_USERNAME', ''),
'password' => env('ES_PASSWORD', ''),
'cloud_id' => env('ES_CLOUD_ID', ''),
'api_id' => env('ES_API_ID', ''),
'api_key' => env('ES_API_KEY', ''),
'ssl_cert' => env('ES_SSL_CA', ''),
'ssl' => [
'cert' => env('ES_SSL_CERT', ''),
'cert_password' => env('ES_SSL_CERT_PASSWORD', ''),
'key' => env('ES_SSL_KEY', ''),
'key_password' => env('ES_SSL_KEY_PASSWORD', ''),
],
'index_prefix' => env('ES_INDEX_PREFIX', false),
'options' => [
'allow_id_sort' => env('ES_OPT_ID_SORTABLE', false),
'ssl_verification' => env('ES_OPT_VERIFY_SSL', true),
'retires' => env('ES_OPT_RETRIES', null),
'meta_header' => env('ES_OPT_META_HEADERS', true),
],
'query_log' => [
'index' => false, //Or provide a name for the logging index ex: 'laravel_query_logs'
'error_only' => true, //If false, then all queries are logged if the query_log index is set
],
],
Full Changelog: https://github.com/pdphilip/laravel-elasticsearch/compare/v3.11.0...v4.0.1
How can I help you explore Laravel packages today?