Html Meta Crawler Bundle Laravel Package

Getting Started

getOpenGraph($url) (added in v0.3.0) is the fastest path to your first result — it fetches the target page using a Facebook-crawler-style user agent and returns the parsed OpenGraph metadata, so you get the same og:title / og:description / og:image payload Facebook itself would see.

1. composer require the package, then check composer.json for the require constraints — this is pre-1.0, so pin to ^0.3 rather than *.
2. Look first at the package's service provider to see what is bound in the container and whether anything is published (config, migrations). If a config file exists, publish it before writing code, otherwise you'll hard-code assumptions that later conflict with the defaults.
3. Run php artisan tinker and call getOpenGraph('https://laravel.com') directly. Confirm the return shape before designing around it — v0.3.0 is the first release with this entry point, so the contract (array vs. DTO vs. Collection) is not yet battle-tested.
4. First real use case: link previews. Paste-a-URL flows in a CMS, comment feed, or chat feature where you store a title/description/thumbnail against a saved URL.

Implementation Patterns

Cache aggressively by URL. Every call is an outbound HTTP request to a third party. Wrap it in Cache::remember("og:{$hash}", now()->addDay(), ...) keyed on a normalized URL (lowercase host, strip tracking params, strip trailing slash) so the same page saved twice doesn't hit the network twice.

Push it to a queue. Never call getOpenGraph inside a request cycle that a user is waiting on. Dispatch a FetchOpenGraphJob on save, store the result on a polymorphic link_previews table or JSON column, and render from your own table. This also gives you retries and backoff for free.

Normalize before persisting. OpenGraph is optional markup; plenty of sites ship only <title> and <meta name="description">. Build a resolver that prefers og:*, falls back to <title> / meta description / <link rel="icon">, then to a placeholder. Same for og:image — relative paths (/images/hero.png) must be resolved against the base URL, or your preview cards render broken.

Fake the network in tests. The package almost certainly goes through Laravel's HTTP client. Use Http::fake(['*' => Http::response($fixtureHtml)]) with 3–4 saved HTML fixtures (full OG, partial OG, no OG, redirect) rather than hitting live sites. Your test suite must not depend on laravel.com being up.

Abstract it behind your own interface. Define OpenGraphFetcher in your app and bind the package to it. Given the 0.x version number, a rename in v0.4 shouldn't ripple through controllers.

Gotchas and Tips

Crawler impersonation is the point and also the failure mode. Many sites (Cloudflare, Akamai, some WordPress security plugins) block or challenge non-browser user agents, or serve different markup to facebookexternalhit than to Chrome. Expect 403s and empty results on a meaningful slice of URLs — always treat the call as fallible and degrade to your fallback resolver rather than throwing a 500.

SSRF is your problem, not the package's. You're passing user-supplied URLs to a server-side HTTP client. Validate scheme is http/https, resolve the host and reject private/loopback/link-local ranges, disable or limit redirects, and set an explicit timeout (5–10s). Do this in a FormRequest or a dedicated validator, not inside the fetch call.

Encoding is the classic silent breakage. DOMDocument::loadHTML mangles UTF-8 unless you force it (<?xml encoding="UTF-8"> prefix or mb_convert_encoding). If you see mojibake in og:title on accented or CJK pages, that's the cause. Also watch for <meta charset> declared differently from what the server sends.

Lazy-loaded images lie. Modern sites use <img data-src> and often omit og:image entirely or point it at a placeholder. If your preview cards look empty, check whether the site actually declares og:image before blaming the parser.

Debugging: log the final resolved URL (after redirects), the status code, and the raw HTML size alongside the parsed output. Nine times out of ten the "parser bug" is a 302 to a consent page or a 200 with a challenge body.

Extension points: if the fetcher isn't injectable, wrap it. If it is, bind your own implementation that adds a User-Agent override or a proxy — useful when a specific target site refuses Facebook's UA but allows a generic one.

Upgrade discipline: v0.3.0 is a feature addition, not a breaking change, but pre-1.0 semver means a minor bump can break you. Read the diff of every 0.x release, and keep the fixture-based tests as your tripwire.