Commit Graph

38 Commits

Author SHA1 Message Date
Bart van Bragt 557604e426 fix: retry fill ES queries on transient connection failure 2026-04-15 14:07:43 +02:00
Bart van Bragt 80357e6a77 fix: include --staging in build conflict suggestions 2026-04-14 16:46:06 +02:00
Bart van Bragt 2fc66b9bf7 perf: eager-load relationships in UpdateSearchJob and cache permissions in DiscussionSeeder
- Add relationships() to Seeder base class; DiscussionSeeder implements it to
  return tags/recipientUsers/recipientGroups for eager loading
- UpdateSearchJob calls loadMissing(relationships()) before the map loop,
  eliminating N×R lazy-load queries per batch
- Remove refresh:true from bulk call — synchronous ES refresh was the primary
  throughput bottleneck during indexing
- Cache viewForum permissions in DiscussionSeeder per job instance, replacing
  N×2 Permission queries per document with a single cached query
- Increase seeder batch size from 1000 to 2500
2026-04-13 22:18:14 +02:00
Bart van Bragt 439c74f0a3 fix: store index compat version in ES mapping _meta for correct rollback behaviour
- Write INDEX_COMPAT_VERSION into mapping _meta.index_compat_version on every
  build/promote so the version travels with the index rather than being a
  separate settings value
- saveIndexedConfig now reads compat version back from _meta alongside the
  existing analyzer/min_gram sync, so rollback automatically restores the
  correct value (or null for pre-feature indexes, triggering the reindex warning)
- Replace InvalidArgumentException in buildIndexSettings with $this->error()
  fallback to avoid stack traces in the console
- Fix stale comment in ViewsSearchJob referencing removed CLI flags
2026-04-10 13:46:08 +02:00
Bart van Bragt f486292f15 feat: make ngram min_gram configurable via admin setting
Adds a 'Minimum search query length' setting (1–4, default 3). The admin
shows a warning when the analyzer or min-search-length have changed since
the last index build.
2026-04-10 13:19:22 +02:00
Bart van Bragt 81e7b6502d feat: Show admin dashboard notice when search index needs rebuilding
After a mapping change (e.g. the parent-child restructure), existing installs
need to rebuild their index. A DashboardWidget appears on the admin page
warning the admin and showing the command to run. Compatibility is tracked
via the blomstra-search.index-compatible setting.
2026-04-10 11:10:53 +02:00
Bart van Bragt 3039b65946 refactor: Simplify index command with better naming/options
The flag-based interface (--rebuild etc) became rather convoluted with the changes. Replace this with commands with more straight forward naming.

Also added an explicit 'rollback' action and updated the README.md
2026-04-10 09:35:49 +02:00
Bart van Bragt e2e2d1428f feat: restructure index as parent-child documents with blue-green rebuilds
- Switch to ES parent/child join_field: discussion docs hold all metadata,
  post docs hold content only. Routing ensures parent and child land on the
  same shard. Removes the old flat-document approach where post data was
  duplicated onto every comment document.

- Add HasChildQuery with inner_hits so the best-matching post ID is
  surfaced as mostRelevantPost without a second DB query.

- Blue-green index rebuilds: --recreate writes into a timestamped pending
  index, --swap atomically promotes it via alias. Interrupted builds are
  resumable with --recreate --continue. --swap requires confirmation before
  proceeding.

- Index hidden posts; non-moderators are filtered at query time via
  is_hidden on the has_child clause. Core\Hidden and Core\Restored trigger
  re-indexing so moderators with post.hide can search hidden posts.
2026-04-09 23:30:57 +02:00
Bart van Bragt 80d66c98c4 feat: index and sync view_count for fof/discussion-views sort support
- Rename SavingJob to UpdateSearchJob
- Add ViewsSearchJob for lightweight partial view_count updates
- Add probabilistic sync throttle in DiscussionSeeder::viewingOn()
  to avoid an ES update on every page view
- Add view_count to mapping, DiscussionSeeder::toDocument(), and
  translateSort
- Log a warning and skip instead of 500ing on unknown sort fields
2026-04-09 12:05:28 +02:00
Bart van Bragt 5929a191f6 fix: --seed-missing stops early when a full range is already indexed
When all documents in a range were already in the index, the empty
collection caused continueAt to be set to null, aborting before
scanning lower ID ranges. Now advances past the range bottom instead.
2026-04-09 11:17:17 +02:00
Bart van Bragt 6699c8cb58 fix: ngram min_gram=1 indexes tokens that can never be matched
Flarum's search UI enforces a minimum query length of 3 characters
(MIN_SEARCH_LEN in Search.tsx), so 1- and 2-character ngram tokens
are indexed but can never be queried. Changed min_gram from 1 to 3
and updated max_ngram_diff accordingly.
2026-04-09 11:04:27 +02:00
Daniël Klabbers 508bb42ac9 chore: clean up buildcommand 2022-11-23 12:46:04 +01:00
StyleCI Bot db74da3e37
Apply fixes from StyleCI 2022-11-23 11:29:48 +00:00
Daniël Klabbers a02d35394b feat: store tags, rawId, support indexing missing objects 2022-11-23 12:09:10 +01:00
StyleCI Bot f8aa3dcf13
Apply fixes from StyleCI 2022-11-18 09:33:13 +00:00
Daniel Klabbers 803d1ecfc5 fix partial matching and weighting 2022-01-10 14:16:42 +01:00
Daniel Klabbers b6ac018a56 fix partial matching 2022-01-03 16:53:45 +01:00
Daniel Klabbers 68fce10d56 support patching the mapping without recreating 2021-11-16 20:33:17 +01:00
Daniël Klabbers 8b1a32af8d add comment count to mapping 2021-11-11 21:46:17 +01:00
Daniël Klabbers 3106667519 attempt to improve results loading with eloquent 2021-11-08 16:04:42 +01:00
Daniël Klabbers b87a4e2328 support continue flag 2021-11-06 21:32:36 +01:00
Daniel Klabbers bcdf625603 added --recreate and --only to command 2021-11-05 20:10:06 +01:00
Daniel Klabbers 614e630fde add throttling and sorting 2021-11-05 13:51:06 +01:00
Daniel Klabbers ff4e6be86a allow bigger chunk sizes 2021-11-04 12:28:19 +01:00
Daniel Klabbers 4c056a4db4 force queue 2021-11-04 10:47:24 +01:00
Daniel Klabbers 4e6f560376 reduce complexity of rebuild command 2021-11-02 14:47:24 +01:00
Daniel Klabbers c04a9f9861 remove string binding 2021-11-02 13:58:59 +01:00
Daniel Klabbers 47fc5c2f1b refactored jobs, cleaned up 2021-11-02 13:55:41 +01:00
Daniel Klabbers 254c58101b indexing now works using language processing 2021-11-02 12:07:27 +01:00
Daniël Klabbers 61e2f90835 wip 2021-10-31 02:53:16 +02:00
Daniel Klabbers 23c3b02201 add flag description 2021-10-20 14:39:47 +02:00
Daniel Klabbers 0574307439 refactored 2021-10-20 14:24:44 +02:00
Daniel Klabbers 4fa4ccce99 wip 2021-10-15 21:21:30 +02:00
Daniel Klabbers 4130e49bee wip 2021-10-15 13:51:36 +02:00
Daniel Klabbers 17bb6ff837 moving to posts 2021-10-14 21:31:50 +02:00
Daniel Klabbers f69541d38a moved to elastic 2021-10-14 12:31:45 +02:00
Daniel Klabbers e2e56831e5 improve permission logic again and allow flushing indices with command 2021-10-14 00:56:32 +02:00
Daniel Klabbers 217e4496be what i have 2021-10-13 20:06:27 +02:00