Commit Graph

62 Commits

Author SHA1 Message Date
JM fc830ea921 Add content autocomplete to search dropdown 2026-04-22 20:52:35 +02:00
JM 5774879331 fuzzy search in thread 2026-04-22 20:19:09 +02:00
JM 7cb4071a54 restore strong title match and keep prefix title fallback 2026-04-22 18:08:12 +02:00
JM e3dfc04192 add low-boost title autocomplete match to full search 2026-04-22 17:50:30 +02:00
Bart van Bragt 0f58abf141 feat: add title.autocomplete instant search path sorted by latest
When filter[autocomplete]=1 is present, bypass the full has_child query
and run a single MatchQuery on title.autocomplete (edge-ngram prefix match).
Results are sorted by updated_at desc to stay consistent with full search.
2026-04-16 11:51:45 +02:00
Bart van Bragt c5643bed31 fix: default search sort to latest and keep dropdown label in sync
- Search with no explicit sort now defaults to latest (updated_at desc)
  on both backend and frontend, matching the forum UX expectation.
- "All discussions" link from the search box routes to /?q=...&sort=latest
  so the sort dropdown shows "Latest" immediately after submission.
- extendDiscussionState injects sort=-lastPostedAt into API params when
  no sort is present, keeping direct-URL navigation (?q=anan) consistent
  with the backend default.
- Backend guard changed from phpSortField===null to empty($sorts), which
  is the correct sentinel since $sorts is the authoritative ES sort list.
2026-04-15 17:21:10 +02:00
Bart van Bragt 08353d2473 perf: sort by date by default, skip scoring on field sorts
- Default sort changed from relevance to updated_at desc. This lets
  has_child use score_mode:none, which skips child scoring entirely
  and lets ES short-circuit early on large corpora.
- When a field sort is requested, score_mode is also none for the same
  reason. score_mode:sum is only used when sorting by relevance.
- track_total_hits:false avoids a full-index count on every query,
  allowing ES to stop once it has collected enough results.
- Strip all gambit operators (tag:foo, author:bar, is:unread, etc.)
  from the ES query string, not just is:private. Leaving them in caused
  operator:and to require the gambit tokens to appear literally in post
  content, producing zero results when gambits were combined with text.
2026-04-14 22:40:43 +02:00
Bart van Bragt d8eebb183a fix: prevent ES CPU saturation from has_child scoring all posts
Without minimum_should_match=1 on the inner post bool query, Elasticsearch
defaults MSM to 0 whenever a filter clause is present. This caused has_child
to score every non-hidden post on every search request, saturating CPU on
shared ES nodes with large corpora (3.7M+ posts).

- Add BoolQuery subclass with create() override and minimumShouldMatch()
  (spatie/elasticsearch-query-builder 1.x uses `new self()` in create(),
  so subclassing requires overriding it)
- Set minimumShouldMatch(1) on the inner post query after adding the
  is_hidden filter, so only genuinely matching posts are scored
- Remove the operator('or') clause from buildShouldClauses() — with
  Turkish min_ngram=2, 'or' generates 2-gram tokens that match nearly
  every post, causing near-total index scans
- Add ES client timeouts (connect: 2s, query: 10s) to prevent Apache
  mod_php worker saturation when ES is slow or unreachable
2026-04-14 16:07:50 +02:00
Bart van Bragt 4709b9d1db fix: is_hidden boolean coercion and forum settings access 2026-04-14 08:59:19 +02:00
Bart van Bragt e2e2d1428f feat: restructure index as parent-child documents with blue-green rebuilds
- Switch to ES parent/child join_field: discussion docs hold all metadata,
  post docs hold content only. Routing ensures parent and child land on the
  same shard. Removes the old flat-document approach where post data was
  duplicated onto every comment document.

- Add HasChildQuery with inner_hits so the best-matching post ID is
  surfaced as mostRelevantPost without a second DB query.

- Blue-green index rebuilds: --recreate writes into a timestamped pending
  index, --swap atomically promotes it via alias. Interrupted builds are
  resumable with --recreate --continue. --swap requires confirmation before
  proceeding.

- Index hidden posts; non-moderators are filtered at query time via
  is_hidden on the has_child clause. Core\Hidden and Core\Restored trigger
  re-indexing so moderators with post.hide can search hidden posts.
2026-04-09 23:30:57 +02:00
Bart van Bragt 80d66c98c4 feat: index and sync view_count for fof/discussion-views sort support
- Rename SavingJob to UpdateSearchJob
- Add ViewsSearchJob for lightweight partial view_count updates
- Add probabilistic sync throttle in DiscussionSeeder::viewingOn()
  to avoid an ES update on every page view
- Add view_count to mapping, DiscussionSeeder::toDocument(), and
  translateSort
- Log a warning and skip instead of 500ing on unknown sort fields
2026-04-09 12:05:28 +02:00
Bart van Bragt eeb091273d fix: tag filter produces empty results when browsing a tag
When navigating to /t/{tag}?q=query the tag filter from the request
was silently ignored. Added a TermsQuery on the tags field using the
tag IDs resolved from the provided slugs.
2026-04-09 08:20:42 +02:00
StyleCI Bot 8b519619e5
Apply fixes from StyleCI 2022-11-24 09:38:21 +00:00
Daniël Klabbers 83afe95d61 chore: fix search and seeding 2022-11-24 10:37:57 +01:00
Daniël Klabbers 846cc348ff Merge branch 'dk/search-improvements' into main 2022-11-24 10:29:18 +01:00
StyleCI Bot db74da3e37
Apply fixes from StyleCI 2022-11-23 11:29:48 +00:00
Daniël Klabbers a02d35394b feat: store tags, rawId, support indexing missing objects 2022-11-23 12:09:10 +01:00
StyleCI Bot f8aa3dcf13
Apply fixes from StyleCI 2022-11-18 09:33:13 +00:00
Daniël Klabbers 9401b677a3 change weighing for exact words match 2022-11-10 12:57:46 +01:00
Daniël Klabbers 4bc89949ec complete testing 2022-11-10 12:54:26 +01:00
Daniël Klabbers 29ba864849 settings configurations 2022-11-09 18:06:48 +01:00
Daniël Klabbers e26d90d7b6 fix ambiguous 2022-03-04 12:37:43 +01:00
Daniël Klabbers cb3d4f5225 fix typo 2022-02-28 20:33:01 +01:00
Daniël Klabbers c6fd942bf1 prevent leaking deleted discussions 2022-02-28 20:32:27 +01:00
Daniël Klabbers cc8145c236 drop partial search 2022-02-21 10:20:46 +01:00
Daniel Klabbers ccfb964aff improve search results 2022-02-14 16:51:55 +01:00
Daniël Klabbers 133ded0684 fixes #5 2022-01-11 10:10:39 +01:00
Daniel Klabbers 803d1ecfc5 fix partial matching and weighting 2022-01-10 14:16:42 +01:00
Daniel Klabbers 502f7a9f86 use partial match with weighting 2022-01-10 13:47:05 +01:00
Daniel Klabbers 9108553f09 add and vs or distinction, but partial isnt working in combination with the others yet 2022-01-06 12:58:50 +01:00
Daniel Klabbers b6ac018a56 fix partial matching 2022-01-03 16:53:45 +01:00
Daniel Klabbers 88b3e57d8f added weighting properly, also different for titles vs posts 2022-01-03 15:13:13 +01:00
Daniel Klabbers f180d66815 Revert "feat: add partial word match with low weighting"
This reverts commit 0d51e5d912.
2021-12-23 13:07:04 +01:00
Daniel Klabbers 013f5162c5 Revert "introduce wildcard, needs further testing"
This reverts commit 877533b43d.
2021-12-23 13:07:03 +01:00
Daniel Klabbers 23074a3a0c Revert "attempt to make partial match filter"
This reverts commit f21dc04a00.
2021-12-23 13:07:00 +01:00
Daniel Klabbers 0ee902b7b7 Revert "allow minimum should match config"
This reverts commit d5eebf6c65.
2021-12-23 13:06:56 +01:00
Daniel Klabbers d5eebf6c65 allow minimum should match config 2021-12-16 15:05:22 +01:00
Daniel Klabbers f21dc04a00 attempt to make partial match filter 2021-12-16 15:02:18 +01:00
Daniel Klabbers 877533b43d introduce wildcard, needs further testing 2021-12-14 16:35:16 +01:00
Daniel Klabbers 0d51e5d912 feat: add partial word match with low weighting 2021-12-14 12:05:32 +01:00
Daniel Klabbers cb7da5d3f8 fix search for the right fields 2021-11-16 20:23:07 +01:00
Daniel Klabbers 868713da11 fix: search for byobu page
Fixes the index for byobu.
2021-11-16 20:22:22 +01:00
Daniël Klabbers cadfc7b9bd seed and use comment count for sorting 2021-11-11 21:40:34 +01:00
Daniel Klabbers 77c406c07f improve scoring by boosting titles, now also major 2021-11-10 16:52:53 +01:00
Daniel Klabbers 6369637145 fix issue with selection of columns 2021-11-08 20:25:02 +01:00
Daniel Klabbers fd11442a76 optimize search query with join 2021-11-08 16:46:15 +01:00
Daniel Klabbers 8977d3d161 fix for issue with search 2021-11-08 16:41:37 +01:00
Daniel Klabbers 33ae86a539 fix issues with discussion search 2021-11-04 10:01:54 +01:00
Daniel Klabbers de7f1096a9 fix issue with search controller due to binding change 2021-11-02 14:57:11 +01:00
Daniel Klabbers c04a9f9861 remove string binding 2021-11-02 13:58:59 +01:00