Compare commits
33 Commits
| Author | SHA1 | Date |
|---|---|---|
flarum-bot
|
bf19eed20f | |
Bart van Bragt
|
2fc66b9bf7 | |
|
Bart van Bragt
|
439c74f0a3 | |
|
Bart van Bragt
|
f486292f15 | |
|
Bart van Bragt
|
4412583cd9 | |
|
Bart van Bragt
|
81e7b6502d | |
|
Bart van Bragt
|
3039b65946 | |
|
Bart van Bragt
|
e2e2d1428f | |
|
Bart van Bragt
|
80d66c98c4 | |
|
Bart van Bragt
|
ff2a252e99 | |
|
Bart van Bragt
|
5929a191f6 | |
|
Bart van Bragt
|
6699c8cb58 | |
|
Bart van Bragt
|
369ee504f3 | |
|
Bart van Bragt
|
eeb091273d | |
Daniël Klabbers
|
d252a17586 | |
|
Daniël Klabbers
|
a9dc8bfb16 | |
StyleCI Bot
|
82e6ec786f | |
|
flarum-bot
|
f981c27e00 | |
|
Daniël Klabbers
|
f6503f5e6a | |
|
flarum-bot
|
ad92d8856b | |
|
StyleCI Bot
|
8b519619e5 | |
|
Daniël Klabbers
|
83afe95d61 | |
|
Daniël Klabbers
|
846cc348ff | |
|
Daniël Klabbers
|
508bb42ac9 | |
|
StyleCI Bot
|
db74da3e37 | |
|
Daniël Klabbers
|
a02d35394b | |
|
flarum-bot
|
b60563f2ec | |
David Wheatley
|
703cbbdd48 | |
|
flarum-bot
|
769b6cd278 | |
Ian Morland
|
f500d7f956 | |
|
Daniël Klabbers
|
9401b677a3 | |
|
Daniël Klabbers
|
4bc89949ec | |
|
Daniël Klabbers
|
29ba864849 |
143
README.md
143
README.md
|
|
@ -5,47 +5,154 @@ fulltext search with one that is completely relying on the proven elasticsearch
|
|||
|
||||
## Features
|
||||
|
||||
- Sync discussions to elastic search using your queue, unobtrusively for the user.
|
||||
- Reduces search loading times to well below 400ms (local tests with 50.000 discussion **260ms**)
|
||||
- Sync discussions and posts to Elasticsearch using your queue, unobtrusively for the user.
|
||||
- Reduces search loading times to well below 400ms (local tests with 50,000 discussions: **260ms**)
|
||||
- Uses Flarum's group permissions and tags system.
|
||||
- Compatible with Friends of Flarum Byōbu.
|
||||
|
||||
## Installation
|
||||
## Requirements
|
||||
|
||||
Use composer:
|
||||
- Elasticsearch 7.x or OpenSearch 1.x+
|
||||
- A non-sync queue driver with a running worker (`php flarum queue:work`) is strongly recommended for production. The extension works with the default sync driver, but index jobs run inline which adds latency to user-facing changes like posting.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
composer require blomstra/search:*
|
||||
```
|
||||
|
||||
Enable the extension inside the admin area and configure the settings.
|
||||
Enable the extension in the admin area and configure the Elasticsearch endpoint and index name in the extension settings.
|
||||
|
||||
### Set up
|
||||
## Setting up the index
|
||||
|
||||
Enable the extension in your admin area. Now to seed your existing discussions use the following command:
|
||||
### First install
|
||||
|
||||
```
|
||||
php flarum blomstra:search:index
|
||||
Run the build command once. It creates a timestamped concrete index, immediately aliases your configured index name to it, and begins queuing documents. Search is available and improves as the queue processes:
|
||||
|
||||
```bash
|
||||
php flarum blomstra:search:index build
|
||||
php flarum queue:work
|
||||
```
|
||||
|
||||
All mutations to discussions are automatically added and removed from the elasticsearch index.
|
||||
### Subsequent rebuilds (zero-downtime)
|
||||
|
||||
### FAQ
|
||||
When you need to rebuild the full index (e.g. after a mapping change):
|
||||
|
||||
*I have another question.*
|
||||
Reach out to us via https://helpdesk.blomstra.net. We will get back to you as soon as we can. If you have a running subscription please mention when you started your plan and/or which plan you are on. Always add sufficient information when reporting errors. We prefer errors being reported here, but understand that sometimes you can't.
|
||||
```bash
|
||||
# Simple rebuild — promotes automatically once all jobs are queued
|
||||
php flarum blomstra:search:index build
|
||||
|
||||
# Or keep a backup of the old index in case you need to roll back
|
||||
php flarum blomstra:search:index build --keep-backup
|
||||
```
|
||||
|
||||
After the queue drains, fill any gaps from content posted during the build:
|
||||
|
||||
```bash
|
||||
php flarum blomstra:search:index fill
|
||||
```
|
||||
|
||||
If you kept a backup and want to roll back:
|
||||
|
||||
```bash
|
||||
php flarum blomstra:search:index rollback
|
||||
```
|
||||
|
||||
Once satisfied with the new index, drop the backup:
|
||||
|
||||
```bash
|
||||
php flarum blomstra:search:index discard --backup
|
||||
```
|
||||
|
||||
### Blue-green rebuild (manual promotion)
|
||||
|
||||
Use `--staging` to keep the old index live until you explicitly promote:
|
||||
|
||||
```bash
|
||||
# 1. Build into a staging index — live index is untouched
|
||||
php flarum blomstra:search:index build --staging
|
||||
|
||||
# 2. Drain the queue
|
||||
php flarum queue:work --stop-when-empty
|
||||
|
||||
# 3. Promote the staging index to live
|
||||
php flarum blomstra:search:index promote
|
||||
# Or keep the old index as a backup:
|
||||
php flarum blomstra:search:index promote --keep-backup
|
||||
```
|
||||
|
||||
### Resuming or cancelling an interrupted build
|
||||
|
||||
If a build is interrupted, re-run it with the appropriate flag:
|
||||
|
||||
```bash
|
||||
# Resume each seeder from where it left off
|
||||
php flarum blomstra:search:index build --resume
|
||||
|
||||
# Drop the staging index and start completely fresh
|
||||
php flarum blomstra:search:index build --fresh
|
||||
|
||||
# Cancel the build without starting a new one
|
||||
php flarum blomstra:search:index discard --pending
|
||||
```
|
||||
|
||||
### Filling gaps in an existing index
|
||||
|
||||
If documents are missing from the live index (e.g. due to queue failures):
|
||||
|
||||
```bash
|
||||
php flarum blomstra:search:index fill
|
||||
```
|
||||
|
||||
### Updating the mapping only
|
||||
|
||||
To push a mapping change to the live index without rebuilding:
|
||||
|
||||
```bash
|
||||
php flarum blomstra:search:index mapping
|
||||
```
|
||||
|
||||
## Command reference
|
||||
|
||||
| Command | Description |
|
||||
|---|---|
|
||||
| `build` | Rebuild the index and promote automatically once all jobs are queued. On first install, aliases immediately so search is live during seeding. |
|
||||
| `build --keep-backup` | Rebuild and promote, retaining the old index as a backup for rollback. |
|
||||
| `build --staging` | Build into a staging index without promoting — use `promote` when ready (blue-green workflow). |
|
||||
| `build --resume` | Resume an interrupted build from where each seeder left off. |
|
||||
| `build --fresh` | Drop the staging index and start completely fresh. |
|
||||
| `promote` | Atomically swap the alias to the staging index. Prompts for confirmation (blue-green workflow). |
|
||||
| `promote --keep-backup` | Promote and retain the replaced live index as a backup for rollback. |
|
||||
| `rollback` | Restore the backup index to live. Deletes the index that was live. |
|
||||
| `discard --pending` | Drop the staging index without promoting (cancels an in-progress build). |
|
||||
| `discard --backup` | Drop the backup index (cleanup after `--keep-backup`). |
|
||||
| `mapping` | Push updated mapping to the live index without rebuilding or reseeding. |
|
||||
| `fill` | Seed only documents missing from the live index. |
|
||||
| `build --only=discussions` | Seed only the specified document type (`discussions` or `posts`). |
|
||||
| `build --throttle=N` | Wait N seconds between batches (reduces queue pressure). |
|
||||
| `build --max-id=N` | Limit seeding to documents with ID ≤ N. |
|
||||
| `promote --i-am-sure` | Skip the promotion confirmation prompt (for scripts and CI). |
|
||||
|
||||
## FAQ
|
||||
|
||||
## Queue configuration
|
||||
|
||||
*"Can I dispatch indexing jobs to a specific queue?"*
|
||||
|
||||
*Can I dispatch the sync jobs to another queue?*
|
||||
Yes:
|
||||
|
||||
```php
|
||||
\Blomstra\Search\Observe\Job::$onQueue = 'sync';
|
||||
\Blomstra\Search\Jobs\Job::$onQueue = 'search';
|
||||
```
|
||||
|
||||
*"I have a different question"*
|
||||
|
||||
Reach out ot us via https://support.on-floxum.com/t/ext-search . If you have an active subscription, please mention what plan you are on.
|
||||
|
||||
---
|
||||
|
||||
- Blomstra provides managed Flarum hosting.
|
||||
- https://blomstra.net
|
||||
- https://blomstra.community/t/ext-search
|
||||
- Floxum provides managed Flarum hosting.
|
||||
- https://floxum.com
|
||||
- https://support.on-floxum.com/t/ext-search
|
||||
|
||||
Icon made by [Freepik](https://www.freepik.com) from [Flaticon](https://www.flaticon.com/).
|
||||
|
|
|
|||
11
extend.php
11
extend.php
|
|
@ -20,14 +20,19 @@ return [
|
|||
(new Flarum\Frontend('forum'))
|
||||
->js(__DIR__.'/js/dist/forum.js'),
|
||||
(new Flarum\Frontend('admin'))
|
||||
->js(__DIR__.'/js/dist/admin.js'),
|
||||
->js(__DIR__.'/js/dist/admin.js')
|
||||
->css(__DIR__.'/resources/less/admin.less'),
|
||||
|
||||
(new Flarum\Locales(__DIR__.'/resources/locale')),
|
||||
new Flarum\Locales(__DIR__.'/resources/locale'),
|
||||
|
||||
(new Flarum\Routes('api'))
|
||||
->get('/blomstra/search/{type}', 'blomstra.search', Api\Controllers\SearchController::class)
|
||||
->put('/blomstra/search/index', 'blomstra.search.index', Api\Controllers\IndexController::class),
|
||||
|
||||
(new Flarum\Console())
|
||||
->command(Commands\BuildCommand::class),
|
||||
|
||||
(new Flarum\Settings())
|
||||
->default('blomstra-search.search-discussion-subjects', true)
|
||||
->default('blomstra-search.search-post-bodies', true)
|
||||
->default('blomstra-search.min-search-length', Commands\BuildCommand::DEFAULT_MIN_SEARCH_LENGTH),
|
||||
];
|
||||
|
|
|
|||
|
|
@ -1,2 +1,2 @@
|
|||
(()=>{var a={n:e=>{var t=e&&e.__esModule?()=>e.default:()=>e;return a.d(t,{a:t}),t},d:(e,t)=>{for(var r in t)a.o(t,r)&&!a.o(e,r)&&Object.defineProperty(e,r,{enumerable:!0,get:t[r]})},o:(a,e)=>Object.prototype.hasOwnProperty.call(a,e),r:a=>{"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(a,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(a,"__esModule",{value:!0})}},e={};(()=>{"use strict";a.r(e);const t=flarum.core.compat["admin/app"];var r=a.n(t);r().initializers.add("blomstra-search",(function(){var a=new Map;["arabic","armenian","basque","bengali","brazilian","bulgarian","catalan","cjk","czech","danish","dutch","english","estonian","finnish","french","galician","german","greek","hindi","hungarian","indonesian","irish","italian","latvian","lithuanian","norwegian","persian","portuguese","romanian","russian","sorani","spanish","swedish","turkish","thai"].forEach((function(e){a.set(e,e)})),r().extensionData.for("blomstra-search").registerSetting({setting:"blomstra-search.elastic-endpoint",label:r().translator.trans("blomstra-search.admin.elastic-endpoint"),type:"input"}).registerSetting({setting:"blomstra-search.elastic-username",label:r().translator.trans("blomstra-search.admin.elastic-username"),type:"input"}).registerSetting({setting:"blomstra-search.elastic-password",label:r().translator.trans("blomstra-search.admin.elastic-password"),type:"password"}).registerSetting({setting:"blomstra-search.elastic-index",label:r().translator.trans("blomstra-search.admin.elastic-index"),default:"flarum",type:"input"}).registerSetting({setting:"blomstra-search.analyzer-language",label:r().translator.trans("blomstra-search.admin.analyzer.label"),help:r().translator.trans("blomstra-search.admin.analyzer.help"),type:"select",options:Object.fromEntries(a.entries()),default:"english"}).registerSetting({setting:"blomstra-search.elastic-index",label:r().translator.trans("blomstra-search.admin.elastic-index"),default:"flarum",type:"input"})}))})(),module.exports=e})();
|
||||
(()=>{var t={n:a=>{var e=a&&a.__esModule?()=>a.default:()=>a;return t.d(e,{a:e}),e},d:(a,e)=>{for(var r in e)t.o(e,r)&&!t.o(a,r)&&Object.defineProperty(a,r,{enumerable:!0,get:e[r]})},o:(t,a)=>Object.prototype.hasOwnProperty.call(t,a)};(()=>{"use strict";function a(t,e){return a=Object.setPrototypeOf?Object.setPrototypeOf.bind():function(t,a){return t.__proto__=a,t},a(t,e)}const e=flarum.core.compat["admin/app"];var r=t.n(e);const n=flarum.core.compat["common/extend"],s=flarum.core.compat["admin/components/DashboardPage"];var i=t.n(s);const o=flarum.core.compat["admin/components/DashboardWidget"];var l=t.n(o);const c=flarum.core.compat["common/components/Alert"];var h=t.n(c),d=function(t){var e,n;function s(){return t.apply(this,arguments)||this}n=t,(e=s).prototype=Object.create(n.prototype),e.prototype.constructor=e,a(e,n);var i=s.prototype;return i.className=function(){return"ReindexWarningWidget"},i.content=function(){return m(h(),{type:"warning",dismissible:!1,icon:"fas fa-exclamation-triangle",title:r().translator.trans("blomstra-search.admin.reindex-required.title")},r().translator.trans("blomstra-search.admin.reindex-required.detail"))},s}(l());r().initializers.add("blomstra-search",function(){var t=r().data.settings["blomstra-search.active-index"],a=r().data.settings["blomstra-search.index-compatible"];t&&"v2"!==a&&(0,n.extend)(i().prototype,"availableWidgets",function(t){t.add("blomstra-search-reindex",m(d),110)});var e=new Map;["arabic","armenian","basque","bengali","brazilian","bulgarian","catalan","cjk","czech","danish","dutch","english","estonian","finnish","french","galician","german","greek","hindi","hungarian","indonesian","irish","italian","latvian","lithuanian","norwegian","persian","portuguese","romanian","russian","sorani","spanish","swedish","turkish","thai"].forEach(function(t){e.set(t,t)}),r().extensionData.for("blomstra-search").registerSetting(function(){var a=r().data.settings["blomstra-search.indexed-analyzer"];if(!t||!a)return null;var e=this.setting("blomstra-search.analyzer-language")()||"english",n=this.setting("blomstra-search.min-search-length")(),s=String(r().data.settings["blomstra-search.indexed-min-search-length"]||r().data.settings["blomstra-search.min-search-length"]);return e===a&&n===s?null:m(h(),{type:"warning",dismissible:!1,icon:"fas fa-exclamation-triangle"},r().translator.trans("blomstra-search.admin.index-settings-changed"))}).registerSetting({setting:"blomstra-search.elastic-endpoint",label:r().translator.trans("blomstra-search.admin.elastic-endpoint"),type:"input"}).registerSetting({setting:"blomstra-search.elastic-username",label:r().translator.trans("blomstra-search.admin.elastic-username"),type:"input"}).registerSetting({setting:"blomstra-search.elastic-password",label:r().translator.trans("blomstra-search.admin.elastic-password"),type:"password"}).registerSetting({setting:"blomstra-search.elastic-index",label:r().translator.trans("blomstra-search.admin.elastic-index"),default:"flarum",type:"input"}).registerSetting({setting:"blomstra-search.analyzer-language",label:r().translator.trans("blomstra-search.admin.analyzer.label"),help:r().translator.trans("blomstra-search.admin.analyzer.help"),type:"select",options:Object.fromEntries(e.entries()),default:"english"}).registerSetting({setting:"blomstra-search.search-discussion-subjects",label:r().translator.trans("blomstra-search.admin.search-discussion-subjects"),type:"switch"}).registerSetting({setting:"blomstra-search.search-post-bodies",label:r().translator.trans("blomstra-search.admin.search-post-bodies"),type:"switch"}).registerSetting({setting:"blomstra-search.min-search-length",label:r().translator.trans("blomstra-search.admin.min-search-length.label"),help:r().translator.trans("blomstra-search.admin.min-search-length.help"),type:"select",options:{1:"1",2:"2",3:"3",4:"4"},default:r().data.settings["blomstra-search.min-search-length"]})})})(),module.exports={}})();
|
||||
//# sourceMappingURL=admin.js.map
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
|
|
@ -1,16 +1,14 @@
|
|||
{
|
||||
"name": "@blomstra/realtime",
|
||||
"name": "@blomstra/search",
|
||||
"private": true,
|
||||
"version": "0.0.0",
|
||||
"prettier": "@flarum/prettier-config",
|
||||
"dependencies": {
|
||||
"@flarum/prettier-config": "^1.0.0",
|
||||
"build": "^0.1.4",
|
||||
"flarum-tsconfig": "^1.0.2",
|
||||
"flarum-webpack-config": "^2.0.0",
|
||||
"pusher-js": "^7.0.3",
|
||||
"webpack": "^5.75.0",
|
||||
"webpack-cli": "^4.10.0"
|
||||
"webpack-cli": "^5.0.0"
|
||||
},
|
||||
"scripts": {
|
||||
"dev": "webpack --mode development --watch",
|
||||
|
|
|
|||
|
|
@ -1,6 +1,40 @@
|
|||
import app from 'flarum/admin/app';
|
||||
import { extend } from 'flarum/common/extend';
|
||||
import DashboardPage from 'flarum/admin/components/DashboardPage';
|
||||
import DashboardWidget from 'flarum/admin/components/DashboardWidget';
|
||||
import Alert from 'flarum/common/components/Alert';
|
||||
|
||||
const REQUIRED_INDEX_COMPAT = 'v2';
|
||||
|
||||
class ReindexWarningWidget extends DashboardWidget {
|
||||
className() {
|
||||
return 'ReindexWarningWidget';
|
||||
}
|
||||
|
||||
content() {
|
||||
return m(
|
||||
Alert,
|
||||
{
|
||||
type: 'warning',
|
||||
dismissible: false,
|
||||
icon: 'fas fa-exclamation-triangle',
|
||||
title: app.translator.trans('blomstra-search.admin.reindex-required.title'),
|
||||
},
|
||||
app.translator.trans('blomstra-search.admin.reindex-required.detail')
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
app.initializers.add('blomstra-search', () => {
|
||||
const activeIndex = app.data.settings['blomstra-search.active-index'];
|
||||
const compatVersion = app.data.settings['blomstra-search.index-compatible'];
|
||||
|
||||
if (activeIndex && compatVersion !== REQUIRED_INDEX_COMPAT) {
|
||||
extend(DashboardPage.prototype, 'availableWidgets', function (items) {
|
||||
items.add('blomstra-search-reindex', m(ReindexWarningWidget), 110);
|
||||
});
|
||||
}
|
||||
|
||||
const languages = new Map();
|
||||
[
|
||||
'arabic',
|
||||
|
|
@ -44,6 +78,25 @@ app.initializers.add('blomstra-search', () => {
|
|||
|
||||
app.extensionData
|
||||
.for('blomstra-search')
|
||||
.registerSetting(function (this: any) {
|
||||
// No index yet, or first build predates this tracking — stay silent.
|
||||
const indexedAnalyzer = app.data.settings['blomstra-search.indexed-analyzer'];
|
||||
if (!activeIndex || !indexedAnalyzer) return null;
|
||||
|
||||
const currentAnalyzer = this.setting('blomstra-search.analyzer-language')() || 'english';
|
||||
const currentMinLength = this.setting('blomstra-search.min-search-length')();
|
||||
const indexedMinLength = String(
|
||||
app.data.settings['blomstra-search.indexed-min-search-length'] || app.data.settings['blomstra-search.min-search-length']
|
||||
);
|
||||
|
||||
if (currentAnalyzer === indexedAnalyzer && currentMinLength === indexedMinLength) return null;
|
||||
|
||||
return m(
|
||||
Alert,
|
||||
{ type: 'warning', dismissible: false, icon: 'fas fa-exclamation-triangle' },
|
||||
app.translator.trans('blomstra-search.admin.index-settings-changed')
|
||||
);
|
||||
})
|
||||
.registerSetting({
|
||||
setting: 'blomstra-search.elastic-endpoint',
|
||||
label: app.translator.trans('blomstra-search.admin.elastic-endpoint'),
|
||||
|
|
@ -74,9 +127,21 @@ app.initializers.add('blomstra-search', () => {
|
|||
default: 'english',
|
||||
})
|
||||
.registerSetting({
|
||||
setting: 'blomstra-search.elastic-index',
|
||||
label: app.translator.trans('blomstra-search.admin.elastic-index'),
|
||||
default: 'flarum',
|
||||
type: 'input',
|
||||
setting: 'blomstra-search.search-discussion-subjects',
|
||||
label: app.translator.trans('blomstra-search.admin.search-discussion-subjects'),
|
||||
type: 'switch',
|
||||
})
|
||||
.registerSetting({
|
||||
setting: 'blomstra-search.search-post-bodies',
|
||||
label: app.translator.trans('blomstra-search.admin.search-post-bodies'),
|
||||
type: 'switch',
|
||||
})
|
||||
.registerSetting({
|
||||
setting: 'blomstra-search.min-search-length',
|
||||
label: app.translator.trans('blomstra-search.admin.min-search-length.label'),
|
||||
help: app.translator.trans('blomstra-search.admin.min-search-length.help'),
|
||||
type: 'select',
|
||||
options: { '1': '1', '2': '2', '3': '3', '4': '4' },
|
||||
default: app.data.settings['blomstra-search.min-search-length'],
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -8,8 +8,8 @@ export default function extendDiscussionState() {
|
|||
override(DiscussionListState.prototype, 'loadPage', async function (this: DiscussionListState, original, page: number = 1) {
|
||||
const preloaded = app.data.apiDocument || null;
|
||||
|
||||
// If existing payload is given fallback on native page.
|
||||
if (preloaded) return original.call(this, page);
|
||||
// If existing payload is given or no search is made, fallback on native page.
|
||||
if (preloaded || !this.requestParams()?.filter?.q) return original.call(this, page);
|
||||
|
||||
const params = this.requestParams();
|
||||
params.page = {
|
||||
|
|
|
|||
|
|
@ -9,8 +9,17 @@ import DiscussionsSearchSource from './SearchSources/DiscussionsSearchSource';
|
|||
import extendDiscussionState from './PaginatedListStates/extendDiscussionState';
|
||||
|
||||
app.initializers.add('blomstra-search', () => {
|
||||
const minLength = parseInt(app.data.settings['blomstra-search.min-search-length'] || String(Search.MIN_SEARCH_LEN), 10);
|
||||
if (minLength !== Search.MIN_SEARCH_LEN) {
|
||||
// Flarum provides no extension point for MIN_SEARCH_LEN, so we overwrite the
|
||||
// static property directly. TypeScript `readonly` is compile-time only — at
|
||||
// runtime this is a plain property assignment and is safe as long as no code
|
||||
// reads MIN_SEARCH_LEN before this initializer runs.
|
||||
(Search as any).MIN_SEARCH_LEN = minLength;
|
||||
}
|
||||
|
||||
extend(Search.prototype, 'sourceItems', function (this: Search<SearchAttrs>, items: ItemList<SearchSource>) {
|
||||
items.setContent('discussions', new DiscussionsSearchSource());
|
||||
items.replace('discussions', new DiscussionsSearchSource());
|
||||
});
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,3 @@
|
|||
.ReindexWarningWidget {
|
||||
padding: 0;
|
||||
}
|
||||
|
|
@ -8,4 +8,18 @@ blomstra-search:
|
|||
label: Analyzer language
|
||||
help: |
|
||||
The analyzer makes search understand stop words and undertakes language
|
||||
specific improvements for indexing.
|
||||
specific improvements for indexing. Changing this requires a full index rebuild to take effect.
|
||||
index-settings-changed: "Search index settings have changed. Run: php flarum blomstra:search:index build"
|
||||
reindex-required:
|
||||
title: Search index rebuild required
|
||||
detail: "The search index is not compatible with this version of the extension. Run: php flarum blomstra:search:index build"
|
||||
search-discussion-subjects: Search inside discussion titles
|
||||
search-post-bodies: Search inside comments
|
||||
match-sentences: Match search term against full sentence
|
||||
match-words: Match search term against full words
|
||||
min-search-length:
|
||||
label: Minimum search query length
|
||||
help: |
|
||||
Minimum number of characters required before a search is triggered. Lower this to 1 or 2
|
||||
for CJK (Chinese, Japanese, Korean) communities where single characters carry full meaning.
|
||||
Changing this requires a full index rebuild to take effect.
|
||||
|
|
|
|||
|
|
@ -12,10 +12,13 @@
|
|||
|
||||
namespace Blomstra\Search\Api\Controllers;
|
||||
|
||||
use Blomstra\Search\Elasticsearch\HasChildQuery;
|
||||
use Blomstra\Search\Elasticsearch\MatchPhraseQuery;
|
||||
use Blomstra\Search\Elasticsearch\MatchQuery;
|
||||
use Blomstra\Search\Elasticsearch\TermsQuery;
|
||||
use Blomstra\Search\Save\Document as ElasticDocument;
|
||||
use Blomstra\Search\Searchers\CommentPostSearcher;
|
||||
use Blomstra\Search\Searchers\DiscussionSearcher;
|
||||
use Blomstra\Search\Searchers\Searcher;
|
||||
use Elasticsearch\Client;
|
||||
use Flarum\Api\Controller\ListDiscussionsController;
|
||||
use Flarum\Api\Serializer\DiscussionSerializer;
|
||||
|
|
@ -24,15 +27,17 @@ use Flarum\Extension\ExtensionManager;
|
|||
use Flarum\Group\Group;
|
||||
use Flarum\Http\RequestUtil;
|
||||
use Flarum\Http\UrlGenerator;
|
||||
use Flarum\Settings\SettingsRepositoryInterface;
|
||||
use Flarum\Tags\Tag;
|
||||
use Flarum\User\User;
|
||||
use Illuminate\Contracts\Container\Container;
|
||||
use Illuminate\Support\Arr;
|
||||
use Illuminate\Support\Collection;
|
||||
use Illuminate\Support\Str;
|
||||
use Psr\Http\Message\ServerRequestInterface;
|
||||
use Psr\Log\LoggerInterface;
|
||||
use Spatie\ElasticsearchQueryBuilder\Builder;
|
||||
use Spatie\ElasticsearchQueryBuilder\Queries\BoolQuery;
|
||||
use Spatie\ElasticsearchQueryBuilder\Queries\Query;
|
||||
use Spatie\ElasticsearchQueryBuilder\Queries\TermQuery;
|
||||
use Spatie\ElasticsearchQueryBuilder\Sorts\Sort;
|
||||
use Tobscure\JsonApi\Document;
|
||||
|
|
@ -45,92 +50,103 @@ class SearchController extends ListDiscussionsController
|
|||
'lastPostedAt' => 'updated_at',
|
||||
'createdAt' => 'created_at',
|
||||
'commentCount' => 'comment_count',
|
||||
'view_count' => 'view_count',
|
||||
];
|
||||
|
||||
public function __construct(protected Client $elastic, protected UrlGenerator $uri)
|
||||
protected bool $matchSentences;
|
||||
protected bool $matchWords;
|
||||
protected ?Searcher $discussionSearcher;
|
||||
protected ?Searcher $postSearcher;
|
||||
|
||||
public function __construct(protected Client $elastic, protected UrlGenerator $uri, Container $container, SettingsRepositoryInterface $settings)
|
||||
{
|
||||
$this->matchSentences = true;
|
||||
$this->matchWords = true;
|
||||
|
||||
$searchers = collect($container->tagged('blomstra.search.searchers'));
|
||||
|
||||
$this->discussionSearcher = $searchers->first(fn ($s) => $s instanceof DiscussionSearcher);
|
||||
$this->postSearcher = $searchers->first(fn ($s) => $s instanceof CommentPostSearcher);
|
||||
}
|
||||
|
||||
protected function data(ServerRequestInterface $request, Document $document)
|
||||
{
|
||||
// Not used for now.
|
||||
$type = Arr::get($request->getQueryParams(), 'type');
|
||||
|
||||
$actor = RequestUtil::getActor($request);
|
||||
|
||||
$actor = RequestUtil::getActor($request);
|
||||
$filters = $this->extractFilter($request);
|
||||
|
||||
$search = $this->getSearch($filters);
|
||||
|
||||
$limit = $this->extractLimit($request);
|
||||
$offset = $this->extractOffset($request);
|
||||
|
||||
$search = $this->getSearch($filters);
|
||||
$limit = $this->extractLimit($request);
|
||||
$offset = $this->extractOffset($request);
|
||||
$include = array_merge($this->extractInclude($request), ['state']);
|
||||
|
||||
$filterQuery = BoolQuery::create();
|
||||
$query = BoolQuery::create()
|
||||
// Always restrict to discussion documents; posts are only searched via has_child.
|
||||
->add(TermQuery::create('join_field', 'discussion'), 'filter');
|
||||
|
||||
if (!empty($search)) {
|
||||
$filterQuery
|
||||
// @todo commented out to use only partial matching for now
|
||||
->add($this->sentenceMatch($search))
|
||||
->add($this->wordMatch($search, 'and'))
|
||||
->add($this->wordMatch($search, 'or'))
|
||||
// ->add($this->partialMatch($search))
|
||||
;
|
||||
$query->add($this->buildTextQuery($search, $actor));
|
||||
}
|
||||
|
||||
$this->addFilters($query, $actor, $filters);
|
||||
|
||||
$builder = (new Builder($this->elastic))
|
||||
->index(resolve('blomstra.search.elastic_index'))
|
||||
->size($limit + 1)
|
||||
->from($this->extractOffset($request))
|
||||
->addQuery(
|
||||
$this->addFilters($filterQuery, $actor, $filters)
|
||||
);
|
||||
->from($offset)
|
||||
->addQuery($query);
|
||||
|
||||
$knownSortFields = array_merge(array_values($this->translateSort), ['rawId']);
|
||||
$logger = resolve(LoggerInterface::class);
|
||||
|
||||
$phpSortField = null;
|
||||
$phpSortDir = 'desc';
|
||||
|
||||
foreach ($this->extractSort($request) as $field => $direction) {
|
||||
$field = $this->translateSort[$field] ?? $field;
|
||||
$builder->addSort(new Sort($field, $direction));
|
||||
$translated = $this->translateSort[$field] ?? $field;
|
||||
|
||||
if (!in_array($translated, $knownSortFields)) {
|
||||
$logger->warning("blomstra/search: unknown sort field \"{$field}\", ignoring.");
|
||||
continue;
|
||||
}
|
||||
|
||||
$builder->addSort(new Sort($translated, $direction));
|
||||
|
||||
if ($phpSortField === null && $translated !== 'rawId') {
|
||||
$phpSortField = $translated;
|
||||
$phpSortDir = $direction;
|
||||
}
|
||||
}
|
||||
|
||||
$response = $builder->search();
|
||||
|
||||
Discussion::setStateUser($actor);
|
||||
|
||||
// Eager load groups for use in the policies (isAdmin check)
|
||||
if (in_array('mostRelevantPost.user', $include)) {
|
||||
$include[] = 'mostRelevantPost.user.groups';
|
||||
|
||||
// If the first level of the relationship wasn't explicitly included,
|
||||
// add it so the code below can look for it
|
||||
if (!in_array('mostRelevantPost', $include)) {
|
||||
$include[] = 'mostRelevantPost';
|
||||
}
|
||||
}
|
||||
|
||||
// we need to retrieve all discussion ids and when the results are posts,
|
||||
// their ids as most relevant post id
|
||||
// All hits are discussion documents. Extract the best-matching post ID from
|
||||
// inner_hits when a has_child clause matched (i.e. the match came from a post).
|
||||
$results = Collection::make(Arr::get($response, 'hits.hits'))
|
||||
->map(function ($hit) {
|
||||
$type = $hit['_source']['type'];
|
||||
$id = Str::after($hit['_source']['id'], "$type:");
|
||||
// _id is "discussions:123" — parse the numeric part directly.
|
||||
$discussionId = Str::after($hit['_id'], 'discussions:');
|
||||
|
||||
if ($type === 'posts') {
|
||||
return [
|
||||
'most_relevant_post_id' => $id,
|
||||
'weight' => Arr::get($hit, 'sort.0'),
|
||||
];
|
||||
} else {
|
||||
return [
|
||||
'discussion_id' => $id,
|
||||
'weight' => Arr::get($hit, 'sort.0'),
|
||||
];
|
||||
}
|
||||
// rawId on the inner hit gives us the integer post ID.
|
||||
$bestPostId = Arr::get($hit, 'inner_hits.best_post.hits.hits.0._source.rawId');
|
||||
|
||||
return [
|
||||
'discussion_id' => $discussionId,
|
||||
'most_relevant_post_id' => $bestPostId,
|
||||
'weight' => Arr::get($hit, 'sort.0', Arr::get($hit, '_score', 0)),
|
||||
];
|
||||
});
|
||||
|
||||
$document->addPaginationLinks(
|
||||
$this->uri->to('api')->route('blomstra.search', [
|
||||
'type' => 'discussions',
|
||||
]),
|
||||
$this->uri->to('api')->route('blomstra.search', ['type' => 'discussions']),
|
||||
$request->getQueryParams(),
|
||||
$offset,
|
||||
$limit,
|
||||
|
|
@ -140,28 +156,25 @@ class SearchController extends ListDiscussionsController
|
|||
$results = $results->take($limit);
|
||||
|
||||
$discussions = Discussion::query()
|
||||
->select('discussions.*')
|
||||
->join('posts', 'posts.discussion_id', 'discussions.id')
|
||||
// Extra safety to prevent leaking hidden discussion (titles) towards search results.
|
||||
->when($actor->isGuest() || !$actor->hasPermission('discussion.hide'), fn ($query) => $query->whereNull('discussions.hidden_at'))
|
||||
->where(function ($query) use ($results) {
|
||||
$query
|
||||
->whereIn('discussions.id', $results->pluck('discussion_id')->filter())
|
||||
->orWhereIn('posts.id', $results->pluck('most_relevant_post_id')->filter());
|
||||
})
|
||||
->when(
|
||||
$actor->isGuest() || !$actor->hasPermission('discussion.hide'),
|
||||
fn ($q) => $q->whereNull('hidden_at')
|
||||
)
|
||||
->whereIn('id', $results->pluck('discussion_id')->filter())
|
||||
->get()
|
||||
->each(function (Discussion $discussion) use ($results) {
|
||||
if (in_array($discussion->id, $results->pluck('discussion_id')->toArray())) {
|
||||
$discussion->most_relevant_post_id = $discussion->first_post_id;
|
||||
$discussion->weight = $results->firstWhere('discussion_id', $discussion->id)['weight'] ?? 0;
|
||||
} else {
|
||||
$post = $discussion->posts()->whereIn('id', $results->pluck('most_relevant_post_id'))->first();
|
||||
$discussion->most_relevant_post_id = $post?->id ?? $discussion->first_post_id;
|
||||
$discussion->weight = $results->firstWhere('most_relevant_post_id', $post?->id)['weight'] ?? 0;
|
||||
}
|
||||
$result = $results->firstWhere('discussion_id', $discussion->id);
|
||||
|
||||
$discussion->most_relevant_post_id = $result['most_relevant_post_id']
|
||||
?? $discussion->first_post_id;
|
||||
$discussion->weight = $result['weight'] ?? 0;
|
||||
})
|
||||
->keyBy('id')
|
||||
->sortByDesc('weight')
|
||||
->when(
|
||||
$phpSortField,
|
||||
fn ($c) => $phpSortDir === 'desc' ? $c->sortByDesc($phpSortField) : $c->sortBy($phpSortField),
|
||||
fn ($c) => $c->sortByDesc('weight')
|
||||
)
|
||||
->unique();
|
||||
|
||||
$this->loadRelations($discussions, $include);
|
||||
|
|
@ -179,33 +192,77 @@ class SearchController extends ListDiscussionsController
|
|||
return $discussions;
|
||||
}
|
||||
|
||||
protected function getDocument(string $type): ?ElasticDocument
|
||||
/**
|
||||
* Build the text-matching portion of the query.
|
||||
*
|
||||
* Discussion titles are matched directly (filtered to join_field=discussion).
|
||||
* Post bodies are matched via has_child with score_mode=sum so discussions
|
||||
* with many matching posts score higher than those with a single strong match.
|
||||
* inner_hits returns the best-scoring post for use as mostRelevantPost.
|
||||
* Hidden posts are only included in matching for users with post.hide permission.
|
||||
*/
|
||||
protected function buildTextQuery(string $search, User $actor): BoolQuery
|
||||
{
|
||||
$documents = resolve(Container::class)->tagged('blomstra.search.documents');
|
||||
$textQuery = BoolQuery::create();
|
||||
|
||||
return collect($documents)->first(function (ElasticDocument $document) use ($type) {
|
||||
return $document->type() === $type;
|
||||
});
|
||||
if ($this->discussionSearcher?->enabled()) {
|
||||
$textQuery->add($this->buildShouldClauses($search, $this->discussionSearcher->boost()), 'should');
|
||||
}
|
||||
|
||||
if ($this->postSearcher?->enabled()) {
|
||||
$postQuery = $this->buildShouldClauses($search, $this->postSearcher->boost());
|
||||
|
||||
// Guests and non-moderators may not see hidden posts; exclude them from child matching.
|
||||
if ($actor->isGuest() || !$actor->hasPermission('post.hide')) {
|
||||
$postQuery->add(TermQuery::create('is_hidden', false), 'filter');
|
||||
}
|
||||
|
||||
$textQuery->add(
|
||||
HasChildQuery::create('post', $postQuery)->withInnerHits(),
|
||||
'should'
|
||||
);
|
||||
}
|
||||
|
||||
return $textQuery;
|
||||
}
|
||||
|
||||
protected function buildShouldClauses(string $search, float $boost): BoolQuery
|
||||
{
|
||||
$should = BoolQuery::create();
|
||||
|
||||
if ($this->matchSentences) {
|
||||
$should->add((new MatchPhraseQuery('content', $search))->boost(2 * $boost), 'should');
|
||||
}
|
||||
if ($this->matchWords) {
|
||||
$should->add((new MatchQuery('content', $search))->operator('and')->boost(1.8 * $boost), 'should');
|
||||
$should->add((new MatchQuery('content', $search))->operator('or')->boost(0.8 * $boost), 'should');
|
||||
}
|
||||
|
||||
return $should;
|
||||
}
|
||||
|
||||
protected function extensionEnabled(string $extension): bool
|
||||
{
|
||||
/** @var ExtensionManager $manager */
|
||||
$manager = resolve(ExtensionManager::class);
|
||||
|
||||
return $manager->isEnabled($extension);
|
||||
return resolve(ExtensionManager::class)->isEnabled($extension);
|
||||
}
|
||||
|
||||
protected function addFilters(BoolQuery $query, User $actor, array $filters = []): BoolQuery
|
||||
protected function addFilters(BoolQuery $query, User $actor, array $filters = []): void
|
||||
{
|
||||
$groups = $this->getGroups($actor);
|
||||
|
||||
$groups = $this->getGroups($actor);
|
||||
$onlyPrivate = Str::contains($filters['q'] ?? '', 'is:private');
|
||||
|
||||
$subQuery = BoolQuery::create()
|
||||
->add(TermQuery::create('is_private', 'false'))
|
||||
->add(TermsQuery::create('groups', $groups->toArray()));
|
||||
|
||||
if ($this->extensionEnabled('flarum-tags') && !empty($filters['tag'])) {
|
||||
$slugs = is_array($filters['tag']) ? $filters['tag'] : explode(',', $filters['tag']);
|
||||
$tagIds = Tag::query()->whereIn('slug', $slugs)->pluck('id')->toArray();
|
||||
if (!empty($tagIds)) {
|
||||
$query->add(TermsQuery::create('tags', $tagIds), 'filter');
|
||||
}
|
||||
}
|
||||
|
||||
if ($this->extensionEnabled('fof-byobu') && $actor->exists) {
|
||||
$byobuQuery = BoolQuery::create()
|
||||
->add(TermQuery::create('is_private', 'true'))
|
||||
|
|
@ -224,85 +281,12 @@ class SearchController extends ListDiscussionsController
|
|||
}
|
||||
}
|
||||
|
||||
$query->add(
|
||||
$subQuery,
|
||||
'filter'
|
||||
);
|
||||
|
||||
return $query;
|
||||
}
|
||||
|
||||
protected function sentenceMatch(string $q): Query
|
||||
{
|
||||
$query = (new MatchPhraseQuery('content', $q));
|
||||
|
||||
return BoolQuery::create()
|
||||
// Discussion titles
|
||||
->add(
|
||||
BoolQuery::create()
|
||||
->add(TermQuery::create('type', 'discussions'), 'filter')
|
||||
->add($query->boost(2)),
|
||||
'should'
|
||||
)
|
||||
// Post bodies
|
||||
->add(
|
||||
BoolQuery::create()
|
||||
->add(TermQuery::create('type', 'posts'), 'filter')
|
||||
->add($query->boost(1.9)),
|
||||
'should'
|
||||
);
|
||||
}
|
||||
|
||||
protected function wordMatch(string $q, string $operator = 'or')
|
||||
{
|
||||
$query = (new MatchQuery('content', $q))
|
||||
->operator($operator);
|
||||
|
||||
$boost = $operator === 'and' ? 1 : .8;
|
||||
|
||||
return BoolQuery::create()
|
||||
// Discussion titles
|
||||
->add(
|
||||
BoolQuery::create()
|
||||
->add(TermQuery::create('type', 'discussions'), 'filter')
|
||||
->add($query->boost($boost * 1.8)),
|
||||
'should'
|
||||
)
|
||||
// Post bodies
|
||||
->add(
|
||||
BoolQuery::create()
|
||||
->add(TermQuery::create('type', 'posts'), 'filter')
|
||||
->add($query->boost($boost * 1.8)),
|
||||
'should'
|
||||
);
|
||||
}
|
||||
|
||||
protected function partialMatch(string $q)
|
||||
{
|
||||
$query = (new MatchQuery('content', $q));
|
||||
|
||||
return BoolQuery::create()
|
||||
// Discussion titles
|
||||
->add(
|
||||
BoolQuery::create()
|
||||
->add(TermQuery::create('type', 'discussions'), 'filter')
|
||||
->add(clone $query->boost(1.6)),
|
||||
'should'
|
||||
)
|
||||
// Post bodies
|
||||
->add(
|
||||
BoolQuery::create()
|
||||
->add(TermQuery::create('type', 'posts'), 'filter')
|
||||
->add(clone $query->boost(1.3)),
|
||||
'should'
|
||||
);
|
||||
$query->add($subQuery, 'filter');
|
||||
}
|
||||
|
||||
protected function getGroups(User $actor): Collection
|
||||
{
|
||||
/** @var Collection $groups */
|
||||
$groups = $actor->groups->pluck('id');
|
||||
|
||||
$groups->add(Group::GUEST_ID);
|
||||
|
||||
if ($actor->is_email_confirmed) {
|
||||
|
|
@ -318,9 +302,7 @@ class SearchController extends ListDiscussionsController
|
|||
|
||||
if ($search) {
|
||||
$q = collect(explode(' ', $search))
|
||||
->filter(function (string $part) {
|
||||
return $part !== 'is:private';
|
||||
})
|
||||
->filter(fn (string $part) => $part !== 'is:private')
|
||||
->filter()
|
||||
->join(' ');
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
namespace Blomstra\Search\Commands;
|
||||
|
||||
use Blomstra\Search\Jobs\Job;
|
||||
use Blomstra\Search\Jobs\SavingJob;
|
||||
use Blomstra\Search\Jobs\UpdateSearchJob;
|
||||
use Blomstra\Search\Seeders\Seeder;
|
||||
use Elasticsearch\Client;
|
||||
use Flarum\Settings\SettingsRepositoryInterface;
|
||||
|
|
@ -21,97 +21,314 @@ use Illuminate\Console\Command;
|
|||
use Illuminate\Contracts\Container\Container;
|
||||
use Illuminate\Contracts\Queue\Queue;
|
||||
use Illuminate\Database\Eloquent\Collection;
|
||||
use Illuminate\Support\Arr;
|
||||
use Spatie\ElasticsearchQueryBuilder\Builder;
|
||||
use Spatie\ElasticsearchQueryBuilder\Queries\BoolQuery;
|
||||
use Spatie\ElasticsearchQueryBuilder\Queries\RangeQuery;
|
||||
use Spatie\ElasticsearchQueryBuilder\Queries\TermQuery;
|
||||
|
||||
class BuildCommand extends Command
|
||||
{
|
||||
/**
|
||||
* Bump this when a mapping change requires a full reindex.
|
||||
* Written into the mapping's _meta.index_compat_version on every build and
|
||||
* read back into blomstra-search.index-compatible by saveIndexedConfig so the
|
||||
* value always reflects the live index — including after a promote/rollback.
|
||||
*/
|
||||
public const INDEX_COMPAT_VERSION = 'v2';
|
||||
|
||||
/** Matches Flarum's Search::MIN_SEARCH_LEN — the default minimum query length. */
|
||||
public const DEFAULT_MIN_SEARCH_LENGTH = 3;
|
||||
|
||||
protected $signature = 'blomstra:search:index
|
||||
{--max-id= : Limits for each object the number of items to seed}
|
||||
{--chunk-size= : Size of the chunks to dispatch into jobs}
|
||||
{--throttle= : Number of seconds to wait between pushing to the queue}
|
||||
{--only= : type to run seeder for, eg discussions or posts}
|
||||
{--recreate : create or recreate the index}
|
||||
{--mapping : recreate the mapping}
|
||||
{--continue : continue each object type where you left off}';
|
||||
protected $description = 'Rebuilds the complete search server with its documents.';
|
||||
{action? : build | promote | rollback | discard | mapping | fill}
|
||||
{--fresh}
|
||||
{--resume}
|
||||
{--staging}
|
||||
{--keep-backup}
|
||||
{--pending}
|
||||
{--backup}
|
||||
{--only=}
|
||||
{--max-id=}
|
||||
{--throttle=}
|
||||
{--i-am-sure}';
|
||||
|
||||
public function handle(Container $container)
|
||||
protected $description = 'Build and manage the Elasticsearch search index.';
|
||||
|
||||
protected $help = <<<'HELP'
|
||||
<comment>Actions and their options:</comment>
|
||||
|
||||
<info>build</info> Queue documents and promote automatically.
|
||||
<comment>--resume</comment> Resume an interrupted build from where it left off
|
||||
<comment>--fresh</comment> Drop the staging index and start completely fresh
|
||||
<comment>--staging</comment> Keep in staging — requires explicit <info>promote</info> (blue-green)
|
||||
<comment>--keep-backup</comment> Retain the replaced index for rollback
|
||||
|
||||
<info>promote</info> Swap alias to staging index (blue-green workflow).
|
||||
<comment>--keep-backup</comment> Retain the replaced index for rollback
|
||||
<comment>--i-am-sure</comment> Skip the confirmation prompt
|
||||
|
||||
<info>rollback</info> Restore the backup index to live.
|
||||
|
||||
<info>discard</info> Drop an index without promoting.
|
||||
<comment>--pending</comment> Drop the staging index (cancel a build)
|
||||
<comment>--backup</comment> Drop the backup index (cleanup after --keep-backup)
|
||||
|
||||
<info>mapping</info> Push updated mapping to the live index only.
|
||||
|
||||
<info>fill</info> Seed only documents missing from the live index.
|
||||
|
||||
<comment>Shared seeding options (build / fill):</comment>
|
||||
<comment>--only=TYPE</comment> Seed only this type: <info>discussions</info> or <info>posts</info>
|
||||
<comment>--throttle=N</comment> Seconds to wait between batches
|
||||
<comment>--max-id=N</comment> Limit seeding to IDs up to this value
|
||||
HELP;
|
||||
|
||||
public function handle(Container $container): void
|
||||
{
|
||||
$index = $container->make('blomstra.search.elastic_index');
|
||||
|
||||
/** @var array $seeders */
|
||||
$seeders = $container->tagged('blomstra.search.seeders');
|
||||
|
||||
/** @var Queue $queue */
|
||||
$queue = $container->make(Queue::class);
|
||||
|
||||
/** @var Client $client */
|
||||
$client = $container->make(Client::class);
|
||||
|
||||
/** @var SettingsRepositoryInterface $settings */
|
||||
$settings = $container->make(SettingsRepositoryInterface::class);
|
||||
|
||||
$properties = [
|
||||
'properties' => [
|
||||
'content' => ['type' => 'text', 'analyzer' => 'flarum_analyzer_partial', 'search_analyzer' => 'flarum_analyzer'],
|
||||
'created_at' => ['type' => 'date'],
|
||||
'updated_at' => ['type' => 'date'],
|
||||
'is_private' => ['type' => 'boolean'],
|
||||
'is_sticky' => ['type' => 'boolean'],
|
||||
'groups' => ['type' => 'integer'],
|
||||
'recipient_groups' => ['type' => 'integer'],
|
||||
'recipient_users' => ['type' => 'integer'],
|
||||
'comment_count' => ['type' => 'integer'],
|
||||
],
|
||||
];
|
||||
/** @var string $alias */
|
||||
$alias = $container->make('blomstra.search.elastic_index');
|
||||
|
||||
if ($this->option('recreate')) {
|
||||
// Flush the index.
|
||||
$client->indices()->delete([
|
||||
'index' => $index,
|
||||
'ignore_unavailable' => true,
|
||||
]);
|
||||
|
||||
// Create a new index.
|
||||
$client->indices()->create([
|
||||
'index' => $index,
|
||||
'body' => [
|
||||
'settings' => [
|
||||
'index.max_ngram_diff' => 10,
|
||||
'analysis' => [
|
||||
'analyzer' => [
|
||||
'flarum_analyzer' => [
|
||||
'type' => $settings->get('blomstra-search.analyzer-language') ?: 'english',
|
||||
],
|
||||
'flarum_analyzer_partial' => [
|
||||
'type' => 'custom',
|
||||
'tokenizer' => 'standard',
|
||||
'filter' => [
|
||||
'lowercase',
|
||||
'partial_search_filter',
|
||||
],
|
||||
],
|
||||
],
|
||||
'filter' => [
|
||||
'partial_search_filter' => [
|
||||
'type' => 'ngram',
|
||||
'min_gram' => 1,
|
||||
'max_gram' => 10,
|
||||
'token_chars' => ['letter', 'digit', 'symbol'],
|
||||
],
|
||||
],
|
||||
],
|
||||
],
|
||||
],
|
||||
]);
|
||||
if (!$this->argument('action')) {
|
||||
$this->call('help', ['command_name' => $this->getName()]);
|
||||
return;
|
||||
}
|
||||
|
||||
if ($this->option('recreate') || $this->option('mapping')) {
|
||||
$client->indices()->putMapping([
|
||||
'index' => $index,
|
||||
'body' => $properties,
|
||||
]);
|
||||
switch ($this->argument('action')) {
|
||||
case 'build':
|
||||
$this->runBuild($client, $alias, $settings, $container);
|
||||
break;
|
||||
|
||||
case 'promote':
|
||||
$this->runPromote($client, $alias, $settings);
|
||||
break;
|
||||
|
||||
case 'rollback':
|
||||
$this->runRollback($client, $alias, $settings);
|
||||
break;
|
||||
|
||||
case 'discard':
|
||||
$this->runDiscard($client, $settings);
|
||||
break;
|
||||
|
||||
case 'mapping':
|
||||
$client->indices()->putMapping([
|
||||
'index' => $alias,
|
||||
'body' => $this->mappingProperties(),
|
||||
]);
|
||||
$this->info('Mapping updated on live index.');
|
||||
break;
|
||||
|
||||
case 'fill':
|
||||
$this->runSeeders(
|
||||
collect($container->tagged('blomstra.search.seeders')),
|
||||
$container->make(Queue::class),
|
||||
$client,
|
||||
$settings,
|
||||
$alias,
|
||||
seedMissing: true
|
||||
);
|
||||
break;
|
||||
|
||||
default:
|
||||
$this->error("Unknown action '{$this->argument('action')}'. Valid actions: build, promote, rollback, discard, mapping, fill.");
|
||||
}
|
||||
}
|
||||
|
||||
protected function runBuild(Client $client, string $alias, SettingsRepositoryInterface $settings, Container $container): void
|
||||
{
|
||||
/** @var Seeder[] $seeders */
|
||||
$seeders = collect($container->tagged('blomstra.search.seeders'));
|
||||
|
||||
// Guard: staging build exists without explicit intent given.
|
||||
$staging = $settings->get('blomstra-search.staging-index');
|
||||
if ($staging && $client->indices()->exists(['index' => $staging])
|
||||
&& !$this->option('resume') && !$this->option('fresh')
|
||||
) {
|
||||
$this->warn("An in-progress build already exists: $staging");
|
||||
$this->line('');
|
||||
$this->line('Choose one of:');
|
||||
$this->line(' blomstra:search:index build --resume Continue from where it left off');
|
||||
$this->line(' blomstra:search:index build --fresh Drop this build and start completely fresh');
|
||||
return;
|
||||
}
|
||||
|
||||
$aliasExists = (bool) $client->indices()->existsAlias(['name' => $alias]);
|
||||
$indexExists = !$aliasExists && (bool) $client->indices()->exists(['index' => $alias]);
|
||||
|
||||
if (!$aliasExists && !$indexExists) {
|
||||
// First install: create a timestamped index, alias it immediately, seed live.
|
||||
$targetIndex = $this->prepareFirstInstall($client, $alias, $settings, $seeders);
|
||||
$stagingBuild = false;
|
||||
} else {
|
||||
// Live system: build into a staging index without touching the live alias.
|
||||
$targetIndex = $this->prepareStagingIndex($client, $alias, $settings, $seeders);
|
||||
$stagingBuild = true;
|
||||
}
|
||||
|
||||
$client->indices()->putMapping([
|
||||
'index' => $targetIndex,
|
||||
'body' => $this->mappingProperties(),
|
||||
]);
|
||||
|
||||
$this->runSeeders($seeders, $container->make(Queue::class), $client, $settings, $targetIndex);
|
||||
|
||||
if ($stagingBuild) {
|
||||
if ($this->option('staging')) {
|
||||
$staging = $settings->get('blomstra-search.staging-index');
|
||||
$this->info("Staging build complete. Drain the queue, then promote '$staging' to live:");
|
||||
$this->line(' php flarum queue:work --stop-when-empty');
|
||||
$this->line(' php flarum blomstra:search:index promote');
|
||||
} else {
|
||||
$this->info('Jobs queued. Promoting now (search improves as the queue drains)...');
|
||||
$this->performPromote($client, $alias, $settings);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
protected function runPromote(Client $client, string $alias, SettingsRepositoryInterface $settings): void
|
||||
{
|
||||
if (!$this->option('i-am-sure')) {
|
||||
$this->warn('promote switches the live index. Ensure all queued jobs have finished first:');
|
||||
$this->line(' php flarum queue:work --stop-when-empty');
|
||||
$this->line('');
|
||||
|
||||
if (!$this->confirm('Queue drained? Proceed?')) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
$this->performPromote($client, $alias, $settings);
|
||||
}
|
||||
|
||||
protected function performPromote(Client $client, string $alias, SettingsRepositoryInterface $settings): void
|
||||
{
|
||||
$staging = $settings->get('blomstra-search.staging-index');
|
||||
|
||||
if (!$staging || !$client->indices()->exists(['index' => $staging])) {
|
||||
$this->error('No staging index ready to promote. Run: blomstra:search:index build');
|
||||
return;
|
||||
}
|
||||
|
||||
$aliasExists = (bool) $client->indices()->existsAlias(['name' => $alias]);
|
||||
$indexExists = !$aliasExists && (bool) $client->indices()->exists(['index' => $alias]);
|
||||
|
||||
if ($aliasExists) {
|
||||
$result = $client->indices()->getAlias(['name' => $alias]);
|
||||
$activeIndex = array_key_first($result);
|
||||
|
||||
$client->indices()->updateAliases([
|
||||
'body' => ['actions' => [
|
||||
['remove' => ['index' => $activeIndex, 'alias' => $alias]],
|
||||
['add' => ['index' => $staging, 'alias' => $alias]],
|
||||
]],
|
||||
]);
|
||||
|
||||
$this->info("'$alias' now points to '$staging'. Promote complete.");
|
||||
|
||||
if ($this->option('keep-backup')) {
|
||||
$settings->set('blomstra-search.backup-index', $activeIndex);
|
||||
$this->info("Kept '$activeIndex' as backup — run rollback to revert, discard --backup to drop it.");
|
||||
} else {
|
||||
$client->indices()->delete(['index' => $activeIndex, 'ignore_unavailable' => true]);
|
||||
$this->info("Deleted old index: $activeIndex");
|
||||
}
|
||||
} else {
|
||||
// One-time migration: concrete index → alias (legacy installs only).
|
||||
if ($indexExists) {
|
||||
$client->indices()->delete(['index' => $alias]);
|
||||
$this->info("Deleted legacy concrete index: $alias");
|
||||
}
|
||||
|
||||
$client->indices()->putAlias(['index' => $staging, 'name' => $alias]);
|
||||
$this->info("'$alias' now points to '$staging'. Promote complete.");
|
||||
}
|
||||
|
||||
$settings->set('blomstra-search.active-index', $staging);
|
||||
$settings->set('blomstra-search.staging-index', null);
|
||||
$this->saveIndexedConfig($client, $settings, $staging);
|
||||
}
|
||||
|
||||
protected function runRollback(Client $client, string $alias, SettingsRepositoryInterface $settings): void
|
||||
{
|
||||
$backup = $settings->get('blomstra-search.backup-index');
|
||||
|
||||
if (!$backup || !$client->indices()->exists(['index' => $backup])) {
|
||||
$this->error('No backup index to roll back to. Was promote run with --keep-backup?');
|
||||
return;
|
||||
}
|
||||
|
||||
$result = $client->indices()->getAlias(['name' => $alias]);
|
||||
$activeIndex = array_key_first($result);
|
||||
|
||||
$client->indices()->updateAliases([
|
||||
'body' => ['actions' => [
|
||||
['remove' => ['index' => $activeIndex, 'alias' => $alias]],
|
||||
['add' => ['index' => $backup, 'alias' => $alias]],
|
||||
]],
|
||||
]);
|
||||
|
||||
$client->indices()->delete(['index' => $activeIndex, 'ignore_unavailable' => true]);
|
||||
$this->info("Rolled back: '$alias' now points to '$backup'. Deleted '$activeIndex'.");
|
||||
|
||||
$settings->set('blomstra-search.active-index', $backup);
|
||||
$settings->set('blomstra-search.backup-index', null);
|
||||
$this->saveIndexedConfig($client, $settings, $backup);
|
||||
}
|
||||
|
||||
protected function runDiscard(Client $client, SettingsRepositoryInterface $settings): void
|
||||
{
|
||||
$pending = $this->option('pending');
|
||||
$backup = $this->option('backup');
|
||||
|
||||
if (!$pending && !$backup) {
|
||||
$this->error('Specify what to discard: --pending (staging build) or --backup (rollback index).');
|
||||
return;
|
||||
}
|
||||
|
||||
if ($pending) {
|
||||
$staging = $settings->get('blomstra-search.staging-index');
|
||||
|
||||
if (!$staging) {
|
||||
$this->warn('No staging index to discard.');
|
||||
} else {
|
||||
if ($client->indices()->exists(['index' => $staging])) {
|
||||
$client->indices()->delete(['index' => $staging]);
|
||||
}
|
||||
$settings->set('blomstra-search.staging-index', null);
|
||||
$this->info("Discarded staging index: $staging");
|
||||
}
|
||||
}
|
||||
|
||||
if ($backup) {
|
||||
$backupIndex = $settings->get('blomstra-search.backup-index');
|
||||
|
||||
if (!$backupIndex) {
|
||||
$this->warn('No backup index to discard.');
|
||||
} else {
|
||||
if ($client->indices()->exists(['index' => $backupIndex])) {
|
||||
$client->indices()->delete(['index' => $backupIndex]);
|
||||
}
|
||||
$settings->set('blomstra-search.backup-index', null);
|
||||
$this->info("Discarded backup index: $backupIndex");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
protected function runSeeders(
|
||||
iterable $seeders,
|
||||
Queue $queue,
|
||||
Client $client,
|
||||
SettingsRepositoryInterface $settings,
|
||||
string $targetIndex,
|
||||
bool $seedMissing = false
|
||||
): void {
|
||||
$only = $this->option('only');
|
||||
|
||||
/** @var Seeder $seeder */
|
||||
|
|
@ -122,33 +339,63 @@ class BuildCommand extends Command
|
|||
|
||||
$total = 0;
|
||||
|
||||
$continueAt = $this->option('continue')
|
||||
? ($this->continueAt($seeder->type()) ?? $seeder->query()->max('id'))
|
||||
: $seeder->query()->max('id');
|
||||
if ($this->option('resume')) {
|
||||
$saved = $this->getContinueAt($settings, $seeder->type());
|
||||
|
||||
if ($saved === 0) {
|
||||
$this->info("Seeder '{$seeder->type()}' already completed in a previous run, skipping.");
|
||||
continue;
|
||||
}
|
||||
|
||||
$continueAt = $saved ?? $seeder->query()->max('id');
|
||||
} else {
|
||||
$continueAt = $seeder->query()->max('id');
|
||||
}
|
||||
|
||||
$seeded = null;
|
||||
|
||||
while ($continueAt !== null) {
|
||||
$rangeFrom = max(1, $continueAt - 2500);
|
||||
$rangeTo = $continueAt;
|
||||
|
||||
if ($seedMissing) {
|
||||
$response = (new Builder($client))
|
||||
->index($targetIndex)
|
||||
->size(2500)
|
||||
->addQuery(
|
||||
(new BoolQuery())
|
||||
->add((new RangeQuery('rawId'))->gte($rangeFrom)->lte($rangeTo))
|
||||
->add(TermQuery::create('join_field', $seeder->joinRelation()))
|
||||
)
|
||||
->search();
|
||||
|
||||
$seeded = Arr::pluck(Arr::get($response, 'hits.hits'), '_source.rawId');
|
||||
}
|
||||
|
||||
/** @var Collection $collection */
|
||||
$collection = $seeder->query()
|
||||
->latest('id')
|
||||
->whereBetween('id', [$continueAt - 1000, $continueAt])
|
||||
->when($this->option('max-id'), function ($query, $id) {
|
||||
$query->where('id', '<=', $id);
|
||||
})
|
||||
->whereBetween('id', [$rangeFrom, $rangeTo])
|
||||
->when($this->option('max-id'), fn ($q, $id) => $q->where('id', '<=', $id))
|
||||
->when($seeded, fn ($q, $seeded) => $q->whereNotIn('id', $seeded))
|
||||
->get();
|
||||
|
||||
$min = $collection->min('id');
|
||||
$continueAt = $min && $min > 2 ? $min - 1 : null;
|
||||
|
||||
$queue->pushOn(Job::$onQueue, new SavingJob($collection, $seeder));
|
||||
if ($seedMissing && $collection->isEmpty()) {
|
||||
$continueAt = $rangeFrom > 2 ? $rangeFrom - 1 : null;
|
||||
} else {
|
||||
$continueAt = $min && $min > 2 ? $min - 1 : null;
|
||||
}
|
||||
|
||||
$this->info("Pushed into the index, type: {$seeder->type()}, amount: {$collection->count()}.");
|
||||
if ($collection->isNotEmpty()) {
|
||||
$queue->pushOn(Job::$onQueue, new UpdateSearchJob($collection, $seeder, $targetIndex));
|
||||
}
|
||||
|
||||
$this->info("IDs {$rangeFrom}–{$rangeTo} | type: {$seeder->type()} | queued: {$collection->count()}.");
|
||||
|
||||
$total += $collection->count();
|
||||
|
||||
$this->continueAt(
|
||||
$seeder->type(),
|
||||
$continueAt
|
||||
);
|
||||
$this->setContinueAt($settings, $seeder->type(), $continueAt);
|
||||
|
||||
if ($throttle = $this->option('throttle')) {
|
||||
$this->info("Throttling for $throttle seconds");
|
||||
|
|
@ -156,21 +403,181 @@ class BuildCommand extends Command
|
|||
}
|
||||
}
|
||||
|
||||
$this->info("Pushed a total of $total into the index.");
|
||||
$this->setContinueAt($settings, $seeder->type(), 0);
|
||||
$this->info("Queued a total of $total {$seeder->type()} for indexing.");
|
||||
}
|
||||
}
|
||||
|
||||
protected function continueAt(string $type, int $at = null)
|
||||
{
|
||||
/** @var SettingsRepositoryInterface $settings */
|
||||
$settings = resolve(SettingsRepositoryInterface::class);
|
||||
/**
|
||||
* First install: create a timestamped concrete index, alias the configured name to it
|
||||
* immediately, and return the alias so seeding goes through it and is live from the start.
|
||||
*/
|
||||
protected function prepareFirstInstall(
|
||||
Client $client,
|
||||
string $alias,
|
||||
SettingsRepositoryInterface $settings,
|
||||
iterable $seeders
|
||||
): string {
|
||||
$concrete = $alias . '_' . date('YmdHis');
|
||||
|
||||
$key = "blomstra-search.continued-at.$type";
|
||||
$client->indices()->create([
|
||||
'index' => $concrete,
|
||||
'body' => ['settings' => $this->buildIndexSettings($settings)],
|
||||
]);
|
||||
|
||||
if ($at) {
|
||||
$settings->set($key, $at);
|
||||
} else {
|
||||
return $settings->get($key);
|
||||
$client->indices()->putAlias(['index' => $concrete, 'name' => $alias]);
|
||||
|
||||
$settings->set('blomstra-search.active-index', $concrete);
|
||||
$this->saveIndexedConfig($client, $settings, $concrete);
|
||||
|
||||
foreach ($seeders as $seeder) {
|
||||
$this->setContinueAt($settings, $seeder->type(), null);
|
||||
}
|
||||
|
||||
$this->info("Created '$concrete', aliased '$alias' → '$concrete'.");
|
||||
$this->info("Index is live — documents become searchable as the queue processes.");
|
||||
|
||||
return $alias;
|
||||
}
|
||||
|
||||
/**
|
||||
* Prepare the staging index for a blue-green build.
|
||||
*
|
||||
* - If a staging build exists and --fresh is not set, resume it.
|
||||
* - Otherwise create a fresh timestamped index and save it as staging.
|
||||
*/
|
||||
protected function prepareStagingIndex(
|
||||
Client $client,
|
||||
string $alias,
|
||||
SettingsRepositoryInterface $settings,
|
||||
iterable $seeders
|
||||
): string {
|
||||
$staging = $settings->get('blomstra-search.staging-index');
|
||||
|
||||
if ($this->option('fresh') && $staging) {
|
||||
if ($client->indices()->exists(['index' => $staging])) {
|
||||
$client->indices()->delete(['index' => $staging]);
|
||||
$this->info("Dropped staging index: $staging");
|
||||
}
|
||||
$staging = null;
|
||||
$settings->set('blomstra-search.staging-index', null);
|
||||
}
|
||||
|
||||
if ($staging && $client->indices()->exists(['index' => $staging])) {
|
||||
$this->info("Resuming staging index build: $staging");
|
||||
return $staging;
|
||||
}
|
||||
|
||||
$staging = $alias . '_' . date('YmdHis');
|
||||
|
||||
$client->indices()->create([
|
||||
'index' => $staging,
|
||||
'body' => ['settings' => $this->buildIndexSettings($settings)],
|
||||
]);
|
||||
|
||||
$settings->set('blomstra-search.staging-index', $staging);
|
||||
|
||||
foreach ($seeders as $seeder) {
|
||||
$this->setContinueAt($settings, $seeder->type(), null);
|
||||
}
|
||||
|
||||
$this->info("Created staging index: $staging");
|
||||
|
||||
return $staging;
|
||||
}
|
||||
|
||||
/**
|
||||
* Persist the analysis config and compat version that are actually live in ES for the
|
||||
* given index. Reading from ES (rather than from Flarum settings) means rollbacks are
|
||||
* also covered: the stored values always reflect the index that is currently aliased,
|
||||
* not the settings at the time the command ran. Indexes built before _meta tracking
|
||||
* existed will yield a null compat version, which correctly triggers the reindex warning.
|
||||
*/
|
||||
protected function saveIndexedConfig(Client $client, SettingsRepositoryInterface $settings, string $indexName): void
|
||||
{
|
||||
$settingsResponse = $client->indices()->getSettings(['index' => $indexName]);
|
||||
$analysis = Arr::get($settingsResponse, "$indexName.settings.index.analysis", []);
|
||||
|
||||
$analyzer = Arr::get($analysis, 'analyzer.flarum_analyzer.type', 'english');
|
||||
$minGram = (int) Arr::get($analysis, 'filter.partial_search_filter.min_gram', self::DEFAULT_MIN_SEARCH_LENGTH);
|
||||
|
||||
$mappingResponse = $client->indices()->getMapping(['index' => $indexName]);
|
||||
$compatVersion = Arr::get($mappingResponse, "$indexName.mappings._meta.index_compat_version");
|
||||
|
||||
$settings->set('blomstra-search.indexed-analyzer', $analyzer);
|
||||
$settings->set('blomstra-search.indexed-min-search-length', $minGram);
|
||||
$settings->set('blomstra-search.index-compatible', $compatVersion);
|
||||
}
|
||||
|
||||
protected function buildIndexSettings(SettingsRepositoryInterface $settings): array
|
||||
{
|
||||
$language = $settings->get('blomstra-search.analyzer-language') ?: 'english';
|
||||
$minGram = max(1, (int) ($settings->get('blomstra-search.min-search-length') ?: self::DEFAULT_MIN_SEARCH_LENGTH));
|
||||
$maxGram = 10;
|
||||
|
||||
if ($minGram >= $maxGram) {
|
||||
$this->error("min_gram ($minGram) must be less than max_gram ($maxGram). Using default.");
|
||||
$minGram = self::DEFAULT_MIN_SEARCH_LENGTH;
|
||||
}
|
||||
|
||||
return [
|
||||
'index.max_ngram_diff' => $maxGram - $minGram,
|
||||
'analysis' => [
|
||||
'analyzer' => [
|
||||
'flarum_analyzer' => [
|
||||
'type' => $language,
|
||||
],
|
||||
'flarum_analyzer_partial' => [
|
||||
'type' => 'custom',
|
||||
'tokenizer' => 'standard',
|
||||
'filter' => ['lowercase', 'partial_search_filter'],
|
||||
],
|
||||
],
|
||||
'filter' => [
|
||||
'partial_search_filter' => [
|
||||
'type' => 'ngram',
|
||||
'min_gram' => $minGram,
|
||||
'max_gram' => $maxGram,
|
||||
'token_chars' => ['letter', 'digit', 'symbol'],
|
||||
],
|
||||
],
|
||||
],
|
||||
];
|
||||
}
|
||||
|
||||
protected function mappingProperties(): array
|
||||
{
|
||||
return [
|
||||
'_meta' => ['index_compat_version' => self::INDEX_COMPAT_VERSION],
|
||||
'properties' => [
|
||||
'join_field' => ['type' => 'join', 'relations' => ['discussion' => 'post']],
|
||||
'discussion_id' => ['type' => 'integer'],
|
||||
'content' => ['type' => 'text', 'analyzer' => 'flarum_analyzer_partial', 'search_analyzer' => 'flarum_analyzer'],
|
||||
'rawId' => ['type' => 'integer'],
|
||||
'created_at' => ['type' => 'date'],
|
||||
'updated_at' => ['type' => 'date'],
|
||||
'is_private' => ['type' => 'boolean'],
|
||||
'is_sticky' => ['type' => 'boolean'],
|
||||
'groups' => ['type' => 'integer'],
|
||||
'tags' => ['type' => 'integer'],
|
||||
'recipient_groups' => ['type' => 'integer'],
|
||||
'recipient_users' => ['type' => 'integer'],
|
||||
'comment_count' => ['type' => 'integer'],
|
||||
'view_count' => ['type' => 'integer'],
|
||||
'is_hidden' => ['type' => 'boolean'],
|
||||
],
|
||||
];
|
||||
}
|
||||
|
||||
protected function getContinueAt(SettingsRepositoryInterface $settings, string $type): ?int
|
||||
{
|
||||
$raw = $settings->get("blomstra-search.continued-at.$type");
|
||||
|
||||
return $raw !== null ? (int) $raw : null;
|
||||
}
|
||||
|
||||
protected function setContinueAt(SettingsRepositoryInterface $settings, string $type, ?int $at): void
|
||||
{
|
||||
$settings->set("blomstra-search.continued-at.$type", $at);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,57 @@
|
|||
<?php
|
||||
|
||||
/*
|
||||
* This file is part of blomstra/search.
|
||||
*
|
||||
* Copyright (c) 2022 Blomstra Ltd.
|
||||
*
|
||||
* For the full copyright and license information, please view the LICENSE.md
|
||||
* file that was distributed with this source code.
|
||||
*
|
||||
*/
|
||||
|
||||
namespace Blomstra\Search\Elasticsearch;
|
||||
|
||||
use Spatie\ElasticsearchQueryBuilder\Queries\Query;
|
||||
|
||||
class HasChildQuery implements Query
|
||||
{
|
||||
protected bool $innerHits = false;
|
||||
|
||||
public function __construct(
|
||||
protected string $type,
|
||||
protected Query $query,
|
||||
protected string $scoreMode = 'sum'
|
||||
) {}
|
||||
|
||||
public static function create(string $type, Query $query, string $scoreMode = 'sum'): static
|
||||
{
|
||||
return new static($type, $query, $scoreMode);
|
||||
}
|
||||
|
||||
/**
|
||||
* Include inner_hits so the best-matching post ID is available in the response.
|
||||
* ES returns the top-scoring child document per parent hit under inner_hits.best_post.
|
||||
*/
|
||||
public function withInnerHits(): static
|
||||
{
|
||||
$this->innerHits = true;
|
||||
|
||||
return $this;
|
||||
}
|
||||
|
||||
public function toArray(): array
|
||||
{
|
||||
$hasChild = [
|
||||
'type' => $this->type,
|
||||
'score_mode' => $this->scoreMode,
|
||||
'query' => $this->query->toArray(),
|
||||
];
|
||||
|
||||
if ($this->innerHits) {
|
||||
$hasChild['inner_hits'] = ['name' => 'best_post', 'size' => 1];
|
||||
}
|
||||
|
||||
return ['has_child' => $hasChild];
|
||||
}
|
||||
}
|
||||
|
|
@ -26,9 +26,10 @@ class DeletingJob extends Job
|
|||
// Preparing body for storing.
|
||||
$body = $this->models->map(function (Model $model) {
|
||||
$document = $this->seeder->toDocument($model);
|
||||
$routing = $this->seeder->routing($model);
|
||||
|
||||
return [
|
||||
['delete' => ['_index' => $this->index, '_id' => $document->id]],
|
||||
['delete' => ['_index' => $this->index, '_id' => $document->id, 'routing' => $routing]],
|
||||
];
|
||||
})->flatten(1);
|
||||
|
||||
|
|
|
|||
|
|
@ -22,9 +22,13 @@ abstract class Job extends AbstractJob
|
|||
|
||||
public static ?string $onQueue = null;
|
||||
|
||||
public function __construct(protected Collection $models, protected Seeder $seeder)
|
||||
/**
|
||||
* @param string|null $targetIndex Explicit index name for blue-green builds.
|
||||
* Defaults to the configured alias when null.
|
||||
*/
|
||||
public function __construct(protected Collection $models, protected Seeder $seeder, ?string $targetIndex = null)
|
||||
{
|
||||
$this->index = resolve('blomstra.search.elastic_index');
|
||||
$this->index = $targetIndex ?? resolve('blomstra.search.elastic_index');
|
||||
|
||||
if (static::$onQueue) {
|
||||
$this->onQueue(static::$onQueue);
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ use Elasticsearch\Client;
|
|||
use Illuminate\Database\Eloquent\Model;
|
||||
use Illuminate\Support\Arr;
|
||||
|
||||
class SavingJob extends Job
|
||||
class UpdateSearchJob extends Job
|
||||
{
|
||||
public function handle(Client $client)
|
||||
{
|
||||
|
|
@ -25,21 +25,23 @@ class SavingJob extends Job
|
|||
return;
|
||||
}
|
||||
|
||||
$this->models->loadMissing($this->seeder->relationships());
|
||||
|
||||
// Preparing body for storing.
|
||||
$body = $this->models->map(function (Model $model) {
|
||||
$document = $this->seeder->toDocument($model);
|
||||
$routing = $this->seeder->routing($model);
|
||||
|
||||
return [
|
||||
['index' => ['_index' => $this->index, '_id' => $document->id]],
|
||||
['index' => ['_index' => $this->index, '_id' => $document->id, 'routing' => $routing]],
|
||||
$document->toArray(),
|
||||
];
|
||||
})
|
||||
->flatten(1);
|
||||
|
||||
$response = $client->bulk([
|
||||
'index' => $this->index,
|
||||
'body' => $body->toArray(),
|
||||
'refresh' => true,
|
||||
'index' => $this->index,
|
||||
'body' => $body->toArray(),
|
||||
]);
|
||||
|
||||
if (Arr::get($response, 'errors') !== true) {
|
||||
|
|
@ -0,0 +1,59 @@
|
|||
<?php
|
||||
|
||||
/*
|
||||
* This file is part of blomstra/search.
|
||||
*
|
||||
* Copyright (c) 2022 Blomstra Ltd.
|
||||
*
|
||||
* For the full copyright and license information, please view the LICENSE.md
|
||||
* file that was distributed with this source code.
|
||||
*
|
||||
*/
|
||||
|
||||
namespace Blomstra\Search\Jobs;
|
||||
|
||||
use Elasticsearch\Client;
|
||||
use Flarum\Api\Serializer\DiscussionSerializer;
|
||||
use Flarum\Discussion\Discussion;
|
||||
use Flarum\Queue\AbstractJob;
|
||||
|
||||
class ViewsSearchJob extends AbstractJob
|
||||
{
|
||||
protected string $index;
|
||||
protected string $documentType;
|
||||
|
||||
public function __construct(protected int $discussionId)
|
||||
{
|
||||
$this->index = resolve('blomstra.search.elastic_index');
|
||||
$this->documentType = resolve(DiscussionSerializer::class)->getType(new Discussion());
|
||||
|
||||
if (Job::$onQueue) {
|
||||
$this->onQueue(Job::$onQueue);
|
||||
}
|
||||
}
|
||||
|
||||
public function handle(Client $client): void
|
||||
{
|
||||
$discussion = Discussion::find($this->discussionId);
|
||||
|
||||
if (!$discussion) {
|
||||
return;
|
||||
}
|
||||
|
||||
$type = $this->documentType;
|
||||
|
||||
try {
|
||||
$client->update([
|
||||
'index' => $this->index,
|
||||
'id' => "$type:{$this->discussionId}",
|
||||
'routing' => (string) $this->discussionId,
|
||||
'retry_on_conflict' => 3,
|
||||
'body' => [
|
||||
'doc' => ['view_count' => (int) $discussion->view_count],
|
||||
],
|
||||
]);
|
||||
} catch (\Elasticsearch\Common\Exceptions\Missing404Exception $e) {
|
||||
// Document not yet indexed; will be picked up on next: blomstra:search:index fill
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -14,7 +14,8 @@ namespace Blomstra\Search;
|
|||
|
||||
use Blomstra\Search\Jobs\DeletingJob;
|
||||
use Blomstra\Search\Jobs\Job;
|
||||
use Blomstra\Search\Jobs\SavingJob;
|
||||
use Blomstra\Search\Jobs\UpdateSearchJob;
|
||||
use Blomstra\Search\Jobs\ViewsSearchJob;
|
||||
use Elasticsearch\Client as Elastic;
|
||||
use Elasticsearch\ClientBuilder;
|
||||
use Flarum\Api\Client;
|
||||
|
|
@ -87,6 +88,11 @@ class Provider extends AbstractServiceProvider
|
|||
return new Api\Client($pipe);
|
||||
}
|
||||
);
|
||||
|
||||
$this->container->tag([
|
||||
Searchers\DiscussionSearcher::class,
|
||||
Searchers\CommentPostSearcher::class,
|
||||
], 'blomstra.search.searchers');
|
||||
}
|
||||
|
||||
public function boot()
|
||||
|
|
@ -103,12 +109,16 @@ class Provider extends AbstractServiceProvider
|
|||
/** @var string|Seeders\Seeder $seeder */
|
||||
foreach ($seeders as $seeder) {
|
||||
$seeder::savingOn($events, function ($model) use ($queue, $seeder) {
|
||||
$queue->pushOn(Job::$onQueue, new SavingJob(Collection::make([$model]), $seeder));
|
||||
$queue->pushOn(Job::$onQueue, new UpdateSearchJob(Collection::make([$model]), $seeder));
|
||||
});
|
||||
|
||||
$seeder::deletingOn($events, function ($model) use ($queue, $seeder) {
|
||||
$queue->pushOn(Job::$onQueue, new DeletingJob(Collection::make([$model]), $seeder));
|
||||
});
|
||||
|
||||
$seeder::viewingOn($events, function (int $discussionId) use ($queue) {
|
||||
$queue->pushOn(Job::$onQueue, new ViewsSearchJob($discussionId));
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -13,11 +13,12 @@
|
|||
namespace Blomstra\Search\Save;
|
||||
|
||||
use Carbon\Carbon;
|
||||
use Illuminate\Support\Arr;
|
||||
use Illuminate\Support\Fluent;
|
||||
|
||||
/**
|
||||
* @property string $type
|
||||
* @property string $id
|
||||
* @property int $rawId
|
||||
* @property string $content
|
||||
* @property Carbon $created_at
|
||||
* @property Carbon $updated_at
|
||||
|
|
@ -30,4 +31,13 @@ use Illuminate\Support\Fluent;
|
|||
*/
|
||||
class Document extends Fluent
|
||||
{
|
||||
/**
|
||||
* Exclude `id` from the body sent to Elasticsearch. The document ID is
|
||||
* already stored as `_id` by the bulk API action; duplicating it in
|
||||
* `_source` wastes space and is redundant.
|
||||
*/
|
||||
public function toArray(): array
|
||||
{
|
||||
return Arr::except($this->attributes, ['id']);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,27 @@
|
|||
<?php
|
||||
|
||||
/*
|
||||
* This file is part of blomstra/search.
|
||||
*
|
||||
* Copyright (c) 2022 Blomstra Ltd.
|
||||
*
|
||||
* For the full copyright and license information, please view the LICENSE.md
|
||||
* file that was distributed with this source code.
|
||||
*
|
||||
*/
|
||||
|
||||
namespace Blomstra\Search\Searchers;
|
||||
|
||||
use Blomstra\Search\Seeders\CommentSeeder;
|
||||
|
||||
class CommentPostSearcher extends Searcher
|
||||
{
|
||||
protected string|null $seeder = CommentSeeder::class;
|
||||
|
||||
public function enabled(): bool
|
||||
{
|
||||
$enabled = $this->setting('blomstra-search.search-post-bodies', true);
|
||||
|
||||
return boolval($enabled);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
<?php
|
||||
|
||||
/*
|
||||
* This file is part of blomstra/search.
|
||||
*
|
||||
* Copyright (c) 2022 Blomstra Ltd.
|
||||
*
|
||||
* For the full copyright and license information, please view the LICENSE.md
|
||||
* file that was distributed with this source code.
|
||||
*
|
||||
*/
|
||||
|
||||
namespace Blomstra\Search\Searchers;
|
||||
|
||||
use Blomstra\Search\Seeders\DiscussionSeeder;
|
||||
|
||||
class DiscussionSearcher extends Searcher
|
||||
{
|
||||
protected string|null $seeder = DiscussionSeeder::class;
|
||||
|
||||
public function enabled(): bool
|
||||
{
|
||||
$enabled = $this->setting('blomstra-search.search-discussion-subjects', true);
|
||||
|
||||
return boolval($enabled);
|
||||
}
|
||||
|
||||
public function boost(): float
|
||||
{
|
||||
return 1.5;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
<?php
|
||||
|
||||
/*
|
||||
* This file is part of blomstra/search.
|
||||
*
|
||||
* Copyright (c) 2022 Blomstra Ltd.
|
||||
*
|
||||
* For the full copyright and license information, please view the LICENSE.md
|
||||
* file that was distributed with this source code.
|
||||
*
|
||||
*/
|
||||
|
||||
namespace Blomstra\Search\Searchers;
|
||||
|
||||
use Blomstra\Search\Seeders\Seeder;
|
||||
use Flarum\Settings\SettingsRepositoryInterface;
|
||||
|
||||
abstract class Searcher
|
||||
{
|
||||
protected string|null $seeder = null;
|
||||
|
||||
public function type(): string
|
||||
{
|
||||
/** @var Seeder $seeder */
|
||||
$seeder = $this->seeder;
|
||||
|
||||
if (empty($seeder)) {
|
||||
throw new \InvalidArgumentException('Implement type or add $seeder');
|
||||
}
|
||||
|
||||
return (new $seeder())->type();
|
||||
}
|
||||
|
||||
public function enabled(): bool
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
public function boost(): float
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
|
||||
protected function setting(string $key, $default = null)
|
||||
{
|
||||
return resolve(SettingsRepositoryInterface::class)->get($key, $default);
|
||||
}
|
||||
}
|
||||
|
|
@ -13,10 +13,11 @@
|
|||
namespace Blomstra\Search\Seeders;
|
||||
|
||||
use Blomstra\Search\Save\Document;
|
||||
use Flarum\Api\Serializer\DiscussionSerializer;
|
||||
use Flarum\Api\Serializer\PostSerializer;
|
||||
use Flarum\Discussion\Discussion;
|
||||
use Flarum\Post\CommentPost;
|
||||
use Flarum\Post\Event\Deleted;
|
||||
use Flarum\Post\Event\Posted;
|
||||
use Flarum\Post\Event as Core;
|
||||
use Illuminate\Contracts\Events\Dispatcher;
|
||||
use Illuminate\Database\Eloquent\Builder;
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
|
|
@ -28,64 +29,68 @@ class CommentSeeder extends Seeder
|
|||
return resolve(PostSerializer::class)->getType(new CommentPost());
|
||||
}
|
||||
|
||||
public function joinRelation(): string
|
||||
{
|
||||
return 'post';
|
||||
}
|
||||
|
||||
public function routing(Model $model): string
|
||||
{
|
||||
return (string) $model->discussion_id;
|
||||
}
|
||||
|
||||
public function query(): Builder
|
||||
{
|
||||
$includes = ['discussion'];
|
||||
|
||||
if ($this->extensionEnabled('flarum-tags')) {
|
||||
$includes[] = 'discussion.tags';
|
||||
}
|
||||
|
||||
if ($this->extensionEnabled('fof-byobu')) {
|
||||
$includes[] = 'discussion.recipientUsers';
|
||||
$includes[] = 'discussion.recipientGroups';
|
||||
}
|
||||
|
||||
return CommentPost::query()
|
||||
->whereNull('hidden_at')
|
||||
->where('type', CommentPost::$type)
|
||||
->with($includes);
|
||||
->where('type', CommentPost::$type);
|
||||
}
|
||||
|
||||
public static function savingOn(Dispatcher $events, callable $callable)
|
||||
{
|
||||
$events->listen(Posted::class, function (Posted $event) use ($callable) {
|
||||
$events->listen([
|
||||
Core\Posted::class,
|
||||
Core\Revised::class,
|
||||
Core\Hidden::class,
|
||||
Core\Restored::class,
|
||||
], function ($event) use ($callable) {
|
||||
$callable($event->post);
|
||||
});
|
||||
}
|
||||
|
||||
public static function deletingOn(Dispatcher $events, callable $callable)
|
||||
{
|
||||
$events->listen(Deleted::class, function (Deleted $event) use ($callable) {
|
||||
$events->listen([
|
||||
Core\Deleted::class
|
||||
], function ($event) use ($callable) {
|
||||
$callable($event->post);
|
||||
});
|
||||
}
|
||||
|
||||
/** Cached discussion type string (e.g. "discussions") — resolved once per job. */
|
||||
private ?string $discussionType = null;
|
||||
|
||||
private function discussionType(): string
|
||||
{
|
||||
return $this->discussionType ??= resolve(DiscussionSerializer::class)->getType(new Discussion());
|
||||
}
|
||||
|
||||
/**
|
||||
* @param CommentPost $model
|
||||
* Post documents only need content for has_child matching and the join
|
||||
* field to establish the parent-child relationship. All discussion-level
|
||||
* fields (groups, tags, comment_count, etc.) live on the discussion
|
||||
* document and are irrelevant here.
|
||||
*
|
||||
* @return Document
|
||||
* @param CommentPost $model
|
||||
*/
|
||||
public function toDocument(Model $model): Document
|
||||
{
|
||||
$document = new Document([
|
||||
'type' => $this->type(),
|
||||
'id' => $this->type().':'.$model->id,
|
||||
'content' => $model->content,
|
||||
'content_partial' => $model->content,
|
||||
'created_at' => $model->created_at?->toAtomString(),
|
||||
'updated_at' => $model->edited_at?->toAtomString(),
|
||||
'is_private' => $model->is_private,
|
||||
'user_id' => $model->user_id,
|
||||
'groups' => $this->groupsForDiscussion($model->discussion),
|
||||
'comment_count' => $model->discussion->comment_count,
|
||||
return new Document([
|
||||
'join_field' => ['name' => $this->joinRelation(), 'parent' => "{$this->discussionType()}:{$model->discussion_id}"],
|
||||
'discussion_id' => $model->discussion_id,
|
||||
'id' => $this->type().':'.$model->id,
|
||||
'rawId' => $model->id,
|
||||
'content' => $model->content,
|
||||
'is_hidden' => $model->hidden_at !== null,
|
||||
]);
|
||||
|
||||
if ($this->extensionEnabled('fof-byobu')) {
|
||||
$document['recipient_users'] = $model->discussion->recipientUsers->pluck('id')->toArray();
|
||||
$document['recipient_groups'] = $model->discussion->recipientGroups->pluck('id')->toArray();
|
||||
}
|
||||
|
||||
return $document;
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -15,12 +15,16 @@ namespace Blomstra\Search\Seeders;
|
|||
use Blomstra\Search\Save\Document;
|
||||
use Flarum\Api\Serializer\DiscussionSerializer;
|
||||
use Flarum\Discussion\Discussion;
|
||||
use Flarum\Discussion\Event\Deleted;
|
||||
use Flarum\Discussion\Event\Hidden;
|
||||
use Flarum\Discussion\Event\Restored;
|
||||
use Flarum\Discussion\Event\Started;
|
||||
use Flarum\Discussion\Event as Core;
|
||||
use Flarum\Extension\ExtensionManager;
|
||||
use Flarum\Group\Group;
|
||||
use Flarum\Group\Permission;
|
||||
use Flarum\Tags\Tag;
|
||||
use FoF\Byobu\Events as Byobu;
|
||||
use FoF\DiscussionViews\Events\DiscussionWasViewed;
|
||||
use Illuminate\Contracts\Events\Dispatcher;
|
||||
use Illuminate\Database\Eloquent\Builder;
|
||||
use Illuminate\Database\Eloquent\Collection;
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
|
||||
class DiscussionSeeder extends Seeder
|
||||
|
|
@ -30,7 +34,23 @@ class DiscussionSeeder extends Seeder
|
|||
return resolve(DiscussionSerializer::class)->getType(new Discussion());
|
||||
}
|
||||
|
||||
public function joinRelation(): string
|
||||
{
|
||||
return 'discussion';
|
||||
}
|
||||
|
||||
public function routing(Model $model): string
|
||||
{
|
||||
return (string) $model->id;
|
||||
}
|
||||
|
||||
public function query(): Builder
|
||||
{
|
||||
return Discussion::query()
|
||||
->whereNull('hidden_at');
|
||||
}
|
||||
|
||||
public function relationships(): array
|
||||
{
|
||||
$includes = [];
|
||||
|
||||
|
|
@ -43,21 +63,48 @@ class DiscussionSeeder extends Seeder
|
|||
$includes[] = 'recipientGroups';
|
||||
}
|
||||
|
||||
return Discussion::query()
|
||||
->whereNull('hidden_at')
|
||||
->with($includes);
|
||||
return $includes;
|
||||
}
|
||||
|
||||
public static function savingOn(Dispatcher $events, callable $callable)
|
||||
{
|
||||
$events->listen([Started::class, Restored::class], function ($event) use ($callable) {
|
||||
$events->listen([
|
||||
// flarum/core events
|
||||
Core\Started::class, Core\Restored::class,
|
||||
// fof/byobu discussion recipients events.
|
||||
Byobu\DiscussionMadePublic::class, Byobu\RemovedSelf::class, Byobu\RecipientsChanged::class,
|
||||
], function ($event) use ($callable) {
|
||||
return $callable($event->discussion);
|
||||
});
|
||||
}
|
||||
|
||||
public static function viewingOn(Dispatcher $events, callable $callable): void
|
||||
{
|
||||
if (!resolve(ExtensionManager::class)->isEnabled('fof-discussion-views')) {
|
||||
return;
|
||||
}
|
||||
|
||||
$events->listen(DiscussionWasViewed::class, function (DiscussionWasViewed $event) use ($callable) {
|
||||
$viewCount = $event->discussion->view_count;
|
||||
|
||||
$shouldSync = match (true) {
|
||||
$viewCount < 15 => true,
|
||||
$viewCount < 100 => rand(1, 3) === 1,
|
||||
default => rand(1, 19) === 1,
|
||||
};
|
||||
|
||||
if ($shouldSync) {
|
||||
$callable($event->discussion->id);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
public static function deletingOn(Dispatcher $events, callable $callable)
|
||||
{
|
||||
$events->listen([Deleted::class, Hidden::class], function ($event) use ($callable) {
|
||||
$events->listen([
|
||||
// flarum/core events.
|
||||
Core\Deleted::class, Core\Hidden::class
|
||||
], function ($event) use ($callable) {
|
||||
return $callable($event->discussion);
|
||||
});
|
||||
}
|
||||
|
|
@ -70,10 +117,10 @@ class DiscussionSeeder extends Seeder
|
|||
public function toDocument(Model $model): Document
|
||||
{
|
||||
$document = new Document([
|
||||
'type' => $this->type(),
|
||||
'join_field' => $this->joinRelation(),
|
||||
'id' => $this->type().':'.$model->id,
|
||||
'rawId' => $model->id,
|
||||
'content' => $model->title,
|
||||
'content_partial' => $model->title,
|
||||
'created_at' => $model->created_at?->toAtomString(),
|
||||
'updated_at' => $model->last_posted_at?->toAtomString(),
|
||||
'is_private' => $model->is_private,
|
||||
|
|
@ -82,15 +129,81 @@ class DiscussionSeeder extends Seeder
|
|||
'comment_count' => $model->comment_count,
|
||||
]);
|
||||
|
||||
if ($this->extensionEnabled('flarum-tags')) {
|
||||
$document['tags'] = $model->tags->pluck('id')->toArray();
|
||||
}
|
||||
|
||||
if ($this->extensionEnabled('fof-byobu')) {
|
||||
$document['recipient_users'] = $model->recipientUsers->pluck('id')->toArray();
|
||||
$document['recipient_groups'] = $model->recipientGroups->pluck('id')->toArray();
|
||||
$document['recipient_users'] = $model->recipientUsers
|
||||
->whereNull('removed_at')
|
||||
->pluck('id')
|
||||
->toArray();
|
||||
$document['recipient_groups'] = $model->recipientGroups
|
||||
->whereNull('removed_at')
|
||||
->pluck('id')
|
||||
->toArray();
|
||||
}
|
||||
|
||||
if ($this->extensionEnabled('flarum-sticky')) {
|
||||
$document['is_sticky'] = (bool) $model->is_sticky;
|
||||
}
|
||||
|
||||
if ($this->extensionEnabled('fof-discussion-views')) {
|
||||
$document['view_count'] = (int) ($model->view_count ?? 0);
|
||||
}
|
||||
|
||||
return $document;
|
||||
}
|
||||
|
||||
/**
|
||||
* All viewForum permissions keyed by permission string, loaded once per seeder instance.
|
||||
* Avoids N×2 Permission queries inside the per-document map loop.
|
||||
*/
|
||||
private ?Collection $cachedPermissions = null;
|
||||
private ?Collection $cachedGlobalPermission = null;
|
||||
|
||||
private function allPermissions(): Collection
|
||||
{
|
||||
if ($this->cachedPermissions === null) {
|
||||
$this->cachedPermissions = Permission::query()
|
||||
->where(function ($q) {
|
||||
$q->where('permission', 'viewForum')
|
||||
->orWhere('permission', 'like', 'tag%.viewForum');
|
||||
})
|
||||
->get();
|
||||
|
||||
$this->cachedGlobalPermission = $this->cachedPermissions
|
||||
->where('permission', 'viewForum')
|
||||
->pluck('group_id');
|
||||
}
|
||||
|
||||
return $this->cachedPermissions;
|
||||
}
|
||||
|
||||
protected function groupsForDiscussion(Discussion $discussion): array
|
||||
{
|
||||
$allPerms = $this->allPermissions();
|
||||
$permissions = collect();
|
||||
|
||||
if ($this->extensionEnabled('flarum-tags')) {
|
||||
/** @var Collection $tags */
|
||||
$tags = $discussion->tags;
|
||||
|
||||
$permissions = $tags->map(function (Tag $tag) use ($allPerms) {
|
||||
$tagPerms = $allPerms->where('permission', "tag$tag->id.viewForum");
|
||||
|
||||
if ($tag->is_restricted) {
|
||||
$tagPerms = $tagPerms->add(['group_id' => Group::ADMINISTRATOR_ID]);
|
||||
}
|
||||
|
||||
return $tagPerms->pluck('group_id');
|
||||
})->flatten();
|
||||
}
|
||||
|
||||
if (!$discussion->is_private && $permissions->isEmpty()) {
|
||||
$permissions = $this->cachedGlobalPermission;
|
||||
}
|
||||
|
||||
return $permissions->toArray();
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -13,65 +13,42 @@
|
|||
namespace Blomstra\Search\Seeders;
|
||||
|
||||
use Blomstra\Search\Save\Document;
|
||||
use Flarum\Discussion\Discussion;
|
||||
use Flarum\Extension\ExtensionManager;
|
||||
use Flarum\Group\Group;
|
||||
use Flarum\Group\Permission;
|
||||
use Flarum\Tags\Tag;
|
||||
use Illuminate\Contracts\Events\Dispatcher;
|
||||
use Illuminate\Database\Eloquent\Builder;
|
||||
use Illuminate\Database\Eloquent\Collection;
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
|
||||
abstract class Seeder
|
||||
{
|
||||
abstract public function type(): string;
|
||||
|
||||
/**
|
||||
* The join relation name for this document type ('discussion' or 'post').
|
||||
* Used in the join_field mapping and for filtering in seed-missing checks.
|
||||
*/
|
||||
abstract public function joinRelation(): string;
|
||||
|
||||
/**
|
||||
* The ES routing key for this model. Parent and child documents for the same
|
||||
* discussion must share a routing key so ES places them on the same shard.
|
||||
*/
|
||||
abstract public function routing(Model $model): string;
|
||||
|
||||
abstract public function query(): Builder;
|
||||
|
||||
abstract public static function savingOn(Dispatcher $events, callable $callable);
|
||||
|
||||
abstract public static function deletingOn(Dispatcher $events, callable $callable);
|
||||
|
||||
/** No-op default; override in seeders that need to react to view events. */
|
||||
public static function viewingOn(Dispatcher $events, callable $callable): void {}
|
||||
|
||||
abstract public function toDocument(Model $model): Document;
|
||||
|
||||
protected function groupsForDiscussion(Discussion $discussion): array
|
||||
/** Relationships to eager-load on the collection before calling toDocument(). */
|
||||
public function relationships(): array
|
||||
{
|
||||
$permissions = collect();
|
||||
|
||||
$globalPermission = Permission::query()
|
||||
->where('permission', 'viewForum')
|
||||
->pluck('group_id');
|
||||
|
||||
if ($this->extensionEnabled('flarum-tags')) {
|
||||
/** @var Collection $tags */
|
||||
$tags = $discussion->tags;
|
||||
|
||||
$filters['tags'] = $tags->pluck('id')->toArray();
|
||||
$tagPermissions = Permission::query()
|
||||
->whereIn(
|
||||
'permission',
|
||||
$tags->pluck('id')->map(function (int $id) {
|
||||
return "tag$id.viewForum";
|
||||
})
|
||||
)->get();
|
||||
|
||||
$permissions = $tags->map(function (Tag $tag) use ($tagPermissions) {
|
||||
$permissions = $tagPermissions->where('permission', "tag$tag->id.viewForum");
|
||||
|
||||
if ($tag->is_restricted) {
|
||||
$permissions = $permissions->add(['group_id' => Group::ADMINISTRATOR_ID]);
|
||||
}
|
||||
|
||||
return $permissions->pluck('group_id');
|
||||
})->flatten();
|
||||
}
|
||||
|
||||
if (!$discussion->is_private && $permissions->isEmpty()) {
|
||||
$permissions = $globalPermission;
|
||||
}
|
||||
|
||||
return $permissions->toArray();
|
||||
return [];
|
||||
}
|
||||
|
||||
protected function extensionEnabled(string $extension): bool
|
||||
|
|
|
|||
Loading…
Reference in New Issue