Skip to content

A platform for community discussion. Free, open, simple.

FEATURE: Support subcategories and IDs in researcher category filter (#39877)

Samc8e5ca633bc0
diff --git a/plugins/discourse-ai/lib/agents/forum_researcher.rb b/plugins/discourse-ai/lib/agents/forum_researcher.rb
index 2ad3c12cdcf..18edabfe2fd 100644
--- a/plugins/discourse-ai/lib/agents/forum_researcher.rb
+++ b/plugins/discourse-ai/lib/agents/forum_researcher.rb
@@ -1,4 +1,4 @@
-#frozen_string_literal: true
+# frozen_string_literal: true
module DiscourseAi
module Agents
@@ -24,25 +24,30 @@ module DiscourseAi
Topic URLs are formatted as: /t/-/TOPIC_ID
Post URLs are formatted as: /t/-/TOPIC_ID/POST_NUMBER
- CRITICAL: Research is extremely expensive. You MUST gather ALL research goals upfront and execute them in a SINGLE request. Never run multiple research operations.
+ CRITICAL: Research is extremely expensive. Gather the full research brief upfront, then use at most one dry run and one final research execution. Never run exploratory research calls one question at a time.
As a forum researcher, follow this structured process:
- 1. UNDERSTAND: Clarify ALL research goals - what insights are they seeking?
- 2. PLAN: Design ONE comprehensive research approach covering all objectives
- 3. TEST: Always begin with dry_run:true to gauge the scope of results
- 4. REFINE: If results are too broad/narrow, suggest filter adjustments (but don't re-run yet)
- 5. EXECUTE: Run the final analysis ONCE when filters are well-tuned for all goals
+ 1. UNDERSTAND: Identify all research goals, constraints, and the decision the user is trying to make
+ 2. PLAN: Design one comprehensive filter and goal statement covering every objective
+ 3. TEST: Begin with dry_run:true to estimate the result count before processing content
+ 4. REFINE: If results are too broad or narrow, explain the proposed filter adjustment before the final run
+ 5. EXECUTE: Run the final analysis once, with all goals in a single request
6. SUMMARIZE: Present findings with links to supporting evidence
- Before any research, ask users to specify:
- - ALL research questions they want answered
+ Before research, ask only for missing information that materially affects the filter or goals:
+ - All research questions they want answered
- Time periods of interest
- - Specific users, categories, or tags to focus on
+ - Specific users, groups, categories, or tags to focus on
+ - Whether subcategories should be excluded from category filters
- Expected scope (broad overview vs. deep dive)
Research filter guidelines:
+ - The filter must not be blank; choose the narrowest useful filter for the request
- Use post date filters (after/before) for analyzing specific posts
- Use topic date filters (topic_after/topic_before) for analyzing entire topics
+ - Category filters include subcategories by default, for example category:support
+ - Prefix a category with = to exclude subcategories, for example category:=support
+ - Prefer category slugs or IDs when names are ambiguous; use parent/child for subcategories when needed
- Combine user/group filters with categories/tags to find specialized contributions
When formatting results:
diff --git a/plugins/discourse-ai/lib/agents/tools/researcher.rb b/plugins/discourse-ai/lib/agents/tools/researcher.rb
index dff051fe689..a425a96efd4 100644
--- a/plugins/discourse-ai/lib/agents/tools/researcher.rb
+++ b/plugins/discourse-ai/lib/agents/tools/researcher.rb
@@ -40,7 +40,9 @@ module DiscourseAi
- keywords:word1,word2 - full-text search in post content
- topic_keywords:word1,word2 - full-text search in topics (returns all posts from matching topics)
- topic:123 or topics:123,456 - specific topics by ID
- - category:name1 or categories:name1,name2 - posts in categories (by name/slug)
+ - category:name1 or categories:name1,name2 - posts in categories and subcategories (by name/slug/id)
+ Prefix a category with = to exclude subcategories, for example category:=bugs.
+ Use parent/child for subcategories when a slug is ambiguous, for example category:support/bugs.
- tag:tag1 or tags:tag1,tag2 - posts in topics with tags
- after:YYYY-MM-DD, before:YYYY-MM-DD - filter by post creation date
- topic_after:YYYY-MM-DD, topic_before:YYYY-MM-DD - filter by topic creation date
@@ -53,7 +55,8 @@ module DiscourseAi
Examples:
- 'username:sam after:2023-01-01' - sam's posts after date
- - 'max_results:50 category:bugs OR tag:urgent' - (≤50 bug posts) OR (all urgent posts)
+ - 'max_results:50 category:bugs OR tag:urgent' - (≤50 bug posts, including bug subcategories) OR (all urgent posts)
+ - 'category:=bugs' - posts directly in bugs, excluding bug subcategories
TEXT
end
diff --git a/plugins/discourse-ai/lib/utils/research/filter.rb b/plugins/discourse-ai/lib/utils/research/filter.rb
index 087077d7c5c..6af9d5d273d 100644
--- a/plugins/discourse-ai/lib/utils/research/filter.rb
+++ b/plugins/discourse-ai/lib/utils/research/filter.rb
@@ -16,6 +16,70 @@ module DiscourseAi
::Search.word_to_date(str)
end
+ def self.category_ids_from_param(category_param)
+ category_param
+ .to_s
+ .split(",")
+ .map(&:strip)
+ .reject(&:blank?)
+ .flat_map { |category_value| category_ids_from_value(category_value) }
+ .uniq
+ end
+
+ def self.category_ids_from_value(category_value)
+ exact = category_value.start_with?("=")
+ category_value = category_value[1..] if exact
+ category_value = strip_surrounding_quotes(category_value.strip)
+
+ category = find_category(category_value)
+ return [] if category.blank?
+
+ category_ids = [category.id]
+ category_ids.concat(Category.subcategory_ids(category.id)) if !exact
+ category_ids
+ end
+
+ def self.find_category(category_value)
+ parent_value, child_value = category_value.split(%r{[/:]}, 2)
+
+ if child_value.present?
+ parent_category_ids = matching_categories(parent_value).select(:id)
+
+ matching_categories(child_value)
+ .where(parent_category_id: parent_category_ids)
+ .order("case when parent_category_id is null then 0 else 1 end")
+ .first
+ else
+ matching_categories(category_value).order(
+ "case when parent_category_id is null then 0 else 1 end",
+ ).first
+ end
+ end
+
+ def self.matching_categories(category_value)
+ categories =
+ Category.where(
+ "#{Category.normalize_sql("slug")} = #{Category.normalize_sql("?")} OR " \
+ "#{Category.normalize_sql("name")} = #{Category.normalize_sql("?")}",
+ category_value,
+ category_value,
+ )
+
+ if category_value.match?(/\A\d{1,10}\z/)
+ categories = categories.or(Category.where(id: category_value.to_i))
+ end
+
+ categories
+ end
+
+ def self.strip_surrounding_quotes(value)
+ if value.length >= 2 && value.start_with?("\"") && value.end_with?("\"")
+ value[1...-1]
+ else
+ value
+ end
+ end
+
attr_reader :term, :filters, :order, :guardian, :limit, :offset, :invalid_filters
register_filter(/\Astatus:open\z/i) do |relation, _, _|
@@ -132,24 +196,12 @@ module DiscourseAi
end
register_filter(/\A(?:categories?|category):(.*)\z/i) do |relation, category_param, _|
- if category_param.include?(",")
- category_names = category_param.split(",").map(&:strip)
-
- found_category_ids = []
- category_names.each do |name|
- category = Category.find_by(slug: name) || Category.find_by(name: name)
- found_category_ids << category.id if category
- end
+ category_ids = Filter.category_ids_from_param(category_param)
- return relation.where("1 = 0") if found_category_ids.empty?
- relation.where(topic_id: Topic.where(category_id: found_category_ids).select(:id))
+ if category_ids.empty?
+ relation.where("1 = 0")
else
- if category =
- Category.find_by(slug: category_param) || Category.find_by(name: category_param)
- relation.where(topic_id: Topic.where(category_id: category.id).select(:id))
- else
- relation.where("1 = 0")
- end
+ relation.where(topic_id: Topic.where(category_id: category_ids).select(:id))
end
end
diff --git a/plugins/discourse-ai/spec/lib/utils/research/filter_spec.rb b/plugins/discourse-ai/spec/lib/utils/research/filter_spec.rb
index 1acaea8db07..3cf6f7f06f4 100644
--- a/plugins/discourse-ai/spec/lib/utils/research/filter_spec.rb
+++ b/plugins/discourse-ai/spec/lib/utils/research/filter_spec.rb
@@ -164,10 +164,55 @@ describe DiscourseAi::Utils::Research::Filter do
end
describe "category filtering" do
+ fab!(:announcement_child_category) do
+ Fabricate(
+ :category,
+ name: "Announcements Child",
+ parent_category_id: announcement_category.id,
+ )
+ end
+ fab!(:announcement_child_topic) do
+ Fabricate(
+ :topic,
+ user: user,
+ category: announcement_child_category,
+ title: "Child Category Discussion",
+ )
+ end
+ fab!(:announcement_child_post) do
+ Fabricate(:post, topic: announcement_child_topic, user: user)
+ end
+
it "correctly filters posts by categories" do
filter = described_class.new("category:Announcements")
+ expect(filter.search.pluck(:id)).to contain_exactly(
+ feature_post.id,
+ bug_post.id,
+ announcement_child_post.id,
+ )
+
+ filter = described_class.new("category:=Announcements")
expect(filter.search.pluck(:id)).to contain_exactly(feature_post.id, bug_post.id)
+ filter = described_class.new("category:#{announcement_category.id}")
+ expect(filter.search.pluck(:id)).to contain_exactly(
+ feature_post.id,
+ bug_post.id,
+ announcement_child_post.id,
+ )
+
+ filter =
+ described_class.new(
+ "category:#{announcement_category.slug}/#{announcement_child_category.slug}",
+ )
+ expect(filter.search.pluck(:id)).to contain_exactly(announcement_child_post.id)
+
+ filter = described_class.new('category:"Announcements Child"')
+ expect(filter.search.pluck(:id)).to contain_exactly(announcement_child_post.id)
+
+ filter = described_class.new('category:="Announcements Child"')
+ expect(filter.search.pluck(:id)).to contain_exactly(announcement_child_post.id)
+
# it can tack on topics
filter =
described_class.new(
@@ -176,6 +221,7 @@ describe DiscourseAi::Utils::Research::Filter do
expect(filter.search.pluck(:id)).to contain_exactly(
feature_post.id,
bug_post.id,
+ announcement_child_post.id,
feature_bug_post.id,
no_tag_post.id,
)
@@ -184,6 +230,7 @@ describe DiscourseAi::Utils::Research::Filter do
expect(filter.search.pluck(:id)).to contain_exactly(
feature_post.id,
bug_post.id,
+ announcement_child_post.id,
feature_bug_post.id,
no_tag_post.id,
)