Skip to content

armgit / memos

Public Imported from GitHub
0

Memos is an open-source, self-hosted note-taking app that brings your everyday thoughts together in a personal timeline. Your data stays yours, and you choose what to keep private or share.

fix(tags): include combining marks in tag character class (#6051)

Santhosh Thottingal281e0dc17c1c
diff --git a/web/src/utils/tag-grammar.ts b/web/src/utils/tag-grammar.ts
index dcd819742..de394b8ea 100644
--- a/web/src/utils/tag-grammar.ts
+++ b/web/src/utils/tag-grammar.ts
@@ -4,10 +4,14 @@
* tagMarkdown.ts) and the read-only renderer (utils/remark-plugins/remark-tag.ts)
* so they can't drift.
*
- * A tag character is any Unicode letter, number, or symbol, plus `_ - / &`.
+ * A tag character is any Unicode letter, mark, number, or symbol, plus
+ * `_ - / &`. The mark class (`\p{M}`) keeps combining marks — Indic vowel
+ * signs, Arabic harakat, Hebrew niqqud, decomposed accents — attached to the
+ * base letters they belong to, so a tag like `#കവിത` isn't cut short at its
+ * first vowel sign.
* A tag run is capped at MAX_TAG_LENGTH characters.
*/
-export const TAG_CHAR_CLASS = "[\\p{L}\\p{N}\\p{S}_\\-/&]";
+export const TAG_CHAR_CLASS = "[\\p{L}\\p{M}\\p{N}\\p{S}_\\-/&]";
export const MAX_TAG_LENGTH = 100;
diff --git a/web/tests/editor-tag.test.ts b/web/tests/editor-tag.test.ts
index 5b7e93928..b5921a412 100644
--- a/web/tests/editor-tag.test.ts
+++ b/web/tests/editor-tag.test.ts
@@ -43,6 +43,16 @@ describe("Tag mark", () => {
expect(tagged?.marks?.find((m) => m.type === "tag")?.attrs?.tag).toBe("work/project-1");
});
+ it("includes combining marks in a tag run", () => {
+ // Malayalam കവിത carries a spacing combining vowel sign (U+0D3F, \p{M});
+ // the tag must cover the whole word, not stop at the first mark.
+ expect(firstParagraphChildren("#കവിത")[0]).toMatchObject({
+ type: "text",
+ text: "#കവിത",
+ marks: [{ type: "tag", attrs: { tag: "കവിത" } }],
+ });
+ });
+
it("does not turn headings into tags", () => {
expect(parseMarkdown("# heading").content?.[0]?.type).toBe("heading");
});
diff --git a/web/tests/remark-tag.test.tsx b/web/tests/remark-tag.test.tsx
index 9b89e8f16..b3fa3de29 100644
--- a/web/tests/remark-tag.test.tsx
+++ b/web/tests/remark-tag.test.tsx
@@ -70,6 +70,15 @@ describe("remarkTag", () => {
expect(html).toContain('data-tag="second"');
});
+ it("tags a whole word containing combining marks", () => {
+ // Malayalam കവിത = ka, va, vowel-sign-i (U+0D3F, a spacing combining mark),
+ // ta. The vowel sign is a \p{M} character, so the tag must not stop at കവ.
+ const html = renderMarkdown("#കവിത");
+
+ expect(html).toContain('data-tag="കവിത"');
+ expect(html).not.toContain('data-tag="കവ"');
+ });
+
it("still tags a hash that shares a text node with an entity reference", () => {
// The source slice ("...&...") differs from the decoded value, so the
// escape-aware path bows out and the tag is detected the original way.