feat: add organized-feedback skill with docs and validation

This commit is contained in:
2026-04-09 15:28:26 +08:00
parent ba0edce38c
commit c0717e9a24
8 changed files with 674 additions and 4 deletions
+2
View File
@@ -67,6 +67,8 @@ This repository includes a reusable skill for other LLMs to operate this CLI cor
- Path: `skills/gitea-pr-review-cli/SKILL.md`
- Scope: standardized usage of `fetch` / `render-md` / `version`, `v1` version contract, and troubleshooting
- Path: `skills/organized-feedback/SKILL.md`
- Scope: organize `gitea-pr-review` PR comments into structured feedback with coverage auditing and unknown-item handling
## Markdown Output Shape (Example)
+2
View File
@@ -67,6 +67,8 @@ gitea-pr-review version
- 路径:`skills/gitea-pr-review-cli/SKILL.md`
- 用途:标准化 `fetch` / `render-md` / `version` 的调用方式、`v1` 版本约束和常见错误排查
- 路径:`skills/organized-feedback/SKILL.md`
- 用途:将 `gitea-pr-review` 生成的 PR 评论整理为结构化反馈,并执行覆盖审计与未知项处理
## Markdown 输出结构(示例)
@@ -0,0 +1,375 @@
# Organized Feedback Skill Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add a reusable skill that organizes noisy PR review comments into structured feedback with mandatory coverage auditing in interaction and file output to a user-specified path.
**Architecture:** Implement this feature as a pure prompt skill under `skills/` with explicit workflow gates: input validation, classify/group, coverage audit loop, unknown reflection loop, and output write gate. Keep runtime behavior in instructions (no binary changes), and provide fixtures plus a validation script so contributors can quickly verify the skill contract after edits.
**Tech Stack:** Markdown skill spec, Bash validation script, existing repository docs
---
## File Structure
- Create: `skills/organized-feedback/SKILL.md`
- Create: `skills/organized-feedback/examples/input-pr-review.md`
- Create: `skills/organized-feedback/examples/output-organized-feedback.md`
- Create: `scripts/validate_organized_feedback_skill.sh`
- Modify: `README.md`
- Modify: `README.en.md`
### Task 1: Create the skill contract file with hard gates
**Files:**
- Create: `skills/organized-feedback/SKILL.md`
- Test: `scripts/validate_organized_feedback_skill.sh`
- [ ] **Step 1: Write the failing validator script first (checks missing skill file)**
```bash
#!/usr/bin/env bash
set -euo pipefail
SKILL_FILE="skills/organized-feedback/SKILL.md"
if [[ ! -f "$SKILL_FILE" ]]; then
echo "FAIL: missing $SKILL_FILE"
exit 1
fi
echo "PASS: skill file exists"
```
- [ ] **Step 2: Run validator to verify it fails before skill creation**
Run: `bash scripts/validate_organized_feedback_skill.sh`
Expected: `FAIL: missing skills/organized-feedback/SKILL.md`
- [ ] **Step 3: Create `SKILL.md` metadata and workflow skeleton**
```markdown
---
name: organized-feedback
description: Organize unstructured PR review comments into classified feedback with mandatory coverage checks and unknown-item reflection.
---
# Organized Feedback Skill
## Overview
Use this skill to convert `gitea-pr-review` markdown into `Organized Feedback` markdown.
## Hard Gates
- Do not write final output until user provides output path.
- Coverage audit must run in interaction; every source comment/reply must be covered.
- Coverage audit content must not be written to final document.
- If unknown count is greater than 3, pause and request user permission before writing file.
```
- [ ] **Step 4: Extend validator with required hard-gate checks and run it**
```bash
#!/usr/bin/env bash
set -euo pipefail
SKILL_FILE="skills/organized-feedback/SKILL.md"
[[ -f "$SKILL_FILE" ]] || { echo "FAIL: missing $SKILL_FILE"; exit 1; }
required_patterns=(
"Do not write final output until user provides output path"
"Coverage audit must run in interaction"
"Coverage audit content must not be written to final document"
"If unknown count is greater than 3, pause and request user permission"
)
for p in "${required_patterns[@]}"; do
if ! rg -Fq "$p" "$SKILL_FILE"; then
echo "FAIL: missing required rule -> $p"
exit 1
fi
done
echo "PASS: organized-feedback hard gates validated"
```
Run: `bash scripts/validate_organized_feedback_skill.sh`
Expected: `PASS: organized-feedback hard gates validated`
- [ ] **Step 5: Commit Task 1**
```bash
git add skills/organized-feedback/SKILL.md scripts/validate_organized_feedback_skill.sh
git commit -m "feat(skill): scaffold organized-feedback skill with hard gates"
```
### Task 2: Implement classification, grouping, and audit workflow in skill instructions
**Files:**
- Modify: `skills/organized-feedback/SKILL.md`
- Test: `scripts/validate_organized_feedback_skill.sh`
- [ ] **Step 1: Add failing checks for taxonomy and audit loop requirements**
```bash
# append to scripts/validate_organized_feedback_skill.sh
required_patterns+=(
"question"
"supplement"
"request-for-change"
"unknown"
"Change-Scope"
"Necessity"
"Missing-Source-Refs"
"unknown>=1),必须执行一次反思复判"
)
```
- [ ] **Step 2: Run validator to verify it fails on missing workflow details**
Run: `bash scripts/validate_organized_feedback_skill.sh`
Expected: FAIL mentioning at least one missing taxonomy/audit pattern.
- [ ] **Step 3: Fill `SKILL.md` with full workflow and output format contract**
```markdown
## Inputs
Required:
- PR review markdown (with Review/Comment/Reply)
- output path for organized feedback markdown
## Classification Taxonomy
- Type: `question | supplement | request-for-change | unknown`
- For `request-for-change` only:
- `Change-Scope`: `local | implement | api-change | requirement-change`
- `Necessity`: `nice-to-have | should-fix | must-fix`
## Processing Flow
1. Parse all source comments/replies into stable refs like `R1.C2`, `R1.C2.P1`.
2. Create organized items (split/merge is allowed).
3. Run coverage audit in interaction:
- show `All-Source-Refs`
- show `Covered-Source-Refs`
- show `Missing-Source-Refs`
4. If missing refs exist, add items and re-audit until missing set is empty.
5. If unknown>=1),必须执行一次反思复判.
6. If unknown count > 3, pause and request user permission before writing output file.
## Final File Format
Write only:
- `## Organized Feedback`
- optional `## Unknown Items`
Do not write coverage audit sections into the output file.
```
- [ ] **Step 4: Re-run validator and ensure full pass**
Run: `bash scripts/validate_organized_feedback_skill.sh`
Expected: `PASS: organized-feedback hard gates validated`
- [ ] **Step 5: Commit Task 2**
```bash
git add skills/organized-feedback/SKILL.md scripts/validate_organized_feedback_skill.sh
git commit -m "feat(skill): add classification and coverage audit workflow"
```
### Task 3: Add realistic examples for maintainers and prompt tuning
**Files:**
- Create: `skills/organized-feedback/examples/input-pr-review.md`
- Create: `skills/organized-feedback/examples/output-organized-feedback.md`
- Test: `scripts/validate_organized_feedback_skill.sh`
- [ ] **Step 1: Add failing validator checks for examples existence**
```bash
example_files=(
"skills/organized-feedback/examples/input-pr-review.md"
"skills/organized-feedback/examples/output-organized-feedback.md"
)
for f in "${example_files[@]}"; do
[[ -f "$f" ]] || { echo "FAIL: missing example file -> $f"; exit 1; }
done
```
- [ ] **Step 2: Run validator to confirm failure before adding examples**
Run: `bash scripts/validate_organized_feedback_skill.sh`
Expected: `FAIL: missing example file -> ...`
- [ ] **Step 3: Add a compact but complete input example**
```markdown
# org/repo `#42` Improve transaction handling
## Review 42.1 (COMMENT)
> reviewer-a
### Comment 42.1.1
service/order.rs:87
reviewer-a:
```md
这里为什么要在循环里每次开启事务?
```
### Reply 42.1.1.1
service/order.rs:87
author:
```md
为了保证每个子任务互不影响。
```
## Review 42.2 (REQUEST_CHANGES)
> reviewer-b
### Comment 42.2.1
api/order.ts:15
reviewer-b:
```md
建议把 `createOrder(input)` 改成 `createOrder(ctx, input)`,否则审计信息拿不到。
```
```
- [ ] **Step 4: Add output example that excludes coverage audit and includes optional unknown section**
```markdown
## Organized Feedback
### Item 1
- Type: question
- Summary: reviewer-a 询问为何循环内重复开启事务。
- Rationale: 该意见请求解释现有设计意图,不是直接改动请求。
- Source-Refs: R1.C1
- Raw-Excerpts: "这里为什么要在循环里每次开启事务?"
### Item 2
- Type: request-for-change
- Change-Scope: api-change
- Necessity: should-fix
- Summary: reviewer-b 建议 API 增加 `ctx` 参数以支持审计信息透传。
- Rationale: 该意见明确要求接口签名调整,影响调用方。
- Source-Refs: R2.C1
- Raw-Excerpts: "建议把 createOrder(input) 改成 createOrder(ctx, input)"
## Unknown Items
### Unknown 1
- Source-Refs: R1.C1.P1
- Reason: 回复仅陈述现状,缺少审阅者意图,不足以单独归类。
- Needed-Info: 需要确认该回复是否用于反驳、接受或补充审阅意见。
```
- [ ] **Step 5: Run validator and commit Task 3**
Run: `bash scripts/validate_organized_feedback_skill.sh`
Expected: `PASS: organized-feedback hard gates validated`
```bash
git add skills/organized-feedback/examples scripts/validate_organized_feedback_skill.sh
git commit -m "docs(skill): add organized-feedback input and output examples"
```
### Task 4: Document skill discoverability in README files
**Files:**
- Modify: `README.md`
- Modify: `README.en.md`
- [ ] **Step 1: Add Chinese README entry under skill section**
```markdown
## 给 LLM 的 Skill
仓库内已提供两个可复用 skill
- `skills/gitea-pr-review-cli/SKILL.md`:标准化 PR 抓取与渲染流程
- `skills/organized-feedback/SKILL.md`:将 review/comment/reply 整理为 `Organized Feedback`,并执行覆盖审计与 unknown 反思机制
```
- [ ] **Step 2: Add matching English README entry**
```markdown
## Skills for LLM Agents
This repository provides reusable skills:
- `skills/gitea-pr-review-cli/SKILL.md`: standardized CLI usage for fetch/render/version flows
- `skills/organized-feedback/SKILL.md`: organizes review/comment/reply text into `Organized Feedback` with coverage auditing and unknown-item reflection
```
- [ ] **Step 3: Verify docs include new skill references**
Run: `rg -n "organized-feedback" README.md README.en.md`
Expected: both files contain exactly one bullet for the new skill.
- [ ] **Step 4: Commit Task 4**
```bash
git add README.md README.en.md
git commit -m "docs: document organized-feedback skill in readmes"
```
### Task 5: Final verification and delivery
**Files:**
- Modify: `skills/organized-feedback/SKILL.md`
- Modify: `scripts/validate_organized_feedback_skill.sh`
- Modify: `README.md`
- Modify: `README.en.md`
- [ ] **Step 1: Run full validation commands**
Run:
```bash
bash scripts/validate_organized_feedback_skill.sh
rg -n "Coverage Audit" skills/organized-feedback/examples/output-organized-feedback.md || true
```
Expected:
- validator prints pass
- second command prints no matches (final output example must not contain coverage audit block)
- [ ] **Step 2: Manual acceptance walkthrough**
Run this checklist during a dry-run prompt session:
1. Start skill without output path -> confirm it asks for path and pauses.
2. Provide path and source markdown -> confirm it prints coverage audit in interaction.
3. Provide sample producing unknown count 1 -> confirm reflection pass runs.
4. Provide sample producing unknown count 4 -> confirm it asks user permission before writing.
Expected: all four checks succeed.
- [ ] **Step 3: Final commit and summary**
```bash
git add skills/organized-feedback scripts/validate_organized_feedback_skill.sh README.md README.en.md
git commit -m "feat(skill): add organized-feedback workflow with audit and guardrails"
```
Run: `git show --name-only --oneline -n 1`
Expected: only skill/docs/validator files included.
## Self-Review
### Spec coverage
- Input path gate: covered in Task 1 + Task 5 walkthrough.
- Classification taxonomy: covered in Task 2.
- Split/merge freedom + source refs: covered in Task 2.
- Coverage audit interaction-only: covered in Task 2 + Task 5.
- Unknown reflection and >3 permission gate: covered in Task 2 + Task 5.
- Output format (`Organized Feedback` + optional `Unknown Items`): covered in Task 2 and Task 3 example.
### Placeholder scan
- No `TBD`/`TODO` placeholders in tasks.
- Every command step contains explicit command and expected result.
### Type consistency
- Taxonomy keys are consistent across tasks:
- `Type`
- `Change-Scope`
- `Necessity`
- `Source-Refs`
- `Unknown Items`
@@ -1,4 +1,4 @@
# 意见整理 Skill 设计v1
# 意见整理 Skill 设计
## 1. 背景与目标
@@ -10,9 +10,6 @@
- 强制执行覆盖审计,保证不遗漏任何原始 `Comment/Reply`
- 最终输出写入用户指定路径。
约束:
- 当前文档版本规则固定为 `v1`,不做版本 bump。
## 2. 范围
本 Skill 仅负责:
+142
View File
@@ -0,0 +1,142 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
SKILL_FILE="$REPO_ROOT/skills/organized-feedback/SKILL.md"
INPUT_EXAMPLE="$REPO_ROOT/skills/organized-feedback/examples/input-pr-review.md"
OUTPUT_EXAMPLE="$REPO_ROOT/skills/organized-feedback/examples/output-organized-feedback.md"
if ! command -v rg >/dev/null 2>&1; then
echo "FAIL: rg is required but was not found in PATH"
exit 1
fi
if [[ ! -f "$SKILL_FILE" ]]; then
echo "FAIL: missing $SKILL_FILE"
exit 1
fi
for example_file in "$INPUT_EXAMPLE" "$OUTPUT_EXAMPLE"; do
if [[ ! -f "$example_file" ]]; then
echo "FAIL: missing $example_file"
exit 1
fi
done
anchored_patterns=(
'^## Inputs$'
'^## Hard Gates$'
'^## Classification Taxonomy$'
'^## RFC Subfields$'
'^## Processing Flow$'
'^## Interaction-Only Coverage Audit$'
'^## Unknown Handling$'
'^## Final File Format$'
'^## Output Discipline$'
'^- `Change-Scope`: how broad the requested change is\.$'
'^ - `local`$'
'^ - `implement`$'
'^ - `api-change`$'
'^ - `requirement-change`$'
'^- `Necessity`: how strongly the change is required\.$'
'^ - `nice-to-have`$'
'^ - `should-fix`$'
'^ - `must-fix`$'
'^- `All-Source-Refs`: every source reference found in the interaction$'
'^- `Covered-Source-Refs`: refs that are represented in the organized output$'
'^- `Missing-Source-Refs`: refs that are present in the interaction but not yet covered$'
'^Source reference format rule:$'
'^- Use `R` \+ PR numbering path from the source markdown\.$'
'^- Example mapping: `Comment 42\.1\.1` -> `R42\.1\.1`, `Reply 42\.1\.1\.1` -> `R42\.1\.1\.1`\.$'
)
for pattern in "${anchored_patterns[@]}"; do
if ! rg -n -q --pcre2 "$pattern" "$SKILL_FILE"; then
echo "FAIL: missing required rule -> $pattern"
exit 1
fi
done
for pattern in "All refs" "Covered refs" "Missing refs"; do
if rg -Fq "$pattern" "$SKILL_FILE"; then
echo "FAIL: legacy alias found -> $pattern"
exit 1
fi
done
for pattern in "output path" "interaction" "coverage audit" "unknown count" "reflection pass" "Unknown Items"; do
if ! rg -Fq -- "$pattern" "$SKILL_FILE"; then
echo "FAIL: missing required rule -> $pattern"
exit 1
fi
done
if ! rg -Fq -- "unknown>=1" "$SKILL_FILE" || ! rg -Fq -- "反思复判" "$SKILL_FILE"; then
echo "FAIL: missing unknown reflection rule semantics (unknown>=1 + 反思复判)"
exit 1
fi
if rg -Fq "Coverage Audit" "$OUTPUT_EXAMPLE"; then
echo "FAIL: coverage audit must not appear in $OUTPUT_EXAMPLE"
exit 1
fi
if ! rg -n -q --pcre2 '^## Organized Feedback$' "$OUTPUT_EXAMPLE"; then
echo "FAIL: missing ## Organized Feedback in $OUTPUT_EXAMPLE"
exit 1
fi
if ! rg -Fq "Source-Refs:" "$OUTPUT_EXAMPLE"; then
echo "FAIL: missing Source-Refs: line in $OUTPUT_EXAMPLE"
exit 1
fi
if ! rg -n -q --pcre2 '^- Source-Refs: R[0-9]+(\.[0-9]+)+(, R[0-9]+(\.[0-9]+)+)*$' "$OUTPUT_EXAMPLE"; then
echo "FAIL: Source-Refs lines must use R-prefixed numeric reference format"
exit 1
fi
awk '
BEGIN {
in_section = 0
item_count = 0
item_has_source_refs = 0
saw_item = 0
}
/^## Organized Feedback$/ {
in_section = 1
next
}
in_section && /^## / {
if (saw_item && !item_has_source_refs) {
bad = 1
}
exit
}
in_section {
if ($0 ~ /^### Item [0-9]+$/) {
if (saw_item && !item_has_source_refs) {
bad = 1
}
saw_item = 1
item_has_source_refs = 0
item_count++
next
}
if ($0 ~ /^- Source-Refs:/ && saw_item) {
item_has_source_refs = 1
}
}
END {
if (in_section && saw_item && !item_has_source_refs) {
bad = 1
}
exit bad
}
' "$OUTPUT_EXAMPLE" || {
echo "FAIL: each ### Item in Organized Feedback must include a Source-Refs line"
exit 1
}
echo "PASS: organized-feedback hard gates validated"
+83
View File
@@ -0,0 +1,83 @@
---
name: organized-feedback
description: Organize PR review comments into structured feedback with coverage auditing and unknown-item handling. Use when converting gitea-pr-review Markdown into a user-specified output document.
---
# Organized Feedback
## Overview
Use this skill to turn a single PR review document into organized feedback.
## Inputs
- PR markdown
- output path
## Hard Gates
- Do not write final output until user provides output path.
- Coverage audit must run in interaction.
- Coverage audit content must not be written to final document.
- If unknown count is greater than 3, pause and request user permission.
## Classification Taxonomy
- `question`: a review item that asks for clarification, confirmation, or intent.
- `supplement`: a review item that adds missing context, examples, or supporting detail without changing the requested behavior.
- `request-for-change`: a review item that asks for a code, behavior, or design change.
- `unknown`: a review item that cannot be classified confidently after one reflection pass.
## RFC Subfields
When a review item is classified as `request-for-change`, annotate it with:
- `Change-Scope`: how broad the requested change is.
- `local`
- `implement`
- `api-change`
- `requirement-change`
- `Necessity`: how strongly the change is required.
- `nice-to-have`
- `should-fix`
- `must-fix`
## Processing Flow
1. Read the PR markdown and identify each review comment or feedback unit.
2. Classify each unit as `question`, `supplement`, `request-for-change`, or `unknown`.
3. If a unit clearly contains multiple intents, split it into separate items before grouping.
4. If multiple units express the same intent, merge them into one grouped item.
5. For each `request-for-change`, assign `Change-Scope` and `Necessity`.
6. Run an interaction-only coverage audit using `All-Source-Refs`, `Covered-Source-Refs`, and `Missing-Source-Refs`.
7. Re-audit until `Missing-Source-Refs` is empty.
8. Apply the unknown reflection rule once before finalizing any `unknown` item.
9. If the count of `unknown` items is greater than 3, stop and request user permission before proceeding.
## Interaction-Only Coverage Audit
The coverage audit must be performed only against the interaction content, not the final document.
Track these lists explicitly:
- `All-Source-Refs`: every source reference found in the interaction
- `Covered-Source-Refs`: refs that are represented in the organized output
- `Missing-Source-Refs`: refs that are present in the interaction but not yet covered
Source reference format rule:
- Use `R` + PR numbering path from the source markdown.
- Example mapping: `Comment 42.1.1` -> `R42.1.1`, `Reply 42.1.1.1` -> `R42.1.1.1`.
Audit rules:
- Compare only against interaction content.
- Re-audit after every grouping pass until `Missing-Source-Refs` is empty.
- Do not treat the final document as an audit source.
## Unknown Handling
- unknown>=1),必须执行一次反思复判
- Use one reflection pass to decide whether an item can be reclassified into `question`, `supplement`, or `request-for-change`.
- If it remains `unknown`, keep it in the `Unknown Items` section.
- If the number of `unknown` items is greater than 3, request user permission before continuing.
## Final File Format
Write only the organized output content:
- `Organized Feedback`
- optional `Unknown Items`
Do not write the coverage audit into the final output file.
## Output Discipline
- Preserve the user-specified output path.
- Keep the final file limited to the agreed format.
- Do not add extra audit notes, scratch work, or intermediate classification logs to the final file.
@@ -0,0 +1,45 @@
# org/repo `#42` Improve transaction handling
## Review 42.1 (COMMENT)
> reviewer-a
### Comment 42.1.1
service/order.rs:87
reviewer-a:
```md
为什么要在循环里每次开启事务?
```
### Reply 42.1.1.1
service/order.rs:87
author:
```md
为了保证每个子任务互不影响。
```
## Review 42.2 (REQUEST_CHANGES)
> reviewer-b
### Comment 42.2.1
api/order.ts:15
reviewer-b:
```md
建议把 `createOrder(input)` 改成 `createOrder(ctx, input)`,否则审计信息拿不到。
```
## Review 42.3 (COMMENT)
> reviewer-c
### Comment 42.3.1
service/order.rs:120
reviewer-c:
```md
这里的缓存失效策略是不是太激进了?
```
### Reply 42.3.1.1
service/order.rs:120
author:
```md
按之前的约定处理。
```
@@ -0,0 +1,24 @@
## Organized Feedback
### Item 1
- Type: question
- Summary: reviewer-a 询问为什么在循环里重复开启事务。
- Rationale: 该意见在确认设计意图,不是直接要求代码修改。
- Source-Refs: R42.1.1
- Raw-Excerpts: "为什么要在循环里每次开启事务?"
### Item 2
- Type: request-for-change
- Change-Scope: api-change
- Necessity: should-fix
- Summary: reviewer-b 建议把 `createOrder(input)` 改成 `createOrder(ctx, input)` 以便传递审计信息。
- Rationale: 该意见明确要求修改 API 签名,影响调用方。
- Source-Refs: R42.2.1
- Raw-Excerpts: "建议把 `createOrder(input)` 改成 `createOrder(ctx, input)`,否则审计信息拿不到。"
## Unknown Items
### Unknown 1
- Source-Refs: R42.3.1, R42.3.1.1
- Reason: 回复只是“按之前的约定处理”,缺少足够上下文来判断是接受建议、补充说明还是保持现状。
- Needed-Info: 需要确认这条回复是否在响应审阅意见,还是仅说明已有实现策略。