gmgauthier/grokkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 11 Wiki Activity

feature/recipe_implementation #5

Merged
gmgauthier merged 54 commits from feature/recipe_implementation into master 2026-03-07 23:17:54 +00:00
Conversation 0 Commits 54 Files Changed 38 +2215 -534
gmgauthier commented 2026-03-07 22:55:57 +00:00
Owner
Copy Link

Summary

This PR introduces the full Recipe Engine — a powerful, transactional, markdown-driven way to run multi-step LLM workflows.

Key Features

  • Recipe system — Define complex, auditable workflows in normal .md files (project-local or global)
  • Parameters — Pass values with --param key=value (e.g. project_name, package_path, dry_run)
  • Generic file discovery — Controlled by project_languages, extensions, and search_pattern in the recipe frontmatter
  • Safe read-only shell access — Explicit "Read-Only Shell" steps with per-command user confirmation
  • Strong safety boundaries — Global ~/.config/grokkit/safe_shell_commands.yaml whitelist + path boundary enforcement
  • Comprehensive documentation — New docs/user-guide/ section with detailed guides

Why this matters

Recipes let you harness Grok for sophisticated, repeatable tasks (refactoring, test generation, analysis, etc.) while keeping the user fully in control and the system safe.

Testing

  • All existing tests pass
  • result-refactor recipe works end-to-end
  • Read-only shell steps now prompt for confirmation
  • Unsafe commands are rejected at load time

This closes the long-running recipe implementation effort.

Ready for review and merge.

### Summary This PR introduces the full **Recipe Engine** — a powerful, transactional, markdown-driven way to run multi-step LLM workflows. #### Key Features - **Recipe system** — Define complex, auditable workflows in normal `.md` files (project-local or global) - **Parameters** — Pass values with `--param key=value` (e.g. `project_name`, `package_path`, `dry_run`) - **Generic file discovery** — Controlled by `project_languages`, `extensions`, and `search_pattern` in the recipe frontmatter - **Safe read-only shell access** — Explicit "Read-Only Shell" steps with per-command user confirmation - **Strong safety boundaries** — Global `~/.config/grokkit/safe_shell_commands.yaml` whitelist + path boundary enforcement - **Comprehensive documentation** — New `docs/user-guide/` section with detailed guides #### Why this matters Recipes let you harness Grok for sophisticated, repeatable tasks (refactoring, test generation, analysis, etc.) while keeping the user fully in control and the system safe. #### Testing - All existing tests pass - `result-refactor` recipe works end-to-end - Read-only shell steps now prompt for confirmation - Unsafe commands are rejected at load time --- **This closes the long-running recipe implementation effort.** Ready for review and merge.
gmgauthier added 54 commits 2026-03-07 22:55:58 +00:00 
Implement recipe loading, parsing, and running infrastructure.
Add initial result-refactor recipe for converting Go error handling to monadic Result[T] style.
Includes markdown recipe definition, YAML frontmatter parsing, step extraction, and basic runner.
Simplify the subRe regular expression by using a non-capturing group for the section labels instead of alternation, and switch the lookahead to a non-capturing group for consistency. Add comments to document the purpose of stepRe and subRe for better maintainability.
Implement the `recipe` command with `run` subcommand for executing transactional recipes.

- Resolve recipe paths: explicit, project-local (.grokkit/recipes), or global (~/.local/share/grokkit/recipes) with user prompt.
- Load and parse recipes using internal/recipe package.
- Integrate with root command and project root detection.
- Updated import path for recipe package to match module name.
- Added error checking for fmt.Scanln to handle potential input errors gracefully.
- Simplify import comments and error handling in cmd/recipe.go
- Streamline regex comment in internal/recipe/loader.go
- Enhance console output and add LLM placeholder in internal/recipe/runner.go
Introduces a new Markdown file in todo/doing that outlines the recipe feature implementation, including key components, parsing logic, CLI integration, and progress checklist.
Add support for executing recipe steps via Grok API streaming, including parameter defaults from YAML, model selection via flags/config, previous step context, and a final summary prompt. Update runner to handle client and model, and refine loader to apply user params with fallbacks.
- Enhance recipe parsing in loader.go: extract overview, use split-based step extraction to avoid duplicates, refine final summary handling, and clean up comments/templates.
- Refine runner.go prompts: add Grok system message, simplify user prompts for conciseness, adjust result joining with separators, and remove unnecessary text.
- Introduce special case in Runner.Run() for steps containing "apply" or "patch" in title.
- Add handleApplyStep to parse code blocks from previous results, support dry-run patch creation, or real application with user confirmation and backups.
- Implement extractCodeBlocks using regex to identify labelled Go code blocks.
- Add createUnifiedPatch for generating unified diff patches in dry-run mode.
- Remove final summary prompt and streamline recipe completion.
- Adjust system prompt for clarity.
- Change default package_path from internal/service to internal for wider applicability.
- Add instruction to output a specific message if no files are found or path does not exist in Step 1.
- Restrict filesystem interactions to apply/patch steps exclusively.
- Remove real apply logic with user confirmation; default to creating a patch file in dry-run mode.
- Update prompts, comments, and regex for better clarity and precision.
- Add unit tests for the extractCodeBlocks function to ensure reliable parsing.
Update the regex in runner.go to use string concatenation for better readability.
Add test output files as fixtures for test verification.
Replace regex-based sub-section extraction with label-indexed parsing to
handle multi-line content, blank lines, and ensure reliability without
duplicates. Add clarifying comments for templates and summary extraction.
- Introduce discoverFiles function to scan Go files in 'internal' for error handling patterns.
- Add special case for "discover" or "find" steps to perform filesystem scans.
- Refine LLM prompting to enforce strict output format and shorten system prompt.
- Update apply/patch handling and unified patch creation with simplifications.
- Import bufio for potential future use and adjust regex for code block extraction.
- Adjust bufio import to blank (likely for side effects or pending use)
- Refine comments for clarity and remove unnecessary ones
- Split regex string to avoid backtick collisions in literals
- Add error handling to patch writing in createUnifiedPatch
- Minor formatting and defer close adjustments
- Update discoverFiles comment to be more concise.
- Enhance blockRe regex to optionally match leading "// " for filenames, supporting varied Grok output formats.
- Revise handleApplyStep comment to reflect regex changes.
- Simplify createUnifiedPatch by removing unnecessary error checks on fmt.Fprintf and defer closure.
Adjust the regex in handleApplyStep to flexibly match both old and new Grok formats for code blocks. Also, remove blank import for bufio as it's now used.
- Blank import bufio to avoid unused warnings.
- Refine regex to handle double-quoted format and fix backtick issues.
- Update comments for clarity on regex changes.
- Update result-refactor.md to output JSON array of file changes instead of code blocks
- Modify runner.go to parse JSON directly, removing regex-based extraction
- Add project_languages and extensions to recipe metadata
- Improve error handling and output consistency in steps
- Move FileChange struct to top-level for better organization.
- Enhance createUnifiedPatch with proper error handling on writes.
- Remove unused bufio import and update comments.
- Delete obsolete runner_test.go file.
Add refactorFiles to process discovered files individually, generating small JSON responses per file to avoid truncation. Update handleApplyStep to collect and parse multiple single-file JSONs into a unified patch. Switch discoverFiles comment to reflect real scanning. Add fallback default in Run for other steps.
Changed the default value of the package_path parameter in result-refactor.md from "internal" to "internal/git".
- Introduce refactorJSONs to collect pure JSON from refactor steps for apply.
- Update discoverFiles to respect recipe's package_path parameter.
- Refine handleApplyStep to parse and apply changes more robustly.
- Remove outdated test output files.
- Introduce --param/-p flag to pass key=value parameters to recipes, parsed into a map with basic bool handling.
- Expand findProjectRoot to detect more project types (e.g., .gitignore, pyproject.toml, CMakeLists.txt).
- Clean up comments and minor refactoring in recipe resolution logic.
- Enhance boolean parsing in runRecipe to handle variations like "1", "yes", "on" for true, and "0", "no", "off" for false, satisfying staticcheck.
- Reorganize and simplify findProjectRoot by removing checks for pyproject.toml and CMakeLists.txt, and adjusting .grokkit position.
- Build allowed extensions from recipe languages and extensions map
- Generalize filepath walking to filter by allowed extensions instead of hardcoding .go
- Retain Go-specific content check for now; can generalize later
- Simplify refactor JSON handling and patch creation by removing redundant comments and error checks
- Adjust comments and minor formatting for clarity
Introduce ProjectLanguages and Extensions fields to the Recipe struct
to support option 2 for generic file discovery. Also update comments
for internal fields populated by the loader.
Handle potential errors from file writes in createUnifiedPatch to prevent silent failures.
Introduce ResolvedParams field to Recipe struct for storing resolved
parameter values from defaults and user overrides. Update loader to
populate and use it for template rendering. Adjust runner to use
ResolvedParams for root path and generalize file discovery.
Enhance discoverFiles to respect explicit package_path param if provided.
If no param is given, intelligently default to 'src' directory if it exists,
otherwise fall back to project root (.).
Update discoverFiles to leverage recipe metadata for extensions and apply smart defaults more cleanly. Generalize logic with comments for future improvements, while retaining err != nil check for now.
- Add SearchPattern field to Recipe struct
- Update discoverFiles to use configurable search pattern, defaulting to "if err != nil"
- Set default search_pattern in result-refactor.md recipe
Changes the comment to describe the default as "Go-style" instead of the specific string for better generality.
- Added new template-recipe.md with structure for creating recipes.
- Changed "**Final Summary**" to "### Final Summary" in result-refactor.md for better formatting.
Update .gitignore to ignore new directories and patch files, and remove obsolete grokkit entry.
Removed the settings file specifying the 'grok-code-fast-1' model to revert to default configuration.
Introduce a new mechanism in the recipe runner to execute whitelisted shell commands for steps like initialization, creation, or running tools (e.g., poetry, git). Commands are generated via AI prompt, parsed from JSON, validated against an allowed list, and executed within a strict working directory to ensure safety.
- Introduce resolvePackagePath to handle ~ expansion, relative paths, and absolutization at the start.
- Update discoverFiles and executeShellCommands to use the resolved workDir.
- Remove redundant path logic from discoverFiles and shell execution for better maintainability.
- Rename and expand resolvePackagePath to resolveWorkDir, which now appends project_name if provided
- Update discoverFiles to use workDir and add default search pattern if none specified
- Modify executeShellCommands prompt to include workDir and enforce relative paths only
- Add safety checks to reject shell commands with ".." or absolute paths to prevent directory escapes
- Minor prompt and formatting adjustments for clarity and consistency
Implement a read-only command whitelist in the recipe loader to reject
potentially dangerous shell commands, ensuring only safe operations like
ls, pwd, cat, etc., are permitted. This enhances security by preventing
execution of unauthorized commands in recipes.
- Consolidate resolveWorkDir and remove resolvePackagePath for cleaner path handling.
- Eliminate executeShellCommands and related logic to disable shell execution in recipes.
- Simplify error messaging in loader for unsafe commands.
Update the error message for unsafe shell commands to be more concise and user-friendly, removing redundant "ERROR:" prefix and "try again" instruction.
- Renamed safeReadOnlyCommands to safeCommands for clarity.
- Added support for additional safe commands including GNU utilities (find, grep, which),
  Git commands (diff, branch), and various test runners (go test, make test/lint, pytest, etc.).
- Updated safety check to allow commands prefixed with any safe command for flexibility.
- Improved error message for unsafe commands.
- Replace hardcoded safeCommands map with sync.OnceValue loading from ~/.config/grokkit/safe_shell_commands.yaml
- Provide fallback built-in safe list if config load fails
- Add safe_shell_commands.yaml.example for user reference
- Update safety check to use loaded map and prefix matching
Add debug print statements to indicate when the safe commands config file
is successfully loaded or when falling back to the built-in list.
Implement executeReadOnlyShell method to safely run whitelisted read-only commands,
prompting for user approval and integrating AI-suggested commands for filesystem context.
- Introduce new Step 1 in result-refactor.md for exploring project structure using safe read-only shell commands (e.g., tree, cat).
- Rename subsequent steps accordingly.
- Add handler in runner.go for read-only shell steps triggered by keywords like "explore" or "inspect".
- Tighten trigger conditions for read-only shell steps to specific phrases
- Add robust JSON extraction with escaped quote handling
- Reorder user confirmation before whitelist check in execution flow
- Relocate FileChange struct and clean up comments
- Update recipe markdown for step title consistency
Update the executeReadOnlyShell function to handle numbers in command arguments,
such as 'tree -L 3', by changing Args to []interface{} and converting them to strings
before execution. Add strconv import for formatting.
Update the command matching logic to require an exact match or the command followed by a space and arguments. Also normalize case and trim whitespace for safe commands to prevent loose prefix matches that could allow unintended commands.
- Introduce global safeCommands() map for command whitelisting.
- Implement case-insensitive prefix checking for allowed commands.
- Simplify argument handling by removing redundant int conversions.
- Update error messages and comments for clarity on security policies.
- Remove outdated comments and adjust prompt text for consistency.
- Reorganize commit commands into commit-msg and commit
- Add sections for scaffold, recipe, and testgen
- Update PR description base branch example
- Add new features to quality section: AI unit test generation, file scaffolding, transactional recipes
- Update directory structure with .grokkit/recipes and internal/recipe
This prevents potential duplicate command registrations if added elsewhere.
docs: refactor README and docs structure for better organization
All checks were successful
CI / Test (pull_request) Successful in 42s
Details
CI / Lint (pull_request) Successful in 27s
Details
CI / Build (pull_request) Successful in 23s
Details
cd47686679 
- Simplified README.md by moving detailed command docs, workflows, and development info to dedicated user-guide/ and developer-guide/ directories.
- Created index.md files for both guides to improve navigation.
- Extracted individual command guides (e.g., chat.md, edit.md) into user-guide/ for focused, maintainable documentation.
- Moved architecture, configuration, and troubleshooting to developer-guide/.
- Updated README links to point to the new docs structure.
gmgauthier merged commit ffc840ca5b into master 2026-03-07 23:17:54 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: gmgauthier/grokkit#5
No description provided.
Delete Branch "feature/recipe_implementation"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?