| .gitignore | ||
| gptel-org-tools.el | ||
| LICENSE | ||
| README.org | ||
gptel-org-tools
Intro
This is a collection of tools I wrote to help me review my life.
To that end, it's proven useful enough.
Summary:
- An explanation of each tool is below.
- The tools are tangled into
gptel-org-tools.elfrom this file. - There are no docstrings unless there are, in which case that's a mistake and I apologize.
Premise:
- LLMs are not very smart and unreliable, but they do okay with basic text comprehension, and can generate completions/ responses which fit the vibe of the request.
- LLMs can then fit the vibe of the request better if there is more relevant and accurate request data in their context window.
- LLMs lose the vibe when there's irrelevant (garbage) data in the context window.
Therefore (philosophy, I may change my mind later):
- Each tool has to limit the garbage data it returns as much as possible.
- Each tool has to have as little LLM-facing documentation as possible (while being enough for the LLM to understand and use the tool.) Extra words is extra garbage.
-
Each tool should handle as wide an range of even remotely valid inputs from an LLM as possible.
- Different models are biased toward different outputs.
user-errorisn't addressable when a model only has 5 minutes worth of memory.- Failure caused by LLM mis-use should be solved in such a way that failure becomes increasingly less likely.
- We never know when an LLM will respond with a string, json, s-exp, or ASCII codes (no, that last one hasn't happened yet).
-
Each tool should work in harmony with other tools to form a toolbox which serves these goals.
- Avoid tool overlap.
- One tool for one task.
- Tool names are documentation.
- Argument names are documentation.
- As few arguments per tool as possible.
- Documentation strings are (ideally) for examples.
These are primarily for my own use. Please don't expect quality.
This forge is for my personal use, and as such: contact, just in case there's questions, issues, whatnot.
My system
I'm running LLMs exclusively locally, on the smaller side. With Qwen 14B, I'm getting reasonably good results with the following set-up:
- Hardware: RX 7900XTX
-
Software:
- llama.cpp compiled with ROCm 6.4.
- Emacs: Doom Emacs, gptel, org-ql.
My set-up
These are the settings I use, for reproducibility.
Yes, this is somewhat odd, that I would use the deepseek option, but I have found that it handles reasoning a little bit better than gptel's openai backend.
I change models a lot, and this just works for most models, even if some aren't compatible outright.
(use-package! gptel)
(setq gptel-model 'llamacpp)
(setq gptel-include-reasoning t)
(setq gptel-backend (gptel-make-deepseek "llamacpp"
:host "localhost:8080"
:protocol "http"
:stream nil
:models '("llamacpp"
:capabilities (reasoning))
:request-params '(:thinking t
:enable_thinking t
:include_reasoning t
:parallel_tool_calls t)))
(setf (alist-get 'org-mode gptel-prompt-prefix-alist) "@user\n")
(setf (alist-get 'org-mode gptel-response-prefix-alist) "@assistant\n")
(setq gptel-default-mode 'org-mode)
(setq gptel-use-tools t)
(setq gptel-log-level 'debug)
(setq gptel--debug t)
(require 'gptel)With that out of the way, let's get to the tools.
Code
Preamble
Stuff, headers, etc.
;;; gptel-org-tools.el --- LLM Tools for org-mode interaction. -*- lexical-binding: t; -*-
;; Copyright (C) 2025 Phil Bajsicki
;; Author: Phil Bajsicki <phil@bajsicki.com>
;; Keywords: extensions, comm, tools, matching, convenience,
;;
;; Author: Phil Bajsicki <phil@bajsicki.com>
;; Version: 0.0.1
;; Package-Requires: ((emacs "30.1") (gptel 0.9.8) (org-ql 0.9))
;; URL: https://github.com/phil/gptel-org-tools
;; SPDX-License-Identifier: GPL-3.0
;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, version 3 of the License.
;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
;; GNU General Public License for more details.
;; You should have received a copy of the GNU General Public License
;; along with this program. If not, see <https://www.gnu.org/licenses/>.
;; This file is NOT part of GNU Emacs.
;; This file is not part of GNU Emacs.
;;; Commentary:
;; All documentation regarding these functions is in the README.org file.
;; Repository: https://git.bajsicki.com/phil/gptel-org-tools
;;; Code:The tools
Emacs
These tools are primarily concerned with Emacs, Emacs Lisp, and files-or-buffers.
eval
Dangerous, but occasionally useful for pure chaos and amusement… I would like to say. But in actuality, especially with the 'smarter' models, they can surprise with the varied approaches they have to problem-solving.
E.g. can't find the right org-ql-select query? Let's use an eldritch abomination of a regular expression while mapping over each heading in the org-mode buffer and do a close enough job.
Highly not recommended, but sometimes an LLM can pull a rabbit out of pure entropy.
(gptel-make-tool
:function (lambda (elisp)
(unless (stringp elisp) (error "elisp code must be a string"))
(with-temp-buffer
(insert (eval (read elisp)))
(buffer-string)))
:name "eval"
:description "Execute arbitrary Emacs Lisp code"
:args (list '(:name "eval"
:type string
:description "The Emacs Lisp code to evaluate."))
:category "emacs")list-buffers
I wanted the assistant to have an easier time finding my files and buffers, and this has proven to be a great choice. I have yet to manage to get rid of the :args, but having them optional/ do nothing works well enough.
The rationale behind using ibuffer is the same as with dired. They both display a lot of data, densely. So instead of trying to use some workaround with buffer-file-name or other functions, I'd rather just grab a 'text capture' of the same UI I'm looking at, and call it a day.
Seems to be one of the most reliable tools in the basket… mostly because
(gptel-make-tool
:function (lambda (arg)
(with-temp-buffer
(ibuffer)
(let ((content (buffer-string)))
(kill-buffer (current-buffer))
content)))
:name "list-buffers"
:description "Access the list of buffers open in Emacs, including file names and full paths."
:args (list '(:name "arg"
:type string
:description "Does nothing."
:optional t))
:category "emacs")dired
See above, same reasoning. There's very little reason to use the directory-files function for this (as in another tool I saw.) The reason is, directory-files doesn't provide nearly as much information about the items in that directory. Not even a distinction between files and directories.
directory-files-and-attributes might, but I personally found its output horrendous to read, and still somehow more expensive context-wise than just plain ol' dired.
Be sure to customize the function to point to your org directory, if you wish. I find it makes a big difference. Having the argument be optional also helps when the LLM starts stumbling, as it gives it a reset point so it can re-orient itself (although by that time it has usually forgotten what it was supposed to be doing…)
(gptel-make-tool
:function (lambda (dir)
(with-temp-buffer
(dired (or dir "~"))
(let ((content (buffer-string)))
(kill-buffer (current-buffer))
content)))
:name "dired"
:description "List directory contents"
:args (list '(:name "dir"
:type string
:description "Directory path"
:optional t))
:category "filesystem")find-buffer-visiting
(gptel-make-tool
:function (lambda (filename)
(bufferp (find-buffer-visiting (expand-file-name filename))))
:name "find-buffer-visiting"
:description "Check if the file is open in a buffer. Usage (find-buffer-visiting filename)"
:args (list '(:name "filename"
:type string
:description "The filename to compare to open buffers."))
:category "org-mode")find-file-noselect
Continuation from above. Open a file into a buffer for processing. Onec it's found by dired-list.
(gptel-make-tool
:function (lambda (file)
(find-file-noselect file))
:name "find-file-noselect"
:description "Open the file in a buffer. This doesn't interfere with the user."
:args (list '(:name "file"
:type string
:description "Path to file.."))
:category "filesystem")Org-mode
And here we start getting into the weeds.
The core of these functions is to ensure that the LLM doesn't go astray.
It will, inevitably.
Then, the core of these functions is to ensure it can be gently prodded into doing what it's asked, instead of being "asked politely" until it complies.
We want to minimize the effect of entropy on the use of these tools. To that end, we're making separate tools for separate purposes.
This isn't proven, by any means, but given what little I know about LLMs, I believe it's easier for them to choose between many single-purpose tools, than from few multi-purpose tools.
Even us humans experience less cognitive load when we're given clear options up-front, instead of being given a multi-tool that we then need to learn our way around to get the job done.
With that secondary purpose in mind, the real aim of these tools is to limit the information that the LLM has access to only to that which is relevant. In the ideal.
In the real world, we'll still see garbage being pulled in, and the LLM being led astray by an unfortunate sentence, or an org-ql query that explodes the context.
At the same time, that will happen significantly less than if we were to give it more freedom.
LLMs are not intelligent, despite claims to the contrary.
org-extract-tags
Pretty simple, does what it says on the tin. It gets all the tags from the buffer. This is useful as a first line of research for the LLM, as it can then get a surface-level understanding of the contents of a file.
This is not, by any means, sufficient, but I do tag people and specific events frequently enough that it helps save on the context window.
(defun gptel-org-tools-org-extract-tags (buffer)
(interactive "bBuffer: ")
(with-current-buffer buffer
(let ((tags '()))
(org-map-entries
(lambda ()
(let* ((components (org-heading-components))
(tag-string (car (last components))))
(when tag-string
(dolist (tag (split-string tag-string ":" t))
(push tag tags))))))
(sort (-uniq tags) #'string<))))
(gptel-make-tool
:function #'gptel-org-tools-org-extract-tags
:name "org-extract-tags"
:description "Extract all unique tags from an org-mode buffer"
:args (list '(:name "buffer"
:type string
:description "The Org buffer to extract tags from."))
:category "org-mode")org-extract-headings
But what if there's no tags related to the topic?
Then we need to pull some information from the buffer, without dragging the entire 500kb in and exploding the context window.
Therefore, headings. A reasonable amount of information, and still keeping the signal-to-noise ratio pretty decent.
(defun gptel-org-tools-org-extract-headings (buffer)
(interactive "bBuffer: ")
(with-current-buffer buffer
(org-map-entries
#'(buffer-substring-no-properties
(line-beginning-position)
(line-end-position))
t
'file)))
(gptel-make-tool
:function #'gptel-org-tools-org-extract-headings
:name "org-extract-headings"
:description "Extract all headings from an org-mode buffer"
:args (list '(:name "buffer"
:type string
:description "The Org buffer to extract headings from."))
:category "org-mode")org-ql-select
This is the big one. It works, and it's ugly, and I'm in the middle of replacing it. The tool argument descriptions do a lot of lifting here, and even then the models stumble and fall on their face very often.
They also sometimes return a sexp, and sometimes a quoted string. So I had to work around that. It works… some of the time.
My current goal is to replace this monstrosity with individual functions for each of the main org-ql predicates, such that the LLMs have an easier time with the syntax, so they can just choose their desired query instead of having to both choose and properly form the syntax.
But in the interim, this works. Kind of.
(defun gptel-org-tools-org-ql-select (buf query)
(org-ql-select
(get-buffer buf)
(if (stringp query)
(read query)
query)
:action #'(lambda ()
(concat
(buffer-substring-no-properties
(line-beginning-position)
(progn
(outline-next-heading)
(line-beginning-position)))))))
(gptel-make-tool
:function #'gptel-org-tools-org-ql-select
:name "org-ql-select"
:description "Run org-ql-select against buffer with query. Using filename fails."
:args (list '(:name "buffer"
:type string
:description "The name of the buffer. Can be multiple buffers. See the NAME column in `emacs-list-buffers`.")
'(:name "query"
:type string
:description "The query to pass into org-ql-select. See org-ql documentation for syntax. Usually `(tags \"tag1\" \"tag2\")` is sufficient. Possible predicates: `tags` (finds both local and inherited tags), `tags-local` (finds only local tags), `rifle` (matches against both heading and body text). This is a sexp, not a string."))
:category "org")Somewhat working tools
org-ql-select-dates
My journal is a single file, with a hierarchy like so:
* [YYYY]
** [YYYY-MM]
*** [YYYY-MM-DD Day HH:MM]I wanted to make sure that if I asked about a time period, the LLM would be able to pull at least roughly around the right time from my journal, without blowing out its context window.
This kinda sorta works. The only time I have seen this fail is when the LLM chooses to apply it to buffers that don't follow that convention.
I doubt it'll be useful for anyone else, but it's here and fairly easy to adapt to other needs.
Notice it pulls to the end of the subtree. So for months, that's at least 28 entries in a year, and during busy months, possibly hundreds.
But, any customizations to tweak this is left to the user, as everyone has their own conventions.
(defun gptel-org-tools-org-ql-select-dates (buf date)
(interactive "fBuffer: \nsDate (YYYY or YYYY-MM): ")
(org-ql-select
(get-buffer buf)
`(heading ,date)
:action #'(lambda ()
(buffer-substring-no-properties
(line-beginning-position)
(org-end-of-subtree)))))
(gptel-make-tool
:function #'gptel-org-tools-org-ql-select-dates
:name "org-ql-select-dates"
:description "Extract org subtree by date in YYYY or YYYY-MM format"
:args (list '(:name "buffer"
:type string
:description "Buffer name.")
'(:name "date"
:type string
:description "Date in YYYY or YYYY-MM format. Can add multiple like so: \"YYYY-MM\" \"YYYY-MM\""))
:category "org")org-agenda-fortnight
This is still work in progress, the idea is to have the LLM check my calendar and see what my plans are. I have not had time to really dig into this yet.
It works, in principle, but I haven't been able to find a use for it yet. The real challenge is in building a context where the tools integrate with each-other in a way that makes sense. For now, this exists.
(gptel-make-tool
:function (lambda (days)
(with-temp-buffer
(org-agenda-list (or days 14))
(let ((content (buffer-string)))
(kill-buffer (current-buffer))
content)))
:name "org-agenda-fortnight"
:description "Get the next 14 days of user's org-agenda."
:args (list '(:name "days"
:type integer
:description "The number of days to look ahead. Default: 14"))
:category "org")Completely WIP tools
The following tools are still very much WIP, and I think they're self-explanatory enough. They have NOT been tested in any way, shape, form, or capacity.
(I mean, some of them work… but they don't sometimes, too.)
org-ql-select-headings
Retrieve the headings where the heading matches query..
(defun gptel-org-tools-org-ql-select-headings (buf query)
(org-ql-select
(get-buffer buf)
`(heading ,query)
:action #'(lambda ()
(concat
(buffer-substring-no-properties
(line-beginning-position)
(line-end-position))))))
(gptel-make-tool
:function #'gptel-org-tools-org-ql-select-headings
:name "org-ql-select-headings"
:description "Retreive matching headings from buffer using org-ql-select. Matches only against heading. Using filename fails."
:args (list '(:name "buffer"
:type string
:description "The name of the buffer. See the NAME column in ~emacs-list-buffers~.")
'(:name "query"
:type string
:description "The string to pass into org-ql-select-headings. This is a bare string. Example: \"searchterm\""))
:category "org-ql")org-ql-select-headings-rifle
Retrieve all the headings where either heading or content matches query.
(defun gptel-org-tools-org-ql-select-headings-rifle (buf query)
(org-ql-select
(get-buffer buf)
`(rifle ,query)
:action :action #'(lambda ()
(concat
(buffer-substring-no-properties
(line-beginning-position)
(line-end-position))))))
(gptel-make-tool
:function #'gptel-org-tools-org-ql-select-headings-rifle
:name "org-ql-select-headings-rifle"
:description "Retreive headings from buffer using org-ql-select. Matches against both heading and content. Using filename fails."
:args (list '(:name "buffer"
:type string
:description "The name of the buffer. See the NAME column in ~emacs-list-buffers~.")
'(:name "query"
:type string
:description "The string to pass into org-ql-select-headings-rifle. This is a bare string. Example: \"searchterm\""))
:category "org-ql")org-ql-select-tags-local
This pulls all the headings (and their contents) when they match tags (without inheritance.)
(defun gptel-org-tools-org-ql-select-tags-local (buf query)
(org-ql-select
(get-buffer buf)
`(tags-local ,query)
:action #'(lambda ()
(concat
(buffer-substring-no-properties
(line-beginning-position)
(progn
(outline-next-heading)
(line-beginning-position)))))))
(gptel-make-tool
:function #'gptel-org-tools-org-ql-select-tags-local
:name "org-ql-select-tags-local"
:description "Run org-ql-select-tags-local against buffer with query. No tag inheritance."
:args (list '(:name "buffer"
:type string
:description "The name of the buffer. See the NAME column in `emacs-list-buffers`. Using filename fails.")
'(:name "query"
:type string
:description "The tags to match entry headings against. Example: \"tag1\" \"tag2\""))
:category "org-ql")org-ql-select-tags
This pulls all the headings (and their contents) when they match tags (with inheritance; if a parent entry has the tag, descendant entries do, too.)
(defun gptel-org-tools-org-ql-select-tags (buf query)
(org-ql-select
(get-buffer buf)
`(tags ,query)
:action #'(lambda ()
(concat
(buffer-substring-no-properties
(line-beginning-position)
(progn
(outline-next-heading)
(line-beginning-position)))))))
(gptel-make-tool
:function #'gptel-org-tools-org-ql-select-tags
:name "org-ql-select-tags"
:description "Run org-ql-select-tags against buffer with query. Supports tag inheritance."
:args (list '(:name "buffer"
:type string
:description "The name of the buffer. See the NAME column in `emacs-list-buffers`. Using filename fails.")
'(:name "query"
:type string
:description "The tags to match entry headings against. Example: \"tag1\" \"tag2\""))
:category "org-ql")org-ql-select-rifle
And, the "grab everything that matches" tool.
(defun gptel-org-tools-org-ql-select-rifle (buf query)
(org-ql-select
(get-buffer buf)
`(rifle ,query)
:action #'(lambda ()
(concat
(buffer-substring-no-properties
(line-beginning-position)
(progn
(outline-next-heading)
(line-beginning-position)))))))
(gptel-make-tool
:function #'gptel-org-tools-org-ql-select-rifle
:name "org-ql-select-rifle"
:description "Run org-ql-select-rifle against buffer with query."
:args (list '(:name "buffer"
:type string
:description "The name of the buffer. See the NAME column in `emacs-list-buffers`. Using filename fails.")
'(:name "query"
:type string
:description "The strings to match entry headings and content against. Example: \"tag1\" \"tag2\""))
:category "org-ql")