Did you know that org-mode's source code contains more than 5000 examples?

2022-03-11

/Tony Aldon/

comment on reddit

emacs revision: b8b2dd17c57b

org-mode revision: 96d91bea658c

Table of content

Org manual
Help buffer
The source code
The Tests (aka. "The Examples")
Where the truth lives
Q&A

Hey org-mode lovers,

Have you ever been in the following situation?

You have tried to understand a specific aspect of org-mode (say a command) and you have done the following:

you have played with the command (writing your own examples),
you have read all the parts in the manual dealing with this command,
you have read the docstring of the command,
you have looked for explanations on the web,
you even have read the source code of the command,

but you still haven't figured it out after all your efforts.

I've been in this situation and I thought it would be easier if there were more examples.

BUT THERE ARE MORE EXAMPLES.

We just have to change our lenses to see them.

What if we looked at org-mode tests as examples of org-mode?

BOOM. We have our 5000 examples.

Before going any further, let's make it clear that we don't need to understand or know how to use ert.el (the built-in package used for testing) to benefit from org-mode testing.

And this post is not about ert.el testing but about the information that we can get from the org-mode test suite.

In this post, we use the example of the command org-insert-heading (bound by default to M-<RET>, that allows to insert a new heading or item with the same depth at point) to "demonstrate" that the tests are indeed examples.

Let's assume we are already familiar with org-insert-heading, we use it every day, but sometimes we don't understand why it behaves the way it does.

What can we do to find out the truth about org-insert-heading?

Org manual

First of all, we can take a look in the manual of org.

In the info node org#Plain Lists we can read:

‘M-<RET>’ (‘org-insert-heading’)
     Insert new item at current level.  With a prefix argument, force a
     new heading (see Structure Editing).  If this command is
     used in the middle of an item, that item is _split_ in two, and the
     second part becomes the new item(5).  If this command is executed
     _before item’s body_, the new item is created _before_ the current
     one.

...

   (5) If you do not want the item to be split, customize the variable
‘org-M-RET-may-split-line’.

That might be enough, but let's say it isn't, and we continue our investigation to find the truth.

Help buffer

Secondly, we can find more information about org-insert-heading by reading its docstring by running:

C-h f org-insert-heading RET

which pops up this help buffer:

org-insert-heading is an interactive compiled Lisp function.

(org-insert-heading &optional ARG INVISIBLE-OK TOP)

Insert a new heading or an item with the same depth at point.

If point is at the beginning of a heading, insert a new heading
or a new headline above the current one.  When at the beginning
of a regular line of text, turn it into a heading.

If point is in the middle of a line, split it and create a new
headline with the text in the current line after point (see
‘org-M-RET-may-split-line’ on how to modify this behavior).  As
a special case, on a headline, splitting can only happen on the
title itself.  E.g., this excludes breaking stars or tags.

With a ‘C-u’ prefix, set ‘org-insert-heading-respect-content’ to
a non-nil value for the duration of the command.  This forces the
insertion of a heading after the current subtree, independently
on the location of point.

With a ‘C-u C-u’ prefix, insert the heading at the end of the tree
above the current heading.  For example, if point is within a
2nd-level heading, then it will insert a 2nd-level heading at
the end of the 1st-level parent subtree.

When INVISIBLE-OK is set, stop at invisible headlines when going
back.  This is important for non-interactive uses of the
command.

When optional argument TOP is non-nil, insert a level 1 heading,
unconditionally.

Now we not only know what the command does but also how to call it.

There is also some information about its non-interactive use.

Reading the Org manual and its docstring may have given us the information we wanted, but let's say we want to know more and continue our investigation to find the truth.

The source code

The truth resides in the source code! Isn't that right?

Ok, let's take a look at the command org-insert-heading defined in the file lisp/org.el as follow:

(defun org-insert-heading (&optional arg invisible-ok top)
  ;; HERE WAS THE DOCSTRING
  (interactive "P")
  (let* ((blank? (org--blank-before-heading-p (equal arg '(16))))
         (level (org-current-level))
         (stars (make-string (if (and level (not top)) level 1) ?*)))
    (cond
     ((or org-insert-heading-respect-content
          (member arg '((4) (16)))
          (and (not invisible-ok)
               (invisible-p (max (1- (point)) (point-min)))))
      ;; Position point at the location of insertion.  Make sure we
      ;; end up on a visible headline if INVISIBLE-OK is nil.
      (org-with-limited-levels
       (if (not level) (outline-next-heading) ;before first headline
         (org-back-to-heading invisible-ok)
         (when (equal arg '(16)) (org-up-heading-safe))
         (org-end-of-subtree)))
      (unless (bolp) (insert "\n"))
      (when (and blank? (save-excursion
                          (backward-char)
                          (org-before-first-heading-p)))
        (insert "\n")
        (backward-char))
      (when (and (not level) (not (eobp)) (not (bobp)))
        (when (org-at-heading-p) (insert "\n"))
        (backward-char))
      (unless (and blank? (org-previous-line-empty-p))
        (org-N-empty-lines-before-current (if blank? 1 0)))
      (insert stars " ")
      ;; When INVISIBLE-OK is non-nil, ensure newly created headline
      ;; is visible.
      (unless invisible-ok
        (pcase (get-char-property-and-overlay (point) 'invisible)
          (`(outline . ,o)
           (move-overlay o (overlay-start o) (line-end-position 0)))
          (_ nil))))
     ;; At a headline...
     ((org-at-heading-p)
      (cond ((bolp)
             (when blank? (save-excursion (insert "\n")))
             (save-excursion (insert stars " \n"))
             (unless (and blank? (org-previous-line-empty-p))
               (org-N-empty-lines-before-current (if blank? 1 0)))
             (end-of-line))
            ((and (org-get-alist-option org-M-RET-may-split-line 'headline)
                  (org-match-line org-complex-heading-regexp)
                  (org-pos-in-match-range (point) 4))
             ;; Grab the text that should moved to the new headline.
             ;; Preserve tags.
             (let ((split (delete-and-extract-region (point) (match-end 4))))
               (if (looking-at "[ \t]*$") (replace-match "")
                 (org-align-tags))
               (end-of-line)
               (when blank? (insert "\n"))
               (insert "\n" stars " ")
               (when (org-string-nw-p split) (insert split))))
            (t
             (end-of-line)
             (when blank? (insert "\n"))
             (insert "\n" stars " "))))
     ;; On regular text, turn line into a headline or split, if
     ;; appropriate.
     ((bolp)
      (insert stars " ")
      (unless (and blank? (org-previous-line-empty-p))
        (org-N-empty-lines-before-current (if blank? 1 0))))
     (t
      (unless (org-get-alist-option org-M-RET-may-split-line 'headline)
        (end-of-line))
      (insert "\n" stars " ")
      (unless (and blank? (org-previous-line-empty-p))
        (org-N-empty-lines-before-current (if blank? 1 0))))))
  (run-hooks 'org-insert-heading-hook))

You can get org-mode's source code by running the following command:

git clone https://git.savannah.gnu.org/git/emacs/org-mode.git

Ok...

The source code helps, but now we need to know a lot more about the implementations of org-mode and emacs/elisp than we want to spend time on.

Indeed, if we want to understand the implementation of org-insert-heading we have to understand:

(org API) org--blank-before-heading-p, org-current-level, org-with-limited-levels, outline-next-heading, org-back-to-heading, org-up-heading-safe, org-end-of-subtree, org-before-first-heading-p, org-N-empty-lines-before-current, etc.,
(emacs/elisp API) make-string, member, invisible-p, bolp, save-excursion, eobp, bobp, pcase, get-char-property-and-overlay, etc.

So, what can we do?

Maybe we can look at the test of the command org-insert-heading.

The Tests (aka. "The Examples")

test-org/insert-heading

In testing directory, doing a search for org-insert-heading using grep (or ripgrep) shows that the command org-insert-heading is tested in the ert.el test test-org/insert-heading defined in the file testing/lisp/test-org.el as follows (we reproduce only the first 4 should forms - it contains 28 should forms and 1 should-not form):

(ert-deftest test-org/insert-heading ()
  "Test `org-insert-heading' specifications."
  ;; In an empty buffer, insert a new headline.
  (should
   (equal "* "
          (org-test-with-temp-text ""
            (org-insert-heading)
            (buffer-string))))
  ;; At the beginning of a line, turn it into a headline.
  (should
   (equal "* P"
          (org-test-with-temp-text "<point>P"
            (org-insert-heading)
            (buffer-string))))
  ;; In the middle of a line, split the line if allowed, otherwise,
  ;; insert the headline at its end.
  (should
   (equal "Para\n* graph"
          (org-test-with-temp-text "Para<point>graph"
            (let ((org-M-RET-may-split-line '((default . t))))
              (org-insert-heading))
            (buffer-string))))
  (should
   (equal "Paragraph\n* "
          (org-test-with-temp-text "Para<point>graph"
            (let ((org-M-RET-may-split-line '((default . nil))))
              (org-insert-heading))
            (buffer-string))))
  ;; ...
  )

What can we observe from those 4 should forms?

they have the same "shape", so we can deduce that if we understand how the first one works, we will understand the others,
they use only a few symbols (much less than in the source code of org-insert-heading):
- should from ert.el package,
- equal, let and buffer-string from Emacs/Elisp,
- org-test-with-temp-text specific to Org test suite and
- org-insert-heading and org-M-RET-may-split-line, the command and the variable we are testing.

Now let's describe these should forms in words as best we can.

After doing this, perhaps we will think of these should forms as small examples, each describing a behavior of the command org-insert-heading.

Example 1

The first should form:

(should
   (equal "* "
          (org-test-with-temp-text ""
            (org-insert-heading)
            (buffer-string))))

could be translated like this:

in a temporary buffer in org-mode (org-test-with-temp-text),
keep the buffer empty (""),
keep the point at the beginning,
call org-insert-heading,
return the string from the temporary buffer (buffer-string),
compare this string to the string "* " (equal),
if they are equal, return t, if not, return the error ert-test-failed (should).

ert and org-test-with-temp-text

To evaluate the preceding form we need to have ert.el loaded (for the macro should) which can be achieve by evaluating the following form:

(require 'ert)

and we need the macro org-test-with-temp-text (defined in the file testing/org-test.el) to be defined which can be done by evaluating its definition:

(defmacro org-test-with-temp-text (text &rest body)
  (declare (indent 1))
  `(let ((inside-text (if (stringp ,text) ,text (eval ,text)))
         (org-mode-hook nil))
     (with-temp-buffer
       (org-mode)
       (let ((point (string-match "<point>" inside-text)))
         (if point
             (progn
               (insert (replace-match "" nil nil inside-text))
               (goto-char (1+ (match-beginning 0))))
           (insert inside-text)
           (goto-char (point-min))))
       (font-lock-ensure (point-min) (point-max))
       ,@body)))

should, should-not, should-error

Our examples only use the macro should but if we want to read more tests it is useful to know the macros should-not and should-error that ert.el package provides.

In the following Elisp snippet, we give some examples of the use of should, should-not and should-error:

(should 3)   ; 3
(should t)   ; t
(should nil)
;; Debugger entered--Lisp error: (ert-test-failed ((should nil) :form nil :value nil))

(should-not nil) ; nil
(should-not t)
;; Debugger entered--Lisp error: (ert-test-failed ((should-not t) :form t :value t))

(should-error (= "1" 1)) ; (wrong-type-argument number-or-marker-p "1")
(should-error t)
;; Debugger entered--Lisp error: (ert-test-failed ((should-error t) :form t :value t :fail-reason "did not signal an error"))

Example 2

The second should form:

(should
 (equal "* P"
        (org-test-with-temp-text "<point>P"
          (org-insert-heading)
          (buffer-string))))

could be translated like this:

in a temporary buffer in org-mode (org-test-with-temp-text),
add the string P and place the point before P ("<point>P"),
call org-insert-heading,
return the string from the temporary buffer (buffer-string),
compare this string to the string "* P" (equal),
if they are equal, return t, if not, return the error ert-test-failed (should).

Example 3

The third should form:

(should
 (equal "Para\n* graph"
        (org-test-with-temp-text "Para<point>graph"
          (let ((org-M-RET-may-split-line '((default . t))))
            (org-insert-heading))
          (buffer-string))))

could be translated like this:

in a temporary buffer in org-mode (org-test-with-temp-text),
add the string Paragraph and place the point between the character a and g ("Para<point>graph"),
locally set the variable org-M-RET-may-split-line to the alist '((default . t)) and call org-insert-heading with this binding ((let ...)),
return the string from the temporary buffer (buffer-string),
compare this string to the string "Para\n* graph" (equal and note that \n is the newline),
if they are equal, return t, if not, return the error ert-test-failed (should).

Example 4

The fourth should form:

(should
 (equal "Paragraph\n* "
        (org-test-with-temp-text "Para<point>graph"
          (let ((org-M-RET-may-split-line '((default . nil))))
            (org-insert-heading))
          (buffer-string))))

could be translated like this:

in a temporary buffer in org-mode (org-test-with-temp-text),
add the string Paragraph and place the point between the character a and g ("Para<point>graph"),
locally set the variable org-M-RET-may-split-line to the alist '((default . nil)) and call org-insert-heading with this binding ((let ...)),
return the string from the temporary buffer (buffer-string),
compare this string to the string "Paragraph\n* " (equal and note that \n is the newline),
if they are equal, return t, if not, return the error ert-test-failed (should).

Where the truth lives

By now you should be convinced (that's what I hope) that the org-mode test suite is a gold mine for us (org-mode users) and that is where part of THE ORG-MODE TRUTH lives.

... More than 5000 examples ...

WE ARE DONE!

Q&A

Check Q&A.

PREV RANDOM NEXT