Added llms.txt and llms-full.txt to My Hugo-built Website

This article is a translation of https://masutaka.net/2025-05-18-1/.

What is llms.txt?

llms.txt is a Markdown-formatted text file proposed to address the limitation that LLMs have small context windows and cannot process entire websites.

It was proposed on September 3, 2024, by Jeremy Howard of Answer.AI at https://llmstxt.org/. It is not defined in an RFC like robots.txt.

The format includes certain specifications such as the website name in an H1 section, a brief summary, and a list of links in H2 sections.
Example: https://llmstxt.org/llms.txt

There's also llms-full.txt, which contains all website information.
Example: https://developers.cloudflare.com/llms-full.txt

There are directory sites for llms.txt as well:

The llms.txt and llms-full.txt I Added

I've added both:

How to Configure in Hugo

The necessary work involves modifying or creating just three files:

config.toml (modify)
layouts/index.llms.txt (create new)
layouts/index.llmsfull.txt (create new)

config.toml

I added the following to config.toml:

Defining outputFormats for llms and llmsfull while adding them to the default value of outputs.home, which is ["html", "rss"].

[outputs]
  home = ["html", "rss", "llms", "llmsfull"]

[outputFormats.llms]
  baseName = "llms"
  isPlainText = true
  mediaType = "text/plain"
  rel = "alternate"
  root = true

[outputFormats.llmsfull]
  baseName = "llms-full"
  isPlainText = true
  mediaType = "text/plain"
  rel = "alternate"
  root = true

layouts/index.llms.txt

I created layouts/index.llms.txt as a template for llms.txt:

# {{ .Site.Title }}

> {{ .Site.Params.Description }}


## Articles
{{ $yearMonthGroups := slice -}}
{{ range where (where (sort (.Site.GetPage "/posts/").Pages "Date" "desc") "Draft" "ne" true) "Sitemap.Disable" "ne" true -}}
{{ $yearMonth := .Date.Format "2006/01" -}}
{{ if not (in $yearMonthGroups $yearMonth) }}
* {{ $yearMonth -}}
{{ $yearMonthGroups = $yearMonthGroups | append $yearMonth -}}
{{ end }}
    * [{{ .Title }}]({{ .Permalink }})
{{- end }}


## Others

{{- $baseURL := .Site.BaseURL | strings.TrimSuffix "/" }}

* [Device]({{ $baseURL }}/device/): 所有デバイス
* [History]({{ $baseURL }}/history/): このサイトの歴史
* [About]({{ $baseURL }}/about/): 自己紹介
* [Privacy Policy]({{ $baseURL }}/privacy/): 当サイトの広告、アフィリエイト、プライバシーポリシー

Since Hugo site structures can vary widely, this is just one example. My blog has both blog posts and other articles, and since other articles don't change much, I hardcoded them in the Others section.

layouts/index.llmsfull.txt

I created layouts/index.llmsfull.txt as a template for llms-full.txt:

# {{ .Site.Title }}

> {{ .Site.Params.Description }}


{{/* Articles */}}
{{ range where (where (sort (.Site.GetPage "/posts/").Pages "Date" "desc") "Draft" "ne" true) "Sitemap.Disable" "ne" true }}
--------------------------------------------------------------------------------
title: "{{ .Title }}"
date: "{{ .Date.Format "2006-01-02" }}"
--------------------------------------------------------------------------------
{{ replaceRE "{{<\\s*comment\\s*>}}(.|\n)*?{{<\\s*/comment\\s*>}}" "" .RawContent -}}
{{ end -}}


{{/* Others */}}
{{ range slice "device.md" "history.md" "about.md" "privacy.md" -}}
{{ with site.GetPage . -}}
--------------------------------------------------------------------------------
title: "{{ .Title }}"
lastmod: "{{ .Date.Format "2006-01-02" }}"
--------------------------------------------------------------------------------
{{ replaceRE "{{<\\s*comment\\s*>}}(.|\n)*?{{<\\s*/comment\\s*>}}" "" .RawContent -}}
{{ end -}}
{{ end -}}

I made the delimiter longer because some articles contained ---.

Also, I made sure comments like the following wouldn't be included in llms-full.txt:

{{<comment>}}
This is a Hugo comment. It won't be output as an HTML comment either.
{{</comment>}}

nginx Configuration

Since llms.txt and llms-full.txt were showing character encoding issues, I added the following to the server directive:

location ~ "^/llms(-full)?\.txt" {
    root /usr/share/nginx/html;
    charset UTF-8;
}

Conclusion

I added llms.txt and llms-full.txt on my Hugo-built blog:

Regardless of whether there's demand for it, I'm satisfied with the result.

As a side note, I initially found a method on Hugo Discourse that used resources.ExecuteAsTemplate in layouts/robots.txt to generate llms.txt. However, I decided not to use this approach since including llms.txt in robots.txt is not yet common practice.

Another side note: in the past, my blog was created by converting a single ChangeLog-formatted file into HTML using a tool called chalow. Creating llms-full.txt reminded me of those days. It feels like I've come full circle.

Takashi Masuda @masutaka