Adding Bookmarks to My Blog Part 2: Publication

With the bookmark data and cover images successfully synchronised from Raindrop.io via the API, the next step was to generate the corresponding Markdown pages for Hugo to process.
My first version of the script generated the Markdown pages into the content/bookmarks directory, paginating the data across 25 items per page.
Once that was working, I updated the script to render each bookmark as a listing card, similar to the Posts page. Rather than generating raw HTML, the script emits Markdown with a Hugo Shortcode, which then renders the content of the bookmark to HTML on build.
Contents
This post covers the Python script used to generate the Markdown pages from the DynamoDB table:
- Markdown pages - how the generated pages are organised
- Bookmark Page Generation Script - Python script used for generating the Markdown pages
- Hugo Shortcodes and CSS - Hugo shortcodes and custom CSS
- Deployment Workflow - GitHub Actions deployment workflow updates
- IAM Policy - updated IAM policy used to deploy the blog site and generate bookmark pages
- Troubleshooting - issues that were encountered during the development
Markdown pages
The pages themselves are organised under folders such as content/bookmarks/_index.md, content/bookmarks/page/2/index.md, content/bookmarks/page/3/index.md and so on, with pagination links at the bottom of each page.
Given my blog already uses the GitHub Primer theme, I wanted to reuse the same CSS styling with minimal additions where possible.
For the bookmark covers, the script downloads the image files from the S3 bucket and then places them in an images subdirectory next to the Markdown file:

Once I’d got the listing and bookmark covers working, I updated the script to generate pages for each tag under paths such as content/bookmarks/tags/tech, content/bookmarks/tags/code, content/bookmarks/tags/ai and so on:

Finally, for listing bookmarks by year, the script generates Markdown pages under paths such as content/bookmarks/years/2024, content/bookmarks/years/2025, content/bookmarks/years/2026 and so on:

Bookmark Page Generation Script
Creating this script took the most tweaking and testing to get right and it consists of the following methods:
| Name | Description |
|---|---|
decimal_safe | Converts the Decimal values DynamoDB returns into ints so they survive YAML front matter and Markdown |
load_records | Retrieves every record for the collection from the DynamoDB table, following pagination tokens |
sort_key | Builds the sort key so records are ordered newest first, falling back to id for undated records |
download_cover | Downloads a bookmark cover from the S3 bucket into the page’s images directory |
download_page_covers | Fans the page’s cover downloads out to a pool of worker threads |
sniff_image | Checks the downloaded bytes against known image signatures so HTML error pages don’t slip through |
slugify | Converts a tag string into a slugified URL with dashes |
sc_param | Escapes a value for use inside a quoted shortcode parameter |
record_card | Generates the bookmark-card shortcode for a listing page |
page_path | Returns the path for a numbered page in the main listing |
write_page | Writes the Markdown and front matter for a listing page, including covers and pagination |
write_year_pages | Groups records by year and generates paginated yearly archive pages |
write_tag_pages | Groups records by tag and generates navigation, front matter and paginated listing pages |
main | Parses command-line arguments and generates the main listing, tag pages and year pages |
The full script is as follows:
"""
Generate Hugo pages from the raindrops DynamoDB table (card listing),
downloading each page's cover images from S3 into a per-page images dir:
content/bookmarks/_index.md + content/bookmarks/images/
content/bookmarks/page/2/index.md + content/bookmarks/page/2/images/
...
Also generates per-tag listing pages:
content/bookmarks/tags/_index.md (overview of all tags)
content/bookmarks/tags/<slug>/_index.md (tag page 1)
content/bookmarks/tags/<slug>/page/2/index.md (tag pagination)
Also generates paginated yearly archives:
content/bookmarks/years/_index.md (overview of all years)
content/bookmarks/years/2025/_index.md (all 2025 bookmarks)
content/bookmarks/years/2025/page/2/index.md (pagination)
content/bookmarks/years/2024/_index.md
...
Year page cards reference covers via absolute URLs pointing at the main
listing pages where the images already live - no duplication.
Usage:
python generate_bookmarks.py [--table raindrops] [--collection-id 12345678] \
[--cover-bucket BUCKET] [--output-dir content/bookmarks] [--page-size 25] \
[--tag-page-size 25] [--year-page-size 25] [--no-tags] [--no-years]
"""
import argparse
import concurrent.futures
import datetime as dt
import decimal
import logging
import re
import shutil
import sys
from collections import defaultdict
from pathlib import Path
import boto3
import yaml
logger = logging.getLogger(__name__)
TABLE_KEY = "collectionId"
DATE_FORMAT = "%Y-%m-%d"
IMAGE_DIR_NAME = "images"
PLACEHOLDER_IMAGE = "/images/bookmark-placeholder.jpg"
DOWNLOAD_WORKERS = 10
PAGE_SIZE = 25
TAG_PAGE_SIZE = 25
TAGS_DIR_NAME = "tags"
YEAR_PAGE_SIZE = 25
YEARS_DIR_NAME = "years"
UNKNOWN_YEAR_SLUG = "unknown"
def decimal_safe(value):
"""DynamoDB returns Numbers as Decimal; convert for YAML/markdown."""
if isinstance(value, decimal.Decimal):
return int(value)
return value
def load_records(table, collection_id):
"""Query every record for the collection, following pagination tokens."""
seen, items = 0, []
kwargs = {
"KeyConditionExpression": boto3.dynamodb.conditions.Key(TABLE_KEY).eq(collection_id),
"ConsistentRead": False,
}
while True:
response = table.query(**kwargs)
items.extend(response.get("Items", []))
seen += response.get("Count", 0)
if "LastEvaluatedKey" not in response:
break
kwargs["ExclusiveStartKey"] = response["LastEvaluatedKey"]
logger.info("Loaded %d records from DynamoDB", seen)
return items
def sort_key(record):
"""Newest first, falling back to id for records missing a created date."""
raindrop = record.get("raindrop", {})
created = raindrop.get("created")
if created:
try:
ts = dt.datetime.fromisoformat(created.replace("Z", "+00:00"))
return (ts, decimal_safe(record.get("id", 0)))
except ValueError:
pass
return (dt.datetime.min.replace(tzinfo=dt.timezone.utc), decimal_safe(record.get("id", 0)))
def download_cover(s3_client, bucket, record, images_dir):
"""
Download one record's S3 cover into images_dir. Returns the local
relative path on success, or None if the record has no S3 cover or
the download failed (caller falls back to the placeholder).
"""
key = record.get("coverKey")
if not key:
return None
filename = Path(key).name # covers/{cid}/{id}.jpg -> {id}.jpg
dest = images_dir / filename
if dest.exists(): # already downloaded on a prior page chunk
return IMAGE_DIR_NAME + "/" + filename
try:
response = s3_client.get_object(Bucket=bucket, Key=key)
body = response["Body"].read()
if not body:
logger.warning("Empty cover object %s", key)
return None
if not sniff_image(body):
logger.warning("Invalid image data in %s - using placeholder", key)
return None
dest.write_bytes(body)
return IMAGE_DIR_NAME + "/" + filename
except Exception as exc:
logger.warning("Cover download failed for %s: %s", key, exc)
return None
def download_page_covers(s3_client, bucket, records, images_dir):
"""Fan the page's cover downloads out to a thread pool."""
images_dir.mkdir(parents=True, exist_ok=True)
results = {}
with concurrent.futures.ThreadPoolExecutor(max_workers=DOWNLOAD_WORKERS) as pool:
futures = {
pool.submit(download_cover, s3_client, bucket, record, images_dir): record
for record in records
}
for future in concurrent.futures.as_completed(futures):
record = futures[future]
local_path = future.result()
if local_path:
results[record["id"]] = local_path
logger.info("Downloaded %d/%d covers into %s",
len(results), len(records), images_dir)
return results
def sniff_image(data: bytes) -> bool:
"""True if the byte stream starts with a known image signature."""
if len(data) < 12:
return False
if data[:3] == b"\xff\xd8\xff": # JPEG
return True
if data[:8] == b"\x89PNG\r\n\x1a\n": # PNG
return True
if data[:6] in (b"GIF87a", b"GIF89a"): # GIF
return True
if data[:4] == b"RIFF" and data[8:12] == b"WEBP": # WebP
return True
if data[4:8] == b"ftyp" and data[8:12].startswith(b"avi"): # AVIF
return True
return False
def slugify(tag: str) -> str:
"""Mimic Hugo's urlize for tag URLs."""
return re.sub(r"[^a-z0-9]+", "-", tag.lower()).strip("-")
def sc_param(value):
"""Escape a value for use inside a quoted shortcode parameter."""
if value is None:
return ""
value = str(value).replace("\n", " ").strip()
# Escape internal quotes and backslashes for the shortcode
return value.replace("\\", "\\\\").replace('"', '\\"')
def record_card(record, local_images, no_tags):
"""
Build one bookmark-card shortcode invocation for a raindrop record.
local_images values are either:
- "images/<file>" (relative - main listing pages; the shortcode
resolves the filename against the page bundle's resources), or
- "/bookmarks/.../<file>" (absolute - tag/year pages referencing covers
that live on the main listing pages).
"""
rd = record.get("raindrop", {})
title = sc_param(rd.get("title") or rd.get("domain") or "(untitled)")
link = sc_param(rd.get("link") or "")
domain = sc_param(rd.get("domain") or "")
image_key = local_images.get(record.get("id"))
if image_key:
image = image_key if image_key.startswith("/") else Path(image_key).name
else:
image = PLACEHOLDER_IMAGE
image = sc_param(image)
excerpt = sc_param(rd.get("excerpt"))[:280]
if no_tags:
tags = ""
else:
tags = ", ".join(t for t in (rd.get("tags") or []) if t)
tags = sc_param(tags)
created = ""
if rd.get("created"):
try:
created = dt.datetime.fromisoformat(
rd["created"].replace("Z", "+00:00")).strftime(DATE_FORMAT)
except ValueError:
created = ""
return (
'{{< bookmark-card '
f'image="{image}" '
f'title="{title}" '
f'link="{link}" '
f'domain="{domain}" '
f'date="{created}" '
f'tags="{tags}" '
f'excerpt="{excerpt}"'
' >}}'
)
def page_path(output_dir, page_number):
"""Return path for a numbered page in the main listing."""
if page_number == 1:
return output_dir / "_index.md"
return output_dir / "page" / str(page_number) / "index.md"
def write_page(path, page_number, total_pages, total_records, records,
front_matter, s3_client, cover_bucket, no_tags):
path.parent.mkdir(parents=True, exist_ok=True)
fm = dict(front_matter)
fm.update({
"page": page_number,
"totalPages": total_pages,
"totalRecords": total_records,
})
# Download this page's covers into its own images/ subdirectory
if cover_bucket:
images_dir = path.parent / IMAGE_DIR_NAME
local_images = download_page_covers(s3_client, cover_bucket, records, images_dir)
else:
local_images = {}
cards = "\n".join(record_card(r, local_images, no_tags) for r in records)
nav = (
'{{< bookmark-pagination '
f'current="{page_number}" '
f'total="{total_pages}" '
f'base="/bookmarks/"'
' >}}'
)
content = (
"---\n"
f"{yaml.safe_dump(fm, sort_keys=False, allow_unicode=True)}"
"---\n\n"
f"{fm.get('description', '')}\n\n"
'<div class="Box">\n\n'
f"{cards}\n\n"
'</div>\n\n'
f"{nav}\n"
)
path.write_text(content, encoding="utf-8")
logger.info("Wrote %s (%d cards)", path, len(records))
# Return the download results so other pages can reference these covers
return local_images
def write_year_pages(output_dir, records, record_image, args):
"""
Generate /bookmarks/years/ - an overview index plus paginated yearly
archives at /bookmarks/years/YYYY/, with cards referencing covers on
the main listing pages. Years are newest-first; undated records go
under /bookmarks/years/unknown/.
"""
years_dir = output_dir / YEARS_DIR_NAME
# Group records by year (records are already newest-first)
year_groups = defaultdict(list)
for record in records:
raindrop = record.get("raindrop", {})
created = raindrop.get("created")
if created:
try:
ts = dt.datetime.fromisoformat(created.replace("Z", "+00:00"))
year_groups[ts.year].append(record)
continue
except ValueError:
pass
year_groups[UNKNOWN_YEAR_SLUG].append(record)
if not year_groups:
logger.info("No dated bookmarks found - skipping year pages")
return 0
# Absolute cover URL for a record, or the placeholder
def cover_for(record):
entry = record_image.get(record["id"])
if entry:
page_no, filename = entry
if page_no == 1:
return f"/bookmarks/{IMAGE_DIR_NAME}/{filename}"
return f"/bookmarks/page/{page_no}/{IMAGE_DIR_NAME}/{filename}"
return PLACEHOLDER_IMAGE
written = 0
# Sort years descending (newest first), with "unknown" last
sorted_years = sorted(
[y for y in year_groups.keys() if y != UNKNOWN_YEAR_SLUG],
reverse=True,
)
if UNKNOWN_YEAR_SLUG in year_groups:
sorted_years.append(UNKNOWN_YEAR_SLUG)
for year in sorted_years:
year_records = year_groups[year]
year_slug = str(year)
display_name = "Undated" if year == UNKNOWN_YEAR_SLUG else year_slug
year_total = (len(year_records) + args.year_page_size - 1) // args.year_page_size
# Deterministic page date: newest bookmark in this group
group_newest = sort_key(year_records[0])[0].astimezone(dt.timezone.utc).replace(tzinfo=None)
for yp in range(1, year_total + 1):
start = (yp - 1) * args.year_page_size
chunk = year_records[start:start + args.year_page_size]
# Branch bundle on page 1 so nested page/N bundles are built
path = (years_dir / year_slug / "_index.md") if yp == 1 else \
(years_dir / year_slug / "page" / str(yp) / "index.md")
path.parent.mkdir(parents=True, exist_ok=True)
cards = []
for record in chunk:
url = cover_for(record)
cards.append(record_card(record, {record["id"]: url}, False))
nav = (
'{{< bookmark-pagination '
f'current="{yp}" '
f'total="{year_total}" '
f'base="/bookmarks/years/{year_slug}/"'
' >}}'
)
fm = {
"title": f"Bookmarks from {display_name}",
"description": f"{len(year_records)} bookmarks from {display_name}",
"date": group_newest.strftime("%Y-%m-%dT%H:%M:%SZ"),
"hideReadingTime": True,
"rssIgnore": True,
"layout": "single",
"sitemap": {"changefreq": "weekly", "priority": 0.4},
}
path.write_text(
"---\n"
f"{yaml.safe_dump(fm, sort_keys=False, allow_unicode=True)}"
"---\n\n"
f"{len(year_records)} bookmarks from {display_name}\n\n"
'<div class="Box">\n\n'
+ "\n".join(cards) + "\n\n"
'</div>\n\n'
f"{nav}\n",
encoding="utf-8",
)
written += 1
# Overview index: /bookmarks/years/
overview = years_dir / "_index.md"
overview.parent.mkdir(parents=True, exist_ok=True)
year_links = "\n".join(
f"- [{('Undated' if y == UNKNOWN_YEAR_SLUG else y)}](/bookmarks/years/{y}/) ({len(year_groups[y])})"
for y in sorted_years
)
# Deterministic date: newest bookmark overall
overview_newest = sort_key(records[0])[0].astimezone(dt.timezone.utc).replace(tzinfo=None)
overview.write_text(
"---\n"
+ yaml.safe_dump({
"title": "Bookmarks by year",
"description": "Browse bookmarks grouped by the year they were saved",
"date": overview_newest.strftime("%Y-%m-%dT%H:%M:%SZ"),
"hideReadingTime": True,
"rssIgnore": True,
"layout": "single",
}, sort_keys=False, allow_unicode=True)
+ "---\n\n"
+ year_links + "\n",
encoding="utf-8",
)
logger.info("Wrote %d year pages across %d years", written, len(year_groups))
return written
def write_tag_pages(output_dir, records, record_image, args):
"""
Generate /bookmarks/tags/ - an overview page plus one listing per tag,
paginated, with cards referencing covers on the main listing pages.
"""
tags_dir = output_dir / TAGS_DIR_NAME
# Group records by tag (records are already newest-first)
tag_counts = {}
for record in records:
for tag in record.get("raindrop", {}).get("tags") or []:
tag_counts.setdefault(tag, []).append(record)
if not tag_counts:
logger.info("No tags present - skipping tag pages")
return 0
# Absolute cover URL for a record, or the placeholder
def cover_for(record):
entry = record_image.get(record["id"])
if entry:
page_no, filename = entry
if page_no == 1:
return f"/bookmarks/{IMAGE_DIR_NAME}/{filename}"
return f"/bookmarks/page/{page_no}/{IMAGE_DIR_NAME}/{filename}"
return PLACEHOLDER_IMAGE
written = 0
for tag, tag_records in sorted(tag_counts.items(), key=lambda kv: kv[0]):
slug = slugify(tag)
tag_total = (len(tag_records) + args.tag_page_size - 1) // args.tag_page_size
tag_newest = sort_key(tag_records[0])[0].astimezone(dt.timezone.utc).replace(tzinfo=None)
for tp in range(1, tag_total + 1):
start = (tp - 1) * args.tag_page_size
chunk = tag_records[start:start + args.tag_page_size]
# Branch bundle on page 1 so nested page/N bundles are built
path = (tags_dir / slug / "_index.md") if tp == 1 else \
(tags_dir / slug / "page" / str(tp) / "index.md")
path.parent.mkdir(parents=True, exist_ok=True)
cards = []
for record in chunk:
url = cover_for(record)
cards.append(record_card(record, {record["id"]: url}, False))
nav = (
'{{< bookmark-pagination '
f'current="{tp}" '
f'total="{tag_total}" '
f'base="/bookmarks/tags/{slug}/"'
' >}}'
)
fm = {
"title": f"Bookmarks tagged \u201c{tag}\u201d",
"description": f"{len(tag_records)} bookmarks tagged {tag}",
"date": tag_newest.strftime("%Y-%m-%dT%H:%M:%SZ"),
"hideReadingTime": True,
"layout": "single",
"rssIgnore": True,
# NOTE: deliberately NO "tags" key here - setting it would
# register these pages in the site-wide post taxonomy
"sitemap": {"changefreq": "weekly", "priority": 0.4},
}
path.write_text(
"---\n"
f"{yaml.safe_dump(fm, sort_keys=False, allow_unicode=True)}"
"---\n\n"
f"{len(tag_records)} bookmarks tagged \u201c{tag}\u201d\n\n"
'<div class="Box">\n\n'
+ "\n".join(cards) + "\n\n"
'</div>\n\n'
f"{nav}\n",
encoding="utf-8",
)
written += 1
# Overview page: /bookmarks/tags/
overview = tags_dir / "_index.md"
overview.parent.mkdir(parents=True, exist_ok=True)
tag_links = "\n".join(
f"- [{tag}](/bookmarks/tags/{slugify(tag)}/) ({len(recs)})"
for tag, recs in sorted(tag_counts.items(), key=lambda kv: kv[0])
)
overview.write_text(
"---\n"
+ yaml.safe_dump({
"title": "Bookmark tags",
"description": "All tags used across bookmarks",
"hideReadingTime": True,
"layout": "single",
"rssIgnore": True,
}, sort_keys=False, allow_unicode=True)
+ "---\n\n"
+ tag_links + "\n",
encoding="utf-8",
)
logger.info("Wrote %d tag pages across %d tags", written, len(tag_counts))
return written
def main():
parser = argparse.ArgumentParser(description="Generate Hugo bookmark pages from DynamoDB")
parser.add_argument("--table", default="maceage-blog-raindrops")
parser.add_argument("--collection-id", type=int, required=True)
parser.add_argument("--profile", type=str, default=None, help="AWS profile to use (local testing)")
parser.add_argument("--region", type=str, default=None, help="AWS region to use (local testing)")
parser.add_argument("--cover-bucket", default=None, help="S3 bucket holding cover images; enables local downloads")
parser.add_argument("--output-dir", default="content/bookmarks")
parser.add_argument("--page-size", type=int, default=PAGE_SIZE)
parser.add_argument("--tag-page-size", type=int, default=TAG_PAGE_SIZE)
parser.add_argument("--year-page-size", type=int, default=YEAR_PAGE_SIZE)
parser.add_argument("--title", default="Bookmarks")
parser.add_argument("--description", default="These are articles and links that I've saved from around the web using [Raindrop.io](https://raindrop.io)\r\n\r\n[View by year](/bookmarks/years/) / [View by tag](/bookmarks/tags/)")
parser.add_argument("--no-tags", action="store_true", help="Omit tag pills from cards and skip tag page generation")
parser.add_argument("--no-years", action="store_true", help="Skip year archive page generation")
parser.add_argument("--keep-output", action="store_true")
args = parser.parse_args()
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
session_kwargs = {}
if args.profile:
session_kwargs["profile_name"] = args.profile
if args.region:
session_kwargs["region_name"] = args.region
output_dir = Path(args.output_dir)
session = boto3.Session(**session_kwargs)
table = session.resource("dynamodb").Table(args.table)
s3_client = session.client("s3") if args.cover_bucket else None
records = sorted(load_records(table, args.collection_id), key=sort_key, reverse=True)
if not records:
logger.warning("No records found for collection %d - nothing to do", args.collection_id)
return 0
# Wipes stale pages AND their images together - removed bookmarks vanish
if not args.keep_output and output_dir.exists():
logger.info("Clearing %s", output_dir)
shutil.rmtree(output_dir)
total_pages = (len(records) + args.page_size - 1) // args.page_size
newest = sort_key(records[0])[0].astimezone(dt.timezone.utc).replace(tzinfo=None)
front_matter = {
"title": args.title,
"description": args.description,
"date": newest.strftime("%Y-%m-%dT%H:%M:%SZ"),
"hideReadingTime": True,
"rssIgnore": True,
"layout": "single",
"sitemap": {"changefreq": "weekly", "priority": 0.5},
}
# Main listing pages; record_image maps record id -> (page_no, filename)
# so tag and year pages can reference covers without re-downloading
record_image = {}
for page_number in range(1, total_pages + 1):
start = (page_number - 1) * args.page_size
chunk = records[start:start + args.page_size]
local_images = write_page(page_path(output_dir, page_number), page_number,
total_pages, len(records), chunk, front_matter,
s3_client, args.cover_bucket, args.no_tags)
for record_id, local_path in local_images.items():
record_image[record_id] = (page_number, Path(local_path).name)
# Tag pages (optional)
tag_pages = 0
if not args.no_tags:
tag_pages = write_tag_pages(output_dir, records, record_image, args)
# Year archive pages (optional)
year_pages = 0
if not args.no_years:
year_pages = write_year_pages(output_dir, records, record_image, args)
print(f"Generated {total_pages} pages ({len(records)} cards) "
f"+ {tag_pages} tag pages + {year_pages} year pages in {output_dir}")
return 0
if __name__ == "__main__":
sys.exit(main())Hugo Shortcodes and CSS
To support the listing and pagination pages, shortcodes are used to generate the bookmark cards and pagination links, along with some custom CSS to tweak the layout.
Bookmark Card
The bookmark card shortcode is relatively straightforward and generates a card with a title, image, and a link to the bookmarked page.
The bookmark title, featured image, saved date, website domain, page excerpt, and tags are used to populate the card:
<div class="Box-row">
<article class="d-flex flex-column flex-md-row">
{{ $link := .Get "link" }}
{{ $image := .Get "image" }}
{{ if and $image (ne $image "") }}
<div class="mr-md-3 mb-2 mb-md-0" style="flex-shrink: 0;">
<a href="{{ $link }}" target="_blank" rel="noopener noreferrer">
{{ if or (hasPrefix $image "/") (hasPrefix $image "http") }}
<img src="{{ $image }}" alt="{{ .Get "title" }}" loading="lazy" class="rounded-2 post-list-image bookmark-card__image">
{{ else }}
{{ $res := .Page.Resources.GetMatch (printf "images/%s" $image) }}
{{ if $res }}
<img src="{{ $res.RelPermalink }}" alt="{{ .Get "title" }}" loading="lazy" class="rounded-2 post-list-image bookmark-card__image">
{{ else }}
<img src="{{ "/images/bookmark-placeholder.jpg" }}" alt="{{ .Get "title" }}" loading="lazy" class="rounded-2 post-list-image bookmark-card__image">
{{ end }}
{{ end }}
</a>
</div>
{{ end }}
<div class="flex-auto">
<h2 class="f3 lh-condensed mb-1">
<a href="{{ $link }}" class="Link--primary" target="_blank" rel="noopener noreferrer">{{ .Get "title" }}</a>
</h2>
<div class="d-flex flex-items-center color-fg-muted f6 mb-2">
{{ with .Get "date" }}
{{ $year := slicestr . 0 4 }}
<a href="/bookmarks/years/{{ $year }}/" class="color-fg-muted">
<time datetime="{{ . }}">Saved on {{ . }}</time>
</a>
<span class="mx-2">·</span>{{ end }}
<span>{{ .Get "domain" }}</span>
</div>
{{ with .Get "excerpt" }}
<p class="color-fg-muted mb-2">{{ . }}</p>
{{ end }}
{{ with .Get "tags" }}
<div class="d-flex flex-wrap">
{{ range $tag := split . "," }}
{{ $tag = trim $tag " " }}
<a href="/bookmarks/tags/{{ $tag | urlize }}/" class="Label Label--secondary mr-1 mb-1">{{ $tag }}</a>
{{ end }}
</div>
{{ end }}
</div>
</article>
</div>Bookmark Pagination
For pagination, the Hugo pagination shortcode is used to generate pagination links - a previous link, the current page number, page numbers either side of the current page, and a next link:
{{- $current := int (.Get "current") -}}
{{- $total := int (.Get "total") -}}
{{- $base := "/bookmarks/" -}}
{{- with .Get "base" }}{{ $base = . }}{{ end -}}
{{- if not (hasSuffix $base "/") }}{{ $base = printf "%s/" $base }}{{ end -}}
{{- $lo := sub $current 1 -}}
{{- $hi := add $current 1 -}}
<nav class="paginate-container mt-4" aria-label="Pagination">
<div class="pagination">
{{- if gt $current 1 -}}
{{- $prev := sub $current 1 -}}
{{- $prevUrl := $base -}}
{{- if ne $prev 1 }}{{ $prevUrl = printf "%spage/%d/" $base $prev }}{{ end -}}
<a class="previous_page" href="{{ $prevUrl }}" rel="prev" aria-label="Previous page">
Previous
</a>
{{- else -}}
<span class="previous_page" aria-disabled="true">
Previous
</span>
{{- end -}}
{{- $inGap := false -}}
{{- range $i := seq 1 $total -}}
{{- $show := or (eq $i 1) (eq $i $total) (and (ge $i $lo) (le $i $hi)) -}}
{{- if $show -}}
{{- $inGap = false -}}
{{- if eq $i $current -}}
<em class="current" aria-current="page">{{ $i }}</em>
{{- else -}}
{{- $url := $base -}}
{{- if ne $i 1 }}{{ $url = printf "%spage/%d/" $base $i }}{{ end -}}
<a href="{{ $url }}" aria-label="Page {{ $i }}">{{ $i }}</a>
{{- end -}}
{{- else if not $inGap -}}
{{- $inGap = true -}}
<span class="gap">…</span>
{{- end -}}
{{- end -}}
{{- if lt $current $total -}}
{{- $next := add $current 1 -}}
<a class="next_page" href="{{ printf "%spage/%d/" $base $next }}" rel="next" aria-label="Next page">
Next
</a>
{{- else -}}
<span class="next_page" aria-disabled="true">
Next
</span>
{{- end -}}
</div>
</nav>Custom CSS
Although the majority of the CSS was covered by the theme, some custom styles were added to tweak the appearance of the bookmark cards and pagination links - adjusting the aspect ratio of the bookmark images and customising the appearance of the bookmark tag pills:
/* Bookmark cards */
.bookmark-card__image {
object-fit: cover;
aspect-ratio: 992 / 525;
}
.Box-row p.color-fg-muted.mb-2 {
display: -webkit-box;
-webkit-line-clamp: 3;
-webkit-box-orient: vertical;
overflow: hidden;
font-size: 14px;
}
/* Bookmark tag pills */
.markdown-body a.Label {
color: var(--fgColor-muted);
text-decoration: none;
}
/* Tag pill hover */
a.Label:hover {
color: var(--fgColor-default);
text-decoration: none;
}Deployment Workflow
Updating the deployment workflow was simply a case of including a step to install the Python dependencies and generate the bookmark pages using the generate_bookmarks.py script:
...
- name: Install dependencies
run: pip install boto3 pyyaml
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v6
with:
role-to-assume: ${{ secrets.BLOG_IAM_ROLE_ARN }}
role-session-name: MaceageBlogDeployment
aws-region: ${{ env.AWS_REGION }}
- name: Generate bookmark pages
run: |
python scripts/generate_bookmarks.py \
--collection-id ${{ secrets.RAINDROP_COLLECTION_ID }} \
--cover-bucket blog.maceage.com-raindrop
...IAM policy
As is the case with most IAM policies, this policy was mostly a matter of trial and error - attempt a deploy, then expand the permissions until the deployment worked:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ManageCloudFormationStacks",
"Effect": "Allow",
"Action": [
"cloudformation:CreateChangeSet",
"cloudformation:ExecuteChangeSet",
"cloudformation:DeleteChangeSet",
"cloudformation:DescribeChangeSet",
"cloudformation:DescribeStacks",
"cloudformation:DescribeEvents",
"cloudformation:DescribeStackEvents",
"cloudformation:GetTemplate",
"cloudformation:GetTemplateSummary",
"cloudformation:ListStackResources",
"cloudformation:ValidateTemplate",
"cloudformation:CreateUploadBucket"
],
"Resource": "*"
},
{
"Sid": "DeployArtifacts",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::blog.maceage.com-raindrop",
"arn:aws:s3:::blog.maceage.com-raindrop/*"
]
},
{
"Sid": "DeployFunction",
"Effect": "Allow",
"Action": [
"lambda:CreateFunction",
"lambda:UpdateFunctionCode",
"lambda:UpdateFunctionConfiguration",
"lambda:GetFunction",
"lambda:GetFunctionConfiguration",
"lambda:PublishVersion",
"lambda:UpdateAlias",
"lambda:GetAlias",
"lambda:AddPermission",
"lambda:RemovePermission",
"lambda:TagResource",
"lambda:UntagResource",
"lambda:DeleteFunction"
],
"Resource": "arn:aws:lambda:us-east-1:[AccountID]:function:raindrop-sync*"
},
{
"Sid": "ManageExecutionRole",
"Effect": "Allow",
"Action": [
"iam:CreateRole",
"iam:GetRole",
"iam:UpdateRole",
"iam:DeleteRole",
"iam:TagRole",
"iam:UntagRole",
"iam:PutRolePolicy",
"iam:DeleteRolePolicy",
"iam:GetRolePolicy",
"iam:AttachRolePolicy",
"iam:DetachRolePolicy"
],
"Resource": "arn:aws:iam::[AccountID]:role/MaceageBlogSyncLambdaStack-*"
},
{
"Sid": "CreateRoles",
"Effect": "Allow",
"Action": [
"iam:CreateRole",
"iam:DeleteRole",
"iam:UpdateRole",
"iam:GetRole",
"iam:TagRole",
"iam:UntagRole",
"iam:PutRolePolicy",
"iam:DeleteRolePolicy",
"iam:AttachRolePolicy",
"iam:DetachRolePolicy"
],
"Resource": "*"
},
{
"Sid": "ManageTable",
"Effect": "Allow",
"Action": [
"dynamodb:CreateTable",
"dynamodb:DeleteTable",
"dynamodb:UpdateTable",
"dynamodb:DescribeTable",
"dynamodb:TagResource",
"dynamodb:UntagResource",
"dynamodb:DescribeContinuousBackups",
"dynamodb:DescribeTimeToLive"
],
"Resource": "arn:aws:dynamodb:us-east-1:[AccountID]:table/maceage-blog-raindrops"
},
{
"Sid": "ManageParameter",
"Effect": "Allow",
"Action": [
"ssm:PutParameter",
"ssm:GetParameter",
"ssm:GetParameters",
"ssm:DeleteParameter",
"ssm:AddTagsToResource",
"ssm:RemoveTagsFromResource",
"ssm:DescribeParameters"
],
"Resource": "arn:aws:ssm:us-east-1:[AccountID]:parameter/maceage-blog/raindrop-sync/bearer-token*"
},
{
"Sid": "EncryptTokenParameter",
"Effect": "Allow",
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Resource": "arn:aws:kms:us-east-1:[AccountID]:alias/aws/ssm",
"Condition": {
"StringEquals": {
"kms:ViaService": "ssm.us-east-1.amazonaws.com"
}
}
},
{
"Sid": "ManageSchedule",
"Effect": "Allow",
"Action": [
"scheduler:CreateSchedule",
"scheduler:GetSchedule",
"scheduler:DeleteSchedule",
"scheduler:UpdateSchedule",
"scheduler:TagResource",
"scheduler:UntagResource"
],
"Resource": "arn:aws:scheduler:us-east-1:[AccountID]:schedule/default/RaindropSyncLambda*"
},
{
"Sid": "ManageLogGroups",
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:DeleteLogGroup",
"logs:DescribeLogGroups",
"logs:PutRetentionPolicy",
"logs:DeleteRetentionPolicy",
"logs:TagResource",
"logs:UntagResource"
],
"Resource": "arn:aws:logs:us-east-1:[AccountID]:log-group:/aws/lambda/raindrop-sync*"
},
{
"Sid": "PassExecutionRoles",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "*",
"Condition": {
"StringEquals": {
"iam:PassedToService": [
"lambda.amazonaws.com",
"scheduler.amazonaws.com"
]
}
}
}
]
}Troubleshooting
While troubleshooting the bookmark sync process I encountered a few issues, mostly related to the bookmark cover images.
logger.info() not working in Lambda
When I first ran the Lambda, nothing was appearing in the CloudWatch logs. This was because the Lambda function was not configured to use the logging module correctly.
The following code snippet shows how to configure the logging module to work with Lambda:
if len(logging.getLogger().handlers) > 0:
logging.getLogger().setLevel(logging.INFO)
else:
logging.basicConfig(level=logging.INFO)Downloading cover images
After I’d got the bookmark pages generated and deployed, I noticed that some of the bookmarks being listed had images specified, but the images themselves were not displaying correctly and the placeholder image was not being shown either.
After some investigation, I found that the problem was caused by the sync Lambda downloading HTML error pages from Raindrop’s CDN, which returned HTTP 200 with a text/html body. The sync then dutifully uploaded these HTML pages as covers/{id}.jpg, and because it only ever does a HeadObject existence check on subsequent runs, the poisoned objects were cached in the bucket forever. The generator downloaded these HTML pages faithfully, and my OS image viewer chokes on a JPEG that starts with <!DOCTYPE html>.
The fix for this (which you may have already noticed in the script above) is to check the content of the image file to determine whether it is actually an image or contains text. Both the raindrop_sync.py and generate_bookmarks.py scripts employ a sniff_image() function that looks at the header of the file to verify that it matches a known image signature. If it doesn’t, the placeholder image is used as the thumbnail instead.
Incorrect URLs for bookmark images
Once I’d got the invalid images issue resolved and deployed the changes, I realised some images were not displaying at all - the path to the cover image was resolving at the root rather than at the /bookmarks/images URL.
Eventually I realised the issue was in how Hugo handles page-bundle resources. When images are placed inside content/bookmarks/page/2/images/, Hugo doesn’t automatically expose them at that URL - they need to be referenced through Hugo’s resource system so it calculates the correct permalink.
The simplest fix was to change the generator to use Hugo’s resource syntax:
...
# Pass the raw filename - the shortcode will resolve it via .Page.Resources.GetMatch
image_key = local_images.get(record.get("id"))
if image_key:
image = Path(image_key).name # "images/55660011.jpg" → "55660011.jpg"
else:
image = PLACEHOLDER_IMAGE
image = sc_param(image)
...And then update the shortcode to resolve it properly:
<div class="mr-md-3 mb-2 mb-md-0" style="flex-shrink: 0;">
<a href="{{ $link }}" rel="noopener noreferrer">
{{ if eq $image "/img/bookmark-placeholder.png" }}
<img src="{{ $image }}" alt="{{ .Get "title" }}" loading="lazy" class="rounded-2 post-list-image bookmark-card__image">
{{ else }}
{{ $res := .Page.Resources.GetMatch (printf "images/%s" $image) }}
{{ if $res }}
<img src="{{ $res.RelPermalink }}" alt="{{ .Get "title" }}" loading="lazy" class="rounded-2 post-list-image bookmark-card__image">
{{ else }}
<img src="{{ "/img/bookmark-placeholder.png" }}" alt="{{ .Get "title" }}" loading="lazy" class="rounded-2 post-list-image bookmark-card__image">
{{ end }}
{{ end }}
</a>
</div>This fix causes the following:
- The generator passes just the filename (e.g.
55660011.jpg) instead of a path likeimages/55660011.jpg. Theimages/prefix is implicit, since Hugo knows where the images directory lives for each page bundle. - The shortcode uses
.Page.Resources.GetMatchto look upimages/<filename>relative to the current page bundle. This ensures Hugo resolves it through its resource pipeline, producing the correct relative permalink (e.g./bookmarks/page/2/images/55660011.jpg). - The placeholder stays absolute (
/images/bookmark-placeholder.jpg) since it’s a static asset, not a page resource.
Index.html files not being created
With the images URL issue fixed, I redeployed the site. When navigating to the second page of bookmarks, I encountered the following error:
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
</Error>When I looked in the S3 bucket, there was an index.html file at the root /bookmarks URL but no index.html files in the bookmarks/page/2, bookmarks/page/3 and bookmarks/page/4 paths.
I then learnt that in Hugo’s model, a directory containing index.md is a self-contained leaf bundle: everything under it - including page/2/index.md, page/3/index.md and all the images/ directories - is treated as the bundle’s resources, not as standalone pages. Therefore, nested content pages inside a leaf bundle simply aren’t built, and Hugo happily emitted /bookmarks/index.html from the page-1 bundle while silently absorbing the other pages as unreachable resources.
The fix was to make the page-1 entry a branch bundle instead - _index.md - a one-line change in the generator:
def page_path(output_dir, page_number):
if page_number == 1:
return output_dir / "_index.md" # branch bundle - nested pages are built
return output_dir / "page" / str(page_number) / "index.md"Bookmark pages in RSS feed
After deploying the site, it took me quite a while to realise that Hugo’s RSS feed generator was including all the bookmark pages as part of the feed.
Searching around, I found a post that explained how to modify the RSS template to exclude certain pages from the feed.
To achieve this I had to modify the RSS template under layouts/_default/index.xml to exclude those pages that have rssIgnore set to true in the page’s front matter:
...
{{- range $pages }}{{ if ne .Params.rssIgnore true }}
<item>
<title>{{ .Title }}</title>
<link>{{ .Permalink }}</link>
...
</item>
{{ end }}{{- end }}
...Wrapping up
While I had a few issues getting the bookmarks and their covers to display correctly, the Lambda function has worked flawlessly since it was set up.
Every morning at 4am, the Lambda function runs and updates the DynamoDB table with the latest bookmarks from Raindrop.io.
Then at 6am, the GitHub Actions workflow for the blog runs automatically, generates the Markdown for the bookmark pages, and Hugo builds and deploys the changes.
To see the result, check out the Bookmarks section
Related Posts
2026-01-09
2026-01-07
