# Adding Bookmarks to My Blog Part 2: Publication

*2026-11-17*

> Synchronising Raindrop.io saved links and pages to my blog


With the bookmark data and cover images successfully [synchronised from Raindrop.io via the API]({{< relref "adding-bookmarks-to-my-blog-1" >}}), the next step was to generate the corresponding Markdown pages for Hugo to process.

My first version of the script generated the Markdown pages into the `content/bookmarks` directory, paginating the data across 25 items per page.

Once that was working, I updated the script to render each bookmark as a listing card, similar to the [Posts](/posts) page. Rather than generating raw HTML, the script emits Markdown with a [Hugo Shortcode](https://gohugo.io/content-management/shortcodes/), which then renders the content of the bookmark to HTML on build.

# Contents

This post covers the Python script used to generate the Markdown pages from the DynamoDB table:

* [**Markdown pages**](#markdown-pages) - how the generated pages are organised
* [**Bookmark Page Generation Script**](#bookmark-page-generation-script) - Python script used for generating the Markdown pages
* [**Hugo Shortcodes and CSS**](#hugo-shortcodes-and-css) - Hugo shortcodes and custom CSS
* [**Deployment Workflow**](#deployment-workflow) - GitHub Actions deployment workflow updates
* [**IAM Policy**](#iam-policy) - updated IAM policy used to deploy the blog site and generate bookmark pages
* [**Troubleshooting**](#troubleshooting) - issues that were encountered during the development

## Markdown pages

The pages themselves are organised under folders such as `content/bookmarks/_index.md`, `content/bookmarks/page/2/index.md`, `content/bookmarks/page/3/index.md` and so on, with pagination links at the bottom of each page.

Given my blog already uses the [GitHub Primer theme](https://github.com/ngs/hugo-primer-blog), I wanted to reuse the same CSS styling with minimal additions where possible.

For the bookmark covers, the script downloads the image files from the S3 bucket and then places them in an `images` subdirectory next to the Markdown file:

![Pages by index](pages-by-index.png#center)

Once I'd got the listing and bookmark covers working, I updated the script to generate pages for each tag under paths such as `content/bookmarks/tags/tech`, `content/bookmarks/tags/code`, `content/bookmarks/tags/ai` and so on:

![Pages by tag](pages-by-tags.png#center)

Finally, for listing bookmarks by year, the script generates Markdown pages under paths such as `content/bookmarks/years/2024`, `content/bookmarks/years/2025`, `content/bookmarks/years/2026` and so on:

![Pages by year](pages-by-year.png#center)

# Bookmark Page Generation Script{#bookmark-page-generation-script}

Creating this script took the most tweaking and testing to get right and it consists of the following methods:

| Name                   | Description                                                                                                        |
|------------------------|--------------------------------------------------------------------------------------------------------------------|
| `decimal_safe`         | Converts the `Decimal` values DynamoDB returns into ints so they survive YAML front matter and Markdown            |
| `load_records`         | Retrieves every record for the collection from the DynamoDB table, following pagination tokens                    |
| `sort_key`            | Builds the sort key so records are ordered newest first, falling back to id for undated records                   |
| `download_cover`       | Downloads a bookmark cover from the S3 bucket into the page's `images` directory                                   |
| `download_page_covers` | Fans the page's cover downloads out to a pool of worker threads                                                    |
| `sniff_image`         | Checks the downloaded bytes against known image signatures so HTML error pages don't slip through                 |
| `slugify`             | Converts a tag string into a slugified URL with dashes                                                            |
| `sc_param`             | Escapes a value for use inside a quoted shortcode parameter                                                        |
| `record_card`         | Generates the bookmark-card shortcode for a listing page                                                           |
| `page_path`           | Returns the path for a numbered page in the main listing                                                          |
| `write_page`          | Writes the Markdown and front matter for a listing page, including covers and pagination                          |
| `write_year_pages`     | Groups records by year and generates paginated yearly archive pages                                                |
| `write_tag_pages`      | Groups records by tag and generates navigation, front matter and paginated listing pages                           |
| `main`                 | Parses command-line arguments and generates the main listing, tag pages and year pages                             |

The full script is as follows:

```python
"""
Generate Hugo pages from the raindrops DynamoDB table (card listing),
downloading each page's cover images from S3 into a per-page images dir:

    content/bookmarks/_index.md            + content/bookmarks/images/
    content/bookmarks/page/2/index.md     + content/bookmarks/page/2/images/
    ...

Also generates per-tag listing pages:

    content/bookmarks/tags/_index.md                  (overview of all tags)
    content/bookmarks/tags/<slug>/_index.md           (tag page 1)
    content/bookmarks/tags/<slug>/page/2/index.md      (tag pagination)

Also generates paginated yearly archives:

    content/bookmarks/years/_index.md      (overview of all years)
    content/bookmarks/years/2025/_index.md        (all 2025 bookmarks)
    content/bookmarks/years/2025/page/2/index.md  (pagination)
    content/bookmarks/years/2024/_index.md
    ...

Year page cards reference covers via absolute URLs pointing at the main
listing pages where the images already live - no duplication.

Usage:
    python generate_bookmarks.py [--table raindrops] [--collection-id 12345678] \
        [--cover-bucket BUCKET] [--output-dir content/bookmarks] [--page-size 25] \
        [--tag-page-size 25] [--year-page-size 25] [--no-tags] [--no-years]
"""

import argparse
import concurrent.futures
import datetime as dt
import decimal
import logging
import re
import shutil
import sys
from collections import defaultdict
from pathlib import Path

import boto3
import yaml

logger = logging.getLogger(__name__)

TABLE_KEY = "collectionId"
DATE_FORMAT = "%Y-%m-%d"
IMAGE_DIR_NAME = "images"
PLACEHOLDER_IMAGE = "/images/bookmark-placeholder.jpg"
DOWNLOAD_WORKERS = 10
PAGE_SIZE = 25
TAG_PAGE_SIZE = 25
TAGS_DIR_NAME = "tags"
YEAR_PAGE_SIZE = 25
YEARS_DIR_NAME = "years"
UNKNOWN_YEAR_SLUG = "unknown"

def decimal_safe(value):
    """DynamoDB returns Numbers as Decimal; convert for YAML/markdown."""
    if isinstance(value, decimal.Decimal):
        return int(value)
    return value

def load_records(table, collection_id):
    """Query every record for the collection, following pagination tokens."""
    seen, items = 0, []
    kwargs = {
        "KeyConditionExpression": boto3.dynamodb.conditions.Key(TABLE_KEY).eq(collection_id),
        "ConsistentRead": False,
    }
    while True:
        response = table.query(**kwargs)
        items.extend(response.get("Items", []))
        seen += response.get("Count", 0)
        if "LastEvaluatedKey" not in response:
            break
        kwargs["ExclusiveStartKey"] = response["LastEvaluatedKey"]
    logger.info("Loaded %d records from DynamoDB", seen)
    return items

def sort_key(record):
    """Newest first, falling back to id for records missing a created date."""
    raindrop = record.get("raindrop", {})
    created = raindrop.get("created")
    if created:
        try:
            ts = dt.datetime.fromisoformat(created.replace("Z", "+00:00"))
            return (ts, decimal_safe(record.get("id", 0)))
        except ValueError:
            pass
    return (dt.datetime.min.replace(tzinfo=dt.timezone.utc), decimal_safe(record.get("id", 0)))

def download_cover(s3_client, bucket, record, images_dir):
    """
    Download one record's S3 cover into images_dir. Returns the local
    relative path on success, or None if the record has no S3 cover or
    the download failed (caller falls back to the placeholder).
    """
    key = record.get("coverKey")
    if not key:
        return None

    filename = Path(key).name           # covers/{cid}/{id}.jpg -> {id}.jpg
    dest = images_dir / filename
    if dest.exists():                  # already downloaded on a prior page chunk
        return IMAGE_DIR_NAME + "/" + filename

    try:
        response = s3_client.get_object(Bucket=bucket, Key=key)
        body = response["Body"].read()
        if not body:
            logger.warning("Empty cover object %s", key)
            return None
        if not sniff_image(body):
            logger.warning("Invalid image data in %s - using placeholder", key)
            return None
        dest.write_bytes(body)
        return IMAGE_DIR_NAME + "/" + filename
    except Exception as exc:
        logger.warning("Cover download failed for %s: %s", key, exc)
        return None

def download_page_covers(s3_client, bucket, records, images_dir):
    """Fan the page's cover downloads out to a thread pool."""
    images_dir.mkdir(parents=True, exist_ok=True)
    results = {}
    with concurrent.futures.ThreadPoolExecutor(max_workers=DOWNLOAD_WORKERS) as pool:
        futures = {
            pool.submit(download_cover, s3_client, bucket, record, images_dir): record
            for record in records
        }
        for future in concurrent.futures.as_completed(futures):
            record = futures[future]
            local_path = future.result()
            if local_path:
                results[record["id"]] = local_path
    logger.info("Downloaded %d/%d covers into %s",
                len(results), len(records), images_dir)
    return results

def sniff_image(data: bytes) -> bool:
    """True if the byte stream starts with a known image signature."""
    if len(data) < 12:
        return False
    if data[:3] == b"\xff\xd8\xff":                       # JPEG
        return True
    if data[:8] == b"\x89PNG\r\n\x1a\n":                  # PNG
        return True
    if data[:6] in (b"GIF87a", b"GIF89a"):                # GIF
        return True
    if data[:4] == b"RIFF" and data[8:12] == b"WEBP":      # WebP
        return True
    if data[4:8] == b"ftyp" and data[8:12].startswith(b"avi"):  # AVIF
        return True
    return False

def slugify(tag: str) -> str:
    """Mimic Hugo's urlize for tag URLs."""
    return re.sub(r"[^a-z0-9]+", "-", tag.lower()).strip("-")

def sc_param(value):
    """Escape a value for use inside a quoted shortcode parameter."""
    if value is None:
        return ""
    value = str(value).replace("\n", " ").strip()
    # Escape internal quotes and backslashes for the shortcode
    return value.replace("\\", "\\\\").replace('"', '\\"')

def record_card(record, local_images, no_tags):
    """
    Build one bookmark-card shortcode invocation for a raindrop record.

    local_images values are either:
      - "images/<file>" (relative - main listing pages; the shortcode
        resolves the filename against the page bundle's resources), or
      - "/bookmarks/.../<file>" (absolute - tag/year pages referencing covers
        that live on the main listing pages).
    """
    rd = record.get("raindrop", {})

    title = sc_param(rd.get("title") or rd.get("domain") or "(untitled)")
    link = sc_param(rd.get("link") or "")
    domain = sc_param(rd.get("domain") or "")

    image_key = local_images.get(record.get("id"))
    if image_key:
        image = image_key if image_key.startswith("/") else Path(image_key).name
    else:
        image = PLACEHOLDER_IMAGE
    image = sc_param(image)

    excerpt = sc_param(rd.get("excerpt"))[:280]

    if no_tags:
        tags = ""
    else:
        tags = ", ".join(t for t in (rd.get("tags") or []) if t)

    tags = sc_param(tags)

    created = ""
    if rd.get("created"):
        try:
            created = dt.datetime.fromisoformat(
                rd["created"].replace("Z", "+00:00")).strftime(DATE_FORMAT)
        except ValueError:
            created = ""

    return (
        '{{</* bookmark-card '
        f'image="{image}" '
        f'title="{title}" '
        f'link="{link}" '
        f'domain="{domain}" '
        f'date="{created}" '
        f'tags="{tags}" '
        f'excerpt="{excerpt}"'
        ' */>}}'
    )

def page_path(output_dir, page_number):
    """Return path for a numbered page in the main listing."""
    if page_number == 1:
        return output_dir / "_index.md"
    return output_dir / "page" / str(page_number) / "index.md"

def write_page(path, page_number, total_pages, total_records, records,
               front_matter, s3_client, cover_bucket, no_tags):
    path.parent.mkdir(parents=True, exist_ok=True)

    fm = dict(front_matter)
    fm.update({
        "page": page_number,
        "totalPages": total_pages,
        "totalRecords": total_records,
    })

    # Download this page's covers into its own images/ subdirectory
    if cover_bucket:
        images_dir = path.parent / IMAGE_DIR_NAME
        local_images = download_page_covers(s3_client, cover_bucket, records, images_dir)
    else:
        local_images = {}

    cards = "\n".join(record_card(r, local_images, no_tags) for r in records)

    nav = (
        '{{</* bookmark-pagination '
        f'current="{page_number}" '
        f'total="{total_pages}" '
        f'base="/bookmarks/"'
        ' */>}}'
    )

    content = (
        "---\n"
        f"{yaml.safe_dump(fm, sort_keys=False, allow_unicode=True)}"
        "---\n\n"
        f"{fm.get('description', '')}\n\n"
        '<div class="Box">\n\n'
        f"{cards}\n\n"
        '</div>\n\n'
        f"{nav}\n"
    )
    path.write_text(content, encoding="utf-8")
    logger.info("Wrote %s (%d cards)", path, len(records))

    # Return the download results so other pages can reference these covers
    return local_images

def write_year_pages(output_dir, records, record_image, args):
    """
    Generate /bookmarks/years/ - an overview index plus paginated yearly
    archives at /bookmarks/years/YYYY/, with cards referencing covers on
    the main listing pages. Years are newest-first; undated records go
    under /bookmarks/years/unknown/.
    """
    years_dir = output_dir / YEARS_DIR_NAME

    # Group records by year (records are already newest-first)
    year_groups = defaultdict(list)
    for record in records:
        raindrop = record.get("raindrop", {})
        created = raindrop.get("created")
        if created:
            try:
                ts = dt.datetime.fromisoformat(created.replace("Z", "+00:00"))
                year_groups[ts.year].append(record)
                continue
            except ValueError:
                pass
        year_groups[UNKNOWN_YEAR_SLUG].append(record)

    if not year_groups:
        logger.info("No dated bookmarks found - skipping year pages")
        return 0

    # Absolute cover URL for a record, or the placeholder
    def cover_for(record):
        entry = record_image.get(record["id"])
        if entry:
            page_no, filename = entry
            if page_no == 1:
                return f"/bookmarks/{IMAGE_DIR_NAME}/{filename}"
            return f"/bookmarks/page/{page_no}/{IMAGE_DIR_NAME}/{filename}"
        return PLACEHOLDER_IMAGE

    written = 0

    # Sort years descending (newest first), with "unknown" last
    sorted_years = sorted(
        [y for y in year_groups.keys() if y != UNKNOWN_YEAR_SLUG],
        reverse=True,
    )
    if UNKNOWN_YEAR_SLUG in year_groups:
        sorted_years.append(UNKNOWN_YEAR_SLUG)

    for year in sorted_years:
        year_records = year_groups[year]
        year_slug = str(year)
        display_name = "Undated" if year == UNKNOWN_YEAR_SLUG else year_slug
        year_total = (len(year_records) + args.year_page_size - 1) // args.year_page_size

        # Deterministic page date: newest bookmark in this group
        group_newest = sort_key(year_records[0])[0].astimezone(dt.timezone.utc).replace(tzinfo=None)

        for yp in range(1, year_total + 1):
            start = (yp - 1) * args.year_page_size
            chunk = year_records[start:start + args.year_page_size]

            # Branch bundle on page 1 so nested page/N bundles are built
            path = (years_dir / year_slug / "_index.md") if yp == 1 else \
                (years_dir / year_slug / "page" / str(yp) / "index.md")
            path.parent.mkdir(parents=True, exist_ok=True)

            cards = []
            for record in chunk:
                url = cover_for(record)
                cards.append(record_card(record, {record["id"]: url}, False))

            nav = (
                '{{</* bookmark-pagination '
                f'current="{yp}" '
                f'total="{year_total}" '
                f'base="/bookmarks/years/{year_slug}/"'
                ' */>}}'
            )

            fm = {
                "title": f"Bookmarks from {display_name}",
                "description": f"{len(year_records)} bookmarks from {display_name}",
                "date": group_newest.strftime("%Y-%m-%dT%H:%M:%SZ"),
                "hideReadingTime": True,
                "rssIgnore": True,
                "layout": "single",
                "sitemap": {"changefreq": "weekly", "priority": 0.4},
            }
            path.write_text(
                "---\n"
                f"{yaml.safe_dump(fm, sort_keys=False, allow_unicode=True)}"
                "---\n\n"
                f"{len(year_records)} bookmarks from {display_name}\n\n"
                '<div class="Box">\n\n'
                + "\n".join(cards) + "\n\n"
                                     '</div>\n\n'
                                     f"{nav}\n",
                encoding="utf-8",
                )
            written += 1

    # Overview index: /bookmarks/years/
    overview = years_dir / "_index.md"
    overview.parent.mkdir(parents=True, exist_ok=True)
    year_links = "\n".join(
        f"- [{('Undated' if y == UNKNOWN_YEAR_SLUG else y)}](/bookmarks/years/{y}/) ({len(year_groups[y])})"
        for y in sorted_years
    )
    # Deterministic date: newest bookmark overall
    overview_newest = sort_key(records[0])[0].astimezone(dt.timezone.utc).replace(tzinfo=None)
    overview.write_text(
        "---\n"
        + yaml.safe_dump({
            "title": "Bookmarks by year",
            "description": "Browse bookmarks grouped by the year they were saved",
            "date": overview_newest.strftime("%Y-%m-%dT%H:%M:%SZ"),
            "hideReadingTime": True,
            "rssIgnore": True,
            "layout": "single",
        }, sort_keys=False, allow_unicode=True)
        + "---\n\n"
        + year_links + "\n",
        encoding="utf-8",
        )
    logger.info("Wrote %d year pages across %d years", written, len(year_groups))
    return written

def write_tag_pages(output_dir, records, record_image, args):
    """
    Generate /bookmarks/tags/ - an overview page plus one listing per tag,
    paginated, with cards referencing covers on the main listing pages.
    """
    tags_dir = output_dir / TAGS_DIR_NAME

    # Group records by tag (records are already newest-first)
    tag_counts = {}
    for record in records:
        for tag in record.get("raindrop", {}).get("tags") or []:
            tag_counts.setdefault(tag, []).append(record)

    if not tag_counts:
        logger.info("No tags present - skipping tag pages")
        return 0

    # Absolute cover URL for a record, or the placeholder
    def cover_for(record):
        entry = record_image.get(record["id"])
        if entry:
            page_no, filename = entry
            if page_no == 1:
                return f"/bookmarks/{IMAGE_DIR_NAME}/{filename}"
            return f"/bookmarks/page/{page_no}/{IMAGE_DIR_NAME}/{filename}"
        return PLACEHOLDER_IMAGE

    written = 0
    for tag, tag_records in sorted(tag_counts.items(), key=lambda kv: kv[0]):
        slug = slugify(tag)
        tag_total = (len(tag_records) + args.tag_page_size - 1) // args.tag_page_size
        tag_newest = sort_key(tag_records[0])[0].astimezone(dt.timezone.utc).replace(tzinfo=None)

        for tp in range(1, tag_total + 1):
            start = (tp - 1) * args.tag_page_size
            chunk = tag_records[start:start + args.tag_page_size]

            # Branch bundle on page 1 so nested page/N bundles are built
            path = (tags_dir / slug / "_index.md") if tp == 1 else \
                (tags_dir / slug / "page" / str(tp) / "index.md")
            path.parent.mkdir(parents=True, exist_ok=True)

            cards = []
            for record in chunk:
                url = cover_for(record)
                cards.append(record_card(record, {record["id"]: url}, False))

            nav = (
                '{{</* bookmark-pagination '
                f'current="{tp}" '
                f'total="{tag_total}" '
                f'base="/bookmarks/tags/{slug}/"'
                ' */>}}'
            )

            fm = {
                "title": f"Bookmarks tagged \u201c{tag}\u201d",
                "description": f"{len(tag_records)} bookmarks tagged {tag}",
                "date": tag_newest.strftime("%Y-%m-%dT%H:%M:%SZ"),
                "hideReadingTime": True,
                "layout": "single",
                "rssIgnore": True,
                # NOTE: deliberately NO "tags" key here - setting it would
                # register these pages in the site-wide post taxonomy
                "sitemap": {"changefreq": "weekly", "priority": 0.4},
            }
            path.write_text(
                "---\n"
                f"{yaml.safe_dump(fm, sort_keys=False, allow_unicode=True)}"
                "---\n\n"
                f"{len(tag_records)} bookmarks tagged \u201c{tag}\u201d\n\n"
                '<div class="Box">\n\n'
                + "\n".join(cards) + "\n\n"
                                     '</div>\n\n'
                                     f"{nav}\n",
                encoding="utf-8",
                )
            written += 1

    # Overview page: /bookmarks/tags/
    overview = tags_dir / "_index.md"
    overview.parent.mkdir(parents=True, exist_ok=True)
    tag_links = "\n".join(
        f"- [{tag}](/bookmarks/tags/{slugify(tag)}/) ({len(recs)})"
        for tag, recs in sorted(tag_counts.items(), key=lambda kv: kv[0])
    )
    overview.write_text(
        "---\n"
        + yaml.safe_dump({
            "title": "Bookmark tags",
            "description": "All tags used across bookmarks",
            "hideReadingTime": True,
            "layout": "single",
            "rssIgnore": True,
        }, sort_keys=False, allow_unicode=True)
        + "---\n\n"
        + tag_links + "\n",
        encoding="utf-8",
        )
    logger.info("Wrote %d tag pages across %d tags", written, len(tag_counts))
    return written

def main():
    parser = argparse.ArgumentParser(description="Generate Hugo bookmark pages from DynamoDB")
    parser.add_argument("--table", default="maceage-blog-raindrops")
    parser.add_argument("--collection-id", type=int, required=True)
    parser.add_argument("--profile", type=str, default=None, help="AWS profile to use (local testing)")
    parser.add_argument("--region", type=str, default=None, help="AWS region to use (local testing)")
    parser.add_argument("--cover-bucket", default=None, help="S3 bucket holding cover images; enables local downloads")
    parser.add_argument("--output-dir", default="content/bookmarks")
    parser.add_argument("--page-size", type=int, default=PAGE_SIZE)
    parser.add_argument("--tag-page-size", type=int, default=TAG_PAGE_SIZE)
    parser.add_argument("--year-page-size", type=int, default=YEAR_PAGE_SIZE)
    parser.add_argument("--title", default="Bookmarks")
    parser.add_argument("--description", default="These are articles and links that I've saved from around the web using [Raindrop.io](https://raindrop.io)\r\n\r\n[View by year](/bookmarks/years/) / [View by tag](/bookmarks/tags/)")
    parser.add_argument("--no-tags", action="store_true", help="Omit tag pills from cards and skip tag page generation")
    parser.add_argument("--no-years", action="store_true", help="Skip year archive page generation")
    parser.add_argument("--keep-output", action="store_true")
    args = parser.parse_args()

    logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")

    session_kwargs = {}
    if args.profile:
        session_kwargs["profile_name"] = args.profile
    if args.region:
        session_kwargs["region_name"] = args.region

    output_dir = Path(args.output_dir)
    session = boto3.Session(**session_kwargs)
    table = session.resource("dynamodb").Table(args.table)
    s3_client = session.client("s3") if args.cover_bucket else None

    records = sorted(load_records(table, args.collection_id), key=sort_key, reverse=True)

    if not records:
        logger.warning("No records found for collection %d - nothing to do", args.collection_id)
        return 0

    # Wipes stale pages AND their images together - removed bookmarks vanish
    if not args.keep_output and output_dir.exists():
        logger.info("Clearing %s", output_dir)
        shutil.rmtree(output_dir)

    total_pages = (len(records) + args.page_size - 1) // args.page_size
    newest = sort_key(records[0])[0].astimezone(dt.timezone.utc).replace(tzinfo=None)

    front_matter = {
        "title": args.title,
        "description": args.description,
        "date": newest.strftime("%Y-%m-%dT%H:%M:%SZ"),
        "hideReadingTime": True,
        "rssIgnore": True,
        "layout": "single",
        "sitemap": {"changefreq": "weekly", "priority": 0.5},
    }

    # Main listing pages; record_image maps record id -> (page_no, filename)
    # so tag and year pages can reference covers without re-downloading
    record_image = {}
    for page_number in range(1, total_pages + 1):
        start = (page_number - 1) * args.page_size
        chunk = records[start:start + args.page_size]
        local_images = write_page(page_path(output_dir, page_number), page_number,
                                  total_pages, len(records), chunk, front_matter,
                                  s3_client, args.cover_bucket, args.no_tags)
        for record_id, local_path in local_images.items():
            record_image[record_id] = (page_number, Path(local_path).name)

    # Tag pages (optional)
    tag_pages = 0
    if not args.no_tags:
        tag_pages = write_tag_pages(output_dir, records, record_image, args)

    # Year archive pages (optional)
    year_pages = 0
    if not args.no_years:
        year_pages = write_year_pages(output_dir, records, record_image, args)

    print(f"Generated {total_pages} pages ({len(records)} cards) "
          f"+ {tag_pages} tag pages + {year_pages} year pages in {output_dir}")
    return 0

if __name__ == "__main__":
    sys.exit(main())
```

# Hugo Shortcodes and CSS{#hugo-shortcodes-and-css}

To support the listing and pagination pages, shortcodes are used to generate the bookmark cards and pagination links, along with some custom CSS to tweak the layout.

## Bookmark Card

The bookmark card shortcode is relatively straightforward and generates a card with a title, image, and a link to the bookmarked page.

The bookmark title, featured image, saved date, website domain, page excerpt, and tags are used to populate the card:

```html
<div class="Box-row">
    <article class="d-flex flex-column flex-md-row">
        {{ $link := .Get "link" }}
        {{ $image := .Get "image" }}
        {{ if and $image (ne $image "") }}
        <div class="mr-md-3 mb-2 mb-md-0" style="flex-shrink: 0;">
            <a href="{{ $link }}" target="_blank" rel="noopener noreferrer">
                {{ if or (hasPrefix $image "/") (hasPrefix $image "http") }}
                <img src="{{ $image }}" alt="{{ .Get "title" }}" loading="lazy" class="rounded-2 post-list-image bookmark-card__image">
                {{ else }}
                {{ $res := .Page.Resources.GetMatch (printf "images/%s" $image) }}
                {{ if $res }}
                <img src="{{ $res.RelPermalink }}" alt="{{ .Get "title" }}" loading="lazy" class="rounded-2 post-list-image bookmark-card__image">
                {{ else }}
                <img src="{{ "/images/bookmark-placeholder.jpg" }}" alt="{{ .Get "title" }}" loading="lazy" class="rounded-2 post-list-image bookmark-card__image">
                {{ end }}
                {{ end }}
            </a>
        </div>
        {{ end }}
        <div class="flex-auto">
            <h2 class="f3 lh-condensed mb-1">
                <a href="{{ $link }}" class="Link--primary" target="_blank" rel="noopener noreferrer">{{ .Get "title" }}</a>
            </h2>
            <div class="d-flex flex-items-center color-fg-muted f6 mb-2">
                {{ with .Get "date" }}
                {{ $year := slicestr . 0 4 }}
                <a href="/bookmarks/years/{{ $year }}/" class="color-fg-muted">
                    <time datetime="{{ . }}">Saved on {{ . }}</time>
                </a>
                <span class="mx-2">&middot;</span>{{ end }}
                <span>{{ .Get "domain" }}</span>
            </div>
            {{ with .Get "excerpt" }}
            <p class="color-fg-muted mb-2">{{ . }}</p>
            {{ end }}
            {{ with .Get "tags" }}
            <div class="d-flex flex-wrap">
                {{ range $tag := split . "," }}
                {{ $tag = trim $tag " " }}
                <a href="/bookmarks/tags/{{ $tag | urlize }}/" class="Label Label--secondary mr-1 mb-1">{{ $tag }}</a>
                {{ end }}
            </div>
            {{ end }}
        </div>
    </article>
</div>
```

## Bookmark Pagination

For pagination, the Hugo pagination shortcode is used to generate pagination links - a previous link, the current page number, page numbers either side of the current page, and a next link:

```html
{{- $current := int (.Get "current") -}}
{{- $total := int (.Get "total") -}}
{{- $base := "/bookmarks/" -}}
{{- with .Get "base" }}{{ $base = . }}{{ end -}}
{{- if not (hasSuffix $base "/") }}{{ $base = printf "%s/" $base }}{{ end -}}

{{- $lo := sub $current 1 -}}
{{- $hi := add $current 1 -}}

<nav class="paginate-container mt-4" aria-label="Pagination">
    <div class="pagination">
        {{- if gt $current 1 -}}
        {{- $prev := sub $current 1 -}}
        {{- $prevUrl := $base -}}
        {{- if ne $prev 1 }}{{ $prevUrl = printf "%spage/%d/" $base $prev }}{{ end -}}
        <a class="previous_page" href="{{ $prevUrl }}" rel="prev" aria-label="Previous page">
            Previous
        </a>
        {{- else -}}
        <span class="previous_page" aria-disabled="true">
      Previous
    </span>
        {{- end -}}

        {{- $inGap := false -}}
        {{- range $i := seq 1 $total -}}
        {{- $show := or (eq $i 1) (eq $i $total) (and (ge $i $lo) (le $i $hi)) -}}
        {{- if $show -}}
        {{- $inGap = false -}}
        {{- if eq $i $current -}}
        <em class="current" aria-current="page">{{ $i }}</em>
        {{- else -}}
        {{- $url := $base -}}
        {{- if ne $i 1 }}{{ $url = printf "%spage/%d/" $base $i }}{{ end -}}
        <a href="{{ $url }}" aria-label="Page {{ $i }}">{{ $i }}</a>
        {{- end -}}
        {{- else if not $inGap -}}
        {{- $inGap = true -}}
        <span class="gap">&hellip;</span>
        {{- end -}}
        {{- end -}}

        {{- if lt $current $total -}}
        {{- $next := add $current 1 -}}
        <a class="next_page" href="{{ printf "%spage/%d/" $base $next }}" rel="next" aria-label="Next page">
        Next
        </a>
        {{- else -}}
        <span class="next_page" aria-disabled="true">
      Next
    </span>
        {{- end -}}
    </div>
</nav>
```

## Custom CSS

Although the majority of the CSS was covered by the theme, some custom styles were added to tweak the appearance of the bookmark cards and pagination links - adjusting the aspect ratio of the bookmark images and customising the appearance of the bookmark tag pills:

```css
/* Bookmark cards */
.bookmark-card__image {
    object-fit: cover;
    aspect-ratio: 992 / 525;
}

.Box-row p.color-fg-muted.mb-2 {
    display: -webkit-box;
    -webkit-line-clamp: 3;
    -webkit-box-orient: vertical;
    overflow: hidden;
    font-size: 14px;
}

/* Bookmark tag pills */
.markdown-body a.Label {
    color: var(--fgColor-muted);
    text-decoration: none;
}

/* Tag pill hover */
a.Label:hover {
    color: var(--fgColor-default);
    text-decoration: none;
}
```

# Deployment Workflow{#deployment-workflow}

Updating the deployment workflow was simply a case of including a step to install the Python dependencies and generate the bookmark pages using the `generate_bookmarks.py` script:

```yaml
...

- name: Install dependencies
  run: pip install boto3 pyyaml

- name: Configure AWS credentials
  uses: aws-actions/configure-aws-credentials@v6
  with:
    role-to-assume: ${{ secrets.BLOG_IAM_ROLE_ARN }}
    role-session-name: MaceageBlogDeployment
    aws-region: ${{ env.AWS_REGION }}

- name: Generate bookmark pages
  run: |
    python scripts/generate_bookmarks.py \
    --collection-id ${{ secrets.RAINDROP_COLLECTION_ID }} \
    --cover-bucket blog.maceage.com-raindrop

...
```

# IAM policy{#iam-policy}

As is the case with most IAM policies, this policy was mostly a matter of trial and error - attempt a deploy, then expand the permissions until the deployment worked:

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ManageCloudFormationStacks",
            "Effect": "Allow",
            "Action": [
                "cloudformation:CreateChangeSet",
                "cloudformation:ExecuteChangeSet",
                "cloudformation:DeleteChangeSet",
                "cloudformation:DescribeChangeSet",
                "cloudformation:DescribeStacks",
                "cloudformation:DescribeEvents",
                "cloudformation:DescribeStackEvents",
                "cloudformation:GetTemplate",
                "cloudformation:GetTemplateSummary",
                "cloudformation:ListStackResources",
                "cloudformation:ValidateTemplate",
                "cloudformation:CreateUploadBucket"
            ],
            "Resource": "*"
        },
        {
            "Sid": "DeployArtifacts",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::blog.maceage.com-raindrop",
                "arn:aws:s3:::blog.maceage.com-raindrop/*"
            ]
        },
        {
            "Sid": "DeployFunction",
            "Effect": "Allow",
            "Action": [
                "lambda:CreateFunction",
                "lambda:UpdateFunctionCode",
                "lambda:UpdateFunctionConfiguration",
                "lambda:GetFunction",
                "lambda:GetFunctionConfiguration",
                "lambda:PublishVersion",
                "lambda:UpdateAlias",
                "lambda:GetAlias",
                "lambda:AddPermission",
                "lambda:RemovePermission",
                "lambda:TagResource",
                "lambda:UntagResource",
                "lambda:DeleteFunction"
            ],
            "Resource": "arn:aws:lambda:us-east-1:[AccountID]:function:raindrop-sync*"
        },
        {
            "Sid": "ManageExecutionRole",
            "Effect": "Allow",
            "Action": [
                "iam:CreateRole",
                "iam:GetRole",
                "iam:UpdateRole",
                "iam:DeleteRole",
                "iam:TagRole",
                "iam:UntagRole",
                "iam:PutRolePolicy",
                "iam:DeleteRolePolicy",
                "iam:GetRolePolicy",
                "iam:AttachRolePolicy",
                "iam:DetachRolePolicy"
            ],
            "Resource": "arn:aws:iam::[AccountID]:role/MaceageBlogSyncLambdaStack-*"
        },
        {
            "Sid": "CreateRoles",
            "Effect": "Allow",
            "Action": [
                "iam:CreateRole",
                "iam:DeleteRole",
                "iam:UpdateRole",
                "iam:GetRole",
                "iam:TagRole",
                "iam:UntagRole",
                "iam:PutRolePolicy",
                "iam:DeleteRolePolicy",
                "iam:AttachRolePolicy",
                "iam:DetachRolePolicy"
            ],
            "Resource": "*"
        },
        {
            "Sid": "ManageTable",
            "Effect": "Allow",
            "Action": [
                "dynamodb:CreateTable",
                "dynamodb:DeleteTable",
                "dynamodb:UpdateTable",
                "dynamodb:DescribeTable",
                "dynamodb:TagResource",
                "dynamodb:UntagResource",
                "dynamodb:DescribeContinuousBackups",
                "dynamodb:DescribeTimeToLive"
            ],
            "Resource": "arn:aws:dynamodb:us-east-1:[AccountID]:table/maceage-blog-raindrops"
        },
        {
            "Sid": "ManageParameter",
            "Effect": "Allow",
            "Action": [
                "ssm:PutParameter",
                "ssm:GetParameter",
                "ssm:GetParameters",
                "ssm:DeleteParameter",
                "ssm:AddTagsToResource",
                "ssm:RemoveTagsFromResource",
                "ssm:DescribeParameters"
            ],
            "Resource": "arn:aws:ssm:us-east-1:[AccountID]:parameter/maceage-blog/raindrop-sync/bearer-token*"
        },
        {
            "Sid": "EncryptTokenParameter",
            "Effect": "Allow",
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:GenerateDataKey"
            ],
            "Resource": "arn:aws:kms:us-east-1:[AccountID]:alias/aws/ssm",
            "Condition": {
                "StringEquals": {
                    "kms:ViaService": "ssm.us-east-1.amazonaws.com"
                }
            }
        },
        {
            "Sid": "ManageSchedule",
            "Effect": "Allow",
            "Action": [
                "scheduler:CreateSchedule",
                "scheduler:GetSchedule",
                "scheduler:DeleteSchedule",
                "scheduler:UpdateSchedule",
                "scheduler:TagResource",
                "scheduler:UntagResource"
            ],
            "Resource": "arn:aws:scheduler:us-east-1:[AccountID]:schedule/default/RaindropSyncLambda*"
        },
        {
            "Sid": "ManageLogGroups",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:DeleteLogGroup",
                "logs:DescribeLogGroups",
                "logs:PutRetentionPolicy",
                "logs:DeleteRetentionPolicy",
                "logs:TagResource",
                "logs:UntagResource"
            ],
            "Resource": "arn:aws:logs:us-east-1:[AccountID]:log-group:/aws/lambda/raindrop-sync*"
        },
        {
            "Sid": "PassExecutionRoles",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "iam:PassedToService": [
                        "lambda.amazonaws.com",
                        "scheduler.amazonaws.com"
                    ]
                }
            }
        }
    ]
}
```

# Troubleshooting{#troubleshooting}

While troubleshooting the bookmark sync process I encountered a few issues, mostly related to the bookmark cover images.

## logger.info() not working in Lambda

When I first ran the Lambda, nothing was appearing in the CloudWatch logs. This was because the Lambda function was not configured to [use the logging module correctly](https://stackoverflow.com/questions/37703609/using-python-logging-with-aws-lambda).

The following code snippet shows how to configure the logging module to work with Lambda:

```python
if len(logging.getLogger().handlers) > 0:
    logging.getLogger().setLevel(logging.INFO)
else:
    logging.basicConfig(level=logging.INFO)
```

## Downloading cover images

After I'd got the bookmark pages generated and deployed, I noticed that some of the bookmarks being listed had images specified, but the images themselves were not displaying correctly and the placeholder image was not being shown either.

After some investigation, I found that the problem was caused by the sync Lambda downloading HTML error pages from Raindrop's CDN, which returned HTTP 200 with a `text/html` body. The sync then dutifully uploaded these HTML pages as `covers/{id}.jpg`, and because it only ever does a `HeadObject` existence check on subsequent runs, the poisoned objects were cached in the bucket forever. The generator downloaded these HTML pages faithfully, and my OS image viewer chokes on a JPEG that starts with `<!DOCTYPE html>`.

[![HTML masquerading as image file](html-masquerading-as-image.png#center)](html-masquerading-as-image.png)

The fix for this (which you may have already noticed in the script above) is to check the content of the image file to determine whether it is actually an image or contains text. Both the `raindrop_sync.py` and `generate_bookmarks.py` scripts employ a `sniff_image()` function that looks at the header of the file to verify that it matches a known image signature. If it doesn't, the placeholder image is used as the thumbnail instead.

## Incorrect URLs for bookmark images

Once I'd got the invalid images issue resolved and deployed the changes, I realised some images were not displaying at all - the path to the cover image was resolving at the root rather than at the `/bookmarks/images` URL.

Eventually I realised the issue was in how Hugo handles page-bundle resources. When images are placed inside `content/bookmarks/page/2/images/`, Hugo doesn't automatically expose them at that URL - they need to be referenced through Hugo's resource system so it calculates the correct permalink.

The simplest fix was to change the generator to use Hugo's resource syntax:

```python
...

# Pass the raw filename - the shortcode will resolve it via .Page.Resources.GetMatch
image_key = local_images.get(record.get("id"))
if image_key:
    image = Path(image_key).name  # "images/55660011.jpg" → "55660011.jpg"
else:
    image = PLACEHOLDER_IMAGE
image = sc_param(image)

...
```

And then update the shortcode to resolve it properly:

```html
<div class="mr-md-3 mb-2 mb-md-0" style="flex-shrink: 0;">
    <a href="{{ $link }}" rel="noopener noreferrer">
        {{ if eq $image "/img/bookmark-placeholder.png" }}
        <img src="{{ $image }}" alt="{{ .Get "title" }}" loading="lazy" class="rounded-2 post-list-image bookmark-card__image">
        {{ else }}
            {{ $res := .Page.Resources.GetMatch (printf "images/%s" $image) }}
            {{ if $res }}
                <img src="{{ $res.RelPermalink }}" alt="{{ .Get "title" }}" loading="lazy" class="rounded-2 post-list-image bookmark-card__image">
            {{ else }}
                <img src="{{ "/img/bookmark-placeholder.png" }}" alt="{{ .Get "title" }}" loading="lazy" class="rounded-2 post-list-image bookmark-card__image">
            {{ end }}
        {{ end }}
    </a>
</div>
```

This fix causes the following:

1. The generator passes just the filename (e.g. `55660011.jpg`) instead of a path like `images/55660011.jpg`. The `images/` prefix is implicit, since Hugo knows where the images directory lives for each page bundle.
2. The shortcode uses `.Page.Resources.GetMatch` to look up `images/<filename>` relative to the current page bundle. This ensures Hugo resolves it through its resource pipeline, producing the correct relative permalink (e.g. `/bookmarks/page/2/images/55660011.jpg`).
3. The placeholder stays absolute (`/images/bookmark-placeholder.jpg`) since it's a static asset, not a page resource.

## Index.html files not being created

With the images URL issue fixed, I redeployed the site. When navigating to the second page of bookmarks, I encountered the following error:

```xml
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
</Error>
```

When I looked in the S3 bucket, there was an `index.html` file at the root `/bookmarks` URL but no `index.html` files in the `bookmarks/page/2`, `bookmarks/page/3` and `bookmarks/page/4` paths.

I then learnt that in Hugo's model, a directory containing `index.md` is a self-contained leaf bundle: everything under it - including `page/2/index.md`, `page/3/index.md` and all the `images/` directories - is treated as the bundle's resources, not as standalone pages. Therefore, nested content pages inside a leaf bundle simply aren't built, and Hugo happily emitted `/bookmarks/index.html` from the page-1 bundle while silently absorbing the other pages as unreachable resources.

The fix was to make the page-1 entry a [branch bundle](https://gohugo.io/content-management/organization/#branch-bundles) instead - `_index.md` - a one-line change in the generator:

```python
def page_path(output_dir, page_number):
    if page_number == 1:
        return output_dir / "_index.md"   # branch bundle - nested pages are built
    return output_dir / "page" / str(page_number) / "index.md"
```

## Bookmark pages in RSS feed

After deploying the site, it took me quite a while to realise that Hugo's RSS feed generator was including all the bookmark pages as part of the feed.

Searching around, I found a [post](https://codingnconcepts.com/hugo/custom-rss-feed-hugo/) that explained how to modify the RSS template to exclude certain pages from the feed.

To achieve this I had to modify the [RSS template](https://bwaycer.github.io/hugo_tutorial.hugo/templates/rss/) under `layouts/_default/index.xml` to exclude those pages that have `rssIgnore` set to `true` in the page's front matter:

```xml
...

{{- range $pages }}{{ if ne .Params.rssIgnore true }}
    <item>
        <title>{{ .Title }}</title>
        <link>{{ .Permalink }}</link>
        ...
    </item>
{{ end }}{{- end }}

...
```

# Wrapping up

While I had a few issues getting the bookmarks and their covers to display correctly, the Lambda function has worked flawlessly since it was set up.

Every morning at 4am, the Lambda function runs and updates the DynamoDB table with the latest bookmarks from Raindrop.io.

Then at 6am, the GitHub Actions workflow for the blog runs automatically, generates the Markdown for the bookmark pages, and Hugo builds and deploys the changes.

To see the result, check out the [Bookmarks](/bookmarks) section
