# RSS feed generator from Markdown files
URL: /blog/rss-feed-generator

Generate RSS feeds from Markdown pages.

---
title: "RSS feed generator from Markdown files"
excerpt: "Generate RSS feeds from Markdown pages."
description: "Generate RSS feeds from Markdown pages."
date: "2024-11-11"
author: surya_oruganti
cover: "/images/blog/rss-feed-generator/cover.png"
---

The [WarpBuild documentation site](https://docs.warpbuilds.com) is built with Docusaurus and hosted on Vercel. The documentation is a collection of markdown files stored in a Github repository.

Here's a simple script to generate RSS feeds for the documentation pages.
I used this script to generate the RSS feed for the [changelog](https://docs.warpbuilds.com/changelog) page so users can subscribe to the changelog via RSS, especially to keep track of breaking changes. This was built heavily leveraging `claude sonnet 3.5 v2` and `cursor`.

[Docusaurus](https://docusaurus.io/showcase) is a static site generator with content in markdown and extensive customization options. It is maintained by Meta Open Source and is used by many popular companies including Meta, The Linux Foundation, and Red Hat.

While Docusaurus has a great [RSS feed generator](https://docusaurus.io/docs/blog#rss-feed) for blog posts, it does not support RSS feeds for the documentation content page type.

Hope you find this useful!

## RSS Feed Generator Usage

The `changelog-to-rss.sh` script generates the `changelog.xml` file, which is the RSS feed for the changelog.

1. Keep the `slug` in the frontmatter of the changelog file the same as the filename.
2. The `slug` is used to generate the permalink for the changelog entry.
3. The `updatedAt` field in the frontmatter is used to set the date of the changelog entry.
4. The permalink points to the different sections in the changelog.
5. Sections starting with `###` in the changelog file are used as the title of the RSS item.
6. All the markdown files are in the `docs/changelog` directory, one file per month. The naming convention is `YYYY-monthname.mdx`. Example: `2024-October.mdx`.

## The Script

The code for the script is available in the [warpbuilds/docs-rss-feed](https://github.com/warpbuilds/docs-rss-feed) repository.

```bash
#!/bin/bash

# Configuration
FEED_TITLE="WarpBuild Changelog"
FEED_DESC="WarpBuild platform updates, improvements, and bug fixes"
FEED_LINK="https://docs.warpbuild.com/ci/changelog"
DOCS_BASE_URL="https://docs.warpbuild.com"
OUTPUT_FILE="static/changelog.xml"
CHANGELOG_DIR="docs/changelog"

# Create RSS header
cat > "$OUTPUT_FILE" << EOF
<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
<channel>
    <title>$FEED_TITLE</title>
    <description>$FEED_DESC</description>
    <link>$FEED_LINK</link>
    <atom:link href="$FEED_LINK/changelog.xml" rel="self" type="application/rss+xml" />
    <lastBuildDate>$(date -R)</lastBuildDate>
EOF

# Function to convert date format for macOS
convert_date() {
    local input_date="$1"
    if [ -z "$input_date" ]; then
        return 1
    fi
    # Convert "Month DD, YYYY" to RFC822 format and strip the time portion
    date -R -j -f "%B %d, %Y" "$input_date" 2>/dev/null | sed 's/ [0-9][0-9]:[0-9][0-9]:[0-9][0-9] .*//'
}

# Function to create anchor-friendly string
create_anchor() {
    local input="$1"
    if [ -z "$input" ]; then
        return 1
    fi
    echo "$input" | tr '[:upper:]' '[:lower:]' | tr ' ' '-' | tr -d ',' 2>/dev/null
}

# Function to extract frontmatter value
get_frontmatter_value() {
    local file="$1"
    local key="$2"
    awk -v key="$key:" '$1 == key {print substr($0, length(key) + 3)}' "$file" | tr -d '"'
}

# Function to process markdown content
process_markdown() {
    local content="$1"
    local processed="$content"

    # Convert markdown to HTML first
    processed=$(echo "$processed" | perl -pe 's|\[([^\]]*)\]\(([^\)]*)\)|<a href="\2">\1</a>|g')

    # Properly escape HTML content
    processed=$(echo "$processed" | sed 's/\\n/<br\/>/g')

    echo "$processed"
}

# Process each changelog file in reverse chronological order
for file in $(ls -r "$CHANGELOG_DIR"/*.mdx); do
    # Skip changelog.mdx
    if [[ $file == *"changelog.mdx" ]]; then
        continue
    fi

    # Get update date from frontmatter
    updated_at=$(get_frontmatter_value "$file" "updatedAt")
    title=$(get_frontmatter_value "$file" "title")

    # Extract the slug from the filename (remove path and extension)
    SLUG=$(basename "$file" .mdx)

    CONTENT=""
    CURRENT_DATE=""

    while IFS= read -r line; do
        # Look for changelog entries starting with ###
        if [[ $line =~ ^###[[:space:]]+(.*,[[:space:]]+[0-9]{4})$ ]]; then
            # If we have accumulated content, create an item
            if [ ! -z "$CURRENT_DATE" ] && [ ! -z "$CONTENT" ]; then
                RFC_DATE=$(convert_date "$CURRENT_DATE")
                PROCESSED_CONTENT=$(process_markdown "$CONTENT")

                # Create anchor-friendly date string with error checking
                ANCHOR_DATE=$(create_anchor "$CURRENT_DATE")
                if [ ! -z "$ANCHOR_DATE" ]; then
                    cat >> "$OUTPUT_FILE" << EOF
    <item>
        <title>WarpBuild Updates - $CURRENT_DATE</title>
        <link>$FEED_LINK/$SLUG#$ANCHOR_DATE</link>
        <guid isPermaLink="false">$FEED_LINK/$SLUG#$ANCHOR_DATE</guid>
        <pubDate>$RFC_DATE</pubDate>
        <description><![CDATA[$PROCESSED_CONTENT]]></description>
    </item>
EOF
                fi
            fi

            CURRENT_DATE="${BASH_REMATCH[1]}"
            CONTENT=""
        elif [[ -n $line && ! $line =~ ^--- && ! $line =~ ^$ ]]; then
            CONTENT+="$line\n"
        fi
    done < "$file"

    # Process the last entry in the file
    if [ ! -z "$CURRENT_DATE" ] && [ ! -z "$CONTENT" ]; then
        RFC_DATE=$(convert_date "$CURRENT_DATE")
        echo "RFC_DATE: $RFC_DATE"
        PROCESSED_CONTENT=$(process_markdown "$CONTENT")
        ANCHOR_DATE=$(create_anchor "$CURRENT_DATE")

        if [ ! -z "$ANCHOR_DATE" ]; then
            cat >> "$OUTPUT_FILE" << EOF
    <item>
        <title>WarpBuild Updates - $CURRENT_DATE</title>
        <link>$FEED_LINK/$SLUG#$ANCHOR_DATE</link>
        <guid isPermaLink="false">$FEED_LINK/$SLUG#$ANCHOR_DATE</guid>
        <pubDate>$RFC_DATE</pubDate>
        <description><![CDATA[$PROCESSED_CONTENT]]></description>
    </item>
EOF
        fi
    fi
done

# Close RSS feed
cat >> "$OUTPUT_FILE" << EOF
</channel>
</rss>
EOF

echo "RSS feed generated at $OUTPUT_FILE"

```

## Example Markdown File

Here's a snippet of the markdown file for the changelog:

```mdx
---
title: "October 2024"
slug: "2024-October"
description: "List of updates in 2024-October"
sidebar_position: -9
createdAt: "2024-10-04"
updatedAt: "2024-10-29"
---

### October 29, 2024

- `Feature`: Custom VM images are now supported for GCP BYOC runners.

### October 21, 2024

- `Feature`: Ubuntu 24.04 arm64 runners are now supported natively as cloud
  runners as well as with AWS and GCP custom runners. These runners are
  compatible with GitHub's Ubuntu 24.04 arm64. Refer to [cloud runner labels](/cloud-runners#linux-arm64)
  for the full list of available labels. Refer to
  [this link](https://github.com/actions/partner-runner-images/blob/main/images/arm-ubuntu-24-image.md) for the details on the packaged tools.

### October 17, 2024

- `Enhancement`: The image for `macos-14` (https://github.com/actions/runner-images/releases/tag/macos-14-arm64%2F20241007.259) has been updated. This fixes the issue with iOS 18 SDK and simulator not being available.

### October 15, 2024

- `Feature`: Docker Layer Caching is now available for GCP BYOC runners.
- `Enhancement`: The images for `ubuntu-2204` for [x86-64](https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20241006.1) for `arm64` architecture have been updated.
- `Enhancement`: [ubuntu-2404 for x86-64](https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20241006.1) image has been updated.

### October 14, 2024

- `Enhancement`: BYOC features do not require a payment method to be added, by default. Credits can be used for BYOC runners.

### October 11, 2024

- `Pricing`: Cost for cache operations has been **reduced** from $0.001 to $0.0001 per operation.

### October 09, 2024

- `Feature`: GCP BYOC is now generally available. Read more here: [BYOC on GCP](/byoc/gcp).

### October 08, 2024

- `Enhancement`: The runner start times are now much faster, with a 90%ile of the start times being under 20 seconds. This is a a significant improvement over the previous 90%ile of 45 seconds.

---
```

## Next steps

It would be fantastic to have this as a Docusaurus plugin so it can be reused for other markdown pages. If you are interested in this, please let me know!

The full script is available as a [GitHub Gist](https://gist.github.com/suryaoruganti/8f520335f3c9c9a705687ac6d3c47b9f).

Example markdown file is available [here](https://gist.github.com/suryaoruganti/792b444fc1f2e1d12831712daf68de69).

---

<Callout title="Tip" type="success">
  Use WarpBuild for blazing fast GitHub actions runners with superior job start times, caching backed by object storage, unlimited concurrency, and easy to use dashboards. Save 50-90% on your GitHub Actions costs while getting 10x the performance. <a href="https://cal.com/suryao/start">Book a call</a> or <a href="https://app.warpbuilds.com">get started</a> today!
</Callout>