diff --git a/README.md b/README.md index ba4585eb..a508f647 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,11 @@ Current Version: V3 Release Preview "Bmad Agent" -Demos of the BMad Agent and the entire workflow can be found soon in [Demos](./demos/). +Demos of the BMad Agent and the entire workflow can be found soon in [Demos](./demos/v3-output-demo-files/readme.md). ## Web Quickstart Project Setup (Recommended) -Orchestrator Uber BMad Agent that does it all - already [pre-built](./web-build-sample/agent-prompt.txt)! Just copy to a Gemini Gem or custom GPT as instructions, and attach the remaining files in the folder to the agent as shown in the following image. +Orchestrator Uber BMad Agent that does it all - already [pre-built](./web-build-sample/agent-prompt.txt)! Just copy to a Gemini Gem or custom GPT as instructions, and attach the remaining files from the web-build-sample folder to the agent as shown in the following image: ![image info](./docs/images/gem-setup.png) diff --git a/demos/readme.md b/demos/readme.md index 3595b199..eab316f4 100644 --- a/demos/readme.md +++ b/demos/readme.md @@ -1 +1,13 @@ -New demo output coming soon. +A simple project run through the Web Gemini BMad Agent - all artifacts from a single chat session (split up into smaller files with the sharding task) + +- The [Project Brief](./v3-output-demo-files/project-brief.md) was first collaborated on and created with the Analyst +- The first [PRD Draft](./v3-output-demo-files/prd.draft.md) was created with the PM +- The [Architecture](./v3-output-demo-files/architecture.md) was created and then we worked on some design artifacts. The architect conversation lead to changes in the PRD reflected later. + +Design Artifacts with the Design Architect: + +- [UX UI Spec](./v3-output-demo-files/ux-ui-spec.md) +- [V0 1 Shot UI Prompt](./v3-output-demo-files/v0-prompt.md) +- [Front End Architecture](./v3-output-demo-files/front-end-architecture.md) + +Then the updated PRD with fixed Expic and Stories after running the PO Checklist. The PO took all changes from the architect and design architect and worked them back into the updated [PRD Final](./v3-output-demo-files/prd.md) diff --git a/demos/v3-output-demo-files/api-reference.md b/demos/v3-output-demo-files/api-reference.md new file mode 100644 index 00000000..0311f5e4 --- /dev/null +++ b/demos/v3-output-demo-files/api-reference.md @@ -0,0 +1,34 @@ +# API Reference + +## External APIs Consumed + +**1. Algolia Hacker News Search API** + + * **Base URL:** `http://hn.algolia.com/api/v1/` + * **Authentication:** None. + * **Endpoints Used:** + * `GET /search_by_date?tags=story&hitsPerPage={N}` (For top posts) + * `GET /items/{POST_ID}` (For comments/post details) + * **Key Data Extracted:** Post title, article URL, HN link, HN Post ID, author, points, creation timestamp; Comment text, author, creation timestamp. + +**2. Play.ai PlayNote API** + + * **Base URL:** `https://api.play.ai/api/v1/` + * **Authentication:** Headers: `Authorization: Bearer `, `X-USER-ID: `. + * **Endpoints Used:** + * `POST /playnotes` (Submit job) + * Request: `application/json` with `sourceText`, `title`, voice params (from env vars: `PLAY_AI_VOICE1_ID`, `PLAY_AI_VOICE1_NAME`, `PLAY_AI_VOICE2_ID`, `PLAY_AI_VOICE2_NAME`), style (`PLAY_AI_STYLE`). + * Response: JSON with `jobId`. + * `GET /playnote/{jobId}` (Poll status) + * Response: JSON with `status`, `audioUrl` (if completed). + +## Internal APIs Provided (by backend for frontend) + + * **Base URL Path Prefix:** `/v1` (Full URL from `NEXT_PUBLIC_BACKEND_API_URL`). + * **Authentication:** Requires "Frontend Read API Key" via `x-api-key` header for GET endpoints. A separate "Admin Action API Key" for trigger endpoint. + * **Endpoints:** + * **`GET /status`**: Health/status check. Response: `{"message": "BMad Daily Digest Backend is operational.", "timestamp": "..."}`. + * **`GET /episodes`**: Lists episodes. Response: `{ "episodes": [EpisodeListItem, ...] }`. + * **`GET /episodes/{episodeId}`**: Episode details. Response: `EpisodeDetail` object. + * **`POST /jobs/daily-digest/trigger`**: (Admin Key) Triggers daily pipeline. Response: `{"message": "...", "executionArn": "..."}`. + * **Common Errors:** 401 Unauthorized, 404 Not Found, 500 Internal Server Error. \ No newline at end of file diff --git a/demos/v3-output-demo-files/architecture.md b/demos/v3-output-demo-files/architecture.md new file mode 100644 index 00000000..6fadc1dd --- /dev/null +++ b/demos/v3-output-demo-files/architecture.md @@ -0,0 +1,571 @@ +# BMad Daily Digest Architecture Document + +**Version:** 0.1 +**Date:** May 20, 2025 +**Author:** Fred (Architect) & User + +## Table of Contents + +1. Introduction / Preamble +2. Technical Summary +3. High-Level Overview + * Backend Architectural Style + * Frontend Architectural Style + * Repository Structure + * Primary Data Flow & User Interaction (Conceptual) + * System Context Diagram (Conceptual) +4. Architectural / Design Patterns Adopted +5. Component View + * Backend Components + * Frontend Components + * External Services + * Component Interaction Diagram (Conceptual Backend Focus) +6. Project Structure + * Backend Repository (`bmad-daily-digest-backend`) + * Frontend Repository (`bmad-daily-digest-frontend`) + * Notes +7. API Reference + * External APIs Consumed + * Internal APIs Provided +8. Data Models + * Core Application Entities / Domain Objects + * API Payload Schemas (Internal API) + * Database Schemas (AWS DynamoDB) +9. Core Workflow / Sequence Diagrams + * Daily Automated Podcast Generation Pipeline (Backend) + * Frontend User Requesting and Playing an Episode +10. Definitive Tech Stack Selections +11. Infrastructure and Deployment Overview +12. Error Handling Strategy +13. Coding Standards (Backend: `bmad-daily-digest-backend`) + * Detailed Language & Framework Conventions (TypeScript/Node.js - Backend Focus) +14. Overall Testing Strategy +15. Security Best Practices +16. Key Reference Documents +17. Change Log +18. Prompt for Design Architect (Jane) - To Produce Frontend Architecture Document + +----- + +## 1\. Introduction / Preamble + +This document outlines the overall project architecture for "BMad Daily Digest," including backend systems, frontend deployment infrastructure, shared services considerations, and non-UI specific concerns. Its primary goal is to serve as the guiding architectural blueprint for AI-driven development and human developers, ensuring consistency and adherence to chosen patterns and technologies as defined in the Product Requirements Document (PRD v0.1) and UI/UX Specification (v0.1). + +**Relationship to Frontend Architecture:** +The frontend application (Next.js) will have its own detailed frontend architecture considerations (component structure, state management, etc.) which will be detailed in a separate Frontend Architecture Document (to be created by the Design Architect, Jane, based on a prompt at the end of this document). This overall Architecture Document will define the backend services the frontend consumes, the infrastructure for hosting the frontend (S3/CloudFront), and ensure alignment on shared technology choices (like TypeScript) and API contracts. The "Definitive Tech Stack Selections" section herein is the single source of truth for all major technology choices across the project. + +## 2\. Technical Summary + +"BMad Daily Digest" is a serverless application designed to automatically produce a daily audio podcast summarizing top Hacker News posts. The backend, built with TypeScript on Node.js 22 and deployed as AWS Lambda functions, will fetch data from the Algolia HN API, scrape linked articles, process content, and use the Play.ai API for audio generation (with job status managed by AWS Step Functions polling). Podcast metadata will be stored in DynamoDB, and audio files in S3. All backend infrastructure will be managed via AWS CDK within its own repository. + +The frontend will be a Next.js (React, TypeScript) application, styled with Tailwind CSS and shadcn/ui to an "80s retro CRT terminal" aesthetic, kickstarted by an AI UI generation tool. It will be a statically exported site hosted on AWS S3 and delivered globally via AWS CloudFront, with its infrastructure also managed by a separate AWS CDK application within its own repository. The frontend will consume data from the backend via an AWS API Gateway secured with API Keys. The entire system aims for cost-efficiency, leveraging AWS free-tier services where possible. + +## 3\. High-Level Overview + +The "BMad Daily Digest" system is architected as a decoupled, serverless application designed for automated daily content aggregation and audio generation, with a statically generated frontend for content consumption. + + * **Backend Architectural Style:** The backend employs a **serverless, event-driven architecture** leveraging AWS Lambda functions for discrete processing tasks. These tasks are orchestrated by AWS Step Functions to manage the daily content pipeline, including interactions with external services. An API layer is provided via AWS API Gateway for frontend consumption. + * **Frontend Architectural Style:** The frontend is a **statically generated site (SSG)** built with Next.js. This approach maximizes performance, security, and cost-effectiveness by serving pre-built files from AWS S3 via CloudFront. + * **Repository Structure:** The project utilizes a **polyrepo structure** with two primary repositories: + * `bmad-daily-digest-backend`: Housing all backend TypeScript code, AWS CDK for backend infrastructure, Lambda functions, Step Function definitions, etc. + * `bmad-daily-digest-frontend`: Housing the Next.js TypeScript application, UI components, styling, and its dedicated AWS CDK application for S3/CloudFront infrastructure. + * **Primary Data Flow & User Interaction (Conceptual):** + 1. **Daily Automated Pipeline (Backend):** + * An Amazon EventBridge Scheduler rule triggers an AWS Step Function state machine daily. + * The Step Function orchestrates a sequence of AWS Lambda functions to: + * Fetch top posts and comments from the Algolia HN API (identifying repeats). + * Scrape and extract content from linked external article URLs (for new posts or if scraping previously failed). + * Aggregate and format the text content (handling new posts, updates, scrape failures, truncation). + * Submit the formatted text to the Play.ai PlayNote API, receiving a `jobId`. + * Poll the Play.ai API (using the `jobId`) for podcast generation status until completion or failure. + * Upon completion, download the generated MP3 audio from Play.ai. + * Store the MP3 file in a designated S3 bucket. + * Store episode metadata (including the S3 audio link, source HN post details, etc.) in a DynamoDB table and update HN post processing states. + 2. **User Consumption (Frontend):** + * The user accesses the "BMad Daily Digest" Next.js web application served via AWS CloudFront from an S3 bucket. + * The frontend application makes API calls (via `axios`) to an AWS API Gateway endpoint (secured with an API Key). + * API Gateway routes these requests to specific AWS Lambda functions that query the DynamoDB table to retrieve episode lists and details. + * The frontend renders the information and provides an HTML5 audio player to stream/play the MP3 from its S3/CloudFront URL. + +**System Context Diagram (Conceptual):** + +```mermaid +graph TD + A[User] -->|Views & Interacts via Browser| B(Frontend Application\nNext.js on S3/CloudFront); + B -->|Fetches Episode Data (HTTPS, API Key)| C(Backend API\nAPI Gateway + Lambda); + C -->|Reads/Writes| D(Episode Metadata\nDynamoDB); + B -->|Streams Audio| E(Podcast Audio Files\nS3 via CloudFront); + + F[Daily Scheduler\nEventBridge] -->|Triggers| G(Orchestration Service\nAWS Step Functions); + G -->|Invokes| H(Data Collection Lambdas\n- Fetch HN via Algolia\n- Scrape Articles); + H -->|Calls| I[Algolia HN API]; + H -->|Scrapes| J[External Article Websites]; + G -->|Invokes| K(Content Processing Lambda); + G -->|Invokes| L(Play.ai Interaction Lambdas\n- Submit Job\n- Poll Status); + L -->|Calls / Gets Status| M[Play.ai PlayNote API]; + G -->|Invokes| N(Storage Lambdas\n- Store Audio to S3\n- Store Metadata to DynamoDB); + N -->|Writes| E; + N -->|Writes| D; + M -->|Returns Audio URL| L; +``` + +## 4\. Architectural / Design Patterns Adopted + +The following key architectural and design patterns have been chosen for this project: + + * **Serverless Architecture:** Entire backend on AWS Lambda, API Gateway, S3, DynamoDB, Step Functions. Rationale: Minimized operations, auto-scaling, pay-per-use, cost-efficiency. + * **Event-Driven Architecture:** Daily pipeline initiated by EventBridge Scheduler; Step Functions orchestrate based on state changes. Rationale: Decoupled components, reactive system for automation. + * **Microservices-like Approach (Backend Lambda Functions):** Each Lambda handles a specific, well-defined task. Rationale: Modularity, independent scalability, easier testing/maintenance. + * **Static Site Generation (SSG) for Frontend:** Next.js frontend exported as static files, hosted on S3/CloudFront. Rationale: Optimal performance, security, scalability, lower hosting costs. + * **Infrastructure as Code (IaC):** AWS CDK in TypeScript for all AWS infrastructure in both repositories. Rationale: Repeatable, version-controlled, automated provisioning. + * **Polling Pattern (External Job Status):** AWS Step Functions implement a polling loop for Play.ai job status. Rationale: Reliable tracking of asynchronous third-party jobs, based on Play.ai docs. + * **Orchestration Pattern (AWS Step Functions):** End-to-end daily backend pipeline managed by a Step Functions state machine. Rationale: Robust workflow automation, state management, error handling for multi-step processes. + +## 5\. Component View + +The system is divided into distinct backend and frontend components. + +**Backend Components (`bmad-daily-digest-backend` repository):** + +1. **Daily Workflow Orchestrator (AWS Step Functions state machine):** Manages the end-to-end daily pipeline. +2. **HN Data Fetcher Service (AWS Lambda):** Fetches HN posts/comments (Algolia), identifies repeats (via DynamoDB). +3. **Article Scraping Service (AWS Lambda):** Scrapes/extracts content from external article URLs, handles fallbacks. +4. **Content Formatting Service (AWS Lambda):** Aggregates and formats text payload for Play.ai. +5. **Play.ai Interaction Service (AWS Lambda functions, orchestrated by Polling Step Function):** Submits job to Play.ai, polls for status. +6. **Podcast Storage Service (AWS Lambda):** Downloads audio from Play.ai, stores to S3. +7. **Metadata Persistence Service (AWS Lambda & DynamoDB Tables):** Manages episode and HN post processing state metadata in DynamoDB. +8. **Backend API Service (AWS API Gateway + AWS Lambda functions):** Exposes endpoints for frontend (episode lists/details). + +**Frontend Components (`bmad-daily-digest-frontend` repository):** + +1. **Next.js Web Application (Static Site on S3/CloudFront):** Renders UI, handles navigation. +2. **Frontend API Client Service (TypeScript module):** Encapsulates communication with the Backend API Service. + +**External Services:** Algolia HN Search API, Play.ai PlayNote API, Various External Article Websites. + +**Component Interaction Diagram (Conceptual Backend Focus):** + +```mermaid +graph LR + subgraph Frontend Application Space + F_App[Next.js App on S3/CloudFront] + F_APIClient[Frontend API Client] + F_App --> F_APIClient + end + + subgraph Backend API Space + APIGW[API Gateway] + API_L[Backend API Lambdas] + APIGW --> API_L + end + + subgraph Backend Daily Pipeline Space + Scheduler[EventBridge Scheduler] --> Orchestrator[Step Functions Orchestrator] + + Orchestrator --> HNFetcher[HN Data Fetcher Lambda] + HNFetcher -->|Reads/Writes Post Status| DDB + HNFetcher --> Algolia[Algolia HN API] + + Orchestrator --> ArticleScraper[Article Scraper Lambda] + ArticleScraper --> ExtWebsites[External Article Websites] + + Orchestrator --> ContentFormatter[Content Formatter Lambda] + + Orchestrator --> PlayAISubmit[Play.ai Submit Lambda] + PlayAISubmit --> PlayAI_API[Play.ai PlayNote API] + + subgraph Polling_SF[Play.ai Polling (Step Functions)] + direction LR + PollTask[Poll Status Lambda] --> PlayAI_API + end + Orchestrator --> Polling_SF + + + Orchestrator --> PodcastStorage[Podcast Storage Lambda] + PodcastStorage --> PlayAI_API + PodcastStorage --> S3Store[S3 Audio Storage] + + Orchestrator --> MetadataService[Metadata Persistence Lambda] + MetadataService --> DDB[DynamoDB Episode/Post Metadata] + end + + F_APIClient --> APIGW + API_L --> DDB + + classDef external fill:#ddd,stroke:#333,stroke-width:2px; + class Algolia,ExtWebsites,PlayAI_API external; +``` + +## 6\. Project Structure + +The project utilizes a polyrepo structure with separate backend and frontend repositories, each with its own CDK application. + +**1. Backend Repository (`bmad-daily-digest-backend`)** +Organized by features within `src/`, using `dash-case` for folders and files (e.g., `src/features/content-ingestion/hn-fetcher-service.ts`). + +```plaintext +bmad-daily-digest-backend/ +├── .github/ +├── cdk/ +│ ├── bin/ +│ ├── lib/ # Backend Stack, Step Function definitions +│ └── test/ +├── src/ +│ ├── features/ +│ │ ├── dailyJobOrchestrator/ # Main Step Function trigger/definition support +│ │ ├── hnContentPipeline/ # Services for Algolia, scraping, formatting +│ │ ├── playAiIntegration/ # Services for Play.ai submit & polling Lambda logic +│ │ ├── podcastPersistence/ # Services for S3 & DynamoDB storage +│ │ └── publicApi/ # Handlers for API Gateway (status, episodes) +│ ├── shared/ +│ │ ├── utils/ +│ │ ├── types/ +│ │ └── services/ # Optional shared low-level AWS SDK wrappers +├── tests/ # Unit/Integration tests, mirroring src/features/ +│ └── features/ +... (root config files: .env.example, .eslintrc.js, .gitignore, .prettierrc.js, jest.config.js, package.json, README.md, tsconfig.json) +``` + +*Key Directories: `cdk/` for IaC, `src/features/` for modular backend logic, `src/shared/` for reusable code, `tests/` for Jest tests.* + +**2. Frontend Repository (`bmad-daily-digest-frontend`)** +Aligns with V0.dev generated Next.js App Router structure, using `dash-case` for custom files/folders where applicable. + +```plaintext +bmad-daily-digest-frontend/ +├── .github/ +├── app/ +│ ├── (pages)/ +│ │ ├── episodes/ +│ │ │ ├── page.tsx # List page +│ │ │ └── [episode-id]/ +│ │ │ └── page.tsx # Detail page +│ │ └── about/ +│ │ └── page.tsx +│ ├── layout.tsx +│ └── globals.css +├── components/ +│ ├── ui/ # shadcn/ui based components +│ └── domain/ # Custom composite components (e.g., episode-card) +├── cdk/ # AWS CDK application for frontend infra (S3, CloudFront) +│ ├── bin/ +│ └── lib/ +├── hooks/ +├── lib/ +│ ├── types.ts +│ ├── utils.ts +│ └── api-client.ts # Backend API communication +├── public/ +├── tests/ # Jest & RTL tests +... (root config files: .env.local.example, .eslintrc.js, components.json, next.config.mjs, package.json, tailwind.config.ts, tsconfig.json) +``` + +*Key Directories: `app/` for Next.js routes, `components/` for UI, `cdk/` for frontend IaC, `lib/` for utilities and `api-client.ts`.* + +## 7\. API Reference + +### External APIs Consumed + +**1. Algolia Hacker News Search API** + + * **Base URL:** `http://hn.algolia.com/api/v1/` + * **Authentication:** None. + * **Endpoints Used:** + * `GET /search_by_date?tags=story&hitsPerPage={N}` (For top posts) + * `GET /items/{POST_ID}` (For comments/post details) + * **Key Data Extracted:** Post title, article URL, HN link, HN Post ID, author, points, creation timestamp; Comment text, author, creation timestamp. + +**2. Play.ai PlayNote API** + + * **Base URL:** `https://api.play.ai/api/v1/` + * **Authentication:** Headers: `Authorization: Bearer `, `X-USER-ID: `. + * **Endpoints Used:** + * `POST /playnotes` (Submit job) + * Request: `application/json` with `sourceText`, `title`, voice params (from env vars: `PLAY_AI_VOICE1_ID`, `PLAY_AI_VOICE1_NAME`, `PLAY_AI_VOICE2_ID`, `PLAY_AI_VOICE2_NAME`), style (`PLAY_AI_STYLE`). + * Response: JSON with `jobId`. + * `GET /playnote/{jobId}` (Poll status) + * Response: JSON with `status`, `audioUrl` (if completed). + +### Internal APIs Provided (by backend for frontend) + + * **Base URL Path Prefix:** `/v1` (Full URL from `NEXT_PUBLIC_BACKEND_API_URL`). + * **Authentication:** Requires "Frontend Read API Key" via `x-api-key` header for GET endpoints. A separate "Admin Action API Key" for trigger endpoint. + * **Endpoints:** + * **`GET /status`**: Health/status check. Response: `{"message": "BMad Daily Digest Backend is operational.", "timestamp": "..."}`. + * **`GET /episodes`**: Lists episodes. Response: `{ "episodes": [EpisodeListItem, ...] }`. + * **`GET /episodes/{episodeId}`**: Episode details. Response: `EpisodeDetail` object. + * **`POST /jobs/daily-digest/trigger`**: (Admin Key) Triggers daily pipeline. Response: `{"message": "...", "executionArn": "..."}`. + * **Common Errors:** 401 Unauthorized, 404 Not Found, 500 Internal Server Error. + +## 8\. Data Models + +### Core Application Entities + +**a. Episode** + + * Attributes: `episodeId` (PK, UUID), `publicationDate` (YYYY-MM-DD), `episodeNumber` (Number), `podcastGeneratedTitle` (String), `audioS3Bucket` (String), `audioS3Key` (String), `audioUrl` (String, derived for API), `playAiJobId` (String), `playAiSourceAudioUrl` (String), `sourceHNPosts` (List of `SourceHNPost`), `status` (String: "PROCESSING", "PUBLISHED", "FAILED"), `createdAt` (ISO Timestamp), `updatedAt` (ISO Timestamp). + +**b. SourceHNPost (object within `Episode.sourceHNPosts`)** + + * Attributes: `hnPostId` (String), `title` (String), `originalArticleUrl` (String), `hnLink` (String), `isUpdateStatus` (Boolean), `oldRank` (Number, Optional), `lastCommentFetchTimestamp` (Number, Unix Timestamp), `articleScrapingFailed` (Boolean), `articleTitleFromScrape` (String, Optional). + +**c. HackerNewsPostProcessState (DynamoDB Table)** + + * Attributes: `hnPostId` (PK, String), `originalArticleUrl` (String), `articleTitleFromScrape` (String, Optional), `lastSuccessfullyScrapedTimestamp` (Number, Optional), `lastCommentFetchTimestamp` (Number, Optional), `firstProcessedDate` (YYYY-MM-DD), `lastProcessedDate` (YYYY-MM-DD), `lastKnownRank` (Number, Optional). + +### API Payload Schemas (Internal API) + +**a. `EpisodeListItem` (for `GET /episodes`)** + + * `episodeId`, `publicationDate`, `episodeNumber`, `podcastGeneratedTitle`. + +**b. `EpisodeDetail` (for `GET /episodes/{episodeId}`)** + + * `episodeId`, `publicationDate`, `episodeNumber`, `podcastGeneratedTitle`, `audioUrl`, `sourceHNPosts` (list of `SourceHNPostDetail` containing `hnPostId`, `title`, `originalArticleUrl`, `hnLink`, `isUpdateStatus`, `oldRank`), `playAiJobId` (optional), `playAiSourceAudioUrl` (optional), `createdAt`. + +### Database Schemas (AWS DynamoDB) + +**a. `BmadDailyDigestEpisodes` Table** + + * PK: `episodeId` (String). + * Attributes: As per `Episode` entity. + * GSI Example (`PublicationDateIndex`): PK: `status`, SK: `publicationDate`. + * Billing: PAY\_PER\_REQUEST. + +**b. `HackerNewsPostProcessState` Table** + + * PK: `hnPostId` (String). + * Attributes: As per `HackerNewsPostProcessState` entity. + * Billing: PAY\_PER\_REQUEST. + +## 9\. Core Workflow / Sequence Diagrams + +### 1\. Daily Automated Podcast Generation Pipeline (Backend) + +*(Mermaid diagram as previously shown, detailing EventBridge -\> Step Functions -\> Lambdas -\> Algolia/External Sites/Play.ai -\> S3/DynamoDB).* + +```mermaid +sequenceDiagram + participant Sched as Scheduler (EventBridge) + participant Orch as Orchestrator (Step Functions) + participant HNF as HN Data Fetcher Lambda + participant Algolia as Algolia HN API + participant ASL as Article Scraper Lambda + participant EAS as External Article Sites + participant CFL as Content Formatter Lambda + participant PSubL as Play.ai Submit Lambda + participant PlayAI as Play.ai API + participant PStatL as Play.ai Status Poller Lambda + participant PSL as Podcast Storage Lambda + participant S3 as S3 Audio Storage + participant MPL as Metadata Persistence Lambda + participant DDB as DynamoDB (Episodes & HNPostState) + + Sched->>Orch: Trigger Daily Workflow + activate Orch + Orch->>HNF: Start: Fetch HN Posts + activate HNF + HNF->>Algolia: Request top N posts + Algolia-->>HNF: Return HN post list + HNF->>DDB: Query HNPostProcessState for repeat status & lastCommentFetchTimestamp + DDB-->>HNF: Return status + HNF-->>Orch: HN Posts Data (with repeat status) + deactivate HNF + Orch->>ASL: For each NEW HN Post: Scrape Article (URL) + activate ASL + ASL->>EAS: Fetch article HTML + EAS-->>ASL: Return HTML + ASL-->>Orch: Scraped Article Content / Scrape Failure+Fallback Flag + deactivate ASL + Orch->>HNF: For each HN Post: Fetch Comments (HN Post ID, isRepeat, lastCommentFetchTimestamp, articleScrapedFailedFlag) + activate HNF + HNF->>Algolia: Request comments for Post ID + Algolia-->>HNF: Return comments + HNF->>DDB: Update HNPostProcessState (lastCommentFetchTimestamp) + DDB-->>HNF: Confirm update + HNF-->>Orch: Selected Comments + deactivate HNF + Orch->>CFL: Format Content for Play.ai (HN Posts, Articles, Comments) + activate CFL + CFL-->>Orch: Formatted Text Payload + deactivate CFL + Orch->>PSubL: Submit to Play.ai (Formatted Text) + activate PSubL + PSubL->>PlayAI: POST /playnotes (text, voice params, auth) + PlayAI-->>PSubL: Return { jobId } + PSubL-->>Orch: Play.ai Job ID + deactivate PSubL + loop Poll for Completion (managed by Orchestrator/Step Functions) + Orch->>Orch: Wait (e.g., M minutes) + Orch->>PStatL: Check Status (Job ID) + activate PStatL + PStatL->>PlayAI: GET /playnote/{jobId} (auth) + PlayAI-->>PStatL: Return { status, audioUrl? } + PStatL-->>Orch: Job Status & audioUrl (if completed) + deactivate PStatL + alt Job Completed + Orch->>PSL: Store Podcast (audioUrl, jobId, episode context) + activate PSL + PSL->>PlayAI: GET audio from audioUrl + PlayAI-->>PSL: Audio Stream/File + PSL->>S3: Upload MP3 + S3-->>PSL: Confirm S3 Upload (s3Key, s3Bucket) + PSL-->>Orch: S3 Location + deactivate PSL + Orch->>MPL: Persist Episode Metadata (S3 loc, HN sources, etc.) + activate MPL + MPL->>DDB: Save Episode Item & Update HNPostProcessState (lastProcessedDate) + DDB-->>MPL: Confirm save + MPL-->>Orch: Success + deactivate MPL + else Job Failed or Timeout + Orch->>Orch: Log Error, Terminate Sub-Workflow for this job + end + end + deactivate Orch +``` + +### 2\. Frontend User Requesting and Playing an Episode + +*(Mermaid diagram as previously shown, detailing User -\> Next.js App -\> API Gateway/Lambda -\> DynamoDB, and User -\> Next.js App -\> S3/CloudFront for audio).* + +```mermaid +sequenceDiagram + participant User as User (Browser) + participant FE_App as Frontend App (Next.js on CloudFront/S3) + participant BE_API as Backend API (API Gateway) + participant API_L as API Lambda + participant DDB as DynamoDB (Episode Metadata) + participant Audio_S3 as Audio Storage (S3 via CloudFront) + + User->>FE_App: Requests page (e.g., /episodes or /episodes/{id}) + activate FE_App + FE_App->>BE_API: GET /v1/episodes (or /v1/episodes/{id}) (includes API Key) + activate BE_API + BE_API->>API_L: Invoke Lambda with request data + activate API_L + API_L->>DDB: Query for episode(s) metadata + activate DDB + DDB-->>API_L: Return episode data + deactivate DDB + API_L-->>BE_API: Return formatted episode data + deactivate API_L + BE_API-->>FE_App: Return API response (JSON) + deactivate BE_API + FE_App->>FE_App: Render page with episode data (list or detail) + FE_App-->>User: Display page + deactivate FE_App + + alt User on Episode Detail Page & Clicks Play + User->>FE_App: Clicks play on HTML5 Audio Player + activate FE_App + Note over FE_App, Audio_S3: Player's src attribute is set to CloudFront URL for audio file in S3. + FE_App->>Audio_S3: Browser requests audio file via CloudFront URL + activate Audio_S3 + Audio_S3-->>FE_App: Stream/Return audio file + deactivate Audio_S3 + FE_App-->>User: Plays audio + deactivate FE_App + end +``` + +## 10\. Definitive Tech Stack Selections + +| Category | Technology | Version / Details | Description / Purpose | Justification (Optional) | +| :------------------- | :----------------------------- | :------------------------------------- | :------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------- | +| **Languages** | TypeScript | Latest stable (e.g., 5.x) | Primary language for backend and frontend. | Consistency, strong typing. | +| **Runtime** | Node.js | 22.x | Server-side environment for backend & Next.js. | User preference, performance. | +| **Frameworks (Frontend)** | Next.js (with React) | Latest stable (e.g., 14.x) | Frontend web application framework. | User preference, SSG, DX. | +| **Frameworks (Backend)** | AWS Lambda (Node.js runtime) | N/A | Execution environment for serverless functions. | Serverless architecture. | +| | AWS Step Functions | N/A | Orchestration of backend workflows. | Robust state management, retries. | +| **Databases** | AWS DynamoDB | N/A | NoSQL database for metadata. | Scalability, serverless, free-tier. | +| **Cloud Platform** | AWS | N/A | Primary cloud provider. | Comprehensive services, serverless. | +| **Cloud Services** | AWS Lambda, API Gateway, S3, CloudFront, EventBridge Scheduler, CloudWatch, IAM, ACM | N/A | Core services for application hosting and operation. | Standard AWS serverless stack. | +| **Infrastructure as Code (IaC)** | AWS CDK (TypeScript) | v2.x Latest stable | Defining cloud infrastructure. | User preference, TypeScript, repeatability. | +| **UI Libraries (Frontend)** | Tailwind CSS | Latest stable (e.g., 3.x) | Utility-first CSS framework. | User preference, customization. | +| | shadcn/ui | Latest stable | Accessible UI components. | User preference, base for themed components. | +| **HTTP Client (Backend)** | axios | Latest stable | Making HTTP requests from backend. | User preference, feature-rich. | +| **SDKs / Core Libraries (Backend)** | AWS SDK for JavaScript/TypeScript | v3.x (Latest stable) | Programmatic interaction with AWS services. | Official AWS SDK, modular. | +| **Scraping / Content Extraction** | Cheerio | Latest stable | Server-side HTML parsing. | Efficient for static HTML. | +| | @mozilla/readability (JS port) | Latest stable | Extracting primary readable article content. | Key for isolating main content. | +| | Playwright (or Puppeteer) | Latest stable | Browser automation (if required for dynamic content). | Handles dynamic sites; use judiciously. | +| **Bundling (Backend)**| esbuild | Latest stable | Bundling TypeScript Lambda functions. | User preference, speed. | +| **Logging (Backend)** | Pino | Latest stable | Structured, low-overhead logging. | Better observability, JSON logs for CloudWatch. | +| **Testing (Backend)**| Jest, ESLint, Prettier | Latest stable | Unit/integration testing, linting, formatting. | Code quality, consistency. | +| **Testing (Frontend)**| Jest, React Testing Library, ESLint, Prettier | Latest stable | Unit/component testing, linting, formatting. | Code quality, consistency. | +| **CI/CD** | GitHub Actions | N/A | Automation of build, test, quality checks. | Integration with GitHub. | +| **External APIs** | Algolia HN Search API, Play.ai PlayNote API | v1 (for both) | Data sources and audio generation. | Core to product functionality. | +*Note: "Latest stable" versions should be pinned to specific versions in `package.json` files during development.* + +## 11\. Infrastructure and Deployment Overview + + * **Cloud Provider:** AWS. + * **Core Services Used:** Lambda, API Gateway (HTTP API), S3, DynamoDB (On-Demand), Step Functions, EventBridge Scheduler, CloudFront, CloudWatch, IAM, ACM (if custom domain). + * **IaC:** AWS CDK (TypeScript), with separate CDK apps in backend and frontend polyrepos. + * **Deployment Strategy (MVP):** CI (GitHub Actions) for build/test/lint. CDK deployment (initially manual or CI-scripted) to a single AWS environment. + * **Environments (MVP):** Local Development; Single Deployed MVP Environment (e.g., "dev" acting as initial production). + * **Rollback Strategy (MVP):** CDK stack rollback, Lambda/S3 versioning, DynamoDB PITR. + +## 12\. Error Handling Strategy + + * **General Approach:** Custom `Error` classes hierarchy. Promises reject with `Error` objects. + * **Logging:** Pino for structured JSON logs to CloudWatch. Standard levels (DEBUG, INFO, WARN, ERROR, CRITICAL). Contextual info (AWS Request ID, business IDs). No sensitive data in logs. + * **Specific Patterns:** + * **External API Calls (`axios`):** Timeouts, retries (e.g., `axios-retry`), wrap errors in custom types. + * **Internal Errors:** Custom error types, detailed server-side logging. + * **API Gateway Responses:** Translate internal errors to appropriate HTTP errors (4xx, 500) with generic client messages. + * **Workflow (Step Functions):** Error handling, retries, catch blocks for states. Failed executions logged. + * **Data Consistency:** Lambdas handle partial failures gracefully. Step Functions manage overall workflow state. + +## 13\. Coding Standards (Backend: `bmad-daily-digest-backend`) + +**Scope:** Applies to `bmad-daily-digest-backend`. Frontend standards are separate. + + * **Primary Language:** TypeScript (Node.js 22). + * **Style:** ESLint, Prettier. + * **Naming:** Variables/Functions: `camelCase`. Constants: `UPPER_SNAKE_CASE`. Classes/Interfaces/Types/Enums: `PascalCase`. Files/Folders: `dash-case` (e.g., `episode-service.ts`, `content-ingestion/`). + * **Structure:** Feature-based (`src/features/feature-name/`). + * **Tests:** Unit/integration tests co-located (`*.test.ts`). E2E tests (if any for backend API) in root `tests/e2e/`. + * **Async:** `async`/`await` for Promises. + * **Types:** `strict: true`. No `any` without justification. JSDoc for exported items. Inline comments for clarity. + * **Dependencies:** `npm` with `package-lock.json`. Pin versions or use tilde (`~`). + * **Detailed Conventions:** Immutability preferred. Functional constructs for stateless logic, classes for stateful services/entities. Custom errors. Strict null checks. ESModules. Pino for logging (structured JSON, levels, context, no secrets). Lambda best practices (lean handlers, env vars, optimize size). `axios` with timeouts. AWS SDK v3 modular imports. Avoid common anti-patterns (deep nesting, large functions, `@ts-ignore`, hardcoded secrets, unhandled promises). + +## 14\. Overall Testing Strategy + + * **Tools:** Jest, React Testing Library (frontend), ESLint, Prettier, GitHub Actions. + * **Unit Tests:** Isolate functions/methods/components. Mock dependencies. Co-located. Developer responsibility. + * **Integration Tests (Backend/Frontend):** Test interactions between internal components with external systems mocked (AWS SDK clients, third-party APIs). + * **End-to-End (E2E) Tests (MVP):** + * Backend API: Automated test for "Hello World"/status. Test daily job trigger verifies DDB/S3 output. + * Frontend UI: Key user flows tested manually for MVP. (Playwright deferred to post-MVP). + * **Coverage:** Guideline \>80% unit test coverage for critical logic. Quality over quantity. Measured by Jest. + * **Mocking:** Jest's built-in system. `axios-mock-adapter` if needed. + * **Test Data:** Inline mocks or small fixtures for unit/integration. + +## 15\. Security Best Practices + + * **Input Validation:** API Gateway basic validation; Zod for detailed payload validation in Lambdas. + * **Output Encoding:** Next.js/React handles XSS for frontend rendering. Backend API is JSON. + * **Secrets Management:** Lambda environment variables via CDK (from local gitignored `.env` for MVP setup). No hardcoding. Pino redaction for logs if needed. + * **Dependency Security:** `npm audit` in CI. Promptly address high/critical vulnerabilities. + * **Authentication/Authorization:** API Gateway API Keys (Frontend Read Key, Admin Action Key). IAM roles with least privilege for service-to-service. + * **Principle of Least Privilege (IAM):** Minimal permissions for all IAM roles (Lambdas, Step Functions, CDK). + * **API Security:** HTTPS enforced by API Gateway/CloudFront. Basic rate limiting on API Gateway. Frontend uses HTTP security headers (via CloudFront/Next.js). + * **Error Disclosure:** Generic errors to client, detailed logs server-side. + * **Infrastructure Security:** S3 bucket access restricted (CloudFront OAC/OAI). + * **Post-MVP:** Consider SAST/DAST, penetration testing. + * **Adherence:** AWS Well-Architected Framework - Security Pillar. + +## 16\. Key Reference Documents + +1. **Product Requirements Document (PRD) - BMad Daily Digest** (Version: 0.1) +2. **UI/UX Specification - BMad Daily Digest** (Version: 0.1) +3. **Algolia Hacker News Search API Documentation** (`https://hn.algolia.com/api`) +4. **Play.ai PlayNote API Documentation** (`https://docs.play.ai/api-reference/playnote/post`) + +## 17\. Change Log + +| Version | Date | Author | Summary of Changes | +| :------ | :----------- | :----------------------- | :---------------------------------------------------------------------------------------------------------------- | +| 0.1 | May 20, 2025 | Fred (Architect) & User | Initial draft of the Architecture Document based on PRD v0.1 and UI/UX Spec v0.1. | diff --git a/demos/v3-output-demo-files/change-log.md b/demos/v3-output-demo-files/change-log.md new file mode 100644 index 00000000..97fcee7d --- /dev/null +++ b/demos/v3-output-demo-files/change-log.md @@ -0,0 +1,8 @@ +# Change Log + +> This document is a granulated shard from the main "PRD.md" focusing on "Change Log". + +| Change | Date | Version | Description | Author | +| :----------------------------------------------------------- | :------------ | :------ | :--------------------------------------------------------------------------------------------------------- | :------------------------------- | +| Initial PRD draft and MVP scope definition. | May 20, 2025 | 0.1 | Created initial PRD based on Project Brief and discussions on goals, requirements, and Epics/Stories (shells). | John (PM) & User | +| Architectural refinements incorporated into Story ACs. | May 20, 2025 | 0.2 | Updated ACs for Stories 2.1 and 3.1 based on System Architecture Document feedback from Fred (Architect). | Sarah (PO) & User | \ No newline at end of file diff --git a/demos/v3-output-demo-files/component-view.md b/demos/v3-output-demo-files/component-view.md new file mode 100644 index 00000000..f2413660 --- /dev/null +++ b/demos/v3-output-demo-files/component-view.md @@ -0,0 +1,90 @@ +# Component View + +The system is divided into distinct backend and frontend components. + +## Backend Components (`bmad-daily-digest-backend` repository) + +1. **Daily Workflow Orchestrator (AWS Step Functions state machine):** Manages the end-to-end daily pipeline. +2. **HN Data Fetcher Service (AWS Lambda):** Fetches HN posts/comments (Algolia), identifies repeats (via DynamoDB). +3. **Article Scraping Service (AWS Lambda):** Scrapes/extracts content from external article URLs, handles fallbacks. +4. **Content Formatting Service (AWS Lambda):** Aggregates and formats text payload for Play.ai. +5. **Play.ai Interaction Service (AWS Lambda functions, orchestrated by Polling Step Function):** Submits job to Play.ai, polls for status. +6. **Podcast Storage Service (AWS Lambda):** Downloads audio from Play.ai, stores to S3. +7. **Metadata Persistence Service (AWS Lambda & DynamoDB Tables):** Manages episode and HN post processing state metadata in DynamoDB. +8. **Backend API Service (AWS API Gateway + AWS Lambda functions):** Exposes endpoints for frontend (episode lists/details). + +## Frontend Components (`bmad-daily-digest-frontend` repository) + +1. **Next.js Web Application (Static Site on S3/CloudFront):** Renders UI, handles navigation. +2. **Frontend API Client Service (TypeScript module):** Encapsulates communication with the Backend API Service. + +## External Services + +- Algolia HN Search API +- Play.ai PlayNote API +- Various External Article Websites + +## Component Interaction Diagram (Conceptual Backend Focus) + +```mermaid +graph LR + subgraph Frontend Application Space + F_App[Next.js App on S3/CloudFront] + F_APIClient[Frontend API Client] + F_App --> F_APIClient + end + + subgraph Backend API Space + APIGW[API Gateway] + API_L[Backend API Lambdas] + APIGW --> API_L + end + + subgraph Backend Daily Pipeline Space + Scheduler[EventBridge Scheduler] --> Orchestrator[Step Functions Orchestrator] + + Orchestrator --> HNFetcher[HN Data Fetcher Lambda] + HNFetcher -->|Reads/Writes Post Status| DDB + HNFetcher --> Algolia[Algolia HN API] + + Orchestrator --> ArticleScraper[Article Scraper Lambda] + ArticleScraper --> ExtWebsites[External Article Websites] + + Orchestrator --> ContentFormatter[Content Formatter Lambda] + + Orchestrator --> PlayAISubmit[Play.ai Submit Lambda] + PlayAISubmit --> PlayAI_API[Play.ai PlayNote API] + + subgraph Polling_SF[Play.ai Polling (Step Functions)] + direction LR + PollTask[Poll Status Lambda] --> PlayAI_API + end + Orchestrator --> Polling_SF + + + Orchestrator --> PodcastStorage[Podcast Storage Lambda] + PodcastStorage --> PlayAI_API + PodcastStorage --> S3Store[S3 Audio Storage] + + Orchestrator --> MetadataService[Metadata Persistence Lambda] + MetadataService --> DDB[DynamoDB Episode/Post Metadata] + end + + F_APIClient --> APIGW + API_L --> DDB + + classDef external fill:#ddd,stroke:#333,stroke-width:2px; + class Algolia,ExtWebsites,PlayAI_API external; +``` + +## Architectural / Design Patterns Adopted + +The following key architectural and design patterns have been chosen for this project: + + * **Serverless Architecture:** Entire backend on AWS Lambda, API Gateway, S3, DynamoDB, Step Functions. Rationale: Minimized operations, auto-scaling, pay-per-use, cost-efficiency. + * **Event-Driven Architecture:** Daily pipeline initiated by EventBridge Scheduler; Step Functions orchestrate based on state changes. Rationale: Decoupled components, reactive system for automation. + * **Microservices-like Approach (Backend Lambda Functions):** Each Lambda handles a specific, well-defined task. Rationale: Modularity, independent scalability, easier testing/maintenance. + * **Static Site Generation (SSG) for Frontend:** Next.js frontend exported as static files, hosted on S3/CloudFront. Rationale: Optimal performance, security, scalability, lower hosting costs. + * **Infrastructure as Code (IaC):** AWS CDK in TypeScript for all AWS infrastructure in both repositories. Rationale: Repeatable, version-controlled, automated provisioning. + * **Polling Pattern (External Job Status):** AWS Step Functions implement a polling loop for Play.ai job status. Rationale: Reliable tracking of asynchronous third-party jobs, based on Play.ai docs. + * **Orchestration Pattern (AWS Step Functions):** End-to-end daily backend pipeline managed by a Step Functions state machine. Rationale: Robust workflow automation, state management, error handling for multi-step processes. \ No newline at end of file diff --git a/demos/v3-output-demo-files/data-models.md b/demos/v3-output-demo-files/data-models.md new file mode 100644 index 00000000..7e7cfcd2 --- /dev/null +++ b/demos/v3-output-demo-files/data-models.md @@ -0,0 +1,40 @@ +# Data Models + +## Core Application Entities + +**a. Episode** + + * Attributes: `episodeId` (PK, UUID), `publicationDate` (YYYY-MM-DD), `episodeNumber` (Number), `podcastGeneratedTitle` (String), `audioS3Bucket` (String), `audioS3Key` (String), `audioUrl` (String, derived for API), `playAiJobId` (String), `playAiSourceAudioUrl` (String), `sourceHNPosts` (List of `SourceHNPost`), `status` (String: "PROCESSING", "PUBLISHED", "FAILED"), `createdAt` (ISO Timestamp), `updatedAt` (ISO Timestamp). + +**b. SourceHNPost (object within `Episode.sourceHNPosts`)** + + * Attributes: `hnPostId` (String), `title` (String), `originalArticleUrl` (String), `hnLink` (String), `isUpdateStatus` (Boolean), `oldRank` (Number, Optional), `lastCommentFetchTimestamp` (Number, Unix Timestamp), `articleScrapingFailed` (Boolean), `articleTitleFromScrape` (String, Optional). + +**c. HackerNewsPostProcessState (DynamoDB Table)** + + * Attributes: `hnPostId` (PK, String), `originalArticleUrl` (String), `articleTitleFromScrape` (String, Optional), `lastSuccessfullyScrapedTimestamp` (Number, Optional), `lastCommentFetchTimestamp` (Number, Optional), `firstProcessedDate` (YYYY-MM-DD), `lastProcessedDate` (YYYY-MM-DD), `lastKnownRank` (Number, Optional). + +## API Payload Schemas (Internal API) + +**a. `EpisodeListItem` (for `GET /episodes`)** + + * `episodeId`, `publicationDate`, `episodeNumber`, `podcastGeneratedTitle`. + +**b. `EpisodeDetail` (for `GET /episodes/{episodeId}`)** + + * `episodeId`, `publicationDate`, `episodeNumber`, `podcastGeneratedTitle`, `audioUrl`, `sourceHNPosts` (list of `SourceHNPostDetail` containing `hnPostId`, `title`, `originalArticleUrl`, `hnLink`, `isUpdateStatus`, `oldRank`), `playAiJobId` (optional), `playAiSourceAudioUrl` (optional), `createdAt`. + +## Database Schemas (AWS DynamoDB) + +**a. `BmadDailyDigestEpisodes` Table** + + * PK: `episodeId` (String). + * Attributes: As per `Episode` entity. + * GSI Example (`PublicationDateIndex`): PK: `status`, SK: `publicationDate`. + * Billing: PAY\_PER\_REQUEST. + +**b. `HackerNewsPostProcessState` Table** + + * PK: `hnPostId` (String). + * Attributes: As per `HackerNewsPostProcessState` entity. + * Billing: PAY\_PER\_REQUEST. \ No newline at end of file diff --git a/demos/v3-output-demo-files/environment-vars.md b/demos/v3-output-demo-files/environment-vars.md new file mode 100644 index 00000000..d48005c1 --- /dev/null +++ b/demos/v3-output-demo-files/environment-vars.md @@ -0,0 +1,19 @@ +# Environment Variables Documentation + +The BMad Daily Digest application uses various environment variables for configuration settings. These variables are referenced throughout the architecture document, particularly in the sections about API interactions with external services. + +## Backend Environment Variables + +### Play.ai API Configuration +- `PLAY_AI_BEARER_TOKEN`: Authentication token for Play.ai API +- `PLAY_AI_USER_ID`: User ID for Play.ai API +- `PLAY_AI_VOICE1_ID`: ID for primary voice used in podcast +- `PLAY_AI_VOICE1_NAME`: Name of primary voice +- `PLAY_AI_VOICE2_ID`: ID for secondary voice used in podcast +- `PLAY_AI_VOICE2_NAME`: Name of secondary voice +- `PLAY_AI_STYLE`: Style parameter for the Play.ai API + +### Frontend API Configuration +- `NEXT_PUBLIC_BACKEND_API_URL`: URL to access the backend API + +Note: The environment variables are managed through AWS Lambda environment variables via CDK (from local gitignored `.env` for MVP setup). \ No newline at end of file diff --git a/demos/v3-output-demo-files/epic-1.md b/demos/v3-output-demo-files/epic-1.md new file mode 100644 index 00000000..31524d05 --- /dev/null +++ b/demos/v3-output-demo-files/epic-1.md @@ -0,0 +1,77 @@ +# Epic 1: Backend Foundation, Tooling & "Hello World" API + +**Goal:** To establish the core backend project infrastructure in its dedicated repository, including robust development tooling and initial AWS CDK setup for essential services. By the end of this epic: +1. A simple "hello world" API endpoint (AWS API Gateway + Lambda) **must** be deployed and testable via `curl`, returning a dynamic message. +2. The backend project **must** have ESLint, Prettier, Jest (unit testing), and esbuild (TypeScript bundling) configured and operational. +3. Basic unit tests **must** exist for the "hello world" Lambda function. +4. Code formatting and linting checks **should** be integrated into a pre-commit hook and/or a basic CI pipeline stub. + +## User Stories + +**Story 1.1: Initialize Backend Project using TS-TEMPLATE-STARTER** +* **User Story Statement:** As a Developer, I want to create the `bmad-daily-digest-backend` Git repository and initialize it using the existing `TS-TEMPLATE-STARTER`, ensuring all foundational tooling (TypeScript, Node.js 22, ESLint, Prettier, Jest, esbuild) is correctly configured and operational for this specific project, so that I have a high-quality, standardized development environment ready for application logic. +* **Acceptance Criteria (ACs):** + 1. A new, private Git repository named `bmad-daily-digest-backend` **must** be created on GitHub. + 2. The contents of the `TS-TEMPLATE-STARTER` project **must** be copied/cloned into this new repository. + 3. `package.json` **must** be updated (project name, version, description). + 4. Project dependencies **must** be installable. + 5. TypeScript setup (`tsconfig.json`) **must** be verified for Node.js 22, esbuild compatibility; project **must** compile. + 6. ESLint and Prettier configurations **must** be operational; lint/format scripts **must** execute successfully. + 7. Jest configuration **must** be operational; test scripts **must** execute successfully with any starter example tests. + 8. Irrelevant generic demo code from starter **should** be removed. `index.ts`/`index.test.ts` can remain as placeholders. + 9. A standard `.gitignore` and an updated project `README.md` **must** be present. + +**Story 1.2: Pre-commit Hook Implementation** +* **User Story Statement:** As a Developer, I want pre-commit hooks automatically enforced in the `bmad-daily-digest-backend` repository, so that code quality standards (like linting and formatting) are checked and applied to staged files before any code is committed, thereby maintaining codebase consistency and reducing trivial errors. +* **Acceptance Criteria (ACs):** + 1. A pre-commit hook tool (e.g., Husky) **must** be installed and configured. + 2. A tool for running linters/formatters on staged files (e.g., `lint-staged`) **must** be installed and configured. + 3. Pre-commit hook **must** trigger `lint-staged` on staged `.ts` files. + 4. `lint-staged` **must** be configured to run ESLint (`--fix`) and Prettier (`--write`). + 5. Attempting to commit files with auto-fixable issues **must** result in fixes applied and successful commit. + 6. Attempting to commit files with non-auto-fixable linting errors **must** abort the commit with error messages. + 7. Committing clean files **must** proceed without issues. + +**Story 1.3: "Hello World" Lambda Function Implementation & Unit Tests** +* **User Story Statement:** As a Developer, I need a simple "Hello World" AWS Lambda function implemented in TypeScript within the `bmad-daily-digest-backend` project. This function, when invoked, should return a dynamic greeting message including the current date and time, and it must be accompanied by comprehensive Jest unit tests, so that our basic serverless compute functionality, testing setup, and TypeScript bundling are validated. +* **Acceptance Criteria (ACs):** + 1. A `src/features/publicApi/statusHandler.ts` file (or similar according to final backend structure) **must** contain the Lambda handler. + 2. Handler **must** be AWS Lambda compatible (event, context, Promise response). + 3. Successful execution **must** return JSON: `statusCode: 200`, body with `message: "Hello from BMad Daily Digest Backend, today is [current_date] at [current_time]."`. + 4. Date and time in message **must** be dynamic. + 5. A corresponding Jest unit test file (e.g., `src/features/publicApi/statusHandler.test.ts`) **must** be created. + 6. Unit tests **must** verify: 200 status, valid JSON body, expected `message` field, "Hello from..." prefix, dynamic date/time portion (use mocked `Date`). + 7. All unit tests **must** pass. + 8. esbuild configuration **must** correctly bundle the handler. + +**Story 1.4: AWS CDK Setup for "Hello World" API (Lambda & API Gateway)** +* **User Story Statement:** As a Developer, I want to define the necessary AWS infrastructure (Lambda function and API Gateway endpoint) for the "Hello World" service using AWS CDK (Cloud Development Kit) in TypeScript, so that the infrastructure is version-controlled, repeatable, and can be deployed programmatically. +* **Acceptance Criteria (ACs):** + 1. AWS CDK (v2) **must** be a development dependency. + 2. CDK app structure **must** be initialized (e.g., in `cdk/`). + 3. A new CDK stack (e.g., `BmadDailyDigestBackendStack`) **must** be defined in TypeScript. + 4. CDK stack **must** define an AWS Lambda resource for the "Hello World" function (Node.js 22, bundled code reference, handler entry point, basic IAM role for CloudWatch logs, free-tier conscious settings). + 5. CDK stack **must** define an AWS API Gateway (HTTP API preferred) with a route (e.g., `GET /status` or `GET /hello`) triggering the Lambda, secured with the "Frontend Read API Key". + 6. CDK stack **must** be synthesizable (`cdk synth`) without errors. + 7. CDK code **must** adhere to project ESLint/Prettier standards. + 8. Mechanism for passing Lambda environment variables via CDK **must** be in place. + +**Story 1.5: "Hello World" API Deployment & Manual Invocation Test** +* **User Story Statement:** As a Developer, I need to deploy the "Hello World" API (defined in AWS CDK) to an AWS environment and successfully invoke its endpoint using a tool like `curl` (including the API Key), so that I can verify the end-to-end deployment process and confirm the basic API is operational in the cloud. +* **Acceptance Criteria (ACs):** + 1. The AWS CDK stack for "Hello World" API **must** deploy successfully to a designated AWS account/region. + 2. The API Gateway endpoint URL for the `/status` (or `/hello`) route **must** be retrievable post-deployment. + 3. A `GET` request to the deployed endpoint, including the correct `x-api-key` header, **must** receive a response. + 4. HTTP response status **must** be 200 OK. + 5. Response body **must** be JSON containing the expected dynamic "Hello..." message. + 6. Basic Lambda invocation logs **must** be visible in AWS CloudWatch Logs. + +**Story 1.6: Basic CI/CD Pipeline Stub with Quality Gates** +* **User Story Statement:** As a Developer, I need a basic Continuous Integration (CI) pipeline established for the `bmad-daily-digest-backend` repository, so that code quality checks (linting, formatting, unit tests) and the build process are automated upon code pushes and pull requests, ensuring early feedback and maintaining codebase health. +* **Acceptance Criteria (ACs):** + 1. A CI workflow file (e.g., GitHub Actions in `.github/workflows/main.yml`) **must** be created. + 2. Pipeline **must** trigger on pushes to `main` and PRs targeting `main`. + 3. Pipeline **must** include steps for: checkout, Node.js 22 setup, dependency install, ESLint check, Prettier format check, Jest unit tests, esbuild bundle. + 4. Pipeline **must** fail if any lint, format, test, or bundle step fails. + 5. A successful CI run on the `main` branch **must** be demonstrated. + 6. CI pipeline for MVP **does not** need to perform AWS deployment. \ No newline at end of file diff --git a/demos/v3-output-demo-files/epic-2.md b/demos/v3-output-demo-files/epic-2.md new file mode 100644 index 00000000..0b767af9 --- /dev/null +++ b/demos/v3-output-demo-files/epic-2.md @@ -0,0 +1,119 @@ +# Epic 2: Automated Content Ingestion & Podcast Generation Pipeline + +**Goal:** To implement the complete automated daily workflow within the backend. This includes fetching Hacker News post data, scraping and extracting content from linked external articles, aggregating and formatting text, submitting it to Play.ai, managing job status via polling, and retrieving/storing the final audio file and associated metadata. This epic delivers the core value proposition of generating the daily audio content and making it ready for consumption via an API. + +## User Stories + +**Story 2.1: AWS CDK Extension for Epic 2 Resources** +* **User Story Statement:** As a Developer, I need to extend the existing AWS CDK stack within the `bmad-daily-digest-backend` project to define and provision all new AWS resources required for the content ingestion and podcast generation pipeline—including the `BmadDailyDigestEpisodes` DynamoDB table (with GSI), the `HackerNewsPostProcessState` DynamoDB table, an S3 bucket for audio storage, and the AWS Step Functions state machine for orchestrating the Play.ai job status polling—so that all backend infrastructure for this epic is managed as code and ready for the application logic. +* **Acceptance Criteria (ACs):** + 1. The existing AWS CDK application (from Epic 1) **must** be extended. + 2. The `BmadDailyDigestEpisodes` DynamoDB table resource **must** be defined in CDK as specified in the System Architecture Document's "Data Models" section (with `episodeId` PK, key attributes like `publicationDate`, `episodeNumber`, `podcastGeneratedTitle`, `audioS3Key`, `audioS3Bucket`, `playAiJobId`, `playAiSourceAudioUrl`, `sourceHNPosts` list, `status`, `createdAt`, `updatedAt`), including a GSI for chronological sorting (e.g., PK `status`, SK `publicationDate`), and PAY_PER_REQUEST billing. + 3. The `HackerNewsPostProcessState` DynamoDB table resource **must** be defined in CDK as specified in the System Architecture Document's "Data Models" section (with `hnPostId` PK and attributes like `lastCommentFetchTimestamp`, `lastSuccessfullyScrapedTimestamp`, `lastKnownRank`), and PAY_PER_REQUEST billing. + 4. An S3 bucket resource (e.g., `bmad-daily-digest-audio-{unique-suffix}`) **must** be defined via CDK for audio storage, with private access by default. + 5. An AWS Step Functions state machine resource **must** be defined via CDK to manage the Play.ai job status polling workflow (as detailed in Story 2.6). + 6. Necessary IAM roles and permissions for Lambda functions within this epic to interact with DynamoDB, S3, Step Functions, CloudWatch Logs **must** be defined via CDK, adhering to least privilege. + 7. The updated CDK stack **must** synthesize (`cdk synth`) and deploy (`cdk deploy`) successfully. + 8. All new CDK code **must** adhere to project ESLint/Prettier standards. + +**Story 2.2: Fetch Top Hacker News Posts & Identify Repeats** +* **User Story Statement:** As the System, I need to reliably fetch the top N (configurable, e.g., 10) current Hacker News posts daily using the Algolia HN API, including their essential metadata. I also need to identify if each fetched post has been processed in a recent digest by checking against the `HackerNewsPostProcessState` table, so that I have an accurate list of stories and their status (new or repeat) to begin generating the daily digest. +* **Acceptance Criteria (ACs):** + 1. A `hackerNewsService.ts` function **must** fetch top N HN posts (stories only) via `axios` from Algolia API (configurable `HN_POSTS_COUNT`). + 2. Extracted metadata per post: Title, Article URL, HN Post URL, HN Post ID (`objectID`), Author, Points, Creation timestamp. + 3. For each post, the function **must** query the `HackerNewsPostProcessState` DynamoDB table to determine its `isUpdateStatus` (true if `lastSuccessfullyScrapedTimestamp` and `lastCommentFetchTimestamp` indicate prior full processing) and retrieve `lastCommentFetchTimestamp` and `lastKnownRank` if available. + 4. Function **must** return an array of HN post objects with metadata, `isUpdateStatus`, `lastCommentFetchTimestamp`, and `lastKnownRank`. + 5. Error handling for Algolia/DynamoDB calls **must** be implemented and logged. + 6. Unit tests (Jest) **must** verify API calls, data extraction, repeat identification (mocked DDB), and error handling. All tests **must** pass. + +**Story 2.3: Article Content Scraping & Extraction (Conditional)** +* **User Story Statement:** As the System, for each Hacker News post identified as *new* (or for which article scraping previously failed and is being retried), I need to robustly fetch its HTML content from the linked article URL and extract the primary textual content and title using libraries like Cheerio and Mozilla Readability. If scraping fails, a fallback mechanism must be triggered. +* **Acceptance Criteria (ACs):** + 1. An `articleScraperService.ts` function **must** accept an article URL and `isUpdateStatus`. + 2. If `isUpdateStatus` is true (article already successfully scraped and stored, though we are not storing full articles long term - this implies we have the article data available from a previous step if it's a repeat post where we don't re-scrape), scraping **must** be skipped. (For MVP, if it's a repeat post, we assume we don't need to re-scrape the article itself, only comments, as per user feedback. This story focuses on *new* scrapes or retries of failed scrapes). + 3. If a new scrape is needed: use `axios` (timeout, User-Agent) to fetch HTML. + 4. Use `Mozilla Readability` (JS port) and/or `Cheerio` to extract main article text and title. + 5. Return `{ success: true, title: string, content: string }` on success. + 6. If scraping fails: log failure, return `{ success: false, error: string, fallbackNeeded: true }`. + 7. No specific "polite" inter-article scraping delays for MVP. + 8. Unit tests (Jest) **must** mock `axios`, test successful extraction, skip logic for non-applicable cases, and failure/fallback scenarios. All tests **must** pass. + +**Story 2.4: Fetch Hacker News Comments (Conditional Logic)** +* **User Story Statement:** As the System, I need to fetch comments for each selected Hacker News post using the Algolia HN API, adjusting the strategy to fetch up to N comments for new posts, only new comments since last fetch for repeat posts, or up to 3N comments if article scraping failed. +* **Acceptance Criteria (ACs):** + 1. `hackerNewsService.ts` **must** be extended to fetch comments for an HN Post ID, accepting `isUpdateStatus`, `lastCommentFetchTimestamp` (from `HackerNewsPostProcessState`), and `articleScrapingFailed` flags. + 2. Use `axios` to call Algolia HN API item endpoint. + 3. **Comment Fetching Logic:** + * If `articleScrapingFailed`: Fetch up to 3 * `HN_COMMENTS_COUNT_PER_POST` available comments. + * If `isUpdateStatus`: Fetch all comments, then filter client-side for comments with `created_at_i` > `lastCommentFetchTimestamp`. Select up to `HN_COMMENTS_COUNT_PER_POST` of these *new* comments. + * Else (new post, successful scrape): Fetch up to `HN_COMMENTS_COUNT_PER_POST`. + 4. For selected comments, extract plain text (HTML stripped), author, creation timestamp. + 5. Return array of comment objects; empty if none. An updated `lastCommentFetchTimestamp` (max `created_at_i` of fetched comments for this post) should be available for updating `HackerNewsPostProcessState`. + 6. Error handling and logging for API calls. + 7. Unit tests (Jest) **must** mock `axios` and verify all conditional fetching logic, comment selection/filtering, data extraction, and error handling. All tests **must** pass. + +**Story 2.5: Content Aggregation and Formatting for Play.ai** +* **User Story Statement:** As the System, I need to aggregate the collected Hacker News post data (titles), associated article content (full, truncated, or fallback summary), and comments (new, updated, or extended sets) for all top stories, and format this combined text according to the specified structure for the play.ai PlayNote API, including special phrasing for different post types (new, update, scrape-failed) and configurable article truncation. +* **Acceptance Criteria (ACs):** + 1. A `contentFormatterService.ts` **must** be implemented. + 2. Inputs: Array of processed HN post objects (with metadata, statuses, content, comments). + 3. Output: A single string. + 4. String starts: "It's a top 10 countdown for [Current Date]". + 5. Posts sequenced in reverse rank order. + 6. **Formatting (new post):** "Story [Rank] - [Article Title]. [Full/Truncated Article Text]. Comments Section. [Number] comments follow. Comment 1: [Text]..." + 7. **Formatting (repeat post):** "Story [Rank] (previously Rank [OldRank] yesterday) - [Article Title]. We're bringing you new comments on this popular story. Comments Section. [Number] new comments follow. Comment 1: [Text]..." + 8. **Formatting (scrape-failed post):** "Story [Rank] - [Article Title]. We couldn't retrieve the full article, but here's a summary if available and the latest comments. [Optional HN Summary]. Comments Section. [Number] comments follow. Comment 1: [Text]..." + 9. **Article Truncation:** If `MAX_ARTICLE_LENGTH` (env var) set and article exceeds, truncate aiming to preserve intro/conclusion. + 10. Graceful handling for missing parts. + 11. Unit tests (Jest) verify all formatting, truncation, data merging, error handling. All tests **must** pass. + +**Story 2.6 (REVISED): Implement Podcast Generation Status Polling via Play.ai API** +* **User Story Statement:** As the System, after submitting a podcast generation job to Play.ai and receiving a `jobId`, I need an AWS Step Function state machine to periodically poll the Play.ai API for the status of this specific job, continuing until the job is reported as "completed" or "failed" (or a configurable max duration/attempts limit is reached), so the system can reliably determine when the podcast audio is ready or if an error occurred. +* **Acceptance Criteria (ACs):** + 1. The AWS Step Function state machine (CDK defined in Story 2.1) **must** manage the polling workflow. + 2. Input: `jobId`. + 3. States: Invoke Poller Lambda (calls Play.ai status GET endpoint with `axios`), Wait (configurable `POLLING_INTERVAL_MINUTES`), Choice (evaluates status: "processing", "completed", "failed"). + 4. Loop if "processing". Stop if "completed" or "failed". + 5. Max polling duration/attempts (configurable env vars `MAX_POLLING_DURATION_MINUTES`, `MAX_POLLING_ATTEMPTS`) **must** be enforced, treating expiry as failure. + 6. If "completed": extract `audioUrl`, trigger next step (Story 2.8 process) with data. + 7. If "failed"/"timeout": log event, record failure (e.g., update episode status in DDB via a Lambda), terminate. + 8. Poller Lambda handles API errors gracefully. + 9. Unit tests for Poller Lambda logic; Step Function definition tested (locally if possible, or via AWS console tests). All tests **must** pass. + +**Story 2.7: Submit Content to Play.ai PlayNote API & Initiate Podcast Generation** +* **User Story Statement:** As the System, I need to securely submit the aggregated and formatted text content (using `sourceText`) to the play.ai PlayNote API via an `application/json` request to initiate the podcast generation process, and I must capture the `jobId` returned by Play.ai, so that this `jobId` can be passed to the status polling mechanism (Step Function). +* **Acceptance Criteria (ACs):** + 1. A `playAiService.ts` function **must** handle submission. + 2. Input: formatted text (from Story 2.5). + 3. Use `axios` for `POST` to Play.ai endpoint (e.g., `https://api.play.ai/api/v1/playnotes`). + 4. Request `Content-Type: application/json`. + 5. JSON body: `sourceText`, and configurable `title`, `voiceId1`, `name1` (default "Angelo"), `voiceId2`, `name2` (default "Deedee"), `styleGuidance` (default "podcast") from env vars. + 6. Headers: `Authorization: Bearer `, `X-USER-ID: ` (from env vars). + 7. No `webHookUrl` sent. + 8. On success: extract `jobId`, log it, initiate polling Step Function (Story 2.6) with `jobId` and other context (like internal `episodeId`). + 9. Error handling for API submission; log and flag failure. + 10. Unit tests (Jest) mock `axios`, verify API call, auth, payload, `jobId` extraction, Step Function initiation (mocked), error handling. All tests **must** pass. + +**Story 2.8: Retrieve, Store Generated Podcast Audio & Persist Episode Metadata** +* **User Story Statement:** As the System, once the podcast generation status polling (Story 2.6) indicates a Play.ai job is "completed," I need to download the generated audio file from the provided `audioUrl`, store this file in our designated S3 bucket, and then save/update all relevant metadata for the episode (including S3 audio location, `episodeNumber`, `podcastGeneratedTitle`, `playAiSourceAudioUrl`, and source HN post information including their `lastCommentFetchTimestamp`) into our DynamoDB tables (`BmadDailyDigestEpisodes` and `HackerNewsPostProcessState`), so that the daily digest is fully processed, archived, and ready for access. +* **Acceptance Criteria (ACs):** + 1. A `podcastStorageService.ts` function **must** be triggered by Step Function (Story 2.6) on "completed" status, receiving `audioUrl`, Play.ai `jobId`, and original context (like internal `episodeId`, list of source HN posts with their metadata and processing status). + 2. Use `axios` to download audio from `audioUrl`. + 3. Upload audio to S3 bucket (from Story 2.1), using key (e.g., `YYYY/MM/DD/episodeId.mp3`). + 4. Prepare `Episode` metadata for `BmadDailyDigestEpisodes` table: `episodeId` (UUID), `publicationDate` (YYYY-MM-DD), `episodeNumber` (sequential logic, TBD), `podcastGeneratedTitle` (from Play.ai or constructed), `audioS3Bucket`, `audioS3Key`, `playAiJobId`, `playAiSourceAudioUrl`, `sourceHNPosts` (array of objects: `{ hnPostId, title, originalArticleUrl, hnLink, isUpdateStatus, oldRank, articleScrapingFailed }`), `status: "Published"`, `createdAt`, `updatedAt`. + 5. For each `hnPostId` in `sourceHNPosts`, update its corresponding item in the `HackerNewsPostProcessState` table with the `lastCommentFetchTimestamp` (current time or max comment time from this run), `lastProcessedDate` (current date), and `lastKnownRank`. If `articleScrapingFailed` was false for this run, update `lastSuccessfullyScrapedTimestamp`. + 6. Save `Episode` metadata to `BmadDailyDigestEpisodes` DynamoDB table. + 7. Error handling for download, S3 upload, DDB writes; failure should result in episode `status: "Failed"`. + 8. Unit tests (Jest) mock `axios`, AWS SDK (S3, DynamoDB); verify data handling, storage, metadata construction for both tables, errors. All tests **must** pass. + +**Story 2.9: Daily Workflow Orchestration & Scheduling** +* **User Story Statement:** As the System Administrator, I need the entire daily backend workflow (Stories 2.2 through 2.8) to be fully orchestrated by the primary AWS Step Function state machine and automatically scheduled to run once per day using Amazon EventBridge Scheduler, ensuring it handles re-runs for the same day by overwriting/starting over (for MVP), so that "BMad Daily Digest" episodes are produced consistently and reliably. +* **Acceptance Criteria (ACs):** + 1. The primary AWS Step Function state machine **must** orchestrate the sequence: Fetch HN Posts & Identify Repeats (2.2); For each post: conditionally Scrape Article (2.3) & Fetch Comments (2.4); then Aggregate & Format Content (2.5); then Submit to Play.ai & get `jobId` (2.7); then initiate/manage Polling (2.6 using `jobId`); on "completed" polling, trigger Retrieve & Store Audio/Metadata (2.8). + 2. State machine **must** manage data flow (inputs/outputs) between steps correctly. + 3. Overall workflow error handling: critical step failure marks state machine execution as "Failed" and logs comprehensively. Steps use retries for transient errors. + 4. **Idempotency (MVP):** Re-running for the same `publicationDate` **must** re-process and effectively overwrite previous data for that date. + 5. Amazon EventBridge Scheduler rule (CDK defined) **must** trigger the main Step Function daily at 12:00 UTC (default, configurable via `DAILY_JOB_SCHEDULE_UTC_CRON`). + 6. Successful end-to-end run **must** be demonstrated (e.g., processing sample data through the pipeline). + 7. Step Function execution history **must** provide a clear audit trail of steps and data. + 8. Unit tests for any new orchestrator-specific Lambda functions (if any not covered). All tests **must** pass. \ No newline at end of file diff --git a/demos/v3-output-demo-files/epic-3.md b/demos/v3-output-demo-files/epic-3.md new file mode 100644 index 00000000..6bf5c2e0 --- /dev/null +++ b/demos/v3-output-demo-files/epic-3.md @@ -0,0 +1,83 @@ +# Epic 3: Web Application MVP & Podcast Consumption + +**Goal:** To set up the frontend project in its dedicated repository and develop and deploy the Next.js frontend application MVP, enabling users to consume the "BMad Daily Digest." This includes initial project setup (AI-assisted UI kickstart from `bmad-daily-digest-ui` scaffold), pages for listing and detailing episodes, an about page, and deployment. + +## User Stories + +**Story 3.1: Frontend Project Repository & Initial UI Setup (AI-Assisted)** +* **User Story Statement:** As a Developer, I need to establish the `bmad-daily-digest-frontend` Git repository with a new Next.js (TypeScript, Node.js 22) project, using the provided `bmad-daily-digest-ui` V0 scaffold as the base. This setup must include all foundational tooling (ESLint, Prettier, Jest with React Testing Library, a basic CI stub), and an initial AWS CDK application structure, ensuring the "80s retro CRT terminal" aesthetic (with Tailwind CSS and shadcn/ui) is operational, so that a high-quality, styled, and standardized frontend development environment is ready. +* **Acceptance Criteria (ACs):** + 1. A new, private Git repository `bmad-daily-digest-frontend` **must** be created on GitHub. + 2. The `bmad-daily-digest-ui` V0 scaffold project files **must** be used as the initial codebase in this repository. + 3. `package.json` **must** be updated (project name, version, description). + 4. Project dependencies **must** be installable. + 5. TypeScript (`tsconfig.json`), Next.js (`next.config.mjs`), Tailwind (`tailwind.config.ts`), ESLint, Prettier, Jest configurations from the scaffold **must** be verified and operational. + 6. The application **must** build successfully (`npm run build`) with the scaffolded UI. + 7. A basic CI pipeline stub (GitHub Actions) for lint, format check, test, build **must** be created. + 8. A standard `.gitignore` and an updated `README.md` **must** be present. + 9. An initial AWS CDK application structure **must** be created within a `cdk/` directory in this repository, ready for defining frontend-specific infrastructure (S3, CloudFront in Story 3.6). + +**Story 3.2: Frontend API Service Layer for Backend Communication** +* **User Story Statement:** As a Frontend Developer, I need a dedicated and well-typed API service layer (e.g., `lib/api-client.ts`) within the Next.js frontend application to manage all HTTP communication with the "BMad Daily Digest" backend API (for fetching episode lists and specific episode details), so that UI components can cleanly and securely consume backend data with robust error handling. +* **Acceptance Criteria (ACs):** + 1. A TypeScript module `lib/api-client.ts` (or similar) **must** encapsulate backend API interactions. + 2. Functions **must** exist for: `getEpisodes(): Promise` and `getEpisodeDetails(episodeId: string): Promise`. + 3. `axios` (or native `Workspace` with a wrapper if preferred for frontend) **must** be used for HTTP requests. + 4. Backend API base URL (`NEXT_PUBLIC_BACKEND_API_URL`) and Frontend Read API Key (`NEXT_PUBLIC_FRONTEND_API_KEY`) **must** be configurable via public environment variables and used in requests. + 5. TypeScript interfaces (`EpisodeListItem`, `EpisodeDetail`, `SourceHNPostDetail` from `lib/types.ts`) for API response data **must** be defined/used, matching backend API. + 6. API functions **must** correctly parse JSON responses and transform data into defined interfaces. + 7. Error handling (network errors, non-2xx responses from backend) **must** be implemented, providing clear error information/objects. + 8. Unit tests (Jest) **must** mock the HTTP client and verify API calls, data parsing/transformation, and error handling. All tests **must** pass. + +**Story 3.3: Episode List Page Implementation** +* **User Story Statement:** As a Busy Tech Executive, I want to view a responsive "Episode List Page" (based on `app/(pages)/episodes/page.tsx` from the scaffold) that clearly displays all available "BMad Daily Digest" episodes in reverse chronological order, showing the episode number, publication date, and podcast title for each, using themed components like `episode-card.tsx`, so that I can quickly find and select an episode. +* **Acceptance Criteria (ACs):** + 1. The existing `app/(pages)/episodes/page.tsx` (or equivalent main list page from scaffold) **must** be updated. + 2. It **must** use the API service layer (Story 3.2) to fetch episodes. + 3. A themed loading state (e.g., using `loading-state.tsx`) **must** be shown. + 4. A themed error message (e.g., using `error-state.tsx`) **must** be shown if fetching fails. + 5. A "No episodes available yet" message **must** be shown for an empty list. + 6. Episodes **must** be listed in reverse chronological order. + 7. Each list item, potentially using a modified `episode-card.tsx` component, **must** display "Episode [EpisodeNumber]: [PublicationDate] - [PodcastGeneratedTitle]". + 8. Each item **must** link to the Episode Detail Page for that episode using its `episodeId`. + 9. Styling **must** adhere to the "80s retro CRT terminal" aesthetic. + 10. The page **must** be responsive. + 11. Unit/integration tests (Jest with RTL) **must** cover all states, data display, order, and navigation. All tests **must** pass. + +**Story 3.4: Episode Detail Page Implementation** +* **User Story Statement:** As a Busy Tech Executive, after selecting an episode, I want to navigate to a responsive "Episode Detail Page" (based on `app/(pages)/episodes/[episodeId]/page.tsx`/page.tsx] from the scaffold) that features an embedded HTML5 audio player, displays the episode title/date/number, a list of the Hacker News stories covered (using components like `story-item.tsx`), and provides clear links to the original articles and HN discussions, so I can listen and explore sources. +* **Acceptance Criteria (ACs):** + 1. The dynamic route page `app/(pages)/episodes/[episodeId]/page.tsx` **must** be implemented. + 2. It **must** accept `episodeId` from the URL. + 3. It **must** use the API service layer (Story 3.2) to fetch episode details. + 4. Loading and error states **must** be handled and displayed with themed components. + 5. If data found, **must** display: `podcastGeneratedTitle`, `publicationDate`, `episodeNumber`. + 6. An embedded HTML5 audio player (`