diff --git a/BETA-V3/docs/preferred-patterns-and-preferences.md b/BETA-V3/docs/preferred-patterns-and-preferences.md new file mode 100644 index 00000000..91ccfb8f --- /dev/null +++ b/BETA-V3/docs/preferred-patterns-and-preferences.md @@ -0,0 +1,81 @@ +# User-Defined Preferred Patterns and Preferences + +This document is intended for you, the user, to maintain a list of your preferred architectural patterns, design choices, technologies, or specific configurations. The Architect Agent will consult this file, when available, to understand your preferences and will aim to incorporate them into its recommendations, still seeking your confirmation. + +You can evolve this document over time as you discover new patterns, solidify your preferences, or wish to guide the Architect Agent more specifically for certain types of projects, platforms, languages, or architectural styles. + +## How to Use This Document + +- **List Your Preferences:** Add entries below detailing specific patterns, technologies, or approaches you favor. +- **Provide Context/Rationale (Optional but Recommended):** Briefly explain _why_ you prefer a certain pattern or technology. This helps the Architect Agent understand the underlying reasons and apply the preference more intelligently. +- **Specify Scope (Optional):** If a preference is specific to a certain context (e.g., "For serverless Node.js projects," "For frontend applications requiring real-time updates"), please note that. +- **Keep it Updated:** As your preferences evolve, update this document. + +## Example Preferences (Illustrative - Please Replace with Your Own) + +--- + +### General Architectural Patterns + +- **Pattern:** Microservices Architecture + + - **Preference Level:** Strongly Preferred for large, complex applications. + - **Rationale:** Promotes scalability, independent deployment, and technology diversity. + - **Notes:** Consider event-driven communication (e.g., Kafka, RabbitMQ) for inter-service communication where appropriate. + +- **Pattern:** Hexagonal Architecture (Ports and Adapters) + - **Preference Level:** Preferred for core business logic components. + - **Rationale:** Enhances testability and decouples business logic from infrastructure concerns. + +--- + +### Technology Preferences + +- **Category:** Cloud Provider + + - **Technology:** AWS (Amazon Web Services) + - **Preference Level:** Preferred + - **Rationale:** Familiarity with the ecosystem, extensive service offerings. + - **Specific Services to Favor (if applicable):** + - Compute: Lambda for serverless, EC2 for general purpose, EKS for Kubernetes. + - Database: RDS (PostgreSQL), DynamoDB. + - Storage: S3. + +- **Category:** Backend Language/Framework + + - **Technology:** Python with FastAPI + - **Preference Level:** Strongly Preferred for new API development. + - **Rationale:** Performance, ease of development, modern features. + +- **Category:** Frontend Framework + - **Technology:** React with TypeScript + - **Preference Level:** Preferred + - **Rationale:** Component-based architecture, strong community support, type safety. + +--- + +### Design Choices & Styles + +- **Factories and Facades:** Especially for third party APIs or libraries + + - Use Facades around APIs, when unit testing a file that uses the facade, mock the facade. when testing the facade, if its a library do not mock the library. if its an api, intercept the requests and mock the responses. + +- **Topic:** API Design + + - **Style:** RESTful APIs + - **Preference Level:** Preferred + - **Notes:** Adhere to standard HTTP methods, clear resource naming, and consistent error handling. Consider OpenAPI for documentation. + - Handlers should only deal with request and response concerns and pass off to a controller for business logic. + +- **Topic:** Infrastructure as Code (IaC) + - **Tool:** Terraform + - **Preference Level:** Strongly Preferred + - **Rationale:** Cloud-agnostic, declarative syntax, strong community. + +--- + +_(Add your own preferences below this line. You can copy and modify the structure of the examples above or create your own format.)_ + +## Your Preferences: + +_(Start adding your preferences here...)_ diff --git a/BETA-V3/templates/prd-tmpl.txt b/BETA-V3/templates/prd-tmpl.txt index 54c63955..f17643bb 100644 --- a/BETA-V3/templates/prd-tmpl.txt +++ b/BETA-V3/templates/prd-tmpl.txt @@ -59,6 +59,48 @@ How will we validate functionality beyond unit testing? Will we want manual scri Anything you and the user agreed it out of scope or can be removed from scope to keep MVP lean. Consider the goals of the PRD and what might be extra gold plating or additional features that could wait until the MVP is completed and delivered to assess functionality and market fit or usage. +## [OPTIONAL: For Simplified PM-to-Development Workflow Only] Core Technical Decisions & Application Structure + +{This section is to be populated ONLY if the PM is operating in the 'Simplified PM-to-Development Workflow'. It captures essential technical foundations that would typically be defined by an Architect, allowing for a more direct path to development. This information should be gathered after initial PRD sections (Goals, Users, etc.) are drafted, and ideally before or in parallel with detailed Epic/Story definition, and updated as needed.} + +### Technology Stack Selections +{Collaboratively define the core technologies. Be specific about choices and versions where appropriate.} +- **Primary Backend Language/Framework:** {e.g., Python/FastAPI, Node.js/Express, Java/Spring Boot} +- **Primary Frontend Language/Framework (if applicable):** {e.g., TypeScript/React (Next.js), JavaScript/Vue.js} +- **Database:** {e.g., PostgreSQL, MongoDB, AWS DynamoDB} +- **Key Libraries/Services (Backend):** {e.g., Authentication (JWT, OAuth provider), ORM (SQLAlchemy), Caching (Redis)} +- **Key Libraries/Services (Frontend, if applicable):** {e.g., UI Component Library (Material-UI, Tailwind CSS + Headless UI), State Management (Redux, Zustand)} +- **Deployment Platform/Environment:** {e.g., Docker on AWS ECS, Vercel, Netlify, Kubernetes} +- **Version Control System:** {e.g., Git with GitHub/GitLab} + +### Proposed Application Structure +{Describe the high-level organization of the codebase. This might include a simple text-based directory layout, a list of main modules/components, and a brief explanation of how they interact. The goal is to provide a clear starting point for developers.} + +Example: +``` +/ +├── app/ # Main application source code +│ ├── api/ # Backend API routes and logic +│ │ ├── v1/ +│ │ └── models.py +│ ├── web/ # Frontend components and pages (if monolithic) +│ │ ├── components/ +│ │ └── pages/ +│ ├── core/ # Shared business logic, utilities +│ └── main.py # Application entry point +├── tests/ # Unit and integration tests +├── scripts/ # Utility scripts +├── Dockerfile +├── requirements.txt +└── README.md +``` +- **Monorepo/Polyrepo:** {Specify if a monorepo or polyrepo structure is envisioned, and briefly why.} +- **Key Modules/Components and Responsibilities:** + - {Module 1 Name}: {Brief description of its purpose and key responsibilities} + - {Module 2 Name}: {Brief description of its purpose and key responsibilities} + - ... +- **Data Flow Overview (Conceptual):** {Briefly describe how data is expected to flow between major components, e.g., Frontend -> API -> Core Logic -> Database.} + ## Change Log | Change | Date | Version | Description | Author | diff --git a/BETA-V3/v3-demos/project1/prd-incremental-full-agile-mode.txt b/BETA-V3/v3-demos/project1/prd-incremental-full-agile-mode.txt new file mode 100644 index 00000000..cba82b77 --- /dev/null +++ b/BETA-V3/v3-demos/project1/prd-incremental-full-agile-mode.txt @@ -0,0 +1,420 @@ +# BMad DiCaster Product Requirements Document (PRD) + +## Goal, Objective and Context + +**Goal:** To develop a web application that provides a daily, concise summary of top Hacker News (HN) posts, delivered as a newsletter and accessible via a web interface. [cite: 1, 2, 3, 4, 5, 6, 7, 8] + +**Objective:** To streamline the consumption of HN content by curating the top stories, providing AI-powered summaries, and offering an optional AI-generated podcast version. [cite: 1, 2, 3, 4, 5, 6, 7, 8] + +**Context:** Busy professionals and enthusiasts want to stay updated on HN but lack the time to sift through numerous posts and discussions. [cite: 17, 18, 19, 20, 21, 22, 67] This application will address this problem by automating the delivery of summarized content. [cite: 2, 3, 9, 10, 11, 12, 13, 14, 15, 16] + +## Functional Requirements (MVP) + +- **HN Content Retrieval & Storage:** + - Daily retrieval of the top 30 Hacker News posts and associated comments using the HN Algolia API. [cite: 20, 21, 22] + - Scraping and storage of up to 10 linked articles per day. [cite: 20, 21, 22] + - Storage of all retrieved data (posts, comments, articles) with date association. [cite: 20, 21, 22] +- **AI-Powered Summarization:** + - AI-powered summarization of the 10 selected articles (2-paragraph summaries). [cite: 23, 24, 25] + - AI-powered summarization of comments for the 10 selected posts (2-paragraph summaries highlighting interesting interactions). [cite: 23, 24, 25] + - Configuration for local or remote LLM usage via environment variables. [cite: 25, 26, 27, 28, 29, 30, 31, 32, 33] +- **Newsletter Generation & Delivery:** + - Generation of a daily newsletter in HTML format, including summaries, links to HN posts and articles, and original post dates/times. [cite: 25, 26, 27, 28, 29, 30, 31, 32, 33] + - Automated delivery of the newsletter to a manually configured list of subscribers in Supabase. [cite: 25, 26, 27, 28, 29, 30, 31, 32, 33] The list of emails will be manually populated in the database. Account information for the Nodemailer service will be provided via environment variables. +- **Podcast Generation & Integration:** + - Integration with Play.ht's PlayNote API for AI-generated podcast creation from the newsletter content. [cite: 28, 29, 30, 31, 32, 33] + - Webhook handler to update the newsletter with the generated podcast link. [cite: 28, 29, 30, 31, 32, 33] +- **Web Interface (MVP):** + - Display of current and past newsletters. [cite: 30, 31, 32, 33] + - Functionality to read the newsletter content within the web page. [cite: 30, 31, 32, 33] + - Download option for newsletters. [cite: 30, 31, 32, 33] + - Web player for listening to generated podcasts. [cite: 30, 31, 32, 33] + - Basic mobile responsiveness for displaying newsletters and podcasts. +- **API & Triggering:** + - Secure API endpoint to manually trigger the daily workflow, secured with API keys. [cite: 32, 33] + - CLI command to manually trigger the daily workflow locally. [cite: 32, 33] + +## Non-Functional Requirements (MVP) + +- **Performance:** + - The system should retrieve HN posts and generate the newsletter within a reasonable timeframe (e.g., under 30 minutes) to ensure timely delivery. + - The web interface should load quickly (e.g., within 2 seconds) to provide a smooth user experience. +- **Scalability:** + - The system is designed for an initial MVP delivery to 3-5 email subscribers. Scalability beyond this will be considered post-MVP. +- **Security:** + - The API endpoint for triggering the daily workflow must be secure, using API keys. + - User data (email addresses) should be stored securely. No other security measures are required for the MVP. +- **Reliability:** + - No specific uptime or availability requirements are defined for the MVP. + - The newsletter generation and delivery process should be robust and handle potential errors gracefully. + - The system must be executable from a local development environment. +- **Maintainability:** + - The codebase should adhere to good quality coding standards, including separation of concerns. + - The system should employ facades and factories to facilitate future expansion. + - The system should be built as an event-driven pipeline, leveraging Supabase to capture data at each stage and trigger subsequent functions asynchronously. This approach aims to mitigate potential timeout issues with Vercel hosting. + +## User Interaction and Design Goals + +This section captures the high-level vision and goals for the User Experience (UX) to guide the Design Architect. [cite: 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57] + +- **Overall Vision & Experience:** + - The desired look and feel is modern and minimalist, with synthwave technical glowing purple vibes. + - Users should have a clean and efficient experience when accessing and consuming newsletter content and podcasts. +- **Key Interaction Paradigms:** + - Interaction paradigms will be determined by the Design Architect. +- **Core Screens/Views (Conceptual):** + - The MVP will consist of two pages: + - A list page to display current and past newsletters. + - A detail page to display the selected newsletter content, including: + - Download option for the newsletter. + - Web player for listening to the generated podcast. + - The article laid out for viewing. +- **Accessibility Aspirations:** + - No specific accessibility goals are defined for the MVP. +- **Branding Considerations (High-Level):** + - A logo for the application will be provided. + - The application will use the name "BMad DiCaster". +- **Target Devices/Platforms:** + - The application will be designed as a mobile-first responsive web app, ensuring it looks good on both mobile and desktop devices. + +## Technical Assumptions + +This section captures any existing technical information that will guide the Architect in the technical design. [cite: 57, 58, 59, 60, 61, 62] + +- The application will be developed using the Next.js/Supabase template and hosted entirely on Vercel. [cite: 39, 40, 41, 42] +- This implies a monorepo structure, as the frontend (Next.js) and backend (Supabase functions) will reside within the same repository. [cite: 59, 60, 61, 62] +- The backend will primarily leverage serverless functions provided by Vercel and Supabase. [cite: 59, 60, 61, 62] +- Frontend development will be in Next.js with React. +- Data storage will be handled by Supabase's PostgreSQL database. +- Separate Supabase instances will be used for development and production environments to ensure data isolation and stability. +- For local development, developers can utilize the Supabase CLI and Vercel CLI to emulate the production environment, primarily for testing functions and deployments, but the development Supabase instance will be the primary source of dev data. +- Testing will include unit tests, integration tests (especially for interactions with Supabase), and end-to-end tests. + +## Epic Overview + +_(Note: Epics will be developed sequentially. Development will start with Epic 1 and proceed to the next epic only after the previous one is fully completed and verified.)_ + +_(Note: All UI development across all epics must adhere to mobile responsiveness and Tailwind CSS/theming principles to ensure a consistent and maintainable user experience.)_ + +- **Epic 1: Project Initialization, Setup, and HN Content Acquisition** + + - Goal: Establish the foundational project structure, including the Next.js application, Supabase integration, and deployment pipeline, implement the initial API and CLI trigger mechanisms, and implement the functionality to retrieve, process, and store Hacker News posts and comments, providing the necessary data for newsletter generation. + +- **Epic 2: Article Scraping** + + - Goal: Implement the functionality to scrape and store linked articles from HN posts, enriching the data available for summarization and the newsletter, and ensure this functionality can be triggered via the API and CLI. + +- **Epic 3: AI-Powered Content Summarization** + + - Goal: Integrate AI summarization capabilities to generate concise summaries of articles and comments, enriching the newsletter content, and ensure this functionality can be triggered via the API and CLI. + +- **Epic 4: Automated Newsletter Creation and Distribution** + + - Goal: Automate the generation and delivery of the daily newsletter, including a placeholder for the podcast URL and conditional logic with configurable delay/retry parameters to handle podcast link availability. Ensure this functionality can be triggered via the API and CLI. + +- **Epic 5: Podcast Generation Integration** + + - Goal: Integrate with the Play.ht's PlayNote API to generate and incorporate podcast versions of the newsletter, including a webhook to update the newsletter data with the podcast URL in Supabase. Ensure this functionality can be triggered via the API and CLI. + +- **Epic 6: Web Interface for Initial Structure and Content Access** + + - Goal: Develop a user-friendly web interface, starting with an initial structure generated by a tool like Vercel v0, to display newsletters and provide access to podcast content. + +**Epic 1: Project Initialization, Setup, and HN Content Acquisition** + +- **Story 1.1:** As a developer, I want to set up the Next.js project with Supabase integration, so that I have a functional foundation for building the application. + + - Acceptance Criteria: + - The Next.js project is initialized using the Vercel/Supabase template. + - Supabase is successfully integrated with the Next.js project. + - The project codebase is initialized in a Git repository. + +- **Story 1.2:** As a developer, I want to configure the deployment pipeline to Vercel with separate development and production environments, so that I can easily deploy and update the application. + + - Acceptance Criteria: + - The project is successfully linked to a Vercel project with separate environments. + - Automated deployments are configured for the main branch to the production environment. + - Environment variables are set up for local development and Vercel deployments. + +- **Story 1.3:** As a developer, I want to implement the API and CLI trigger mechanisms, so that I can manually trigger the workflow during development and testing. + + - Acceptance Criteria: + - A secure API endpoint is created. + - The API endpoint requires authentication (API key). + - The API endpoint triggers the execution of the functionality implemented in this epic. + - The API endpoint returns an appropriate response to indicate success or failure. + - A CLI command is created. + - The CLI command triggers the execution of the functionality implemented in this epic. + - The CLI command provides informative output to the console. + - All API requests and CLI command executions are logged, including timestamps and any relevant data. + - The API and CLI interfaces adhere to mobile responsiveness and Tailwind/theming principles. + +- **Story 1.4:** As a system, I want to retrieve the top 30 Hacker News posts and associated comments daily, so that the data is available for summarization and newsletter generation. + + - Acceptance Criteria: + - The system retrieves the top 30 Hacker News posts daily using the HN Algolia API. + - The system retrieves associated comments for the top 30 posts. + - Retrieved data (posts and comments) is stored in the Supabase database with date association. + - This functionality can be triggered via the API and CLI. + - The system logs the start and completion of the retrieval process, including any errors. + +**Epic 2: Article Scraping** + +- **Story 2.1:** As a system, I want to identify URLs within the top 30 (configurable via environment variable) Hacker News posts, so that I can extract the content of linked articles. + + - Acceptance Criteria: + - The system parses the top N (configurable via env var) Hacker News posts to identify URLs. + - The system filters out any URLs that are not relevant to article scraping (e.g., links to images, videos, etc.). + +- **Story 2.2:** As a system, I want to scrape the content of the identified article URLs using Cheerio, so that I can provide summaries in the newsletter. + + - Acceptance Criteria: + - The system scrapes the content from the identified article URLs using Cheerio. + - The system extracts relevant content such as the article title, author, publication date, and main text. + - The system handles potential issues during scraping, such as website errors or changes in website structure, logging errors for review. + +- **Story 2.3:** As a system, I want to store the scraped article content in the Supabase database, associated with the corresponding Hacker News post, so that it can be used for summarization and newsletter generation. + + - Acceptance Criteria: + - The system stores the scraped article content in the Supabase database, associated with the corresponding Hacker News post. + - The system ensures that the stored data includes all extracted information (title, author, date, text). + +- **Story 2.4:** As a developer, I want to trigger the article scraping process via the API and CLI, so that I can manually initiate it for testing and debugging. + + - Acceptance Criteria: + - The API endpoint can trigger the article scraping process. + - The CLI command can trigger the article scraping process locally. + - The system logs the start and completion of the scraping process, including any errors encountered. + - All API requests and CLI command executions are logged, including timestamps and any relevant data. + - The system handles partial execution gracefully (i.e., if triggered before Epic 1 is complete, it logs a message and exits). + +**Epic 3: AI-Powered Content Summarization** + +- **Story 3.1:** As a system, I want to integrate an AI summarization library or API, so that I can generate concise summaries of articles and comments. + + - Acceptance Criteria: + - The system integrates a suitable AI summarization library or API (e.g., a local LLM or a cloud-based service). + - The integration is configurable via environment variables (e.g., to switch between local and remote LLMs). + +- **Story 3.2:** As a system, I want to retrieve summarization prompts from the database, and then use them to generate 2-paragraph summaries of the scraped articles, so that users can quickly grasp the main content and the prompts can be easily updated. + + - Acceptance Criteria: + - The system retrieves the appropriate summarization prompt from the database. + - The system generates a 2-paragraph summary for each scraped article using the retrieved prompt. + - The summaries are accurate and capture the key information from the article. + +- **Story 3.3:** As a system, I want to retrieve summarization prompts from the database, and then use them to generate 2-paragraph summaries of the comments for the selected HN posts, so that users can understand the main discussions and the prompts can be easily updated. + + - Acceptance Criteria: + - The system retrieves the appropriate summarization prompt from the database. + - The system generates a 2-paragraph summary of the comments for each selected HN post using the retrieved prompt. + - The summaries highlight interesting interactions and key points from the discussion. + +- **Story 3.4:** As a developer, I want to trigger the AI summarization process via the API and CLI, so that I can manually initiate it for testing and debugging. + + - Acceptance Criteria: + - The API endpoint can trigger the AI summarization process. + - The CLI command can trigger the AI summarization process locally. + - The system logs the input and output of the summarization process, including the summarization prompt used and any errors. + - All API requests and CLI command executions are logged, including timestamps and any relevant data. + - The system handles partial execution gracefully (i.e., if triggered before Epic 2 is complete, it logs a message and exits). + +**Epic 4: Automated Newsletter Creation and Distribution** + +- **Story 4.1:** As a system, I want to retrieve the newsletter template from the database, so that the newsletter's design and structure can be updated without code changes. + + - Acceptance Criteria: + - The system retrieves the newsletter template from the database. + +- **Story 4.2:** As a system, I want to generate a daily newsletter in HTML format using the retrieved template, so that users can receive a concise summary of Hacker News content. + + - Acceptance Criteria: + - The system generates a newsletter in HTML format using the template retrieved from the database. + - The newsletter includes summaries of selected articles and comments. + - The newsletter includes links to the original HN posts and articles. + - The newsletter includes the original post dates/times. + +- **Story 4.3:** As a system, I want to send the generated newsletter to a list of subscribers using Nodemailer, with credentials securely provided via environment variables, so that users receive the daily summary in their inbox. + + - Acceptance Criteria: + - The system retrieves the list of subscriber email addresses from the Supabase database. + - The system sends the HTML newsletter to all active subscribers using Nodemailer. + - Nodemailer credentials (e.g., Gmail account information) are securely accessed via environment variables. + - The system logs the delivery status for each subscriber. + - The initial email template includes a placeholder for the podcast URL. + - The system implements conditional logic and a configurable delay/retry mechanism (via environment variables) to handle podcast link availability before sending the email. + +- **Story 4.4:** As a developer, I want to trigger the newsletter generation and distribution process via the API and CLI, so that I can manually initiate it for testing and debugging. + + - Acceptance Criteria: + - The API endpoint can trigger the newsletter generation and distribution process. + - The CLI command can trigger the newsletter generation and distribution process locally. + - The system logs the start and completion of the process, including any errors. + - All API requests and CLI command executions are logged, including timestamps and any relevant data. + - The system handles partial execution gracefully (i.e., if triggered before Epic 3 is complete, it logs a message and exits). + +**Epic 5: Podcast Generation Integration** + +- **Story 5.1:** As a system, I want to integrate with the Play.ht's PlayNote API, so that I can generate AI-powered podcast versions of the newsletter content. + + - Acceptance Criteria: + - The system integrates with the Play.ht's PlayNote API using webhooks for asynchronous updates. + - The integration is configurable via environment variables (e.g., API key, user ID, webhook URL). + +- **Story 5.2:** As a system, I want to send the newsletter content to the Play.ht API to initiate podcast generation, and receive a job ID or initial response, so that I can track the podcast creation process. + + - Acceptance Criteria: + - The system sends the newsletter content to the Play.ht API to generate a podcast. + - The system receives a job ID or initial response from the Play.ht API. + - The system stores the job ID for tracking purposes. + +- **Story 5.3:** As a system, I want to implement a webhook handler to receive the podcast URL from Play.ht, and update the newsletter data, so that the podcast link can be included in the newsletter and web interface. + + - Acceptance Criteria: + - The system implements a webhook handler to receive the podcast URL from Play.ht. + - The system extracts the podcast URL from the webhook payload. + - The system updates the newsletter data in the Supabase database with the podcast URL. + +- **Story 5.4:** As a developer, I want to trigger the podcast generation process via the API and CLI, so that I can manually initiate it for testing and debugging. + + - Acceptance Criteria: + - The API endpoint can trigger the podcast generation process. + - The CLI command can trigger the podcast generation process locally. + - The system logs the start and completion of the process, including any intermediate steps, responses from the Play.ht API, and webhook interactions. + - All API requests and CLI command executions are logged, including timestamps and any relevant data. + - The system handles partial execution gracefully (i.e., if triggered before Epic 4 is complete, it logs a message and exits). + - _(Note for Architect: The Play.ht API - [https://docs.play.ai/api-reference/playnote/post](https://docs.play.ai/api-reference/playnote/post) - supports both webhook and polling mechanisms for receiving the podcast URL. We will use the webhook approach for real-time updates and efficiency.)_ + +**Epic 6: Web Interface for Initial Structure and Content Access** + +- **Story 6.1:** As a developer, I want to use a tool like Vercel v0 to generate the initial structure of the web interface, so that I have a basic layout to work with. + + - Acceptance Criteria: + - A tool (e.g., Vercel v0) is used to generate the initial HTML/CSS structure for the web interface. + - The generated structure includes basic layouts for the newsletter list and detail pages, as well as podcast display. + +- **Story 6.2:** As a user, I want to see a list of current and past newsletters, so that I can easily browse available content. + + - Acceptance Criteria: + - The web interface displays a list of newsletters. + - The list includes relevant information such as the newsletter title and date. + - The list is paginated or provides scrolling functionality to handle a large number of newsletters. + +- **Story 6.3:** As a user, I want to be able to read the newsletter content within the web page, so that I can conveniently consume the information. + + - Acceptance Criteria: + - Clicking on a newsletter in the list displays the full newsletter content. + - The newsletter content is displayed in a readable format. + +- **Story 6.4:** As a user, I want to have the option to download newsletters, so that I can access them offline. + + - Acceptance Criteria: + - The web interface provides a download option for each newsletter. + - Newsletters can be downloaded in a suitable format (e.g., HTML, PDF). + +- **Story 6.5:** As a user, I want to listen to generated podcasts within the web interface, so that I can consume the content in audio format. + + - Acceptance Criteria: + - The web interface includes a web player for listening to podcasts. + - The player includes standard playback controls (e.g., play, pause, volume). + - The podcast player is integrated with the display of the corresponding newsletter. + +## Key Reference Documents + +_(This section will be created later, from the sections prior to this being carved up into smaller documents)_ + +## Out of Scope Ideas Post MVP + +- User Authentication and Management [cite: 34, 35, 36, 37, 38, 67] +- Subscription Management [cite: 34, 35, 36, 37, 38, 67] +- Admin Dashboard [cite: 34, 35, 36, 37, 38, 67] + - Viewing and updating daily podcast settings + - Prompt management for summarization + - UI for template modification +- Enhanced Newsletter Customization [cite: 34, 35, 36, 37, 38, 67] +- Additional Content Digests [cite: 34, 35, 36, 37, 38, 67] + - Configuration and creation of different digests + - Support for content sources beyond Hacker News +- Advanced scraping techniques (e.g., Playwright) + +## Change Log + +| Change | Date | Version | Description | Author | +| ------------- | ---------- | ------- | -------------------------------------------------- | ------ | +| Initial Draft | 2024-07-26 | 0.1 | Initial draft of the Product Requirements Document | 2-pm | + +----- END PRD START CHECKLIST OUTPUT ------ + +## Checklist Results Report + +**Category Statuses** + +| Category | Status | Critical Issues | +| -------------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1. Problem Definition & Context | PARTIAL | Quantification of problem impact, baseline measurements, timeframe for goals, user research, competitive analysis, market context | +| 2. MVP Scope Definition | PASS | None | +| 3. User Experience Requirements | PARTIAL | User flows, entry/exit points, decision points, critical path, edge cases, accessibility, performance expectations, error handling, information architecture, visual design guidelines, content, navigation | +| 4. Functional Requirements | PASS | None | +| 5. Non-Functional Requirements | PARTIAL | Compliance, security testing, privacy | +| 6. Epic & Story Structure | PASS | None | +| 7. Technical Guidance | PARTIAL | Technical decision framework details | +| 8. Cross-Functional Requirements | PARTIAL | Data migration | +| 9. Clarity & Communication | PARTIAL | Diagrams/visuals, stakeholder details, disagreement resolution, communication plan, approval process | + +**Critical Deficiencies** + +- Lack of quantification of problem impact, baseline measurements, timeframe for achieving goals, user research findings, competitive analysis, and market context. +- Absence of details on user flows, entry/exit points, decision points, critical path, edge cases, accessibility considerations, performance expectations from the user perspective, error handling and recovery approaches, information architecture, visual design guidelines, content requirements, and high-level navigation structure. +- Missing information on compliance requirements, security testing requirements, and privacy considerations. +- Absence of a detailed technical decision framework. +- Lack of details on data migration needs. +- Absence of diagrams/visuals and information on key stakeholders, disagreement resolution, a communication plan for updates, and the approval process. + +**Recommendations** + +- For "Problem Definition & Context," consider adding quantitative data to demonstrate the impact of the problem. If possible, include baseline measurements and a timeline for achieving the stated goals. +- For "User Experience Requirements," collaborate with the Design Architect to define user flows, address usability concerns, and outline UI requirements in more detail. +- For "Non-Functional Requirements," consult with relevant experts to address compliance, security testing, and privacy considerations. +- For "Technical Guidance," work with the Architect to establish a technical decision framework. +- For "Cross-Functional Requirements," determine if data migration will be necessary and document the requirements accordingly. +- For "Clarity & Communication," enhance the document with diagrams/visuals where appropriate and define key stakeholders, a disagreement resolution process, a communication plan, and an approval process. + +**Final Decision** + +READY FOR ARCHITECT: The PRD and epics are comprehensive, properly structured, and ready for architectural design. + +----- END Checklist START Architect Prompt ------ + +## Initial Architect Prompt + +Based on our discussions and requirements analysis for the BMad DiCaster, I've compiled the following technical guidance to inform your architecture analysis and decisions to kick off Architecture Creation Mode: + +### Technical Infrastructure + +- **Repository & Service Architecture Decision:** Monorepo with Next.js frontend and Supabase functions within the same repo. +- **Starter Project/Template:** Vercel/Supabase template +- **Hosting/Cloud Provider:** Vercel +- **Frontend Platform:** Next.js with React +- **Backend Platform:** Supabase functions (serverless) +- **Database Requirements:** Supabase's PostgreSQL + +### Technical Constraints + +- The application will be developed using the Next.js/Supabase template and hosted entirely on Vercel. +- Separate Supabase instances will be used for development and production environments. + +### Deployment Considerations + +- Automated deployments from the main branch to the production environment. +- Separate Vercel environments for development and production. + +### Local Development & Testing Requirements + +- Developers can use the Supabase CLI and Vercel CLI for local development and testing. +- Testing will include unit tests, integration tests, and end-to-end tests. + +### Other Technical Considerations + +- The system should be built as an event-driven pipeline, leveraging Supabase to capture data at each stage and trigger subsequent functions asynchronously to mitigate potential timeout issues with Vercel. + +----- END Prompt ----- diff --git a/BETA-V3/v3-demos/project1/simulated-prd-no-architect-later.txt b/BETA-V3/v3-demos/project1/simulated-prd-no-architect-later.txt new file mode 100644 index 00000000..ff7076bb --- /dev/null +++ b/BETA-V3/v3-demos/project1/simulated-prd-no-architect-later.txt @@ -0,0 +1,508 @@ +# BMad DiCaster Product Requirements Document (PRD) + +## Goal, Objective and Context + +**Goal:** To develop a web application that provides a daily, concise summary of top Hacker News (HN) posts, delivered as a newsletter and accessible via a web interface. + +**Objective:** To streamline the consumption of HN content by curating the top stories, providing AI-powered summaries, and offering an optional AI-generated podcast version. + +**Context:** Busy professionals and enthusiasts want to stay updated on HN but lack the time to sift through numerous posts and discussions. This application will address this problem by automating the delivery of summarized content. + +## Functional Requirements (MVP) + +- **HN Content Retrieval & Storage:** + - Daily retrieval of the top 30 Hacker News posts and associated comments using the HN Algolia API. + - Scraping and storage of up to 10 linked articles per day. + - Storage of all retrieved data (posts, comments, articles) with date association. +- **AI-Powered Summarization:** + - AI-powered summarization of the 10 selected articles (2-paragraph summaries). + - AI-powered summarization of comments for the 10 selected posts (2-paragraph summaries highlighting interesting interactions). + - Configuration for local or remote LLM usage via environment variables, including specifying the LLM API endpoint and any necessary authentication credentials. +- **Newsletter Generation & Delivery:** + - Generation of a daily newsletter in HTML format, including summaries, links to HN posts and articles, and original post dates/times. + - Automated delivery of the newsletter to a manually configured list of subscribers in Supabase. The list of emails will be manually populated in the "subscribers" table. Account information for the Nodemailer service (including SMTP server, port, username, and password) will be provided via environment variables. +- **Podcast Generation & Integration:** + - Integration with Play.ht's PlayNote API for AI-generated podcast creation from the newsletter content. + - Webhook handler to update the newsletter with the generated podcast link. +- **Web Interface (MVP):** + - Display of current and past newsletters. + - Functionality to read the newsletter content within the web page. + - Download option for newsletters (in HTML format). + - Web player for listening to generated podcasts. +- **API & Triggering:** + - Secure API endpoint (`/api/trigger-workflow`) to manually trigger the daily workflow, secured with API keys passed in the `X-API-Key` header. + - CLI command (`node trigger.js`) to manually trigger the daily workflow locally. + +## Non-Functional Requirements (MVP) + +- **Performance:** + - The system should retrieve HN posts and generate the newsletter within a reasonable timeframe (e.g., under 30 minutes) to ensure timely delivery. + - The web interface should load quickly (e.g., within 2 seconds) to provide a smooth user experience. +- **Scalability:** + - The system is designed for an initial MVP delivery to 3-5 email subscribers. Scalability beyond this will be considered post-MVP. +- **Security:** + - The API endpoint (`/api/trigger-workflow`) for triggering the daily workflow must be secure, using API keys passed in the `X-API-Key` header. + - User data (email addresses) should be stored securely in the Supabase database, with appropriate access controls. No other security measures are required for the MVP. +- **Reliability:** + - No specific uptime or availability requirements are defined for the MVP. + - The newsletter generation and delivery process should be robust and handle potential errors gracefully, with logging and retry mechanisms where appropriate. + - The system must be executable from a local development environment using Node.js and the Supabase CLI. +- **Maintainability:** + - The codebase should adhere to good quality coding standards, including separation of concerns and following SOLID principles. + - The system should employ design patterns like facades and factories to facilitate future expansion. + - The system should be built as an event-driven pipeline, leveraging Supabase to capture data at each stage and trigger subsequent functions asynchronously. This approach aims to mitigate potential timeout issues with Vercel hosting. + - Key parameters (e.g., API keys, email settings, LLM configuration) should be configurable via environment variables. + +## User Interaction and Design Goals + +This section captures the high-level vision and goals for the User Experience (UX) to guide the Design Architect. + +- **Overall Vision & Experience:** + - The desired look and feel is modern and minimalist, with synthwave technical glowing purple vibes. + - Users should have a clean and efficient experience when accessing and consuming newsletter content and podcasts. +- **Key Interaction Paradigms:** + - The web interface will primarily involve browsing lists and reading detailed content. + - Users will interact with the podcast player using standard playback controls. +- **Core Screens/Views (Conceptual):** + - The MVP will consist of two pages: + - A list page (`/newsletters`) to display current and past newsletters, with links to the detail page. + - A detail page (`/newsletters/{date}`) to display the selected newsletter content, including: + - Download option for the newsletter (in HTML format). + - Web player for listening to the generated podcast. + - The article laid out for viewing. +- **Accessibility Aspirations:** + - Basic semantic HTML should be used to ensure a degree of screen reader compatibility. +- **Branding Considerations (High-Level):** + - A logo for the application will be provided. + - The application will use the name "BMad DiCaster". +- **Target Devices/Platforms:** + - The application will be designed as a mobile-first responsive web app, ensuring it looks good on both mobile and desktop devices. The primary target is modern web browsers (Chrome, Firefox, Safari, Edge). + +## Technical Assumptions + +This section captures any existing technical information that will guide the development. + +- The application will be developed using the Next.js framework for the frontend and Vercel serverless functions for the backend, leveraging the Vercel/Supabase template as a starting point. +- This implies a monorepo structure, as the frontend (Next.js) and backend (Vercel functions) will reside within the same repository. +- Frontend development will be in Next.js with React, utilizing Tailwind CSS for styling and theming. +- Data storage will be handled by Supabase's PostgreSQL database. +- Separate Supabase instances will be used for development and production environments to ensure data isolation and stability. Supabase CLI will be used for local development. +- Testing will include unit tests (Jest), integration tests (testing API calls and Supabase interactions), and end-to-end tests (Cypress or Playwright). + +## Technology Choices + +| Technology | Version | Purpose | +| :----------------- | :------ | :------------------- | +| Next.js | Latest | Frontend Framework | +| React | Latest | UI Library | +| Tailwind CSS | 3.x | CSS Framework | +| Vercel | N/A | Hosting Platform | +| Supabase | N/A | Backend as a Service | +| Node.js | 18.x | Backend Runtime | +| HN Algolia API | N/A | HN Data Source | +| Play.ht API | N/A | Podcast Generation | +| Nodemailer | 6.x | Email Sending | +| Cheerio | 1.x | HTML Parsing | +| Jest | 29.x | Unit Testing | +| Cypress/Playwright | Latest | E2E Testing | + +## Proposed Directory Structure + +``` +├── app/ # Next.js app directory +│ ├── api/ # API routes +│ │ └── trigger-workflow.js +│ ├── newsletters/ # Newsletter list and detail pages +│ │ ├── [date]/ +│ │ │ └── page.jsx +│ │ └── page.jsx +│ ├── layout.jsx # Root layout +│ └── page.jsx # Home page (redirect to /newsletters) +├── components/ # React components +│ ├── NewsletterList.jsx +│ ├── NewsletterDetail.jsx +│ └── PodcastPlayer.jsx +├── lib/ # Utility functions +│ ├── hn.js # HN API interactions +│ ├── scraping.js # Article scraping +│ ├── summarization.js # LLM integration +│ ├── email.js # Nodemailer +│ └── db.js # Supabase client +├── public/ # Static assets (logo, etc.) +├── styles/ # Global CSS (if needed) +├── .env.local # Local environment variables +├── next.config.js # Next.js config +├── package.json +├── tailwind.config.js # Tailwind CSS config +└── trigger.js # CLI trigger script +``` + +## Data Models + +### Hacker News Post + +| Field | Type | Description | +| :--------- | :------- | :--------------------------------- | +| id | String | HN Post ID (objectID from Algolia) | +| title | String | Post Title | +| url | String | Post URL | +| created_at | DateTime | Post Creation Date/Time | +| author | String | Post Author | +| date | Date | Retrieval Date | + +### Hacker News Comment + +| Field | Type | Description | +| :--------- | :------- | :------------------------------------ | +| id | String | HN Comment ID (objectID from Algolia) | +| text | String | Comment Text | +| author | String | Comment Author | +| created_at | DateTime | Comment Creation Date/Time | +| parent_id | String | ID of the parent post/comment | +| date | Date | Retrieval Date | +| summary | String | AI-generated summary | + +### Article + +| Field | Type | Description | +| :--------------- | :----- | :----------------------------------- | +| id | UUID | Primary Key | +| post_id | String | HN Post ID (Foreign Key to hn_posts) | +| title | String | Article Title | +| author | String | Article Author | +| publication_date | Date | Article Publication Date | +| content | String | Article Content | +| summary | String | AI-generated summary | + +### Newsletter + +| Field | Type | Description | +| :---------- | :----- | :---------------------------- | +| date | Date | Newsletter Date (Primary Key) | +| content | String | HTML Newsletter Content | +| podcast_url | String | URL of the generated podcast | + +### Subscriber + +| Field | Type | String | +| :---- | :----- | :----------------------------- | +| email | String | Subscriber Email (Primary Key) | + +### Template + +| Field | Type | Description | +| :------ | :----- | :----------------------------------------------------------------------- | +| type | String | Template Type (e.g., 'newsletter', 'article_summary', 'comment_summary') | +| content | String | Template Content (HTML or Prompt) | + +## Database Schema + +```sql +CREATE TABLE hn_posts ( + id VARCHAR(255) PRIMARY KEY, + title TEXT, + url TEXT, + created_at TIMESTAMPTZ, + author VARCHAR(255), + date DATE NOT NULL +); + +CREATE TABLE hn_comments ( + id VARCHAR(255) PRIMARY KEY, + text TEXT, + author VARCHAR(255), + created_at TIMESTAMPTZ, + parent_id VARCHAR(255), + date DATE NOT NULL, + summary TEXT +); + +CREATE TABLE articles ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + post_id VARCHAR(255) NOT NULL REFERENCES hn_posts(id), + title TEXT, + author VARCHAR(255), + publication_date DATE, + content TEXT, + summary TEXT +); + +CREATE TABLE newsletters ( + date DATE PRIMARY KEY, + content TEXT NOT NULL, + podcast_url TEXT +); + +CREATE TABLE subscribers ( + email VARCHAR(255) PRIMARY KEY +); + +CREATE TABLE templates ( + type VARCHAR(255) PRIMARY KEY, + content TEXT NOT NULL +); +``` + +## Epic Overview + +- **Epic 1: Project Initialization, Setup, and HN Content Acquisition** + - Goal: Establish the foundational project structure, including the Next.js application, Supabase integration, and deployment pipeline, implement the initial API and CLI trigger mechanisms, and implement the functionality to retrieve, process, and store Hacker News posts and comments, providing the necessary data for newsletter generation. +- **Epic 2: Article Scraping** + - Goal: Implement the functionality to scrape and store linked articles from HN posts, enriching the data available for summarization and the newsletter, and ensure this functionality can be triggered via the API and CLI. +- **Epic 3: AI-Powered Content Summarization** + - Goal: Integrate AI summarization capabilities to generate concise summaries of articles and comments, retrieving prompts from the database for flexibility, and ensure this functionality can be triggered via the API and CLI. +- **Epic 4: Automated Newsletter Creation and Distribution** + - Goal: Automate the generation and delivery of the daily newsletter, including a placeholder for the podcast URL and conditional logic with configurable delay/retry parameters (configurable via environment variables) to handle podcast link availability. Ensure this functionality can be triggered via the API and CLI. +- **Epic 5: Podcast Generation Integration** + - Goal: Integrate with the Play.ht's PlayNote API (using webhooks) to generate and incorporate podcast versions of the newsletter, including a webhook to update the newsletter data with the podcast URL in Supabase. Ensure this functionality can be triggered via the API and CLI. +- **Epic 6: Web Interface Development** + - Goal: Develop a user-friendly and mobile-responsive web interface, starting with an initial structure generated by a tool like Vercel v0 and styled with Tailwind CSS using design tokens, to display newsletters and provide access to podcast content. + +## Epic and User Stories + +**Epic 1: Project Initialization, Setup, and HN Content Acquisition** + +- Goal: Establish the foundational project structure, including the Next.js application, Supabase integration, and deployment pipeline, implement the initial API and CLI trigger mechanisms, and implement the functionality to retrieve, process, and store Hacker News posts and comments, providing the necessary data for newsletter generation. +- Story 1.1: Project Setup + - As a developer, I want to set up the Next.js project with Supabase integration and initialize a Git repository, so that I have a functional foundation for building the application. + - Acceptance Criteria: + 1. The Next.js project is initialized using the Vercel/Supabase template (`npx create-next-app@latest --example with-supabase`). + 2. Supabase client libraries are installed (`npm install @supabase/supabase-js`). + 3. The Supabase client is initialized with the Supabase URL and Anon Key from environment variables (`NEXT_PUBLIC_SUPABASE_URL`, `NEXT_PUBLIC_SUPABASE_ANON_KEY`). + 4. The project codebase is initialized in a Git repository with a `.gitignore` file. +- Story 1.2: Deployment Pipeline + - As a developer, I want to configure the deployment pipeline to Vercel with separate development and production environments, so that I can easily deploy and update the application. + - Acceptance Criteria: + 1. The project is successfully linked to a Vercel project using the Vercel CLI (`vercel link`). + 2. Separate Vercel projects are set up for development and production environments. + 3. Automated deployments are configured for the `main` branch to the production environment. + 4. Environment variables are set up for local development (`.env.local`) and Vercel deployments (via the Vercel dashboard) for Supabase URL/Key, API key, and other sensitive configuration. +- Story 1.3: API and CLI Trigger Mechanisms + - As a developer, I want to implement a secure API endpoint and a CLI command to trigger the workflow, so that I can manually trigger the process during development and testing. + - Acceptance Criteria: + 1. A secure API endpoint (`/api/trigger-workflow`) is created as a Vercel API Route. + 2. The API endpoint requires authentication using an API key passed in the `X-API-Key` header. + 3. The API endpoint triggers the execution of the main workflow (HN data retrieval). + 4. The API endpoint returns a JSON response with a success/failure status code and message. + 5. A CLI command (`node trigger.js`) is created using Node.js and a library like `commander` or `yargs` to trigger the workflow locally. + 6. The CLI command reads necessary configuration (e.g., Supabase credentials) from environment variables. + 7. Both the API endpoint and CLI command log all requests with timestamps, source (API/CLI), and status. +- Story 1.4: HN Data Retrieval + - As a system, I want to retrieve the top 30 Hacker News posts and associated comments daily using the HN Algolia API, and store them in the Supabase database, so that the data is available for summarization and newsletter generation. + - Acceptance Criteria: + 1. The system retrieves the top 30 Hacker News posts daily using the HN Algolia API (`https://hn.algolia.com/api/v1/search_by_date?tags=front_page`). + 2. The system retrieves associated comments for the top 30 posts using the HN Algolia API (using the post's `objectID`). + 3. Retrieved data (posts and comments) is stored in the Supabase database in tables named `hn_posts` and `hn_comments`, with appropriate fields (e.g., `title`, `url`, `created_at`, `author` for posts; `text`, `author`, `created_at`, `parent_id` for comments) and a `date` field to associate the data with the retrieval date. + 4. This functionality can be triggered via the API endpoint (`/api/trigger-workflow`) and the CLI command (`node trigger.js`). + 5. The system logs the start and completion of the retrieval process, including the number of posts/comments retrieved and any errors encountered (e.g., API request failures, database connection errors). + +**Epic 2: Article Scraping** + +- Goal: Implement the functionality to scrape and store linked articles from HN posts, enriching the data available for summarization and the newsletter, and ensure this functionality can be triggered via the API and CLI. +- Story 2.1: URL Identification + - As a system, I want to identify URLs within the retrieved Hacker News posts, so that I can extract the content of linked articles. + - Acceptance Criteria: + 1. The system parses the `url` field of each post in the `hn_posts` table. + 2. The system filters out any URLs that are not relevant to article scraping (e.g., links to images, videos, or known non-article sites) using a basic filter or a library like `url-parse`. + 3. The system limits the number of URLs to be scraped per post to 1, prioritizing the first valid URL. +- Story 2.2: Article Content Scraping + - As a system, I want to scrape the content of the identified article URLs using Cheerio, so that I can provide summaries in the newsletter. + - Acceptance Criteria: + 1. The system scrapes the content from the identified article URLs using the Cheerio library. + 2. The system extracts the following relevant content: + - Article title (using `` tag or schema.org metadata) + - Article author (if available, using metadata) + - Article publication date (if available, using metadata) + - Main article text (by identifying and extracting content from `<p>` tags within the main article body, excluding navigation and ads). + 3. The system handles potential issues during scraping: + - Website errors (e.g., 404 status codes) are logged. + - Changes in website structure are handled gracefully (e.g., by providing alternative CSS selectors or logging a warning). + - A timeout is implemented for scraping requests. +- Story 2.3: Scraped Data Storage + - As a system, I want to store the scraped article content in the Supabase database, associated with the corresponding Hacker News post, so that it can be used for summarization and newsletter generation. + - Acceptance Criteria: + 1. The system stores the scraped article content in a table named `articles` in the Supabase database. + 2. The `articles` table includes fields for `post_id` (foreign key referencing `hn_posts`), `title`, `author`, `publication_date`, and `content`. + 3. The system ensures that the stored data includes all extracted information and is correctly associated with the corresponding Hacker News post. +- Story 2.4: API/CLI Triggering + - As a developer, I want to trigger the article scraping process via the API and CLI, so that I can manually initiate it for testing and debugging. + - Acceptance Criteria: + 1. The existing API endpoint (`/api/trigger-workflow`) can trigger the article scraping process. + 2. The existing CLI command (`node trigger.js`) can trigger the article scraping process locally. + 3. The system logs the start and completion of the scraping process, including the number of articles scraped, any errors encountered, and the execution time. + +**Epic 3: AI-Powered Content Summarization** + +- Goal: Integrate AI summarization capabilities to generate concise summaries of articles and comments, retrieving prompts from the database for flexibility, and ensure this functionality can be triggered via the API and CLI. +- Story 3.1: LLM Integration + - As a system, I want to integrate an AI summarization library or API, so that I can generate concise summaries of articles and comments. + - Acceptance Criteria: + 1. The system integrates with a suitable AI summarization library (e.g., `langchain` with a local LLM like `Ollama`) or a cloud-based API (e.g., OpenAI API). + 2. The LLM or API endpoint and any necessary authentication credentials (API keys) are configurable via environment variables (`LLM_API_URL`, `LLM_API_KEY`). + 3. The system implements error handling for LLM/API requests, including retries and fallback mechanisms. +- Story 3.2: Article Summarization + - As a system, I want to retrieve summarization prompts from the database, and then use them to generate 2-paragraph summaries of the scraped articles, so that users can quickly grasp the main content and the prompts can be easily updated. + - Acceptance Criteria: + 1. The system retrieves the appropriate summarization prompt from a table named `prompts` in the Supabase database, using a query to select the prompt based on `type = 'article_summary'`. The prompt should include placeholders for article title and content. + 2. The system generates a 2-paragraph summary (approximately 100-150 words) for each scraped article using the retrieved prompt and the LLM/API. + 3. The summaries are accurate, concise, and capture the key information from the article. + 4. The generated summaries are stored in the `articles` table in a field named `summary`. +- Story 3.3: Comment Summarization + - As a system, I want to retrieve summarization prompts from the database, and then use them to generate 2-paragraph summaries of the comments for the selected HN posts, so that users can understand the main discussions and the prompts can be easily updated. + - Acceptance Criteria: + 1. The system retrieves the appropriate summarization prompt from the `prompts` table, using a query to select the prompt based on `type = 'comment_summary'`. The prompt should include placeholders for comment text. + 2. The system generates a 2-paragraph summary (approximately 100-150 words) of the comments for each selected HN post using the retrieved prompt and the LLM/API. + 3. The summaries highlight interesting interactions and key points from the discussion, focusing on distinct viewpoints and arguments. + 4. The generated comment summaries are stored in the `hn_comments` table in a field named `summary`. +- Story 3.4: API/CLI Triggering and Logging + - As a developer, I want to trigger the AI summarization process via the API and CLI, and log all summarization requests, prompts used, and results, so that I can manually initiate it for testing/debugging and monitor the process. + - Acceptance Criteria: + 1. The existing API endpoint (`/api/trigger-workflow`) can trigger the AI summarization process. + 2. The existing CLI command (`node trigger.js`) can trigger the AI summarization process locally. + 3. The system logs the following information for each summarization: + - Timestamp + - Source (API/CLI) + - Article title or HN post ID + - Summarization prompt used + - Generated summary + - LLM/API response time + - Any errors encountered + 4. The system handles partial execution gracefully (i.e., if triggered before Epic 2 is complete, it logs a message and exits). + +**Epic 4: Automated Newsletter Creation and Distribution** + +- Goal: Automate the generation and delivery of the daily newsletter, including a placeholder for the podcast URL and conditional logic with configurable delay/retry parameters (configurable via environment variables) to handle podcast link availability. Ensure this functionality can be triggered via the API and CLI. +- Story 4.1: Newsletter Template Retrieval + - As a system, I want to retrieve the newsletter template from the database, so that the newsletter's design and structure can be updated without code changes. + - Acceptance Criteria: + 1. The system retrieves the newsletter template from the `templates` table in the Supabase database, using a query to select the template based on `type = 'newsletter'`. The template is an HTML string with placeholders for article summaries, comment summaries, HN post links, and the podcast URL placeholder (`{{podcast_url}}`). +- Story 4.2: Newsletter Generation + - As a system, I want to generate a daily newsletter in HTML format using the retrieved template and the summarized data, so that users can receive a concise summary of Hacker News content. + - Acceptance Criteria: + 1. The system generates a newsletter in HTML format using the template retrieved from the `templates` table. + 2. The newsletter includes: + - Summaries of the top 10 articles from the `articles` table. + - Summaries of comments for the top 10 HN posts from the `hn_comments` table. + - Links to the original HN posts and articles. + - Original post dates/times. + - A placeholder for the podcast URL (`{{podcast_url}}`). + 3. The generated newsletter is stored in the `newsletters` table with a `date` field. +- Story 4.3: Newsletter Delivery + - As a system, I want to send the generated newsletter to a list of subscribers using Nodemailer, with credentials securely provided via environment variables, so that users receive the daily summary in their inbox. + - Acceptance Criteria: + 1. The system retrieves the list of subscriber email addresses from the `subscribers` table in the Supabase database. + 2. The system sends the HTML newsletter to all active subscribers using Nodemailer. + 3. Nodemailer is configured using the following environment variables: `SMTP_HOST`, `SMTP_PORT`, `SMTP_USER`, `SMTP_PASS`. + 4. The system logs the delivery status (success/failure, timestamp) for each subscriber in a table named `newsletter_logs`. + 5. The system implements conditional logic and a configurable delay/retry mechanism (using `EMAIL_DELAY` in milliseconds and `EMAIL_RETRIES` as the number of retries, both from environment variables) to handle podcast link availability before sending the email. If the podcast URL is not available after the delay and retries, the email is sent without the podcast link. +- Story 4.4: API/CLI Triggering and Logging + - As a developer, I want to trigger the newsletter generation and distribution process via the API and CLI, and log all relevant actions, so that I can manually initiate it for testing and debugging, and monitor the process. + - Acceptance Criteria: + 1. The existing API endpoint (`/api/trigger-workflow`) can trigger the newsletter generation and distribution process. + 2. The existing CLI command (`node trigger.js`) can trigger the newsletter generation and distribution process locally. + 3. The system logs the following information: + - Timestamp + - Source (API/CLI) + - Newsletter generation start/end times + - Number of emails sent successfully/failed + - Podcast URL availability status + - Email delay/retry attempts + - Any errors encountered + 4. The system handles partial execution gracefully (i.e., if triggered before Epic 3 is complete, it logs a message and exits). + +**Epic 5: Podcast Generation Integration** + +- Goal: Integrate with the Play.ht's PlayNote API (using webhooks) to generate and incorporate podcast versions of the newsletter, including a webhook to update the newsletter data with the podcast URL in Supabase. Ensure this functionality can be triggered via the API and CLI. +- _(Note for Architect: The Play.ht API - [https://docs.play.ai/api-reference/playnote/post](https://docs.play.ai/api-reference/playnote/post) - supports both webhook and polling mechanisms for receiving the podcast URL. We will use the webhook approach for real-time updates and efficiency.)_ +- Story 5.1: Play.ht API Integration + - As a system, I want to integrate with the Play.ht's PlayNote API, so that I can generate AI-powered podcast versions of the newsletter content. + - Acceptance Criteria: + 1. The system integrates with the Play.ht's PlayNote API using webhooks for asynchronous updates. + 2. The Play.ht API key and user ID are configured via environment variables (`PLAYHT_API_KEY`, `PLAYHT_USER_ID`). + 3. The Play.ht webhook URL is configured via the `PLAYHT_WEBHOOK_URL` environment variable. +- Story 5.2: Podcast Generation Initiation + - As a system, I want to send the newsletter content to the Play.ht API to initiate podcast generation, and receive a job ID or initial response, so that I can track the podcast creation process. + - Acceptance Criteria: + 1. The system sends the HTML newsletter content to the Play.ht API's `POST /v1/convert` endpoint to generate a podcast. + 2. The system receives a JSON response from the Play.ht API containing a `jobId`. + 3. The system stores the `jobId` in the `newsletters` table along with the newsletter data. +- Story 5.3: Webhook Handling + - As a system, I want to implement a webhook handler to receive the podcast URL from Play.ht, and update the newsletter data, so that the podcast link can be included in the newsletter and web interface. + - Acceptance Criteria: + 1. The system implements a webhook handler at the URL specified in the `PLAYHT_WEBHOOK_URL` environment variable. + 2. The webhook handler receives a `POST` request from Play.ht with a JSON payload containing the `jobId` and the `audioUrl`. + 3. The system extracts the `audioUrl` from the webhook payload. + 4. The system updates the `podcast_url` field in the `newsletters` table for the corresponding `jobId`. +- Story 5.4: API/CLI Triggering and Logging + - As a developer, I want to trigger the podcast generation process via the API and CLI, and log all interactions with the Play.ht API, so that I can manually initiate it for testing and debugging, and monitor the process. + - Acceptance Criteria: + 1. The existing API endpoint (`/api/trigger-workflow`) can trigger the podcast generation process. + 2. The existing CLI command (`node trigger.js`) can trigger the podcast generation process locally. + 3. The system logs the following information: + - Timestamp + - Source (API/CLI) + - Play.ht API requests (endpoint, payload) and responses (status code, body) + - Webhook requests received from Play.ht (payload) + - Updates to the `newsletters` table + - Any errors encountered + 4. The system handles partial execution gracefully (i.e., if triggered before Epic 4 is complete, it logs a message and exits). + +**Epic 6: Web Interface Development** + +- Goal: Develop a user-friendly and mobile-responsive web interface, starting with an initial structure generated by a tool like Vercel v0 and styled with Tailwind CSS using design tokens, to display newsletters and provide access to podcast content. +- Story 6.1: Initial UI Structure Generation + - As a developer, I want to use a tool like Vercel v0 to generate the initial structure of the web interface, so that I have a basic layout to work with. + - Acceptance Criteria: + 1. A tool (e.g., Vercel v0) is used to generate the initial HTML/CSS/React component structure for the web interface. + 2. The generated structure includes basic layouts for the newsletter list page (`/newsletters`) and detail page (`/newsletters/{date}`), as well as a basic podcast player component. + 3. The generated structure incorporates the desired "modern and minimalist, with synthwave technical glowing purple vibes" aesthetic, using initial Tailwind CSS classes. +- Story 6.2: Newsletter List Page + - As a user, I want to see a list of current and past newsletters, so that I can easily browse available content. + - Acceptance Criteria: + 1. The `/newsletters` page displays a list of newsletters fetched from the `newsletters` table. + 2. Each item in the list displays the newsletter title (e.g., "Daily Digest - YYYY-MM-DD") and date. + 3. Each item is a link to the corresponding detail page (`/newsletters/{date}`). + 4. The list is paginated (e.g., 10 newsletters per page) or provides infinite scrolling. + 5. The page is styled using Tailwind CSS, adhering to the project's design tokens. +- Story 6.3: Newsletter + Detail Page + _ As a user, I want to be able to read the newsletter content and download it, so that I can conveniently consume the information and access it offline. + _ Acceptance Criteria: 1. The `/newsletters/{date}` page displays the full newsletter content from the `newsletters` table. 2. The newsletter content is displayed in a readable format, preserving HTML formatting. 3. A download option (e.g., a button) is provided to download the newsletter in HTML format. 4. The page is styled using Tailwind CSS, adhering to the project's design tokens. +- Story 6.4: Podcast Player Integration + - As a user, I want to listen to generated podcasts within the web interface, so that I can consume the content in audio format. + - Acceptance Criteria: + 1. The `/newsletters/{date}` page includes a web player component. + 2. The web player uses the `podcast_url` from the `newsletters` table as the audio source. + 3. The player includes standard playback controls (play, pause, volume, progress bar). + 4. The player is styled using Tailwind CSS, adhering to the project's design tokens. +- Story 6.5: Mobile Responsiveness + - As a developer, I want the web interface to be mobile-responsive, so that it is accessible and usable on various devices. + - Acceptance Criteria: + 1. The layout of both the `/newsletters` and `/newsletters/{date}` pages adapts to different screen sizes and orientations, using Tailwind CSS responsive modifiers. + 2. The interface is usable and readable on both desktop and mobile devices, with appropriate font sizes, spacing, and navigation. + +## Key Reference Documents + +_(This section will be created later, from the sections prior to this being carved up into smaller documents)_ + +## Out of Scope Ideas Post MVP + +- User Authentication and Management +- Subscription Management +- Admin Dashboard + - Viewing and updating daily podcast settings + - Prompt management for summarization + - UI for template modification +- Enhanced Newsletter Customization +- Additional Content Digests + - Configuration and creation of different digests + - Support for content sources beyond Hacker News +- Advanced scraping techniques (e.g., Playwright) + +## Change Log + +| Change | Date | Version | Description | Author | +| :------------ | :--------- | :------ | :------------------------------------------------------------------------------ | :----- | +| Initial Draft | 2024-07-26 | 0.1 | Initial draft of the Product Requirements Document (YOLO, Simplified PM-to-Dev) | 2-pm | + +-- checklists still follow diff --git a/BETA-V3/web-agent-modes/2-pm.md b/BETA-V3/web-agent-modes/2-pm.md index 2fd7fbc7..3d981ae7 100644 --- a/BETA-V3/web-agent-modes/2-pm.md +++ b/BETA-V3/web-agent-modes/2-pm.md @@ -122,7 +122,7 @@ Remember as you follow the upcoming instructions: - Before PRD generation, ask the user to choose their intended workflow: A. **Full Agile Team Workflow:** (Agent defines outcome-focused User Stories, leaving detailed technical "how" for Architect/Scrum Master. Capture nuances as "Notes for Architect/Scrum Master.") - B. **Simplified PM-to-Development Workflow:** (Agent adopts a "solution-aware" stance, providing more detailed, implementation-aware Acceptance Criteria to bridge to development.) + B. **Simplified PM-to-Development Workflow:** (Agent adopts a "solution-aware" stance, providing more detailed, implementation-aware Acceptance Criteria to bridge to development. <important_note>When this workflow is selected, you are also responsible for collaboratively defining and documenting key technical foundations—such as technology stack choices and proposed application structure—directly within a new, dedicated section of the PRD template titled '[OPTIONAL: For Simplified PM-to-Development Workflow Only] Core Technical Decisions & Application Structure'.</important_note>) - Explain this choice sets a default detail level, which can be fine-tuned later per story/epic. 2. **Determine Interaction Mode (for PRD Structure & Detail):** @@ -139,6 +139,17 @@ Remember as you follow the upcoming instructions: <important_note>When working on the "Technical Assumptions" section of the PRD, explicitly guide the user through discussing and deciding on the repository structure (Monorepo vs. Polyrepo) and the high-level service architecture (e.g., Monolith, Microservices, Serverless functions within a Monorepo). Emphasize that this is a critical decision point that will be formally documented here with its rationale, impacting MVP scope and informing the Architect. Ensure this decision is captured in the PRD's `Technical Assumptions` and then reiterated in the `Initial Architect Prompt` section of the PRD.</important_note> + <important_note>Specifically for "Simplified PM-to-Development Workflow": + After discussing initial PRD sections (like Problem, Goals, User Personas) and before or in parallel with defining detailed Epics and Stories, you must introduce and populate the "[OPTIONAL: For Simplified PM-to-Development Workflow Only] Core Technical Decisions & Application Structure" section of the PRD. + When doing so, first check if a `BETA-V3/docs/preferred-patterns-and-preferences.md` file exists. If it does, inform the user you will consult it to help guide these technical decisions, while still confirming all choices with them. Ask targeted questions such as: + + 1. "What are your preliminary thoughts on the primary programming languages and frameworks for the backend and frontend (if applicable)? (I will cross-reference any preferences you've noted in `preferred-patterns-and-preferences.md`.)" + 2. "Which database system are you considering? (Checking preferences...)" + 3. "Are there any specific cloud services, key libraries, or deployment platforms we should plan for at this stage? (Checking preferences...)" + 4. "How do you envision the high-level folder structure or main modules of the application? Could you describe the key components and their responsibilities? (I'll consider any structural preferences noted.)" + 5. "Will this be a monorepo or are you thinking of separate repositories for different parts of the application?" + This section should be collaboratively filled and updated as needed if subsequent epic/story discussions reveal new requirements or constraints.</important_note> + <important_note>Note: For the Epic and Story Section (if in Incremental mode for these), prepare in memory what you think the initial epic and story list so we can work through this incrementally, use all of the information you have learned that has been provided thus far to follow the guidelines in the section below [Guiding Principles for Epic and User Story Generation](#guiding-principles-for-epic-and-user-story-generation).</important_note> 4A. (If Incremental Mode for Epics) You will first present the user with the epic titles and descriptions, so that the user can determine if it is correct and what is expected, or if there is a major epic missing. diff --git a/BETA-V3/web-agent-modes/3-architect.md b/BETA-V3/web-agent-modes/3-architect.md index 0b9268c2..20513f57 100644 --- a/BETA-V3/web-agent-modes/3-architect.md +++ b/BETA-V3/web-agent-modes/3-architect.md @@ -24,7 +24,7 @@ - Always explain the rationale behind architectural decisions or recommendations. - Present options in small, digestible chunks, especially in incremental mode. - Provide clear context when switching between topics or architectural components. - - Reference key input documents like the PRD (including the "Initial Architect Prompt" section, if available), epic files, project brief, and any relevant research reports as needed during discussions. The `architecture-tmpl.txt` and `architect-checklist.txt` are core guiding documents for Phase 2. + - Reference key input documents like the PRD (including the "Initial Architect Prompt" section, if available), epic files, project brief, any relevant research reports, and the user's `preferred-patterns-and-preferences.md` (if available in `BETA-V3/docs/`) as needed during discussions. The `architecture-tmpl.txt` and `architect-checklist.txt` are core guiding documents for Phase 2. --- @@ -107,7 +107,7 @@ To perform deep research effectively, please be aware: 1. **Input Analysis & Dialogue Establishment:** - - Ensure you have all necessary inputs: PRD document (specifically checking for the 'Technical Assumptions' and 'Initial Architect Prompt' sections for the decided repository and service architecture), project brief, and any deep research reports. Request any missing critical documents.</ + - Ensure you have all necessary inputs: PRD document (specifically checking for the 'Technical Assumptions' and 'Initial Architect Prompt' sections for the decided repository and service architecture), project brief, any deep research reports, and potentially a `preferred-patterns-and-preferences.md` file located in `BETA-V3/docs/`. Request any missing critical documents.</ - Thoroughly review all inputs. - Summarize key technical requirements, constraints, NFRs (Non-Functional Requirements), and the decided repository/service architecture derived from the inputs. Present this summary to the user for confirmation and to ensure mutual understanding. - Share initial architectural observations, potential challenges, or areas needing clarification based on the inputs. @@ -116,14 +116,20 @@ To perform deep research effectively, please be aware: 2. **Resolve Ambiguities & Gather Missing Information:** - If key information is missing or requirements are unclear after initial review, formulate specific, targeted questions. + - **External API Details:** If the project involves integration with external APIs, especially those that are less common or where you lack high confidence in your training data regarding their specific request/response schemas, and if a "Deep Research" phase was not conducted for these APIs: + - Proactively ask the user to provide precise details. This includes: + - Links to the official API documentation. + - Example request structures (e.g., cURL commands, JSON payloads). + - Example response structures (e.g., JSON responses for typical scenarios, including error responses). + - Explain that this information is crucial for accurately defining API interaction contracts within the architecture, for creating robust facades/adapters, and for enabling accurate technical planning (e.g., for technical stories or epic refinements). - Present questions to the user (batched logically if multiple) and await their input. - Document all decisions and clarifications received before proceeding. 3. **Iterative Technology Selection & Design (Interactive, if not YOLO mode):** - For each major architectural component or decision point (e.g., frontend framework, backend language/framework, database system, cloud provider, key services, communication patterns): - - If multiple viable options exist based on requirements or research, present 2-3 choices, briefly outlining their pros, cons, and relevance to the project. - - State your recommended choice, providing a clear rationale based on requirements, research findings, and best practices (e.g., scalability, cost, team familiarity, ecosystem). + - If multiple viable options exist based on requirements or research, present 2-3 choices, briefly outlining their pros, cons, and relevance to the project. Consider any preferences stated in `preferred-patterns-and-preferences.md` when formulating these options and your recommendation. + - State your recommended choice, providing a clear rationale based on requirements, research findings, user preferences (if known), and best practices (e.g., scalability, cost, team familiarity, ecosystem). - Ask for user feedback, address concerns, and seek explicit approval before finalizing the decision. - Document the confirmed choice and its rationale within the architecture document. - **Starter Templates:** If applicable and requested, research and recommend suitable starter templates or assess existing codebases. Explain alignment with project goals and seek user confirmation.