News Aggregator API (MCP Server)

A modular, scalable news aggregation backend built with MCP server conventions and TypeScript. This API provides a unified interface to TheNewsAPI for retrieving current and historical news articles with advanced filtering capabilities. Designed specifically for consumption by AI agents, this API prioritizes structured data and consistent patterns.

Features

• Unified API: Consistent interface for accessing news data
• TypeScript: Fully typed for reliable development
• MCP Architecture: Scalable, modular design with separation of concerns
• Multiple Endpoints: Access various types of news content
• Environment Variables: Secure configuration management
• Error Handling: Consistent error formats with appropriate status codes
• Filtering: Support for various filtering parameters
• Caching: Intelligent caching system with management endpoints
• Interactive Documentation: Swagger-based API documentation
• AI-Friendly Responses: Structured data optimized for machine consumption

Endpoints

News Endpoints

Endpoint	Description	Example
/api/news/top	Get top news headlines	/api/news/top?categories=business
/api/news/all	Get all news with advanced search	/api/news/all?search=technology
/api/news/similar/:uuid	Get similar articles to a specific one	/api/news/similar/cc11e3ab-ced0-4a42-9146-e426505e2e67
/api/news/uuid/:uuid	Get a specific article by UUID	/api/news/uuid/cc11e3ab-ced0-4a42-9146-e426505e2e67
/api/news/sources	Get available news sources	/api/news/sources?language=en

Cache Management Endpoints

Endpoint	Description	Example
/api/cache/stats	Get cache statistics	/api/cache/stats
/api/cache/clear	Clear all cache	DELETE /api/cache/clear
/api/cache/clear/:type	Clear cache by type	DELETE /api/cache/clear/top

Utility Endpoints

Endpoint	Description
/health	Health check endpoint
/docs	Interactive API documentation
/docs.json	OpenAPI specification
/examples	Usage examples

Testing

This project implements a comprehensive testing strategy with multiple layers of tests to ensure reliability and correctness.

Test Types

1. Unit Tests

   • Isolated tests for individual components
   • Mocked dependencies for true unit isolation
   • Focused on business logic validation

2. Controller Tests

   • Verify controller behavior with mocked services
   • Test proper request handling and response formatting
   • Confirm error handling patterns are consistent

3. Integration Tests

   • Test the complete request-response cycle
   • Verify API endpoints with supertest
   • Mock external API calls for reliability

Testing Best Practices

This project follows these testing best practices:

• Isolation - Tests don't depend on each other or external services
• Repeatable - Tests produce the same results on any environment
• Fast - Mocking external dependencies keeps tests efficient
• Comprehensive - Core functionality has high test coverage
• Maintainable - Tests follow consistent patterns and structure

Running Tests

# Run all tests
npm test

# Run tests with coverage
npm run test:coverage

# Run a specific test file
npx jest path/to/test.ts

Installation

# Clone the repository
git clone <repository-url>

# Install dependencies
npm install

# Set up environment variables
cp .env.example .env
# Edit .env file with your own API key and database URL

# Generate Prisma client
npx prisma generate

# Build the project
npm run build

# Start the server
npm start

Configuration

Create a .env file with the following variables:

# News Aggregator API Environment Variables

# API Configuration
NEWS_API_TOKEN=your_api_token_here

# Server Configuration
PORT=3000
NODE_ENV=development

# Database Configuration
DATABASE_URL="file:./dev.db"

Database Configuration

The application uses Prisma ORM with SQLite by default. The DATABASE_URL environment variable should point to your database file. For production, you can configure it to use PostgreSQL or MySQL.

Development

# Run in development mode with hot reload
npm run dev

# Run linting
npm run lint

# Run tests
npm test

Testing Infrastructure

The project includes a comprehensive test suite built with Jest and Supertest, covering API endpoints, database integration, and server connectivity checks.

Running Tests

# Run all tests
npm test

# Run only API endpoint tests
npm run test:api

# Run only database integration tests
npm run test:db

# Run tests with coverage report
npm run test:coverage

# Run tests in watch mode (for development)
npm run test:watch

# Check server connectivity
npm run check-server

Test Structure

• API Endpoint Tests: Located in src/__tests__/api/, these tests verify that all API endpoints return the expected responses, handle errors correctly, and apply appropriate filtering.

• Database Integration Tests: Located in src/__tests__/database.test.ts, these tests verify database connectivity, query execution, and transaction support.

• Server Connectivity Script: Located in scripts/check-server.ts, this script tests connectivity to a running server instance, verifying that critical endpoints are functioning correctly.

Test Environment

Tests use a separate environment configuration specified by setting NODE_ENV=test. This ensures tests don't interfere with development or production environments. The setup includes:

• Automatic database connection setup and teardown
• Cache clearing between tests to ensure isolation
• Server startup and shutdown for endpoint testing

Future Enhancements Todo List

Priority 1: Foundation Improvements

• [x] Caching System

  • Implement in-memory caching
  • Cache frequently accessed data like top news and sources
  • Add cache invalidation strategies
  • Configure TTL (Time-To-Live) for different content types
  • Add cache management endpoints

• [x] Data Persistence Layer

  • Add database integration (SQLite with Prisma)
  • Create schema for storing enhanced article metadata
  • Implement repository pattern for data access
  • Add migration system for schema changes

• [x] API Documentation

  • Generate OpenAPI/Swagger specification
  • Add interactive API documentation
  • Create usage examples for each endpoint

API Documentation

The API now includes comprehensive documentation using OpenAPI/Swagger:

• Interactive API Documentation: Available at /docs when the server is running
• OpenAPI Specification: JSON format available at /docs.json
• Usage Examples: Available at /examples with code samples in multiple languages

How to Access the Documentation

1. Start the server with npm run dev or npm start
2. Open your browser to http://localhost:3000/docs for the interactive Swagger UI
3. Browse through the available endpoints, request parameters, and response formats
4. Reference implementation examples at http://localhost:3000/examples

API Documentation Features

• Complete endpoint descriptions with parameters and response types
• Interactive "Try it out" capability to test endpoints directly
• Request/response schema definitions
• Code snippets for API consumption
• Examples of integrating with the MCP server architecture

Priority 2: Security & Performance

• [ ] Authentication & User Management

  • Implement JWT-based authentication
  • Create user registration and login endpoints
  • Add role-based access control
  • Implement secure password handling

• [ ] Rate Limiting

  • Add request throttling middleware
  • Implement tiered access levels
  • Create fair usage policies
  • Add rate limit headers in responses

• [ ] Monitoring & Analytics

  • Set up request logging and monitoring
  • Add performance metrics collection
  • Create usage dashboards
  • Implement error tracking and alerting

Priority 3: Enhanced Functionality

• [ ] Content Processing

  • Add text summarization capabilities
  • Implement entity extraction
  • Add sentiment analysis
  • Create topic categorization beyond source categories
  • Implement duplicate detection and grouping

• [ ] Personalization

  • Add user preference tracking
  • Create personalized endpoints based on reading history
  • Implement topic following functionality
  • Add recommendation engine

• [ ] Webhooks & Real-time Updates

  • Implement webhook registration
  • Create event-based notification system
  • Add SSE (Server-Sent Events) for real-time updates
  • Implement topic subscription functionality

Contributing

Contributions to the News Aggregator API project are welcome! This project is designed to be a friendly, well-documented codebase for those interested in MCP server architecture, especially with AI consumption in mind.

How to Contribute

1. Fork the repository
2. Create a new feature branch (git checkout -b feature/amazing-feature)
3. Make your changes
4. Run tests to ensure everything works (npm test)
5. Commit your changes (git commit -m 'Add some amazing feature')
6. Push to the branch (git push origin feature/amazing-feature)
7. Open a Pull Request

Code Style

Please follow the existing code style and patterns in the project. This project uses ESLint for code quality and formatting.

Security Considerations

When contributing, please ensure:

• No API keys or credentials are hardcoded
• Environment variables are used for configuration
• No sensitive data is logged
• Input validation is properly implemented

Testing

All new features should include appropriate tests. Please follow the existing testing patterns in the project.

• [ ] Search History & Bookmarks
  • Add saved searches functionality
  • Implement article bookmarking
  • Create reading history tracking
  • Add user collections feature

Contributing

1. Fork the repository
2. Create your feature branch: git checkout -b feature/my-new-feature
3. Commit your changes: git commit -am 'Add some feature'
4. Push to the branch: git push origin feature/my-new-feature
5. Submit a pull request

License

ISC