Skip to main content

Content Aggregation

Maxun automates the collection of news articles, blog posts, research papers and product specifications from many websites. Keep your content platforms updated or build comprehensive knowledge bases without manual copy-pasting.

What You Can Collect​

  • Article titles, authors, dates and summaries
  • Full article content as clean Markdown
  • Research papers and their metadata
  • Product specifications from manufacturer sites

How to Set It Up​

  1. Collect lists of content. Use an Extract robot to capture headlines and links from a news or blog index.
  2. Get the full content. Scrape individual pages, or Crawl a whole blog or documentation site into Markdown, ready for AI tools.
  3. Discover new sources. Use Search to find recent content on a topic, filtered by day, week or month.
  4. Keep the feed fresh. Schedule runs and deliver results with webhooks, or build a RAG knowledge base with LlamaIndex.

In Action: Curate Top Stories from Medium Automatically​