Open Skills Logo
Open Skills

baoyu-url-to-markdown

Fetch any URL and convert to markdown using Chrome CDP. Supports two modes - auto-capture on page load, or wait for user signal (for pages requiring login). Use when user wants to save a webpage as markdown.

Author

@JimLiu

Category

Document Processing

Install

Hot:31

Download and extract to your skills directory

Copy command and send to OpenClaw for auto-install:

Download and install this skill https://openskills.cc/api/download?slug=jimliu-skills-baoyu-url-to-markdown&locale=en&source=copy

baoyu-url-to-markdown

Skill Overview


baoyu-url-to-markdown is a web scraping tool based on Chrome CDP. It can convert any URL into a well-formatted Markdown file, supporting both automatic scraping and manual triggering modes.

Use Cases


  • Save content from pages that require login — Use the wait mode: log in manually in the browser, then trigger scraping yourself. Suitable for member-only articles, paid content, and similar scenarios.

  • Scrape dynamically rendered web pages — Fully execute JavaScript via Chrome CDP to retrieve the final rendered page content. Ideal for SPA apps and lazy-loaded pages.

  • Archive and organize web content — Automatically extract page metadata (title, description, publish time, etc.), and generate a directory structure organized by domain and title. Useful for building knowledge bases and collecting content.

    • Core Features


  • Deep scraping with Chrome CDP — Use the Chrome DevTools Protocol to control the browser and fully execute page JavaScript, ensuring you capture the real rendered content rather than static HTML.

  • Dual-mode scraping support — In automatic mode, scraping runs immediately when the network is idle (best for public pages). In wait mode, the tool waits for a user signal (best for pages that require login or complex interactions).

  • Smart output organization — Automatically extract page metadata to generate YAML front matter, and create kebab-case filenames based on the domain and title. Supports conflict detection and timestamp-based renaming.

    • FAQ

    What is baoyu-url-to-markdown?


    baoyu-url-to-markdown is a command-line tool that uses Chrome CDP (Chrome DevTools Protocol) to open a browser to access a specified URL and convert the web page content into well-formatted Markdown. Compared with traditional crawlers, it can fully execute JavaScript to obtain dynamically rendered content.

    How do I use it?


    Basic usage: npx -y bun ${SKILL_DIR}/scripts/main.ts <url>
    Wait mode (for login pages): npx -y bun ${SKILL_DIR}/scripts/main.ts <url> --wait
    Specify the output file: npx -y bun ${SKILL_DIR}/scripts/main.ts <url> -o output.md

    Does it support pages that require login?


    Yes. Start it in wait mode using the --wait parameter. The tool opens a Chrome browser and waits for you to log in manually or complete page interactions. Once the page is ready, press Enter in the terminal to trigger the scrape. This is the best approach for pages that require login, multi-step verification, or complex interaction flows.