autoskill

autoskill — Automated Skill Generation Tool

Skill Overview

autoskill is an intelligent skill-generation tool based on observing screen activity. It can automatically detect recurring patterns in a user’s everyday research workflows and convert them into reusable AI skills or skill-combination plans.

Use Cases

Suitable for researchers who frequently perform tasks such as literature searching, academic writing, and data analysis. autoskill can identify repetitive processes like literature management, paper writing, and chart creation, and generate corresponding automation skill suggestions to help researchers turn everyday actions into reusable AI workflows.

When you notice you often perform similar sequences of screen operations but aren’t sure whether those operations are already covered by existing skills, autoskill can analyze your screen activity logs, identify repeated patterns, match them against the existing skill library, and tell you which workflows can be automated.

Suitable for researchers who prioritize privacy protection and want all data processing to be done locally. autoskill runs entirely on the local screenpipe daemon and an optional local LLM (such as LM Studio). Sensitive screen data never needs to be uploaded to the cloud to get personalized skill suggestions.

Core Features

Using screen activity logs captured by screenpipe, autoskill uses session-splitting and clustering algorithms to automatically identify recurring workflow patterns. It analyzes application usage duration, window titles, and OCR text, groups continuous action sequences into meaningful “sessions,” and detects repeated workflows.

autoskill includes an embedded index for 135+ existing scientific skills. It performs semantic matching between detected workflows and existing skills. For workflows already covered by existing skills, it recommends reuse; for workflows that require multiple skills in combination, it generates a composition plan; for entirely new scenarios, it drafts a complete new skill.

All sensitive data processing is done locally: screenpipe filters sensitive applications during capture, and autoskill’s redact scripts further remove sensitive information from OCR text (email addresses, API keys, phone numbers). Clustering and matching run entirely on the user’s machine. Users can also choose to use a local LLM (LM Studio) for skill generation, ensuring that no raw screen data leaves the device.

Frequently Asked Questions

How does autoskill protect my privacy and data security?

autoskill uses a multi-layer privacy protection mechanism. First, screenpipe excludes sensitive applications during capture based on the configuration file (such as password managers, banking apps, etc.). Second, autoskill’s redact scripts remove sensitive information from OCR text before analysis (email addresses, API keys, Bearer tokens, phone numbers). Most importantly, all clustering and matching are performed locally. You can choose to use a local LLM (such as LM Studio) for skill generation, so any raw screen data never leaves your machine.

What prerequisites and dependencies are needed to use autoskill?

Using autoskill requires three main components: (1) the screenpipe daemon running locally and bound to localhost:3030; (2) a Python environment with httpx, pyyaml, and sentence-transformers installed via pipenv; (3) an optional LLM backend. By default, it uses a locally running LM Studio (you need to download the Gemma-4-31B-it model). You can also choose the Claude API (requires ANTHROPIC_API_KEY) or a Foundry gateway.

What types of workflows can autoskill recognize?

autoskill is especially good at identifying repetitive research workflows, including literature search and management (e.g., Zotero, PubMed-related operations), academic writing (e.g., LaTeX, Word editing modes), research plotting (e.g., Python matplotlib, R ggplot2), and presentation creation (e.g., PowerPoint, Beamer). It can detect complex workflows spanning multiple applications as long as these operations repeat over time.

How should generated skill suggestions be reviewed and published?

autoskill stores generated suggestions in the directory ~/.autoskill/proposed/<timestamp>/, including a report and a drafted SKILL.md file. You need to manually review and edit these drafts, then use the promote command to move approved suggestions into the official skills directory. The promote command refuses to overwrite skills that already exist, ensuring that you don’t accidentally lose existing content.

Are the skills generated by autoskill reused existing skills or created as new skills?

This depends on how the LLM judges the workflow. If the detected workflow closely matches existing skills, it recommends “reuse.” If the workflow requires multiple existing skills executed in sequence, it generates a “compose” plan and outputs a SKILL.md containing multiple skill-call sequences. Only when existing skills cannot cover the workflow does it generate a complete draft for a “novel” new skill.

How do I configure filtering rules for sensitive applications?

screenpipe uses a configuration file to manage application filtering. autoskill provides references/screenpipe-config.yaml as a starting template, which includes a rejection list of common sensitive applications. You need to copy this file into the screenpipe configuration directory and add or remove applications according to your needs. Then screenpipe will skip those applications’ screen content during capture.

What hardware is needed to run autoskill locally?

Hardware requirements for running autoskill locally mainly depend on the chosen LLM backend. If you use the default local LM Studio backend, you need a GPU capable of running the Gemma-4-31B-it model (recommended: 8GB+ VRAM). If you don’t have enough GPU resources, you can use the Claude API or a Foundry cloud backend, in which case hardware requirements can be greatly reduced. screenpipe’s OCR clustering and matching computations do not require much CPU or memory, and modern laptops should run it smoothly.