# Musebook Narrative Spider

A Chrome Manifest V3 companion adapted from the MIT-licensed Gem Search extension. It reads rendered X posts only after you release it on a tab, keeps up to 200 deduplicated captures in Chrome's local extension storage, and exports a JSON research file. It has no host permissions, credentials, pairing, background collection, or network transport. The spider and all scripts are packaged with the extension.

## Install

1. Download `https://musebook.trade/crawler/musebook-spider.zip` and extract it to a folder you will keep.
2. Open `chrome://extensions`, enable **Developer mode**, and choose **Load unpacked**.
3. Select the extracted folder containing `manifest.json`. Pin **Musebook — Narrative Spider** if useful.
4. Open an X feed, profile, or search page. Open the extension popup and choose **Release spider on this tab**.
5. Scroll normally. **Auto-scroll** is optional and must be checked before release; it pauses while the tab is hidden or you are typing.
6. Open the popup and choose **Export JSON**. Open `https://musebook.trade/crawler#research`, choose **Import JSON**, and select that file.

For a source checkout, load `integrations/crawler-spider/extension/` directly. Chrome grants access to the selected tab when you invoke the extension; there are no persistent site permissions. Refresh a tab after updating/reloading the extension.

## Controls and privacy

- **Pause this tab** stops its collector and retains existing captures. The overlay's Pause/Resume controls do the same. Dismissing the spider stops that tab's collector.
- **Stop every spider & clear captured posts** pauses all injected collectors and removes the saved capture queue. Downloaded JSON files remain on your computer; remove those separately when finished. Remove the extension in `chrome://extensions` to delete its browser storage.
- Captures contain only a post permalink, rendered text, displayed timestamp, and up to eight external links. Post IDs are kept internally for deduplication and omitted from exports.
- Messages/chat, settings, account/login/compose, notifications, and Grok routes are excluded. Hidden tabs do no work. No cookies, passwords, DMs, browser history, wallet access, account actions, or network interception are used.
- Visible posts may be protected or contain personal information. Activate collection only where you intend to use the material, review your exported file, and import only material you intend to share with the workbench. Exporting does not upload the file; importing is a separate action governed by Musebook's research mode.
- The 200-post queue persists across Chrome restarts. Reaching the limit pauses collection; export, then stop and clear before gathering another batch. Chrome sync is not used.
- Auto-scroll is off when a new collector starts. A tab's running collector retains its explicitly selected setting until reconfigured. Reduced-motion users receive immediate spider movement.

## Export format

```json
{
  "posts": [{
    "url": "https://x.com/researcher/status/123456789",
    "text": "A public project release with documentation",
    "created_at": "2026-10-11T12:00:00.000Z",
    "links": ["https://example.com/docs"]
  }]
}
```

This is the strict browser-capture format. Gem Search's generic collector import uses a separate `id`, `author`, `project_url` format; the Musebook research importer accepts browser captures directly.

## Build and check

```sh
python3 integrations/crawler-spider/package_extension.py
node --test integrations/crawler-spider/extension.test.mjs
```

The packager writes `cloudflare/bot-assets/crawler/musebook-spider.zip` from an explicit file allowlist with fixed ZIP metadata. Only extension assets, this guide, and the MIT license enter the archive.

## Source reference and launch boundary

Source: `/Users/8bit/Downloads/gem-search-main/extension/` and the supplied Gem Search documentation. The original local application uses a Python engine, SQLite, optional local xAI credentials, and a separately enabled autolaunch executor. Its optional executor manages treasury and isolated wallet keys, enforces a 0.025 SOL attempt cap, and reconciles submitted transactions. Those local signing and autolaunch components are not part of this browser extension.

Musebook uses reviewed research drafts as the handoff from a finding. Capturing a cashtag, address, phrase, or project does not authorize a launch, wallet connection, signature, or funded transaction. The source guides bundled with the web reference describe the original local app and are labeled as such.

Copyright (c) 2026 Gem Search contributors. Adapted for Musebook under the MIT license in `LICENSE`.