What's new

1.0.0-rc.1

14 November 2026 · Files on GitHub ↗

Release candidate for 1.0: fixes from the beta.4 feedback and separate connections per host for faster crawls on lossy networks.

• Connections per host: OpenSpider now opens several connections to a host and spreads the requests across them. Before, all threads shared a single connection under HTTP/2, so one lost packet held up every request. The number of simultaneous requests and the URL rate stay the same; the server does not receive more requests than before. The new setting is under Configuration › Speed › "Connections per host": "auto" (default) uses as many connections as threads, at most 6; a fixed value from 1 to 6 sets the limit, and 1 restores the previous behaviour under HTTP/2. If a server refuses an additional connection, the crawl continues on the existing ones without error rows. Under HTTP/1.1 a host receives at most as many simultaneous requests as it has connections, so at most 6 with "auto". The debug info names the value of the crawl.
• Live crawl table: when the table has to reload its data during a fast crawl, it now does so silently. The rows stay visible until the new data has arrived, and the multiple selection, the anchor of a Shift selection and the scroll position are kept. Before, the table could show empty rows, lose the selection or jump to the top. A Shift+click that is under way during such a reload is completed instead of being dropped.
• Live crawl counters: a table opened while the crawl is writing no longer stays behind permanently (for example "18 of 20"). The status no longer shows a finished crawl while the counters are still running.
• Starting a new crawl in the same view keeps the old rows until the new ones arrive, without a loading bar.
• Enter and "Start" respond right after a short crawl has finished. Before, the start command could stay blocked while the start URL was being saved to the suggestion list. The question "Replace" keeps its default action.
• "Recently opened": different spellings of the path of one crawl file are merged into one entry; duplicate entries from beta.4 disappear when the list is loaded. Entries show name or domain and date even if the remembered path is spelled differently from the stored one. This fixes the known issue from beta.4 with crawl storage locations behind a symbolic link.
• "Clear" releases the crawl file completely. On Windows the file could not be deleted, moved or replaced until OpenSpider was closed.
• Crawl management shows start time, duration, number of URLs and size in the format of the UI language.
• Status bar: every Crawl-delay hint from a robots.txt appears only once.
• Security update: rustix 1.1.6 (RUSTSEC-2026-0486).

1.0.0-beta.4

12 November 2026 · Files on GitHub ↗

A new update dialog, live counters that keep up with the crawl, clear messages when the start URL cannot be reached, and a repaired comparison view.

• New update dialog: it shows a short summary of the new version in the UI language and offers "Skip this version", "Remind me later" and "Install on quit". A skipped version is only reported again when you check manually; newer versions are still announced. Switching off the automatic check keeps an already found release available.
• The version now carries a build number, for example "1.0.0-beta.4 (Build 2790)", in the About dialog, the debug info, the update dialog and openspider --version.
• Live crawl: the counters and the table footer follow the running crawl within one second instead of jumping with the next batch of rows. "Completed" only appears once the final counts are exact and the automatic crawl analysis has finished, so the overview and the issues are complete when the status changes.
• Unreachable start URL: if the start URL or its robots.txt cannot be fetched, the crawl ends as "Failed" and a dialog names the cause (DNS, connection refused, timeout, TLS) and the address. Before, the crawl simply ended with 0 URLs.
• Start URL blocked by robots.txt: the dialog names the robots.txt and the blocking line. It appears once per crawl and start URL.
• Failure mail: scheduled and command-line crawls that fail at the start URL send a mail marked [FAILED] with the error class, the address and the cause. The scheduler history lists the run as "Failed", and the automatic comparison is skipped for it.
• Comparison view repaired: it fills the table, detail and sidebar areas, uses the normal grouped tab bar and the same columns as the normal view with the comparison status next to the address, scrolls through all rows without pages, and shows "Comparison" as the mode. The values of the comparison status follow the UI language; copies and exports keep the canonical values.
• Crawl management: typing into the form fields and the search works again. "Recently opened" is now one list with the name or domain, date and time, and a path shortened in the middle. A crawl file that no longer exists is reported in the status bar with its path instead of a dialog with a raw error code. Ctrl/Cmd+F focuses the search in the whole dialog.
• Enter in the start URL field starts a crawl and resumes a paused one; with a changed URL OpenSpider asks before starting a new crawl. In dialogs with OK and Cancel, Enter confirms; destructive actions still need the focused button.
• Menu "Configuration": every entry now opens the configuration dialog on its own page; before, most entries stayed disabled. The list of unavailable menu actions in the status bar opens at its trigger, wraps long lines and names the actual reason.
• Sidebar overview: after a finished crawl the overview shows "Up to date" instead of a growing age.
• MCP: agents can pause, resume and stop crawls (pause_crawl, resume_crawl, stop_crawl) when the option "Agent may start crawls" is on. crawl_status reports the state of the run, including crawls started by another process.
• Detail window in a live crawl: range selections with Shift stay bound to one consistent state of the data; selecting "all" in very large tables is limited to 20,000 rows with a hint to use the export; errors while refreshing the detail window are shown and recover on their own; a column choice is saved before the window closes.
• Windows: the updater detects whether OpenSpider was installed with the MSI or the EXE installer and downloads the matching update.
• Form login: the login is confirmed before the login browser closes.
• Crawl files: the backup made before a file migration no longer drops the file lock; crawl management checks all locks before opening a file for writing.
• Security updates: SQLite 3.54.1 (RUSTSEC-2026-0459) and the MCP library rmcp 3.9.0 (RUSTSEC-2026-0457, -0461, -0462).

Known issue on macOS: if the crawl storage location is reached through a symbolic link (for example /tmp), "Recently opened" can list the same crawl file twice. A fix follows in the next version.

1.0.0-beta.3

9 November 2026 · Files on GitHub ↗

Faster crawls, Chromium on demand, one-click agent connections and a report for checking deployments.

• Faster large crawls: crawl data is written in bigger batches, about 30 % less time for 100,000 and 1 million URLs in our benchmarks. The link score analysis now runs in memory (1 million URLs: 113 s → 23 s).
• JavaScript rendering no longer needs an installed Chrome: OpenSpider downloads a tested Chromium version on request and can remove it again.
• Connect Claude Desktop, Claude Code, Codex or Cursor to the MCP server in one step. Agents can only read by default; starting crawls needs the "Agent may start crawls" option. Existing configuration files keep their format, a backup is created, and read-only files are never changed.
• New report "Crawl Changes" compares a crawl with the previous crawl of the same project: removed pages, new redirects, changed canonicals and indexability. Useful after a deployment; the sub-filters can be exported.
• Crash reports are stored locally and only sent if you agree.
• Visualisations: 3D crawl and directory graphs, word cloud, and a chart for the selected entry of the overview.
• New "About OpenSpider" dialog with developer, website and source code links.
• Docker: a Dockerfile for running the CLI, exports and the MCP server on Linux servers (build the image yourself, see the user guide).
• Live crawl: the detail window stays responsive while a crawl runs; Shift+click and Shift+arrow selections are no longer lost during a running crawl; the Crawl Path dialog shows its rows again; a column choice in a detail table is kept when you select another URL; scrolling up in tail mode takes effect at once.
• Fixed possible crawl file damage: listing crawls from a second process (CLI, MCP server or a second app instance) while a crawl was running could damage that crawl's file. Listing now only reads such files and never changes them; locking a running crawl is no longer possible.
• Fixes: a stopped crawl stays "Stopped" after a restart; the status card shows "Finished (limit reached)" like the status bar; the findings sidebar follows the UI language; repairing a crawl file no longer counts write errors as lost rows; more reliable "Check for updates"; updated spreadsheet import library (security advisory RUSTSEC-2026-0436).

1.0.0-beta.2

4 November 2026 · Files on GitHub ↗

Maintenance release without new features: it checks the automatic update from beta.1 to a newer version.

• App features unchanged from beta.1.
• Known: the Chrome download on request for JavaScript rendering follows with beta.3 (not with beta.2 as announced).

1.0.0-beta.1

4 November 2026 · Files on GitHub ↗

First open beta. Please send feedback via Help → "Send feedback…" or openspider.de/en/beta/.

• Desktop app for macOS (Apple silicon and Intel, signed and notarised) and Windows x64 (signed with Azure Artifact Signing).
• Automatic update check (can be switched off) with download in the background; installation at once or on quit, never during a running crawl.
• Smoother live crawl: shared, throttled counting for the overview, issues and filters; the overview only redraws changed rows.
• "Top issues" and "Findings" load quickly after a crawl ends; pause, resume and stop keep the crawl file clean.
• Old crawls and settings are carried over when opened.
• Known: JavaScript rendering uses an installed Chrome in this beta; the download on request follows with beta.2.