Skip to content

A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.

Storage inspector experimental (#8299)

Shahmir Varqha82fa744670cf
diff --git a/docs/_static/docs-remote-storage.mp4 b/docs/_static/docs-remote-storage.mp4
new file mode 100644
index 0000000000..c709ce5458
Binary files /dev/null and b/docs/_static/docs-remote-storage.mp4 differ
diff --git a/docs/guides/editor_features/panels.md b/docs/guides/editor_features/panels.md
index e65788be65..fbb1228ccc 100644
--- a/docs/guides/editor_features/panels.md
+++ b/docs/guides/editor_features/panels.md
@@ -17,7 +17,7 @@ The sidebar provides quick access to panels you'll use frequently while working
| Panel | Description |
|-------|-------------|
-| **Files** | Browse and open files in your workspace |
+| **Files** | Browse workspace files and [inspect remote storage connections](../working_with_data/remote_storage.md) |
| **Variables** | Explore variables and data sources in your notebook |
| **Packages** | View installed packages and manage dependencies |
| **AI** | Chat with AI assistants and use agents |
diff --git a/docs/guides/working_with_data/index.md b/docs/guides/working_with_data/index.md
index d8e8be7783..a233e6ca5a 100644
--- a/docs/guides/working_with_data/index.md
+++ b/docs/guides/working_with_data/index.md
@@ -1,11 +1,12 @@
# Working with data
These guides introduce you to marimo's features for working with data,
-including SQL cells, no-code dataframe transformation tools, and plots whose
-selections are automatically sent back to Python.
+including SQL cells, no-code dataframe transformation tools, plots whose
+selections are automatically sent back to Python, and browsing remote storage.
| Guide | Description |
|-------|-------------|
| [SQL](sql.md) | Use SQL to query dataframes, databases, CSVs, etc. |
| [Dataframes](dataframes.md) | Filter, search, and transform dataframes without code |
| [Plotting](plotting.md) | Send plot selections to Python |
+| [Remote Storage](remote_storage.md) | Browse and download files from S3, GCS, Azure, and more |
diff --git a/docs/guides/working_with_data/remote_storage.md b/docs/guides/working_with_data/remote_storage.md
new file mode 100644
index 0000000000..750a801a06
--- /dev/null
+++ b/docs/guides/working_with_data/remote_storage.md
@@ -0,0 +1,73 @@
+# Remote Storage
+
+marimo makes it easy to work with cloud storage and remote filesystems by automatically detecting [Obstore](https://developmentseed.org/obstore/) and [fsspec](https://filesystem-spec.readthedocs.io/) storage connections in your notebook. From the Files panel, you can browse directories, search entries, copy URLs, and download files—all without leaving the editor.
+
+!!! example "Experimental"
+
+ Remote storage is an experimental feature. Enable it in **Settings > Experimental > Storage Inspector**.
+
+<div align="center">
+<video autoplay muted loop playsinline width="100%" height="100%" align="center">
+ <source src="/_static/docs-remote-storage.mp4" type="video/mp4">
+</video>
+</div>
+
+## Supported libraries
+
+marimo auto-discovers variables that are instances of:
+
+| Library | Base class | Example stores |
+|---------|-----------|----------------|
+| [Obstore](https://developmentseed.org/obstore/) | `obstore.store.ObjectStore` | `S3Store`, `GCSStore`, `AzureStore`, `HTTPStore`, `LocalStore`, `MemoryStore` |
+| [fsspec](https://filesystem-spec.readthedocs.io/) | `fsspec.AbstractFileSystem` | `S3FileSystem`, `GithubFileSystem`, `FTPFileSystem`, `DatabricksFileSystem`, and [many more](https://filesystem-spec.readthedocs.io/en/latest/api.html#built-in-implementations) |
+
+
+## Quick start
+
+Create a storage connection in any cell. marimo will pick it up automatically after the cell executes.
+
+### Obstore
+
+```python
+from obstore.store import S3Store
+
+store = S3Store.from_url(
+ "s3://my-bucket",
+ access_key_id="...",
+ secret_access_key="...",
+)
+```
+
+### fsspec
+
+```python
+from fsspec.implementations.github import GithubFileSystem
+
+repo = GithubFileSystem(org="marimo-team", repo="marimo")
+```
+
+After the cell runs, the **Remote Storage** section in the Files sidebar panel will show your connection with its detected protocol and root path.
+
+
+## Using the panel
+
+The storage inspector appears as a collapsible **Remote storage** section at the top of the **Files** sidebar panel.
+
+| Action | How |
+|--------|-----|
+| **Browse** | Click a directory to expand it. Entries are fetched lazily. |
+| **Search** | Type in the search box to filter loaded entries by name. Expand directories first to include their contents. |
+| **Copy URL** | Right-click or use the `⋮` menu on any entry to copy its full URL (e.g., `s3://bucket/path/to/file.parquet`). |
+| **Download** | Use the `⋮` menu on a file to download it through the marimo server. |
+| **Refresh** | Click the refresh icon on a namespace header to re-fetch its entries. |
+
+## Multiple connections
+
+You can have multiple storage connections in the same notebook — each one appears as a separate namespace. The panel header shows the variable name so you can tell them apart.
+
+```python
+from obstore.store import S3Store
+
+prod = S3Store.from_url("s3://prod-bucket")
+staging = S3Store.from_url("s3://staging-bucket")
+```
diff --git a/examples/storage/general.py b/examples/storage/general.py
new file mode 100644
index 0000000000..07acafd8ed
--- /dev/null
+++ b/examples/storage/general.py
@@ -0,0 +1,77 @@
+# /// script
+# requires-python = ">=3.13"
+# dependencies = [
+# "fsspec==2026.2.0",
+# "marimo>=0.19.9",
+# "obstore==0.8.2",
+# "python-dotenv==1.2.1",
+# "requests==2.32.5",
+# "s3fs==2026.2.0",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.19.11"
+app = marimo.App(width="medium")
+
+
+def _():
+ import marimo as mo
+ from fsspec.implementations.local import LocalFileSystem
+ from fsspec.implementations.github import GithubFileSystem
+ import s3fs
+ import obstore
+ import os
+ from dotenv import load_dotenv
+
+ load_dotenv()
+ return GithubFileSystem, LocalFileSystem, os, s3fs
+
+
+def _(LocalFileSystem):
+ local = LocalFileSystem(asynchronous=True)
+ return
+
+
+def _(os):
+ from obstore.store import S3Store
+
+ access_key_id = os.environ.get("CLOUDFLARE_ACCESS_KEY_ID")
+ secret_access_key = os.environ.get("CLOUDFLARE_SECRET_ACCESS_KEY")
+ url = os.environ.get("CLOUDFLARE_MARIMO_URL")
+ store = S3Store.from_url(
+ f"{url}/marimo-artifacts",
+ access_key_id=access_key_id,
+ secret_access_key=secret_access_key,
+ )
+ return (url,)
+
+
+def _(GithubFileSystem):
+ marimo_repo = GithubFileSystem(org="marimo-team", repo="marimo")
+ return (marimo_repo,)
+
+
+def _(marimo_repo):
+ print(marimo_repo.read_text("github://AGENTS.md"))
+ return
+
+
+def _(os, s3fs, url):
+ s3_client = s3fs.S3FileSystem(
+ endpoint_url=url,
+ key=os.getenv("CLOUDFLARE_ACCESS_KEY_ID"),
+ secret=os.getenv("CLOUDFLARE_SECRET_ACCESS_KEY"),
+ )
+ return
+
+
+if __name__ == "__main__":
+ app.run()
diff --git a/frontend/src/__mocks__/requests.ts b/frontend/src/__mocks__/requests.ts
index 2a2b5ef999..15dafcc0b1 100644
--- a/frontend/src/__mocks__/requests.ts
+++ b/frontend/src/__mocks__/requests.ts
@@ -74,6 +74,8 @@ export const MockRequestClient = {
invokeAiTool: vi.fn().mockResolvedValue({}),
clearCache: vi.fn().mockResolvedValue(null),
getCacheInfo: vi.fn().mockResolvedValue(null),
+ listStorageEntries: vi.fn().mockResolvedValue(null),
+ downloadStorage: vi.fn().mockResolvedValue(null),
...overrides,
};
},
diff --git a/frontend/src/components/app-config/user-config-form.tsx b/frontend/src/components/app-config/user-config-form.tsx
index 1084791b1e..f066b3caea 100644
--- a/frontend/src/components/app-config/user-config-form.tsx
+++ b/frontend/src/components/app-config/user-config-form.tsx
@@ -1404,6 +1404,33 @@ export const UserConfigForm: React.FC = () => {
</div>
)}
/>
+ <FormField
+ control={form.control}
+ name="experimental.storage_inspector"
+ render={({ field }) => (
+ <div className="flex flex-col gap-y-1">
+ <FormItem className={formItemClasses}>
+ <FormLabel className="font-normal">
+ Storage Inspector
+ </FormLabel>
+ <FormControl>
+ <Checkbox
+ data-testid="storage-inspector-checkbox"
+ checked={field.value === true}
+ onCheckedChange={field.onChange}
+ />
+ </FormControl>
+ </FormItem>
+ <IsOverridden
+ userConfig={config}
+ name="experimental.storage_inspector"
+ />
+ <FormDescription>
+ Enable experimental storage inspector.
+ </FormDescription>
+ </div>
+ )}
+ />
</SettingGroup>
);
}
diff --git a/frontend/src/components/databases/engine-variable.tsx b/frontend/src/components/databases/engine-variable.tsx
index d3385c3706..c242aa56b6 100644
--- a/frontend/src/components/databases/engine-variable.tsx
+++ b/frontend/src/components/databases/engine-variable.tsx
@@ -16,7 +16,8 @@ export const EngineVariable: React.FC<Props> = ({
variableName,
className,
}) => {
- const onClick = () => {
+ const onClick = (e: React.MouseEvent<HTMLButtonElement>) => {
+ e.stopPropagation();
const cellId = findCellId(variableName);
if (!cellId) {
return;
diff --git a/frontend/src/components/databases/icons/google-cloud-storage.svg b/frontend/src/components/databases/icons/google-cloud-storage.svg
new file mode 100644
index 0000000000..842c121a7b
--- /dev/null
+++ b/frontend/src/components/databases/icons/google-cloud-storage.svg
@@ -0,0 +1,38 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg id="standard_product_icon" xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 512 512">
+ <!-- Generator: Adobe Illustrator 29.1.0, SVG Export Plug-In . SVG Version: 2.1.0 Build 142) -->
+ <defs>
+ <style>
+ .st0 {
+ fill: none;
+ }
+
+ .st1 {
+ fill: #4285f4;
+ }
+
+ .st2 {
+ fill: #34a853;
+ }
+
+ .st3 {
+ fill: #fbbc04;
+ }
+
+ .st4 {
+ fill: #ea4335;
+ }
+ </style>
+ </defs>
+ <g id="bounding_box">
+ <rect class="st0" width="512" height="512"/>
+ </g>
+ <g id="art">
+ <path class="st2" d="M442,277.9H70c-8.8,0-16,7.2-16,16v148.1c0,8.8,7.2,16,16,16h107.2c1.5.5,3.1.7,4.8.7s3.3-.3,4.8-.7h255.2c8.8,0,16-7.2,16-16v-148.1c0-8.8-7.2-16-16-16ZM86,309.9h80v116.1h-80v-116.1ZM426,425.9h-228v-116.1h228v116.1Z"/>
+ <path class="st3" d="M442,54H70c-8.8,0-16,7.2-16,16v148.8c0,8.8,7.2,16,16,16h372c8.8,0,16-7.2,16-16V70c0-8.8-7.2-16-16-16ZM86,86h80v116.8h-80v-116.8ZM426,202.8h-228v-116.8h228v116.8Z"/>
+ <path class="st4" d="M442,234.8h-16V86H54v-16c0-8.8,7.2-16,16-16h372c8.8,0,16,7.2,16,16v148.8c0,8.8-7.2,16-16,16Z"/>
+ <path class="st1" d="M442,457.9h-16v-148.1H54v-16c0-8.8,7.2-16,16-16h372c8.8,0,16,7.2,16,16v148.1c0,8.8-7.2,16-16,16Z"/>
+ <circle class="st4" cx="349" cy="144.4" r="37"/>
+ <circle class="st1" cx="349" cy="367.9" r="37"/>
+ </g>
+</svg>
\ No newline at end of file
diff --git a/frontend/src/components/editor/chrome/panels/components.tsx b/frontend/src/components/editor/chrome/panels/components.tsx
new file mode 100644
index 0000000000..548e74377d
--- /dev/null
+++ b/frontend/src/components/editor/chrome/panels/components.tsx
@@ -0,0 +1,88 @@
+/* Copyright 2026 Marimo. All rights reserved. */
+
+import * as React from "react";
+import {
+ AccordionContent,
+ AccordionItem,
+ AccordionTrigger,
+} from "@/components/ui/accordion";
+import { Badge } from "@/components/ui/badge";
+import { cn } from "@/utils/cn";
+
+/**
+ * Styled AccordionItem for sidebar panels.
+ * Applies border-b by default; set `lastItem` to remove it on the final item.
+ */
+const PanelAccordionItem = React.forwardRef<
+ React.ComponentRef<typeof AccordionItem>,
+ React.ComponentPropsWithoutRef<typeof AccordionItem> & {
+ lastItem?: boolean;
+ }
+>(({ className, lastItem, ...props }, ref) => (
+ <AccordionItem
+ ref={ref}
+ className={cn(lastItem && "border-b-0", className)}
+ {...props}
+ />
+));
+PanelAccordionItem.displayName = "PanelAccordionItem";
+
+/**
+ * Styled AccordionTrigger for sidebar panels.
+ * Applies compact uppercase styling and wraps children in a flex container
+ * with gap for icon + label layout.
+ */
+const PanelAccordionTrigger = React.forwardRef<
+ React.ComponentRef<typeof AccordionTrigger>,
+ React.ComponentPropsWithoutRef<typeof AccordionTrigger>
+>(({ className, children, ...props }, ref) => (
+ <AccordionTrigger
+ ref={ref}
+ className={cn(
+ "px-3 py-2 text-xs font-semibold uppercase tracking-wide hover:no-underline",
+ className,
+ )}
+ {...props}
+ >
+ <span className="flex items-center gap-2">{children}</span>
+ </AccordionTrigger>
+));
+PanelAccordionTrigger.displayName = "PanelAccordionTrigger";
+
+/**
+ * Styled AccordionContent for sidebar panels.
+ * Removes default wrapper padding.
+ */
+const PanelAccordionContent = React.forwardRef<
+ React.ComponentRef<typeof AccordionContent>,
+ React.ComponentPropsWithoutRef<typeof AccordionContent>
+>(({ wrapperClassName, ...props }, ref) => (
+ <AccordionContent
+ ref={ref}
+ wrapperClassName={cn("p-0", wrapperClassName)}
+ {...props}
+ />
+));
+PanelAccordionContent.displayName = "PanelAccordionContent";
+
+/**
+ * Styled Badge for sidebar panels.
+ */
+const PanelBadge = ({
+ className,
+ ...props
+}: React.ComponentPropsWithoutRef<typeof Badge>) => (
+ <Badge
+ variant="secondary"
+ className={cn("py-0 px-1.5 text-[10px]", className)}
+ {...props}
+ />
+);
+PanelBadge.displayName = "PanelBadge";
+
+export {
+ PanelAccordionItem,
+ PanelAccordionTrigger,
+ PanelAccordionContent,
+ PanelBadge,
+};
diff --git a/frontend/src/components/editor/chrome/panels/file-explorer-panel.tsx b/frontend/src/components/editor/chrome/panels/file-explorer-panel.tsx
index 3625b68ab2..094c24826f 100644
--- a/frontend/src/components/editor/chrome/panels/file-explorer-panel.tsx
+++ b/frontend/src/components/editor/chrome/panels/file-explorer-panel.tsx
@@ -1,14 +1,40 @@
/* Copyright 2026 Marimo. All rights reserved. */
-import React from "react";
+import { useAtom, useAtomValue } from "jotai";
+import { atomWithStorage } from "jotai/utils";
+import { FileIcon, HardDrive } from "lucide-react";
+import React, { useCallback, useMemo } from "react";
import useResizeObserver from "use-resize-observer";
+import { StorageInspector } from "@/components/storage/storage-inspector";
+import { Accordion } from "@/components/ui/accordion";
+import { getFeatureFlag } from "@/core/config/feature-flag";
+import { storageNamespacesAtom } from "@/core/storage/state";
import { cn } from "@/utils/cn";
+import { jotaiJsonStorage } from "@/utils/storage/jotai";
import { TreeDndProvider } from "../../file-tree/dnd-wrapper";
import { FileExplorer } from "../../file-tree/file-explorer";
import { useFileExplorerUpload } from "../../file-tree/upload";
+import {
+ PanelAccordionContent,
+ PanelAccordionItem,
+ PanelAccordionTrigger,
+ PanelBadge,
+} from "./components";
-const FileExplorerPanel: React.FC = () => {
- const { ref, height = 1 } = useResizeObserver<HTMLDivElement>();
+type OpenSections = "files" | "remote-storage";
+
+interface FileExplorerPanelState {
+ openSections: OpenSections[];
+ hasUserInteracted: boolean;
+}
+
+const fileExplorerPanelAtom = atomWithStorage<FileExplorerPanelState>(
+ "marimo:file-explorer-panel:state",
+ { openSections: ["files"], hasUserInteracted: false },
+ jotaiJsonStorage,
+);
+
+const FileExplorerComponent: React.FC<{ height: number }> = ({ height }) => {
const { getRootProps, getInputProps, isDragActive } = useFileExplorerUpload({
noClick: true,
noKeyboard: true,
@@ -18,7 +44,8 @@ const FileExplorerPanel: React.FC = () => {
<TreeDndProvider>
<div
{...getRootProps()}
- className={cn("flex flex-col flex-1 overflow-hidden relative")}
+ className={cn("flex flex-col overflow-hidden relative")}
+ style={{ height }}
>
<input {...getInputProps()} />
{isDragActive && (
@@ -27,12 +54,98 @@ const FileExplorerPanel: React.FC = () => {
</div>
)}
- <div ref={ref} className="flex flex-col flex-1 overflow-hidden">
- <FileExplorer height={height} />
- </div>
+ <FileExplorer height={height} />
</div>
</TreeDndProvider>
);
};
+// Height of each accordion trigger (px-3 py-2 text-xs = ~33px)
+const TRIGGER_HEIGHT = 33;
+
+const FileExplorerPanel: React.FC = () => {
+ const { ref: panelRef, height: panelHeight = 500 } =
+ useResizeObserver<HTMLDivElement>();
+ const [state, setState] = useAtom(fileExplorerPanelAtom);
+
+ const storageNamespaces = useAtomValue(storageNamespacesAtom);
+ const remoteStorageConnections = storageNamespaces.length;
+
+ const openSections = useMemo<OpenSections[]>(() => {
+ if (!state.hasUserInteracted && remoteStorageConnections > 0) {
+ if (state.openSections.includes("remote-storage")) {
+ return state.openSections;
+ }
+ return [...state.openSections, "remote-storage"];
+ }
+ return state.openSections;
+ }, [state.hasUserInteracted, state.openSections, remoteStorageConnections]);
+
+ const handleValueChange = useCallback(
+ (value: OpenSections[]) => {
+ setState({
+ openSections: value,
+ hasUserInteracted: true,
+ });
+ },
+ [setState],
+ );
+
+ const availableContent = panelHeight - TRIGGER_HEIGHT * 2;
+ const storageIsOpen = openSections.includes("remote-storage");
+ const bothOpen = storageIsOpen && openSections.includes("files");
+
+ const storageMaxHeight = bothOpen
+ ? Math.round(availableContent * 0.4)
+ : availableContent;
+ const fileTreeHeight = Math.max(
+ 200,
+ bothOpen ? availableContent - storageMaxHeight : availableContent,
+ );
+
+ const storageInspectorEnabled = getFeatureFlag("storage_inspector");
+ if (!storageInspectorEnabled) {
+ return (
+ <div ref={panelRef} className="h-full overflow-auto">
+ <FileExplorerComponent height={panelHeight} />
+ </div>
+ );
+ }
+
+ return (
+ <div ref={panelRef} className="h-full overflow-auto">
+ <Accordion
+ type="multiple"
+ value={openSections}
+ onValueChange={handleValueChange}
+ >
+ <PanelAccordionItem value="remote-storage">
+ <PanelAccordionTrigger>
+ <HardDrive className="w-4 h-4" /> Remote storage
+ {remoteStorageConnections > 0 && (
+ <PanelBadge>{remoteStorageConnections}</PanelBadge>
+ )}
+ </PanelAccordionTrigger>
+ <PanelAccordionContent
+ className="overflow-auto"
+ style={{ maxHeight: storageMaxHeight }}
+ >
+ <StorageInspector />
+ </PanelAccordionContent>
+ </PanelAccordionItem>
+
+ <PanelAccordionItem value="files">
+ <PanelAccordionTrigger>
+ <FileIcon className="w-4 h-4" />
+ Files
+ </PanelAccordionTrigger>
+ <PanelAccordionContent>
+ <FileExplorerComponent height={fileTreeHeight} />
+ </PanelAccordionContent>
+ </PanelAccordionItem>
+ </Accordion>
+ </div>
+ );
+};
+
export default FileExplorerPanel;
diff --git a/frontend/src/components/editor/chrome/panels/session-panel.tsx b/frontend/src/components/editor/chrome/panels/session-panel.tsx
index 2726278369..8f02e137dd 100644
--- a/frontend/src/components/editor/chrome/panels/session-panel.tsx
+++ b/frontend/src/components/editor/chrome/panels/session-panel.tsx
@@ -8,18 +8,18 @@ import {
connectionsAtom,
DataSources,
} from "@/components/datasources/datasources";
-import {
- Accordion,
- AccordionContent,
- AccordionItem,
- AccordionTrigger,
-} from "@/components/ui/accordion";
-import { Badge } from "@/components/ui/badge";
+import { Accordion } from "@/components/ui/accordion";
import { VariableTable } from "@/components/variables/variables-table";
import { useCellIds } from "@/core/cells/cells";
import { datasetTablesAtom } from "@/core/datasets/state";
import { useVariables } from "@/core/variables/state";
import { jotaiJsonStorage } from "@/utils/storage/jotai";
+import {
+ PanelAccordionContent,
+ PanelAccordionItem,
+ PanelAccordionTrigger,
+ PanelBadge,
+} from "./components";
type OpenSections = "variables" | "datasources";
@@ -69,34 +69,23 @@ const SessionPanel: React.FC = () => {
onValueChange={handleValueChange}
className="flex flex-col h-full overflow-auto"
>
- <AccordionItem value="datasources" className="border-b">
- <AccordionTrigger className="px-3 py-2 text-xs font-semibold uppercase tracking-wide hover:no-underline">
- <span className="flex items-center gap-2">
- <DatabaseIcon className="w-4 h-4" />
- Data sources
- {showDatasourcesBadge && (
- <Badge
- variant="secondary"
- className="ml-1 px-1.5 py-0 mb-px text-[10px]"
- >
- {datasourcesCount}
- </Badge>
- )}
- </span>
- </AccordionTrigger>
- <AccordionContent wrapperClassName="p-0">
+ <PanelAccordionItem value="datasources">
+ <PanelAccordionTrigger>
+ <DatabaseIcon className="w-4 h-4" />
+ Data sources
+ {showDatasourcesBadge && <PanelBadge>{datasourcesCount}</PanelBadge>}
+ </PanelAccordionTrigger>
+ <PanelAccordionContent>
<DataSources />
- </AccordionContent>
- </AccordionItem>
+ </PanelAccordionContent>
+ </PanelAccordionItem>
- <AccordionItem value="variables" className="border-b-0">
- <AccordionTrigger className="px-3 py-2 text-xs font-semibold uppercase tracking-wide hover:no-underline">
- <span className="flex items-center gap-2">
- <VariableIcon className="w-4 h-4" />
- Variables
- </span>
- </AccordionTrigger>
- <AccordionContent wrapperClassName="p-0">
+ <PanelAccordionItem value="variables" lastItem={true}>
+ <PanelAccordionTrigger>
+ <VariableIcon className="w-4 h-4" />
+ Variables
+ </PanelAccordionTrigger>
+ <PanelAccordionContent>
{Object.keys(variables).length === 0 ? (
<div className="px-3 py-4 text-sm text-muted-foreground">
No variables defined
@@ -104,8 +93,8 @@ const SessionPanel: React.FC = () => {
) : (
<VariableTable cellIds={cellIds.inOrderIds} variables={variables} />
)}
- </AccordionContent>
- </AccordionItem>
+ </PanelAccordionContent>
+ </PanelAccordionItem>
</Accordion>
);
};
diff --git a/frontend/src/components/editor/file-tree/__tests__/file-expolorer.test.ts b/frontend/src/components/editor/file-tree/__tests__/file-explorer.test.ts
similarity index 100%
rename from frontend/src/components/editor/file-tree/__tests__/file-expolorer.test.ts
rename to frontend/src/components/editor/file-tree/__tests__/file-explorer.test.ts
diff --git a/frontend/src/components/storage/components.tsx b/frontend/src/components/storage/components.tsx
new file mode 100644
index 0000000000..d201cae590
--- /dev/null
+++ b/frontend/src/components/storage/components.tsx
@@ -0,0 +1,73 @@
+/* Copyright 2026 Marimo. All rights reserved. */
+
+import AwsIcon from "@marimo-team/llm-info/icons/aws.svg?inline";
+import AzureIcon from "@marimo-team/llm-info/icons/azure.svg?inline";
+import {
+ DatabaseZapIcon,
+ FileCodeIcon,
+ FileIcon,
+ FileSpreadsheetIcon,
+ FileTextIcon,
+ FileVideoIcon,
+ GlobeIcon,
+ HardDriveIcon,
+ ImageIcon,
+} from "lucide-react";
+import GoogleCloudIcon from "@/components/databases/icons/google-cloud-storage.svg?inline";
+import type { KnownStorageProtocol } from "@/core/storage/types";
+
+export function renderFileIcon(name: string): React.ReactNode {
+ const ext = name.split(".").pop()?.toLowerCase();
+ switch (ext) {
+ case "png":
+ case "jpg":
+ case "jpeg":
+ case "gif":
+ case "svg":
+ case "webp":
+ return <ImageIcon className="h-3.5 w-3.5 text-purple-500" />;
+ case "csv":
+ case "parquet":
+ case "arrow":
+ case "xlsx":
+ return <FileSpreadsheetIcon className="h-3.5 w-3.5 text-green-500" />;
+ case "py":
+ case "js":
+ case "ts":
+ case "json":
+ return <FileCodeIcon className="h-3.5 w-3.5 text-blue-500" />;
+ case "mp4":
+ case "mpeg":
+ return <FileVideoIcon className="h-3.5 w-3.5 text-orange-500" />;
+ case "txt":
+ case "md":
+ case "log":
+ return <FileTextIcon className="h-3.5 w-3.5 text-muted-foreground" />;
+ default:
+ return <FileIcon className="h-3.5 w-3.5 text-muted-foreground" />;
+ }
+}
+
+const PROTOCOL_ICONS: Record<
+ KnownStorageProtocol,
+ string | React.ComponentType<{ className?: string }>
+> = {
+ s3: AwsIcon,
+ azure: AzureIcon,
+ gcs: GoogleCloudIcon,
+ http: GlobeIcon,
+ file: HardDriveIcon,
+ "in-memory": DatabaseZapIcon,
+} as const;
+
+export function renderProtocolIcon(
+ protocol: KnownStorageProtocol | (string & {}),
+): React.ReactNode {
+ const Icon =
+ PROTOCOL_ICONS[protocol.toLowerCase() as KnownStorageProtocol] ??
+ HardDriveIcon;
+ if (typeof Icon === "string") {
+ return <img src={Icon} alt={protocol} className="h-3.5 w-3.5" />;
+ }
+ return <Icon className="h-3.5 w-3.5" />;
+}
diff --git a/frontend/src/components/storage/storage-inspector.tsx b/frontend/src/components/storage/storage-inspector.tsx
new file mode 100644
index 0000000000..46cc4257ae
--- /dev/null
+++ b/frontend/src/components/storage/storage-inspector.tsx
@@ -0,0 +1,560 @@
+/* Copyright 2026 Marimo. All rights reserved. */
+
+import { CommandList } from "cmdk";
+import {
+ ChevronRightIcon,
+ CopyIcon,
+ DownloadIcon,
+ FolderIcon,
+ HardDriveIcon,
+ HelpCircleIcon,
+ LoaderCircle,
+ MoreVerticalIcon,
+ RefreshCwIcon,
+ XIcon,
+} from "lucide-react";
+import React, { useCallback, useState } from "react";
+import { useLocale } from "react-aria";
+import { EngineVariable } from "@/components/databases/engine-variable";
+import { PanelEmptyState } from "@/components/editor/chrome/panels/empty-state";
+import { Command, CommandInput, CommandItem } from "@/components/ui/command";
+import {
+ DropdownMenu,
+ DropdownMenuContent,
+ DropdownMenuItem,
+ DropdownMenuTrigger,
+} from "@/components/ui/dropdown-menu";
+import { Tooltip } from "@/components/ui/tooltip";
+import { toast } from "@/components/ui/use-toast";
+import { DownloadStorage } from "@/core/storage/request-registry";
+import {
+ useStorage,
+ useStorageActions,
+ useStorageEntries,
+} from "@/core/storage/state";
+import type {
+ StorageEntry,
+ StorageNamespace,
+ StoragePathKey,
+} from "@/core/storage/types";
+import { storagePathKey, storageUrl } from "@/core/storage/types";
+import type { VariableName } from "@/core/variables/types";
+import { cn } from "@/utils/cn";
+import { copyToClipboard } from "@/utils/copy";
+import { downloadByURL } from "@/utils/download";
+import { formatBytes } from "@/utils/formatting";
+import { Logger } from "@/utils/Logger";
+import { Button } from "../ui/button";
+import { renderFileIcon, renderProtocolIcon } from "./components";
+
+// Pixels per depth level. Applied as paddingLeft on each full-width item
+// so the selection highlight still spans the entire panel.
+const INDENT_PX = 16;
+
+function indentStyle(depth: number): React.CSSProperties {
+ return { paddingLeft: depth * INDENT_PX };
+}
+
+function formatDate(timestamp: number, locale: string): string {
+ const date = new Date(timestamp * 1000);
+ return date.toLocaleDateString(locale, {
+ month: "short",
+ day: "numeric",
+ hour: "2-digit",
+ minute: "2-digit",
+ });
+}
+
+/** Extract display name from a full path (e.g., "folder/subfolder/" -> "subfolder") */
+function displayName(path: string): string {
+ // Remove trailing slash
+ const trimmed = path.endsWith("/") ? path.slice(0, -1) : path;
+ const parts = trimmed.split("/");
+ return parts[parts.length - 1] || trimmed;
+}
+
+/**
+ * Recursively check whether an entry (or any of its loaded descendants)
+ * matches the search query.
+ */
+function entryMatchesSearch(
+ entry: StorageEntry,
+ namespace: string,
+ searchValue: string,
+ entriesByPath: ReadonlyMap<StoragePathKey, StorageEntry[]>,
+): boolean {
+ const query = searchValue.toLowerCase();
+
+ if (displayName(entry.path).toLowerCase().includes(query)) {
+ return true;
+ }
+
+ // For directories, check loaded children recursively
+ if (entry.kind === "directory") {
+ const children = entriesByPath.get(storagePathKey(namespace, entry.path));
+ if (children) {
+ return children.some((child) =>
+ entryMatchesSearch(child, namespace, searchValue, entriesByPath),
+ );
+ }
+ }
+
+ return false;
+}
+
+/**
+ * Filter entries to those matching the search (or having loaded descendants
+ * that match). Returns all entries when there is no active search.
+ */
+function filterEntries(
+ entries: StorageEntry[],
+ namespace: string,
+ searchValue: string,
+ entriesByPath: ReadonlyMap<StoragePathKey, StorageEntry[]>,
+): StorageEntry[] {
+ if (!searchValue.trim()) {
+ return entries;
+ }
+ return entries.filter((entry) =>
+ entryMatchesSearch(entry, namespace, searchValue, entriesByPath),
+ );
+}
+
+/**
+ * Lazily loaded children of a directory entry.
+ * Caches fetched entries in the Jotai store so re-expanding doesn't re-fetch.
+ */
+const StorageEntryChildren: React.FC<{
+ namespace: string;
+ protocol: string;
+ rootPath: string;
+ prefix: string;
+ depth: number;
+ locale: string;
+ searchValue: string;
+}> = ({
+ namespace,
+ protocol,
+ rootPath,
+ prefix,
+ depth,
+ locale,
+ searchValue,
+}) => {
+ const { entriesByPath } = useStorage();
+ const {
+ entries: children,
+ isPending,
+ error,
+ } = useStorageEntries(namespace, prefix);
+
+ if (isPending) {
+ return (
+ <div
+ className="flex items-center gap-1.5 py-1 text-xs text-muted-foreground"
+ style={indentStyle(depth)}
+ >
+ <LoaderCircle className="h-3 w-3 animate-spin" />
+ Loading...
+ </div>
+ );
+ }
+
+ if (error) {
+ return (
+ <div className="py-1 text-xs text-destructive" style={indentStyle(depth)}>
+ Failed to load: {error.message}
+ </div>
+ );
+ }
+
+ if (children.length === 0) {
+ return (
+ <div
+ className="py-1 text-xs text-muted-foreground italic"
+ style={indentStyle(depth)}
+ >
+ Empty
+ </div>
+ );
+ }
+
+ const filtered = filterEntries(
+ children,
+ namespace,
+ searchValue,
+ entriesByPath,
+ );
+
+ return (
+ <>
+ {filtered.map((child) => (
+ <StorageEntryRow
+ key={child.path}
+ entry={child}
+ namespace={namespace}
+ protocol={protocol}
+ rootPath={rootPath}
+ depth={depth}
+ locale={locale}
+ searchValue={searchValue}
+ />
+ ))}
+ </>
+ );
+};
+
+const StorageEntryRow: React.FC<{
+ entry: StorageEntry;
+ namespace: string;
+ protocol: string;
+ rootPath: string;
+ depth: number;
+ locale: string;
+ searchValue: string;
+}> = ({ entry, namespace, protocol, rootPath, depth, locale, searchValue }) => {
+ const [isExpanded, setIsExpanded] = useState(false);
+ const { entriesByPath } = useStorage();
+ const isDir = entry.kind === "directory";
+ const name = displayName(entry.path);
+ const hasSearch = !!searchValue.trim();
+
+ // During a search, auto-expand directories whose loaded descendants match
+ const hasMatchingDescendants =
+ isDir &&
+ hasSearch &&
+ !!entriesByPath
+ .get(storagePathKey(namespace, entry.path))
+ ?.some((child) =>
+ entryMatchesSearch(child, namespace, searchValue, entriesByPath),
+ );
+
+ // Folder is shown expanded by manual toggle OR by search auto-expand
+ const effectiveExpanded = isExpanded || hasMatchingDescendants;
+
+ const handleDownload = useCallback(async () => {
+ try {
+ const result = await DownloadStorage.request({
+ namespace,
+ path: entry.path,
+ });
+ if (result.error) {
+ toast({
+ title: "Download failed",
+ description: result.error,
+ variant: "danger",
+ });
+ return;
+ }
+ if (result.url) {
+ downloadByURL(result.url, result.filename ?? "download");
+ }
+ } catch (error) {
+ Logger.error("Failed to download storage entry", error);
+ toast({
+ title: "Download failed",
+ description: String(error),
+ variant: "danger",
+ });
+ }
+ }, [namespace, entry.path]);
+
+ return (
+ <>
+ <CommandItem
+ className={cn(
+ "text-xs flex items-center gap-1.5 cursor-pointer rounded-none group h-6.5",
+ isDir && "font-medium",
+ )}
+ style={indentStyle(depth)}
+ value={`${namespace}:${entry.path}`}
+ onSelect={() => {
+ if (isDir) {
+ setIsExpanded(!effectiveExpanded);
+ }
+ }}
+ >
+ {isDir ? (
+ <ChevronRightIcon
+ className={cn(
+ "h-3 w-3 shrink-0 transition-transform",
+ effectiveExpanded && "rotate-90",
+ )}
+ />
+ ) : (
+ <span className="w-3 shrink-0" />
+ )}
+ {isDir ? (
+ <FolderIcon className="h-3.5 w-3.5 text-amber-500 shrink-0" />
+ ) : (
+ renderFileIcon(name)
+ )}
+ <span className="truncate flex-1 text-left">{name}</span>
+ <div className="flex items-center">
+ {entry.size > 0 && (
+ <span className="text-[10px] text-muted-foreground pr-2 opacity-0 group-hover:opacity-100 transition-opacity tabular-nums">
+ {formatBytes(entry.size, locale)}
+ </span>
+ )}
+ {entry.lastModified != null && (
+ <Tooltip
+ content={`Last modified: ${new Date(entry.lastModified * 1000).toLocaleString()}`}
+ >
+ <span className="text-[10px] text-muted-foreground pr-1 opacity-0 group-hover:opacity-100 transition-opacity whitespace-nowrap">
+ {formatDate(entry.lastModified, locale)}
+ </span>
+ </Tooltip>
+ )}
+ <DropdownMenu>
+ <DropdownMenuTrigger asChild={true}>
+ <Button
+ variant="text"
+ size="icon"
+ className="opacity-0 group-hover:opacity-100 transition-opacity hover:shadow-none hover:text-link text-muted-foreground"
+ onClick={(e) => e.stopPropagation()}
+ >
+ <MoreVerticalIcon className="h-3 w-3" />
+ </Button>
+ </DropdownMenuTrigger>
+ <DropdownMenuContent
+ align="end"
+ onClick={(e) => e.stopPropagation()}
+ onCloseAutoFocus={(e) => e.preventDefault()}
+ >
+ <DropdownMenuItem
+ onSelect={async () => {
+ const url = storageUrl(protocol, rootPath, entry.path);
+ await copyToClipboard(url.toString());
+ toast({ title: "Copied to clipboard" });
+ }}
+ >
+ <CopyIcon className="h-3.5 w-3.5 mr-2" />
+ Copy URL
+ </DropdownMenuItem>
+ {!isDir && (
+ <DropdownMenuItem onSelect={() => handleDownload()}>
+ <DownloadIcon className="h-3.5 w-3.5 mr-2" />
+ Download
+ </DropdownMenuItem>
+ )}
+ </DropdownMenuContent>
+ </DropdownMenu>
+ </div>
+ </CommandItem>
+ {isDir && effectiveExpanded && (
+ <StorageEntryChildren
+ namespace={namespace}
+ protocol={protocol}
+ rootPath={rootPath}
+ prefix={entry.path}
+ depth={depth + 1}
+ locale={locale}
+ searchValue={searchValue}
+ />
+ )}
+ </>
+ );
+};
+
+const StorageNamespaceSection: React.FC<{
+ namespace: StorageNamespace;
+ locale: string;
+ searchValue: string;
+}> = ({ namespace, locale, searchValue }) => {
+ const [isExpanded, setIsExpanded] = useState(true);
+ const [isSpinning, setIsSpinning] = useState(false);
+ const { entriesByPath } = useStorage();
+ const { clearNamespaceCache } = useStorageActions();
+ const namespaceName = namespace.name ?? namespace.displayName;
+
+ const {
+ entries: fetchedEntries,
+ isPending,
+ error,
+ refetch,
+ } = useStorageEntries(namespaceName);
+
+ const handleRefresh = useCallback(
+ (e: React.MouseEvent) => {
+ e.stopPropagation();
+ setIsSpinning(true);
+ clearNamespaceCache(namespaceName);
+ refetch();
+ setTimeout(() => setIsSpinning(false), 500);
+ },
+ [namespaceName, clearNamespaceCache, refetch],
+ );
+
+ // While loading, fall back to initial entries from the namespace notification
+ const entries = isPending ? namespace.storageEntries : fetchedEntries;
+ const filtered = filterEntries(
+ entries,
+ namespaceName,
+ searchValue,
+ entriesByPath,
+ );
+
+ return (
+ <>
+ <CommandItem
+ value={namespace.name}
+ onSelect={() => setIsExpanded(!isExpanded)}
+ className="flex flex-row font-semibold h-7 text-xs gap-1.5 bg-(--slate-2) text-muted-foreground rounded-none"
+ >
+ <ChevronRightIcon
+ className={cn(
+ "h-3 w-3 shrink-0 transition-transform",
+ isExpanded && "rotate-90",
+ )}
+ />
+ {renderProtocolIcon(namespace.protocol)}
+ <span>{namespace.displayName}</span>
+ {namespace.name && (
+ <span className="text-xs text-muted-foreground font-normal">
+ (<EngineVariable variableName={namespace.name as VariableName} />)
+ </span>
+ )}
+ <Tooltip content="Refresh storage connection">
+ <Button
+ variant="ghost"
+ size="icon"
+ className="hover:bg-transparent hover:shadow-none"
+ onClick={handleRefresh}
+ >
+ <RefreshCwIcon
+ className={cn(
+ "h-3 w-3 text-muted-foreground hover:text-foreground",
+ isSpinning && "animate-[spin_0.5s]",
+ )}
+ />
+ </Button>
+ </Tooltip>
+ <span className="text-[10px] text-muted-foreground font-normal tabular-nums ml-auto">
+ {namespace.rootPath || "(root)"}
+ </span>
+ </CommandItem>
+ {isExpanded && (
+ <>
+ {isPending && entries.length === 0 && (
+ <div
+ className="flex items-center gap-1.5 py-1 text-xs text-muted-foreground"
+ style={indentStyle(1)}
+ >
+ <LoaderCircle className="h-3 w-3 animate-spin" />
+ Loading...
+ </div>
+ )}
+ {error && entries.length === 0 && (
+ <div
+ className="py-1 text-xs text-destructive"
+ style={indentStyle(1)}
+ >
+ Failed to load entries: {error.message}
+ </div>
+ )}
+ {!isPending && entries.length === 0 && !error && (
+ <div
+ className="py-1 text-xs text-muted-foreground italic"
+ style={indentStyle(1)}
+ >
+ No entries
+ </div>
+ )}
+ {searchValue && filtered.length === 0 && entries.length > 0 && (
+ <div
+ className="py-1 text-xs text-muted-foreground italic"
+ style={indentStyle(1)}
+ >
+ No matches
+ </div>
+ )}
+ {filtered.map((entry) => (
+ <StorageEntryRow
+ key={entry.path}
+ entry={entry}
+ namespace={namespaceName}
+ protocol={namespace.protocol}
+ rootPath={namespace.rootPath}
+ depth={1}
+ locale={locale}
+ searchValue={searchValue}
+ />
+ ))}
+ </>
+ )}
+ </>
+ );
+};
+
+export const StorageInspector: React.FC = () => {
+ const { namespaces } = useStorage();
+ const { locale } = useLocale();
+ const [searchValue, setSearchValue] = useState("");
+ const hasSearch = !!searchValue.trim();
+
+ if (namespaces.length === 0) {
+ return (
+ <PanelEmptyState
+ title="No storage connected"
+ description={
+ <span>
+ Create an Obstore or fsspec connection in your notebook. See the{" "}
+ <a
+ className="text-link"
+ href="https://docs.marimo.io/guides/working_with_data/remote_storage/#quick-start"
+ target="_blank"
+ rel="noopener noreferrer"
+ >
+ docs
+ </a>
+ .
+ </span>
+ }
+ icon={<HardDriveIcon className="h-8 w-8" />}
+ />
+ );
+ }
+
+ return (
+ <Command
+ className="border-b bg-background rounded-none h-full pb-10 overflow-auto outline-hidden scrollbar-thin"
+ shouldFilter={false}
+ >
+ <div className="flex items-center w-full border-b">
+ <CommandInput
+ placeholder="Search entries..."
+ className="h-6 m-1"
+ value={searchValue}
+ onValueChange={setSearchValue}
+ rootClassName="flex-1 border-b-0"
+ />
+ {hasSearch && (
+ <Button
+ variant="text"
+ size="xs"
+ className="float-right border-none px-2 m-0 h-full"
+ onClick={() => setSearchValue("")}
+ >
+ <XIcon className="h-4 w-4" />
+ </Button>
+ )}
+ <Tooltip
+ content="Filters loaded entries only. Expand directories to include their contents in the search."
+ delayDuration={200}
+ >
+ <HelpCircleIcon className="h-3.5 w-3.5 mr-2 shrink-0 cursor-help text-muted-foreground hover:text-foreground" />
+ </Tooltip>
+ </div>
+ <CommandList className="flex flex-col">
+ {namespaces.map((ns) => (
+ <StorageNamespaceSection
+ key={ns.name ?? ns.displayName}
+ namespace={ns}
+ locale={locale}
+ searchValue={searchValue}
+ />
+ ))}
+ </CommandList>
+ </Command>
+ );
+};
diff --git a/frontend/src/core/config/feature-flag.tsx b/frontend/src/core/config/feature-flag.tsx
index 2cf08f0cca..c27a64eb2b 100644
--- a/frontend/src/core/config/feature-flag.tsx
+++ b/frontend/src/core/config/feature-flag.tsx
@@ -12,6 +12,7 @@ export interface ExperimentalFeatures {
cache_panel: boolean;
external_agents: boolean;
server_side_pdf_export: boolean;
+ storage_inspector: boolean;
// Add new feature flags here
}
@@ -22,6 +23,7 @@ const defaultValues: ExperimentalFeatures = {
cache_panel: false,
external_agents: import.meta.env.DEV,
server_side_pdf_export: true,
+ storage_inspector: false,
};
export function getFeatureFlag<T extends keyof ExperimentalFeatures>(
diff --git a/frontend/src/core/islands/bridge.ts b/frontend/src/core/islands/bridge.ts
index feb9538d1f..3bb1aaba72 100644
--- a/frontend/src/core/islands/bridge.ts
+++ b/frontend/src/core/islands/bridge.ts
@@ -200,6 +200,8 @@ export class IslandsPyodideBridge implements RunRequests, EditRequests {
invokeAiTool = throwNotImplemented;
clearCache = throwNotImplemented;
getCacheInfo = throwNotImplemented;
+ listStorageEntries = throwNotImplemented;
+ downloadStorage = throwNotImplemented;
// The kernel uses msgspec to parse control requests, which requires a 'type'
// field for discriminated union deserialization.
diff --git a/frontend/src/core/islands/main.ts b/frontend/src/core/islands/main.ts
index ad750b12d6..1729cd2f8b 100644
--- a/frontend/src/core/islands/main.ts
+++ b/frontend/src/core/islands/main.ts
@@ -127,6 +127,8 @@ export async function initialize() {
case "data-source-connections":
case "validate-sql-result":
case "storage-namespaces":
+ case "storage-entries":
+ case "storage-download-ready":
case "secret-keys-result":
case "startup-logs":
// Unsupported
diff --git a/frontend/src/core/network/requests-lazy.ts b/frontend/src/core/network/requests-lazy.ts
index 86bb7faf8f..5d4b3d34c1 100644
--- a/frontend/src/core/network/requests-lazy.ts
+++ b/frontend/src/core/network/requests-lazy.ts
@@ -113,6 +113,8 @@ const ACTIONS: Record<keyof AllRequests, Action> = {
previewSQLTableList: "waitForConnectionOpen",
previewDataSourceConnection: "waitForConnectionOpen",
validateSQL: "waitForConnectionOpen",
+ listStorageEntries: "waitForConnectionOpen",
+ downloadStorage: "waitForConnectionOpen",
// Sync operations that wait for connection
syncCellIds: "waitForConnectionOpen",
diff --git a/frontend/src/core/network/requests-network.ts b/frontend/src/core/network/requests-network.ts
index a26c16d7db..86fed0daeb 100644
--- a/frontend/src/core/network/requests-network.ts
+++ b/frontend/src/core/network/requests-network.ts
@@ -488,5 +488,21 @@ export function createNetworkRequests(): EditRequests & RunRequests {
})
.then(handleResponseReturnNull);
},
+ listStorageEntries: (request) => {
+ return getClient()
+ .POST("/api/storage/list_entries", {
+ body: request,
+ params: getParams(),
+ })
+ .then(handleResponseReturnNull);
+ },
+ downloadStorage: (request) => {
+ return getClient()
+ .POST("/api/storage/download", {
+ body: request,
+ params: getParams(),
+ })
+ .then(handleResponseReturnNull);
+ },
};
}
diff --git a/frontend/src/core/network/requests-static.ts b/frontend/src/core/network/requests-static.ts
index a82e971e0d..34844f4731 100644
--- a/frontend/src/core/network/requests-static.ts
+++ b/frontend/src/core/network/requests-static.ts
@@ -90,5 +90,7 @@ export function createStaticRequests(): EditRequests & RunRequests {
invokeAiTool: throwNotInEditMode,
clearCache: throwNotInEditMode,
getCacheInfo: throwNotInEditMode,
+ listStorageEntries: throwNotInEditMode,
+ downloadStorage: throwNotInEditMode,
};
}
diff --git a/frontend/src/core/network/requests-toasting.tsx b/frontend/src/core/network/requests-toasting.tsx
index 5cd32078bb..ddaeb31b3a 100644
--- a/frontend/src/core/network/requests-toasting.tsx
+++ b/frontend/src/core/network/requests-toasting.tsx
@@ -75,6 +75,8 @@ export function createErrorToastingRequests(
invokeAiTool: "Failed to invoke AI tool",
clearCache: "Failed to clear cache",
getCacheInfo: "", // No toast
+ listStorageEntries: "Failed to list storage entries",
+ downloadStorage: "Failed to download storage entry",
};
const handlers = {} as EditRequests & RunRequests;
diff --git a/frontend/src/core/network/types.ts b/frontend/src/core/network/types.ts
index 02415698dc..b0e5878e34 100644
--- a/frontend/src/core/network/types.ts
+++ b/frontend/src/core/network/types.ts
@@ -105,6 +105,9 @@ export type LspRestartRequest = schemas["LspRestartRequest"];
export type LspRestartResponse = schemas["LspRestartResponse"];
export type LspServerHealth = schemas["LspServerHealth"];
+export type StorageListEntriesRequest = schemas["StorageListEntriesRequest"];
+export type StorageDownloadRequest = schemas["StorageDownloadRequest"];
+
/**
* Requests sent to the BE during run/edit mode.
*/
@@ -198,6 +201,9 @@ export interface EditRequests {
// Cache requests
clearCache: () => Promise<null>;
getCacheInfo: () => Promise<null>;
+ // Storage requests
+ listStorageEntries: (request: StorageListEntriesRequest) => Promise<null>;
+ downloadStorage: (request: StorageDownloadRequest) => Promise<null>;
}
export type RequestKey = keyof (EditRequests & RunRequests);
diff --git a/frontend/src/core/storage/__tests__/state.test.ts b/frontend/src/core/storage/__tests__/state.test.ts
new file mode 100644
index 0000000000..d10e65b00d
--- /dev/null
+++ b/frontend/src/core/storage/__tests__/state.test.ts
@@ -0,0 +1,345 @@
+/* Copyright 2026 Marimo. All rights reserved. */
+import { beforeEach, describe, expect, it } from "vitest";
+import type { VariableName } from "../../variables/types";
+import { exportedForTesting } from "../state";
+import type { StorageEntry, StorageNamespace, StorageState } from "../types";
+
+const { initialState, reducer, createActions } = exportedForTesting;
+
+function makeNamespace(
+ overrides: Partial<StorageNamespace> & { name: string },
+): StorageNamespace {
+ return {
+ displayName: overrides.displayName ?? overrides.name,
+ name: overrides.name,
+ protocol: overrides.protocol ?? "s3",
+ rootPath: overrides.rootPath ?? "/",
+ storageEntries: overrides.storageEntries ?? [],
+ };
+}
+
+function makeEntry(
+ overrides: Partial<StorageEntry> & { path: string },
+): StorageEntry {
+ return {
+ kind: overrides.kind ?? "file",
+ lastModified: overrides.lastModified ?? null,
+ metadata: overrides.metadata ?? {},
+ path: overrides.path,
+ size: overrides.size ?? 0,
+ };
+}
+
+describe("storage state", () => {
+ let state: StorageState;
+
+ const actions = createActions((action) => {
+ state = reducer(state, action);
+ });
+
+ beforeEach(() => {
+ state = initialState();
+ });
+
+ describe("initialState", () => {
+ it("should start with empty namespaces and entries", () => {
+ expect(state).toEqual({
+ namespaces: [],
+ entriesByPath: new Map(),
+ });
+ });
+ });
+
+ describe("setNamespaces", () => {
+ it("should add namespaces from an empty state", () => {
+ const ns1 = makeNamespace({ name: "my_s3", protocol: "s3" });
+ const ns2 = makeNamespace({ name: "my_gcs", protocol: "gcs" });
+
+ actions.setNamespaces({ namespaces: [ns1, ns2] });
+
+ expect(state.namespaces).toEqual([ns1, ns2]);
+ });
+
+ it("should merge namespaces by name, replacing existing ones", () => {
+ const nsOld = makeNamespace({
+ name: "my_s3",
+ protocol: "s3",
+ rootPath: "/old",
+ });
+ const nsOther = makeNamespace({ name: "my_gcs", protocol: "gcs" });
+ actions.setNamespaces({ namespaces: [nsOld, nsOther] });
+
+ const nsUpdated = makeNamespace({
+ name: "my_s3",
+ protocol: "s3",
+ rootPath: "/new",
+ });
+ actions.setNamespaces({ namespaces: [nsUpdated] });
+
+ expect(state.namespaces).toHaveLength(2);
+ expect(state.namespaces).toEqual([nsUpdated, nsOther]);
+ });
+
+ it("should add new namespaces alongside existing ones", () => {
+ const ns1 = makeNamespace({ name: "ns1" });
+ actions.setNamespaces({ namespaces: [ns1] });
+
+ const ns2 = makeNamespace({ name: "ns2" });
+ actions.setNamespaces({ namespaces: [ns2] });
+
+ expect(state.namespaces).toHaveLength(2);
+ expect(state.namespaces).toEqual([ns1, ns2]);
+ });
+
+ it("should not affect entriesByPath", () => {
+ const entry = makeEntry({ path: "file.txt" });
+ actions.setEntries({
+ namespace: "ns",
+ prefix: null,
+ entries: [entry],
+ });
+
+ actions.setNamespaces({
+ namespaces: [makeNamespace({ name: "ns" })],
+ });
+
+ expect(state.entriesByPath.get("ns::")).toEqual([entry]);
+ });
+ });
+
+ describe("setEntries", () => {
+ it("should store entries keyed by namespace and prefix", () => {
+ const entries = [
+ makeEntry({ path: "a.txt", size: 100 }),
+ makeEntry({ path: "b.txt", size: 200 }),
+ ];
+
+ actions.setEntries({
+ namespace: "my_s3",
+ prefix: "data/",
+ entries,
+ });
+
+ expect(state.entriesByPath.get("my_s3::data/")).toEqual(entries);
+ });
+
+ it("should use empty string for null prefix", () => {
+ const entries = [makeEntry({ path: "root.txt" })];
+
+ actions.setEntries({
+ namespace: "my_s3",
+ prefix: null,
+ entries,
+ });
+
+ expect(state.entriesByPath.get("my_s3::")).toEqual(entries);
+ });
+
+ it("should use empty string for undefined prefix", () => {
+ const entries = [makeEntry({ path: "root.txt" })];
+
+ actions.setEntries({
+ namespace: "my_s3",
+ prefix: undefined,
+ entries,
+ });
+
+ expect(state.entriesByPath.get("my_s3::")).toEqual(entries);
+ });
+
+ it("should overwrite entries for the same namespace and prefix", () => {
+ const oldEntries = [makeEntry({ path: "old.txt" })];
+ const newEntries = [makeEntry({ path: "new.txt" })];
+
+ actions.setEntries({
+ namespace: "ns",
+ prefix: "p/",
+ entries: oldEntries,
+ });
+ actions.setEntries({
+ namespace: "ns",
+ prefix: "p/",
+ entries: newEntries,
+ });
+
+ expect(state.entriesByPath.get("ns::p/")).toEqual(newEntries);
+ });
+
+ it("should store entries for different namespaces independently", () => {
+ const entriesA = [makeEntry({ path: "a.txt" })];
+ const entriesB = [makeEntry({ path: "b.txt" })];
+
+ actions.setEntries({
+ namespace: "ns_a",
+ prefix: null,
+ entries: entriesA,
+ });
+ actions.setEntries({
+ namespace: "ns_b",
+ prefix: null,
+ entries: entriesB,
+ });
+
+ expect(state.entriesByPath.get("ns_a::")).toEqual(entriesA);
+ expect(state.entriesByPath.get("ns_b::")).toEqual(entriesB);
+ });
+
+ it("should store entries for different prefixes independently", () => {
+ const entriesRoot = [makeEntry({ path: "root.txt" })];
+ const entriesSub = [makeEntry({ path: "sub/file.txt" })];
+
+ actions.setEntries({
+ namespace: "ns",
+ prefix: null,
+ entries: entriesRoot,
+ });
+ actions.setEntries({
+ namespace: "ns",
+ prefix: "sub/",
+ entries: entriesSub,
+ });
+
+ expect(state.entriesByPath.get("ns::")).toEqual(entriesRoot);
+ expect(state.entriesByPath.get("ns::sub/")).toEqual(entriesSub);
+ });
+
+ it("should not affect namespaces", () => {
+ const ns = makeNamespace({ name: "ns" });
+ actions.setNamespaces({ namespaces: [ns] });
+
+ actions.setEntries({
+ namespace: "ns",
+ prefix: null,
+ entries: [makeEntry({ path: "file.txt" })],
+ });
+
+ expect(state.namespaces).toEqual([ns]);
+ });
+ });
+
+ describe("clearNamespaceCache", () => {
+ it("should remove all entries for the given namespace", () => {
+ actions.setEntries({
+ namespace: "my_s3",
+ prefix: null,
+ entries: [makeEntry({ path: "root.txt" })],
+ });
+ actions.setEntries({
+ namespace: "my_s3",
+ prefix: "data/",
+ entries: [makeEntry({ path: "data/file.csv" })],
+ });
+ actions.setEntries({
+ namespace: "my_s3",
+ prefix: "data/nested/",
+ entries: [makeEntry({ path: "data/nested/deep.txt" })],
+ });
+
+ actions.clearNamespaceCache("my_s3");
+
+ expect(state.entriesByPath.get("my_s3::")).toBeUndefined();
+ expect(state.entriesByPath.get("my_s3::data/")).toBeUndefined();
+ expect(state.entriesByPath.get("my_s3::data/nested/")).toBeUndefined();
+ });
+
+ it("should not affect entries from other namespaces", () => {
+ const otherEntries = [makeEntry({ path: "other.txt" })];
+ actions.setEntries({
+ namespace: "my_s3",
+ prefix: null,
+ entries: [makeEntry({ path: "root.txt" })],
+ });
+ actions.setEntries({
+ namespace: "my_gcs",
+ prefix: null,
+ entries: otherEntries,
+ });
+
+ actions.clearNamespaceCache("my_s3");
+
+ expect(state.entriesByPath.get("my_s3::")).toBeUndefined();
+ expect(state.entriesByPath.get("my_gcs::")).toEqual(otherEntries);
+ });
+
+ it("should not affect namespaces", () => {
+ const ns = makeNamespace({ name: "my_s3" });
+ actions.setNamespaces({ namespaces: [ns] });
+ actions.setEntries({
+ namespace: "my_s3",
+ prefix: null,
+ entries: [makeEntry({ path: "file.txt" })],
+ });
+
+ actions.clearNamespaceCache("my_s3");
+
+ expect(state.namespaces).toEqual([ns]);
+ });
+
+ it("should be a no-op for a namespace with no cached entries", () => {
+ actions.setEntries({
+ namespace: "my_gcs",
+ prefix: null,
+ entries: [makeEntry({ path: "file.txt" })],
+ });
+
+ actions.clearNamespaceCache("nonexistent");
+
+ expect(state.entriesByPath.get("my_gcs::")).toEqual([
+ makeEntry({ path: "file.txt" }),
+ ]);
+ });
+ });
+
+ describe("filterFromVariables", () => {
+ it("should keep namespaces whose variable is still in scope", () => {
+ const ns1 = makeNamespace({ name: "var_a" });
+ const ns2 = makeNamespace({ name: "var_b" });
+ actions.setNamespaces({ namespaces: [ns1, ns2] });
+
+ actions.filterFromVariables([
+ "var_a" as VariableName,
+ "var_b" as VariableName,
+ ]);
+
+ expect(state.namespaces).toEqual([ns1, ns2]);
+ });
+
+ it("should remove namespaces whose variable is no longer in scope", () => {
+ const ns1 = makeNamespace({ name: "var_a" });
+ const ns2 = makeNamespace({ name: "var_b" });
+ actions.setNamespaces({ namespaces: [ns1, ns2] });
+
+ actions.filterFromVariables(["var_a" as VariableName]);
+
+ expect(state.namespaces).toEqual([ns1]);
+ });
+
+ it("should remove all named namespaces when given an empty variable list", () => {
+ const ns1 = makeNamespace({ name: "var_a" });
+ const ns2 = makeNamespace({ name: "var_b" });
+ actions.setNamespaces({ namespaces: [ns1, ns2] });
+
+ actions.filterFromVariables([]);
+
+ expect(state.namespaces).toEqual([]);
+ });
+
+ it("should not affect entriesByPath", () => {
+ const ns = makeNamespace({ name: "ns" });
+ actions.setNamespaces({ namespaces: [ns] });
+
+ const entry = makeEntry({ path: "file.txt" });
+ actions.setEntries({
+ namespace: "ns",
+ prefix: null,
+ entries: [entry],
+ });
+
+ actions.filterFromVariables([]);
+
+ // Namespace removed, but entries remain (they are keyed independently)
+ expect(state.namespaces).toEqual([]);
+ expect(state.entriesByPath.get("ns::")).toEqual([entry]);
+ });
+ });
+});
diff --git a/frontend/src/core/storage/__tests__/types.test.ts b/frontend/src/core/storage/__tests__/types.test.ts
new file mode 100644
index 0000000000..61fe2b5e6c
--- /dev/null
+++ b/frontend/src/core/storage/__tests__/types.test.ts
@@ -0,0 +1,83 @@
+/* Copyright 2026 Marimo. All rights reserved. */
+import { describe, expect, it } from "vitest";
+import { storageNamespacePrefix, storagePathKey, storageUrl } from "../types";
+
+describe("storagePathKey", () => {
+ it("should join namespace and prefix", () => {
+ expect(storagePathKey("ns", "data/")).toBe("ns::data/");
+ });
+
+ it("should use empty string for null prefix", () => {
+ expect(storagePathKey("ns", null)).toBe("ns::");
+ });
+
+ it("should use empty string for undefined prefix", () => {
+ expect(storagePathKey("ns", undefined)).toBe("ns::");
+ });
+});
+
+describe("storageNamespacePrefix", () => {
+ it("should append separator", () => {
+ expect(storageNamespacePrefix("my_s3")).toBe("my_s3::");
+ });
+});
+
+describe("storageUrl", () => {
+ it("should combine protocol, rootPath, and entryPath", () => {
+ expect(storageUrl("s3", "my-bucket", "data/file.csv")).toEqual(
+ new URL("s3://my-bucket/data/file.csv"),
+ );
+ });
+
+ it("should handle empty rootPath", () => {
+ expect(storageUrl("s3", "", "marimo-artifacts/file.csv")).toEqual(
+ new URL("s3://marimo-artifacts/file.csv"),
+ );
+ });
+
+ it("should handle rootPath with trailing slash", () => {
+ expect(storageUrl("s3", "my-bucket/", "data/file.csv")).toEqual(
+ new URL("s3://my-bucket/data/file.csv"),
+ );
+ });
+
+ it("should handle entryPath with leading slash", () => {
+ expect(storageUrl("s3", "my-bucket", "/data/file.csv")).toEqual(
+ new URL("s3://my-bucket/data/file.csv"),
+ );
+ });
+
+ it("should collapse multiple slashes between rootPath and entryPath", () => {
+ expect(storageUrl("s3", "my-bucket/", "/data/file.csv")).toEqual(
+ new URL("s3://my-bucket/data/file.csv"),
+ );
+ });
+
+ it("should handle directory paths with trailing slash", () => {
+ expect(storageUrl("gcs", "my-bucket", "data/")).toEqual(
+ new URL("gcs://my-bucket/data/"),
+ );
+ });
+
+ it("should handle empty entryPath", () => {
+ expect(storageUrl("s3", "my-bucket", "")).toEqual(
+ new URL("s3://my-bucket"),
+ );
+ });
+
+ it("should handle both rootPath and entryPath empty", () => {
+ expect(storageUrl("s3", "", "")).toEqual(new URL("s3://"));
+ });
+
+ it("should work with file protocol", () => {
+ expect(storageUrl("file", "/home/user", "docs/readme.md")).toEqual(
+ new URL("file:///home/user/docs/readme.md"),
+ );
+ });
+
+ it("should work with nested rootPath", () => {
+ expect(storageUrl("s3", "bucket/prefix", "sub/file.txt")).toEqual(
+ new URL("s3://bucket/prefix/sub/file.txt"),
+ );
+ });
+});
diff --git a/frontend/src/core/storage/__tests__/useStorageEntries.test.tsx b/frontend/src/core/storage/__tests__/useStorageEntries.test.tsx
new file mode 100644
index 0000000000..29e60fdafb
--- /dev/null
+++ b/frontend/src/core/storage/__tests__/useStorageEntries.test.tsx
@@ -0,0 +1,168 @@
+/* Copyright 2026 Marimo. All rights reserved. */
+
+import { renderHook, waitFor } from "@testing-library/react";
+import { createStore, Provider } from "jotai";
+import type { ReactNode } from "react";
+import * as React from "react";
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import { storageAtom, useStorageEntries } from "../state";
+import type { StorageEntry, StorageState } from "../types";
+
+const mockRequest = vi.fn();
+
+vi.mock("../request-registry", () => ({
+ ListStorageEntries: {
+ request: (...args: unknown[]) => mockRequest(...args),
+ },
+}));
+
+function makeEntry(
+ overrides: Partial<StorageEntry> & { path: string },
+): StorageEntry {
+ return {
+ kind: overrides.kind ?? "file",
+ lastModified: overrides.lastModified ?? null,
+ metadata: overrides.metadata ?? {},
+ path: overrides.path,
+ size: overrides.size ?? 0,
+ };
+}
+
+describe("useStorageEntries", () => {
+ let store: ReturnType<typeof createStore>;
+
+ const wrapper = ({ children }: { children: ReactNode }) =>
+ React.createElement(Provider, { store }, children);
+
+ beforeEach(() => {
+ vi.clearAllMocks();
+ store = createStore();
+ });
+
+ afterEach(() => {
+ vi.restoreAllMocks();
+ });
+
+ function setStorageState(state: Partial<StorageState>) {
+ const current = store.get(storageAtom);
+ store.set(storageAtom, { ...current, ...state });
+ }
+
+ it("should fetch entries when not cached", async () => {
+ const entries = [
+ makeEntry({ path: "a.txt" }),
+ makeEntry({ path: "b.txt" }),
+ ];
+ mockRequest.mockResolvedValue({ entries });
+
+ const { result } = renderHook(() => useStorageEntries("my_s3", "data/"), {
+ wrapper,
+ });
+
+ expect(result.current.isPending).toBe(true);
+ expect(result.current.entries).toEqual([]);
+
+ await waitFor(() => {
+ expect(result.current.isPending).toBe(false);
+ });
+
+ expect(result.current.entries).toEqual(entries);
+ expect(mockRequest).toHaveBeenCalledWith({
+ namespace: "my_s3",
+ prefix: "data/",
+ limit: 150,
+ });
+ });
+
+ it("should return cached entries without fetching", async () => {
+ const entries = [makeEntry({ path: "cached.txt" })];
+ setStorageState({
+ entriesByPath: new Map([["my_s3::data/", entries]]),
+ });
+
+ const { result } = renderHook(() => useStorageEntries("my_s3", "data/"), {
+ wrapper,
+ });
+
+ expect(result.current.entries).toEqual(entries);
+ expect(result.current.isPending).toBe(false);
+ expect(result.current.error).toBeUndefined();
+ expect(mockRequest).not.toHaveBeenCalled();
+ });
+
+ it("should normalize null prefix to empty string", async () => {
+ const entries = [makeEntry({ path: "root.txt" })];
+ mockRequest.mockResolvedValue({ entries });
+
+ const { result } = renderHook(() => useStorageEntries("ns"), { wrapper });
+
+ await waitFor(() => {
+ expect(result.current.isPending).toBe(false);
+ });
+
+ expect(result.current.entries).toEqual(entries);
+ expect(mockRequest).toHaveBeenCalledWith(
+ expect.objectContaining({ prefix: "" }),
+ );
+ });
+
+ it("should normalize undefined prefix to empty string", async () => {
+ const entries = [makeEntry({ path: "root.txt" })];
+ mockRequest.mockResolvedValue({ entries });
+
+ const { result } = renderHook(() => useStorageEntries("ns", undefined), {
+ wrapper,
+ });
+
+ await waitFor(() => {
+ expect(result.current.isPending).toBe(false);
+ });
+
+ expect(mockRequest).toHaveBeenCalledWith(
+ expect.objectContaining({ prefix: "" }),
+ );
+ });
+
+ it("should return the error when fetch fails and nothing is cached", async () => {
+ mockRequest.mockRejectedValue(new Error("network failure"));
+
+ const { result } = renderHook(() => useStorageEntries("ns", "pfx/"), {
+ wrapper,
+ });
+
+ await waitFor(() => {
+ expect(result.current.error).toBeDefined();
+ });
+
+ expect(result.current.error?.message).toBe("network failure");
+ expect(result.current.entries).toEqual([]);
+ expect(result.current.isPending).toBe(false);
+ });
+
+ it("should suppress error when entries are cached", async () => {
+ const entries = [makeEntry({ path: "ok.txt" })];
+ setStorageState({
+ entriesByPath: new Map([["ns::pfx/", entries]]),
+ });
+
+ const { result } = renderHook(() => useStorageEntries("ns", "pfx/"), {
+ wrapper,
+ });
+
+ expect(result.current.error).toBeUndefined();
+ expect(result.current.entries).toEqual(entries);
+ expect(result.current.isPending).toBe(false);
+ });
+
+ it("should store fetched entries in the atom", async () => {
+ const entries = [makeEntry({ path: "new.txt" })];
+ mockRequest.mockResolvedValue({ entries });
+
+ renderHook(() => useStorageEntries("ns", "sub/"), { wrapper });
+
+ await waitFor(() => {
+ const state = store.get(storageAtom);
+ expect(state.entriesByPath.get("ns::sub/")).toEqual(entries);
+ });
+ });
+});
diff --git a/frontend/src/core/storage/request-registry.ts b/frontend/src/core/storage/request-registry.ts
new file mode 100644
index 0000000000..50f9faf571
--- /dev/null
+++ b/frontend/src/core/storage/request-registry.ts
@@ -0,0 +1,36 @@
+/* Copyright 2026 Marimo. All rights reserved. */
+
+import type { NotificationMessageData } from "../kernel/messages";
+import { DeferredRequestRegistry } from "../network/DeferredRequestRegistry";
+import { getRequestClient } from "../network/requests";
+import type {
+ StorageDownloadRequest,
+ StorageListEntriesRequest,
+} from "../network/types";
+
+export type StorageEntriesResult = NotificationMessageData<"storage-entries">;
+
+export type StorageDownloadReadyResult =
+ NotificationMessageData<"storage-download-ready">;
+
+export const ListStorageEntries = new DeferredRequestRegistry<
+ Omit<StorageListEntriesRequest, "requestId">,
+ StorageEntriesResult
+>("storage-list-entries", async (requestId, req) => {
+ const client = getRequestClient();
+ await client.listStorageEntries({
+ requestId: requestId,
+ ...req,
+ });
+});
+
+export const DownloadStorage = new DeferredRequestRegistry<
+ Omit<StorageDownloadRequest, "requestId">,
+ StorageDownloadReadyResult
+>("storage-download", async (requestId, req) => {
+ const client = getRequestClient();
+ await client.downloadStorage({
+ requestId: requestId,
+ ...req,
+ });
+});
diff --git a/frontend/src/core/storage/state.ts b/frontend/src/core/storage/state.ts
new file mode 100644
index 0000000000..780c63183b
--- /dev/null
+++ b/frontend/src/core/storage/state.ts
@@ -0,0 +1,129 @@
+/* Copyright 2026 Marimo. All rights reserved. */
+
+import { atom, useAtomValue } from "jotai";
+import { useAsyncData } from "@/hooks/useAsyncData";
+import { createReducerAndAtoms } from "@/utils/createReducer";
+import type { NotificationMessageData } from "../kernel/messages";
+import type { VariableName } from "../variables/types";
+import { ListStorageEntries } from "./request-registry";
+import type { StorageEntry, StorageState } from "./types";
+import {
+ DEFAULT_FETCH_LIMIT,
+ ROOT_PATH,
+ storageNamespacePrefix,
+ storagePathKey,
+} from "./types";
+
+function initialState(): StorageState {
+ return {
+ namespaces: [],
+ entriesByPath: new Map(),
+ };
+}
+
+const {
+ reducer,
+ createActions,
+ valueAtom: storageAtom,
+ useActions,
+} = createReducerAndAtoms(initialState, {
+ setNamespaces: (
+ state,
+ data: NotificationMessageData<"storage-namespaces">,
+ ) => {
+ // Merge/replace namespaces by name
+ const existingMap = new Map(state.namespaces.map((ns) => [ns.name, ns]));
+ for (const ns of data.namespaces) {
+ existingMap.set(ns.name, ns);
+ }
+ return {
+ ...state,
+ namespaces: [...existingMap.values()],
+ };
+ },
+
+ setEntries: (
+ state,
+ opts: {
+ namespace: string;
+ prefix: string | null | undefined;
+ entries: StorageEntry[];
+ },
+ ) => {
+ const key = storagePathKey(opts.namespace, opts.prefix);
+ const entriesByPath = new Map(state.entriesByPath);
+ entriesByPath.set(key, opts.entries);
+ return { ...state, entriesByPath };
+ },
+
+ clearNamespaceCache: (state, namespace: string) => {
+ const entriesByPath = new Map(state.entriesByPath);
+ const prefix = storageNamespacePrefix(namespace);
+ for (const key of entriesByPath.keys()) {
+ if (key.startsWith(prefix)) {
+ entriesByPath.delete(key);
+ }
+ }
+ return { ...state, entriesByPath };
+ },
+
+ filterFromVariables: (state, variableNames: VariableName[]) => {
+ const names = new Set(variableNames);
+ // Filter out namespaces whose backing variable is no longer in scope
+ const namespaces = state.namespaces.filter((ns) => {
+ return names.has(ns.name as VariableName);
+ });
+ return { ...state, namespaces };
+ },
+});
+
+/**
+ * React hook to get the storage state.
+ */
+export const useStorage = () => useAtomValue(storageAtom);
+
+export const storageNamespacesAtom = atom((get) => get(storageAtom).namespaces);
+
+/**
+ * React hook to get the storage actions.
+ */
+export function useStorageActions() {
+ return useActions();
+}
+
+export { storageAtom };
+
+/**
+ * Hook that fetches and caches storage entries for a given namespace/prefix.
+ * Entries are fetched on first access and cached in the store for subsequent renders.
+ */
+export function useStorageEntries(namespace: string, prefix?: string) {
+ const { entriesByPath } = useStorage();
+ const { setEntries } = useStorageActions();
+ const cached = entriesByPath.get(storagePathKey(namespace, prefix));
+
+ const { isPending, error, refetch } = useAsyncData(async () => {
+ if (cached) {
+ return;
+ }
+ const result = await ListStorageEntries.request({
+ namespace,
+ prefix: prefix ?? ROOT_PATH,
+ limit: DEFAULT_FETCH_LIMIT,
+ });
+ setEntries({ namespace, prefix, entries: result.entries });
+ }, [namespace, prefix, cached === undefined]);
+
+ return {
+ entries: cached ?? [],
+ isPending: isPending && !cached,
+ error: cached ? undefined : error,
+ refetch,
+ };
+}
+
+export const exportedForTesting = {
+ reducer,
+ createActions,
+ initialState,
+};
diff --git a/frontend/src/core/storage/types.ts b/frontend/src/core/storage/types.ts
new file mode 100644
index 0000000000..824ff68925
--- /dev/null
+++ b/frontend/src/core/storage/types.ts
@@ -0,0 +1,55 @@
+/* Copyright 2026 Marimo. All rights reserved. */
+import type { components } from "@marimo-team/marimo-api";
+
+export type StorageEntry = components["schemas"]["StorageEntry"];
+export type StorageNamespace = components["schemas"]["StorageNamespace"];
+
+/**
+ * Key for looking up lazy-loaded entries: "namespace::prefix"
+ */
+export type StoragePathKey = `${string}::${string}`;
+
+export const STORAGE_PATH_SEPARATOR = "::";
+
+export function storagePathKey(
+ namespace: string,
+ prefix: string | null | undefined,
+): StoragePathKey {
+ return `${namespace}${STORAGE_PATH_SEPARATOR}${prefix ?? ""}`;
+}
+
+export function storageNamespacePrefix(namespace: string): string {
+ return `${namespace}${STORAGE_PATH_SEPARATOR}`;
+}
+
+/**
+ * The storage state.
+ */
+export interface StorageState {
+ /** Namespaces from the broadcast notification */
+ namespaces: StorageNamespace[];
+ /** Lazy-loaded entries keyed by "namespace::prefix" */
+ entriesByPath: ReadonlyMap<StoragePathKey, StorageEntry[]>;
+}
+
+export function storageUrl(
+ protocol: string,
+ rootPath: string,
+ entryPath: string,
+): URL {
+ const parts = [rootPath, entryPath].filter(Boolean);
+ const path = parts.join("/").replaceAll(/\/+/g, "/");
+ return new URL(`${protocol}://${path}`);
+}
+
+export const ROOT_PATH = "";
+export const DEFAULT_FETCH_LIMIT = 150;
+
+/** Non-exhaustive list of known storage protocols */
+export type KnownStorageProtocol =
+ | "s3"
+ | "gcs"
+ | "azure"
+ | "http"
+ | "file"
+ | "in-memory";
diff --git a/frontend/src/core/wasm/bridge.ts b/frontend/src/core/wasm/bridge.ts
index 5acd892d72..0d2e5e6151 100644
--- a/frontend/src/core/wasm/bridge.ts
+++ b/frontend/src/core/wasm/bridge.ts
@@ -596,6 +596,8 @@ export class PyodideBridge implements RunRequests, EditRequests {
invokeAiTool = throwNotImplemented;
clearCache = throwNotImplemented;
getCacheInfo = throwNotImplemented;
+ listStorageEntries = throwNotImplemented;
+ downloadStorage = throwNotImplemented;
private async putControlRequest(operation: CommandMessage) {
await this.rpc.proxy.request.bridge({
diff --git a/frontend/src/core/websocket/useMarimoKernelConnection.tsx b/frontend/src/core/websocket/useMarimoKernelConnection.tsx
index 92078b18f5..758e08f514 100644
--- a/frontend/src/core/websocket/useMarimoKernelConnection.tsx
+++ b/frontend/src/core/websocket/useMarimoKernelConnection.tsx
@@ -56,6 +56,11 @@ import type { RequestId } from "../network/DeferredRequestRegistry";
import { useRuntimeManager } from "../runtime/config";
import { SECRETS_REGISTRY } from "../secrets/request-registry";
import { isStaticNotebook } from "../static/static-state";
+import {
+ DownloadStorage,
+ ListStorageEntries,
+} from "../storage/request-registry";
+import { useStorageActions } from "../storage/state";
import { useVariablesActions } from "../variables/state";
import type { VariableName } from "../variables/types";
import { isWasm } from "../wasm/utils";
@@ -105,6 +110,10 @@ export function useMarimoKernelConnection(opts: {
const runtimeManager = useRuntimeManager();
const setCacheInfo = useSetAtom(cacheInfoAtom);
const setKernelStartupError = useSetAtom(kernelStartupErrorAtom);
+ const {
+ setNamespaces: setStorageNamespaces,
+ filterFromVariables: filterStorageFromVariables,
+ } = useStorageActions();
const handleMessage = (e: MessageEvent<JsonString<NotificationPayload>>) => {
const msg = jsonParseWithSpecialChar(e.data);
@@ -196,6 +205,9 @@ export function useMarimoKernelConnection(opts: {
filterDataSourcesFromVariables(
msg.data.variables.map((v) => v.name as VariableName),
);
+ filterStorageFromVariables(
+ msg.data.variables.map((v) => v.name as VariableName),
+ );
return;
case "variable-values":
setMetadata(
@@ -285,7 +297,13 @@ export function useMarimoKernelConnection(opts: {
});
return;
case "storage-namespaces":
- // TODO: Handle storage namespaces
+ setStorageNamespaces(msg.data);
+ return;
+ case "storage-entries":
+ ListStorageEntries.resolve(msg.data.request_id as RequestId, msg.data);
+ return;
+ case "storage-download-ready":
+ DownloadStorage.resolve(msg.data.request_id as RequestId, msg.data);
return;
case "reconnected":
diff --git a/marimo/_cli/development/commands.py b/marimo/_cli/development/commands.py
index 6baed3a6ce..f6abf3ad87 100644
--- a/marimo/_cli/development/commands.py
+++ b/marimo/_cli/development/commands.py
@@ -239,6 +239,8 @@ def _generate_server_api_schema() -> dict[str, Any]:
models.MCPStatusResponse,
models.PreviewDatasetColumnRequest,
models.PreviewSQLTableRequest,
+ models.StorageListEntriesRequest,
+ models.StorageDownloadRequest,
models.ReadCodeResponse,
models.RenameNotebookRequest,
models.ExecuteCellsRequest,
diff --git a/marimo/_data/_external_storage/get_storage.py b/marimo/_data/_external_storage/get_storage.py
index bec2529429..1449e76ec3 100644
--- a/marimo/_data/_external_storage/get_storage.py
+++ b/marimo/_data/_external_storage/get_storage.py
@@ -37,7 +37,7 @@ def storage_backend_to_storage_namespace(
"""
return StorageNamespace(
name=storage_backend.variable_name,
- display_name=storage_backend.variable_name or "",
+ display_name=storage_backend.display_name,
protocol=storage_backend.protocol,
root_path=storage_backend.root_path or "",
storage_entries=[],
diff --git a/marimo/_data/_external_storage/models.py b/marimo/_data/_external_storage/models.py
index 5f05df9e54..7f84cbf1a8 100644
--- a/marimo/_data/_external_storage/models.py
+++ b/marimo/_data/_external_storage/models.py
@@ -2,11 +2,13 @@
from __future__ import annotations
import abc
-from typing import Any, Generic, Literal, TypeVar
+from dataclasses import dataclass
+from typing import Any, Generic, Literal, TypeVar, get_args
import msgspec
from marimo._types.ids import VariableName
+from marimo._utils.assert_never import log_never
KNOWN_STORAGE_TYPES = Literal[
"s3", "gcs", "azure", "http", "file", "in-memory"
@@ -43,7 +45,7 @@ class StorageNamespace(msgspec.Struct, rename="camel"):
storage_entries: The storage entries in the storage namespace.
"""
- name: VariableName | None
+ name: VariableName
display_name: str
protocol: str
root_path: str
@@ -52,13 +54,27 @@ class StorageNamespace(msgspec.Struct, rename="camel"):
DEFAULT_FETCH_LIMIT = 100
+
+@dataclass
+class DownloadResult:
+ """Result of downloading a file from external storage.
+
+ Attributes:
+ file_bytes: The raw bytes of the downloaded file.
+ filename: The suggested filename extracted from the path.
+ ext: The file extension (without dot), or "bin" if none.
+ """
+
+ file_bytes: bytes
+ filename: str
+ ext: str
+
+
Backend = TypeVar("Backend")
class StorageBackend(abc.ABC, Generic[Backend]):
- def __init__(
- self, store: Backend, variable_name: VariableName | None
- ) -> None:
+ def __init__(self, store: Backend, variable_name: VariableName) -> None:
self.store = store
self.variable_name = variable_name
@@ -82,11 +98,49 @@ class StorageBackend(abc.ABC, Generic[Backend]):
async def download(self, path: str) -> bytes:
"""Download the file at the given path."""
+ async def download_file(self, path: str) -> DownloadResult:
+ """Download the file at the given path with extracted metadata.
+
+ Calls download() and extracts the filename and extension from the path.
+ Subclasses can override this to customize filename extraction
+ (e.g., using content-disposition headers).
+ """
+ file_bytes = await self.download(path)
+ filename = path.rsplit("/", 1)[-1] or "download"
+ _, sep, suffix = filename.rpartition(".")
+ ext = suffix if sep and suffix else "bin"
+ return DownloadResult(
+ file_bytes=file_bytes,
+ filename=filename,
+ ext=ext,
+ )
+
@property
@abc.abstractmethod
def protocol(self) -> KNOWN_STORAGE_TYPES | str:
"""Return the protocol of the storage backend."""
+ @property
+ def display_name(self) -> str:
+ protocol = self.protocol
+ if protocol not in get_args(KNOWN_STORAGE_TYPES):
+ return protocol.capitalize()
+ if protocol == "s3":
+ return "Amazon S3"
+ elif protocol == "gcs":
+ return "Google Cloud Storage"
+ elif protocol == "azure":
+ return "Azure Blob Storage"
+ elif protocol == "http":
+ return "HTTP"
+ elif protocol == "file":
+ return "File"
+ elif protocol == "in-memory":
+ return "In-memory"
+ else:
+ log_never(protocol) # type: ignore[arg-type]
+ return protocol
+
@property
@abc.abstractmethod
def root_path(self) -> str | None:
diff --git a/marimo/_messaging/notification.py b/marimo/_messaging/notification.py
index 74e5860754..6fbe61cc97 100644
--- a/marimo/_messaging/notification.py
+++ b/marimo/_messaging/notification.py
@@ -17,7 +17,10 @@ import msgspec
from marimo import _loggers as loggers
from marimo._ast.app_config import _AppConfig
from marimo._ast.cell import CellConfig, RuntimeStateType
-from marimo._data._external_storage.models import StorageNamespace
+from marimo._data._external_storage.models import (
+ StorageEntry,
+ StorageNamespace,
+)
from marimo._data.models import (
ColumnStats,
DataSourceConnection,
@@ -589,6 +592,46 @@ class StorageNamespacesNotification(Notification, tag="storage-namespaces"):
namespaces: list[StorageNamespace]
+class StorageEntriesNotification(Notification, tag="storage-entries"):
+ """Result of a storage operation that returns entries.
+
+ Attributes:
+ request_id: Request ID this responds to.
+ entries: Storage entries returned by the operation.
+ namespace: Variable name of the storage backend.
+ prefix: The prefix that was listed (set by list_entries).
+ query: The search query that was used (set by search).
+ error: Error message if the operation failed.
+ """
+
+ name: ClassVar[str] = "storage-entries"
+ request_id: RequestId
+ entries: list[StorageEntry]
+ namespace: str
+ prefix: Optional[str] = None
+ query: Optional[str] = None
+ error: Optional[str] = None
+
+
+class StorageDownloadReadyNotification(
+ Notification, tag="storage-download-ready"
+):
+ """Signals that a storage file download is ready as a virtual file.
+
+ Attributes:
+ request_id: Request ID this responds to.
+ url: Virtual file URL to download from.
+ filename: Suggested filename for the download.
+ error: Error message if the download failed.
+ """
+
+ name: ClassVar[str] = "storage-download-ready"
+ request_id: RequestId
+ url: Optional[str] = None
+ filename: Optional[str] = None
+ error: Optional[str] = None
+
+
class ValidateSQLResultNotification(Notification, tag="validate-sql-result"):
"""SQL query validation result.
@@ -770,6 +813,8 @@ NotificationMessage = Union[
ValidateSQLResultNotification,
# Storage
StorageNamespacesNotification,
+ StorageEntriesNotification,
+ StorageDownloadReadyNotification,
# Secrets
SecretKeysResultNotification,
# Cache
diff --git a/marimo/_runtime/commands.py b/marimo/_runtime/commands.py
index c33cbc1367..c48907c0dc 100644
--- a/marimo/_runtime/commands.py
+++ b/marimo/_runtime/commands.py
@@ -670,6 +670,42 @@ class ValidateSQLCommand(Command):
dialect: Optional[str] = None
+class StorageListEntriesCommand(Command):
+ """List storage entries at a prefix.
+
+ Navigates storage like a folder tree using delimiter-based listing.
+ Returns entries (files/objects) and virtual directories at one level.
+
+ Attributes:
+ request_id: Unique identifier for this request.
+ namespace: Variable name identifying the storage backend.
+ limit: Max entries to return.
+ prefix: Path prefix to list (None = root).
+ """
+
+ request_id: RequestId
+ namespace: str
+ limit: int
+ prefix: Optional[str] = None
+
+
+class StorageDownloadCommand(Command):
+ """Download a storage entry.
+
+ Downloads file bytes from storage and creates a virtual file
+ so the frontend can fetch the contents.
+
+ Attributes:
+ request_id: Unique identifier for this request.
+ namespace: Variable name identifying the storage backend.
+ path: Full path of the entry to download.
+ """
+
+ request_id: RequestId
+ namespace: str
+ path: str
+
+
class ListSecretKeysCommand(Command):
"""List available secret keys.
@@ -812,6 +848,9 @@ CommandMessage = Union[
ListSQLTablesCommand,
ValidateSQLCommand,
ListDataSourceConnectionCommand,
+ # Storage operations
+ StorageListEntriesCommand,
+ StorageDownloadCommand,
# Secrets management
ListSecretKeysCommand,
RefreshSecretsCommand,
diff --git a/marimo/_runtime/runtime.py b/marimo/_runtime/runtime.py
index 79279bf230..06d8be6fb1 100644
--- a/marimo/_runtime/runtime.py
+++ b/marimo/_runtime/runtime.py
@@ -35,6 +35,7 @@ from marimo._ast.variables import BUILTINS, is_local
from marimo._ast.visitor import ImportData, Name, VariableData
from marimo._config.config import ExecutionType, MarimoConfig, OnCellChangeType
from marimo._config.settings import GLOBAL_SETTINGS
+from marimo._data._external_storage.models import StorageBackend, StorageEntry
from marimo._data.preview_column import (
get_column_preview_for_dataframe,
get_column_preview_for_duckdb,
@@ -69,6 +70,8 @@ from marimo._messaging.notification import (
SQLMetadata,
SQLTableListPreviewNotification,
SQLTablePreviewNotification,
+ StorageDownloadReadyNotification,
+ StorageEntriesNotification,
ValidateSQLResultNotification,
VariableDeclarationNotification,
VariablesNotification,
@@ -127,6 +130,8 @@ from marimo._runtime.commands import (
RefreshSecretsCommand,
RenameNotebookCommand,
StopKernelCommand,
+ StorageDownloadCommand,
+ StorageListEntriesCommand,
SyncGraphCommand,
UpdateCellConfigCommand,
UpdateUIElementCommand,
@@ -533,6 +538,7 @@ class Kernel:
self.packages_callbacks = PackagesCallbacks(self)
self.sql_callbacks = SqlCallbacks(self)
self.cache_callbacks = CacheCallbacks(self)
+ self.external_storage_callbacks = ExternalStorageCallbacks(self)
# Apply pythonpath from config at initialization
pythonpath = user_config["runtime"].get("pythonpath")
@@ -2346,6 +2352,14 @@ class Kernel:
)
# SQL
handler.register(ValidateSQLCommand, self.sql_callbacks.validate_sql)
+ # External storage
+ handler.register(
+ StorageListEntriesCommand,
+ self.external_storage_callbacks.list_entries,
+ )
+ handler.register(
+ StorageDownloadCommand, self.external_storage_callbacks.download
+ )
# Secrets
handler.register(
ListSecretKeysCommand, self.secrets_callbacks.list_secrets
@@ -2648,6 +2662,127 @@ class DatasetCallbacks:
)
+class ExternalStorageCallbacks:
+ def __init__(self, kernel: Kernel):
+ self._kernel = kernel
+
+ def _get_storage_backend(
+ self, namespace: str
+ ) -> tuple[StorageBackend[Any] | None, str | None]:
+ """Look up a storage backend by variable name from kernel globals.
+
+ Returns (backend, error). If there is error, backend is None.
+ """
+ from marimo._data._external_storage.get_storage import STORAGE_BACKENDS
+
+ variable_name = VariableName(namespace)
+ if variable_name not in self._kernel.globals:
+ return None, f"Variable '{namespace}' not found"
+
+ var = self._kernel.globals[variable_name]
+
+ for backend in STORAGE_BACKENDS:
+ if backend.is_compatible(var):
+ return backend(var, variable_name), None
+ return None, (
+ f"Variable '{namespace}' is not a compatible "
+ "storage backend (expected obstore or fsspec)"
+ )
+
+ @kernel_tracer.start_as_current_span("storage_list_entries")
+ async def list_entries(self, request: StorageListEntriesCommand) -> None:
+ """List storage entries at a given prefix."""
+ backend, error = self._get_storage_backend(request.namespace)
+ if error is not None or backend is None:
+ broadcast_notification(
+ StorageEntriesNotification(
+ request_id=request.request_id,
+ entries=[],
+ namespace=request.namespace,
+ prefix=request.prefix,
+ error=error,
+ ),
+ )
+ return
+
+ # list_entries is synchronous, so we wrap it in asyncio.to_thread
+ def list_entries() -> list[StorageEntry]:
+ return backend.list_entries(
+ prefix=request.prefix, limit=request.limit
+ )
+
+ try:
+ entries = await asyncio.to_thread(list_entries)
+ broadcast_notification(
+ StorageEntriesNotification(
+ request_id=request.request_id,
+ entries=entries,
+ namespace=request.namespace,
+ prefix=request.prefix,
+ ),
+ )
+ except Exception as e:
+ LOGGER.exception(
+ "Failed to list entries for %s at prefix %s",
+ request.namespace,
+ request.prefix,
+ )
+ broadcast_notification(
+ StorageEntriesNotification(
+ request_id=request.request_id,
+ entries=[],
+ namespace=request.namespace,
+ prefix=request.prefix,
+ error=f"Failed to list entries: {e}",
+ ),
+ )
+
+ @kernel_tracer.start_as_current_span("storage_download")
+ async def download(self, request: StorageDownloadCommand) -> None:
+ """Download a storage entry and create a virtual file."""
+ backend, error = self._get_storage_backend(request.namespace)
+ if error is not None or backend is None:
+ broadcast_notification(
+ StorageDownloadReadyNotification(
+ request_id=request.request_id,
+ url=None,
+ filename=None,
+ error=error,
+ ),
+ )
+ return
+
+ try:
+ from marimo._runtime.virtual_file.virtual_file import VirtualFile
+
+ result = await backend.download_file(request.path)
+ vfile = VirtualFile.create_and_register(
+ result.file_bytes, result.ext
+ )
+
+ broadcast_notification(
+ StorageDownloadReadyNotification(
+ request_id=request.request_id,
+ url=vfile.url,
+ filename=result.filename,
+ ),
+ )
+ except Exception as e:
+ LOGGER.exception(
+ "Failed to download %s from %s",
+ request.path,
+ request.namespace,
+ )
+ broadcast_notification(
+ StorageDownloadReadyNotification(
+ request_id=request.request_id,
+ url=None,
+ filename=None,
+ error=f"Failed to download: {e}",
+ ),
+ )
+
+
class SqlCallbacks:
def __init__(self, kernel: Kernel):
self._kernel = kernel
diff --git a/marimo/_runtime/virtual_file/virtual_file.py b/marimo/_runtime/virtual_file/virtual_file.py
index 149f2d9f9e..f2420b5e7f 100644
--- a/marimo/_runtime/virtual_file/virtual_file.py
+++ b/marimo/_runtime/virtual_file/virtual_file.py
@@ -79,6 +79,39 @@ class VirtualFile:
url=url,
)
+ @staticmethod
+ def create_and_register(buffer: bytes, ext: str) -> VirtualFile:
+ """Create a virtual file and register it in the current context.
+
+ Falls back to a data URL if no runtime context is available,
+ virtual files aren't supported, or the buffer is empty.
+ """
+ from marimo._runtime.context import get_context
+
+ vfile_name = random_filename(ext)
+
+ def return_data_url() -> VirtualFile:
+ return VirtualFile(
+ filename=vfile_name, buffer=buffer, as_data_url=True
+ )
+
+ # Empty buffers can't be served via the file registry, so use a
+ # data URL instead to ensure the URL is always resolvable.
+ if len(buffer) == 0:
+ return return_data_url()
+
+ try:
+ ctx = get_context()
+ except ContextNotInitializedError:
+ return return_data_url()
+
+ if not ctx.virtual_files_supported:
+ return return_data_url()
+
+ vfile = VirtualFile(filename=vfile_name, buffer=buffer)
+ ctx.virtual_file_registry.add(vfile, ctx)
+ return vfile
+
EMPTY_VIRTUAL_FILE = VirtualFile(
filename="empty.txt",
diff --git a/marimo/_server/api/endpoints/storage.py b/marimo/_server/api/endpoints/storage.py
new file mode 100644
index 0000000000..a519f6d685
--- /dev/null
+++ b/marimo/_server/api/endpoints/storage.py
@@ -0,0 +1,74 @@
+# Copyright 2026 Marimo. All rights reserved.
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+from starlette.authentication import requires
+
+from marimo import _loggers
+from marimo._server.api.utils import dispatch_control_request
+from marimo._server.models.models import (
+ BaseResponse,
+ StorageDownloadRequest,
+ StorageListEntriesRequest,
+)
+from marimo._server.router import APIRouter
+
+if TYPE_CHECKING:
+ from starlette.requests import Request
+
+LOGGER = _loggers.marimo_logger()
+
+router = APIRouter()
+
+
[email protected]("/list_entries")
+@requires("edit")
+async def list_entries(request: Request) -> BaseResponse:
+ """
+ parameters:
+ - in: header
+ name: Marimo-Session-Id
+ schema:
+ type: string
+ required: true
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: "#/components/schemas/StorageListEntriesRequest"
+ responses:
+ 200:
+ description: List storage entries at a prefix
+ content:
+ application/json:
+ schema:
+ $ref: "#/components/schemas/SuccessResponse"
+ """
+ return await dispatch_control_request(request, StorageListEntriesRequest)
+
+
[email protected]("/download")
+@requires("edit")
+async def download(request: Request) -> BaseResponse:
+ """
+ parameters:
+ - in: header
+ name: Marimo-Session-Id
+ schema:
+ type: string
+ required: true
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: "#/components/schemas/StorageDownloadRequest"
+ responses:
+ 200:
+ description: Download a storage entry
+ content:
+ application/json:
+ schema:
+ $ref: "#/components/schemas/SuccessResponse"
+ """
+ return await dispatch_control_request(request, StorageDownloadRequest)
diff --git a/marimo/_server/api/router.py b/marimo/_server/api/router.py
index 6abed452e4..9318f58acc 100644
--- a/marimo/_server/api/router.py
+++ b/marimo/_server/api/router.py
@@ -28,6 +28,7 @@ from marimo._server.api.endpoints.mpl import router as mpl_router
from marimo._server.api.endpoints.packages import router as packages_router
from marimo._server.api.endpoints.secrets import router as secrets_router
from marimo._server.api.endpoints.sql import router as sql_router
+from marimo._server.api.endpoints.storage import router as storage_router
from marimo._server.api.endpoints.terminal import router as terminal_router
from marimo._server.api.endpoints.ws_endpoint import router as ws_router
from marimo._server.router import APIRouter
@@ -63,6 +64,9 @@ def build_routes(base_url: str = "") -> list[BaseRoute]:
datasources_router, prefix="/api/datasources", name="datasources"
)
app_router.include_router(sql_router, prefix="/api/sql", name="sql")
+ app_router.include_router(
+ storage_router, prefix="/api/storage", name="storage"
+ )
app_router.include_router(ai_router, prefix="/api/ai", name="ai")
app_router.include_router(home_router, prefix="/api/home", name="home")
app_router.include_router(login_router, prefix="/auth", name="auth")
diff --git a/marimo/_server/models/models.py b/marimo/_server/models/models.py
index a9b26914f2..f0d7b5d107 100644
--- a/marimo/_server/models/models.py
+++ b/marimo/_server/models/models.py
@@ -23,6 +23,8 @@ from marimo._runtime.commands import (
ModelCommand,
PreviewDatasetColumnCommand,
PreviewSQLTableCommand,
+ StorageDownloadCommand,
+ StorageListEntriesCommand,
UpdateCellConfigCommand,
UpdateUIElementCommand,
UpdateUserConfigCommand,
@@ -135,6 +137,25 @@ class ValidateSQLRequest(ValidateSQLCommand, tag=False):
)
+class StorageListEntriesRequest(StorageListEntriesCommand, tag=False):
+ def as_command(self) -> StorageListEntriesCommand:
+ return StorageListEntriesCommand(
+ request_id=self.request_id,
+ namespace=self.namespace,
+ limit=self.limit,
+ prefix=self.prefix,
+ )
+
+
+class StorageDownloadRequest(StorageDownloadCommand, tag=False):
+ def as_command(self) -> StorageDownloadCommand:
+ return StorageDownloadCommand(
+ request_id=self.request_id,
+ namespace=self.namespace,
+ path=self.path,
+ )
+
+
class UpdateUserConfigRequest(UpdateUserConfigCommand, tag=False):
def as_command(self) -> UpdateUserConfigCommand:
return UpdateUserConfigCommand(config=self.config)
diff --git a/marimo/_session/state/session_view.py b/marimo/_session/state/session_view.py
index 0b01e3dded..efd713e92c 100644
--- a/marimo/_session/state/session_view.py
+++ b/marimo/_session/state/session_view.py
@@ -23,6 +23,7 @@ from marimo._messaging.notification import (
SQLTableListPreviewNotification,
SQLTablePreviewNotification,
StartupLogsNotification,
+ StorageNamespacesNotification,
UIElementMessageNotification,
UpdateCellCodesNotification,
UpdateCellIdsNotification,
@@ -151,6 +152,10 @@ class SessionView:
self.data_connectors = DataSourceConnectionsNotification(
connections=[]
)
+ # The most recent external storage namespaces notification
+ self.external_storage_namespaces = StorageNamespacesNotification(
+ namespaces=[]
+ )
# The most recent Variables notification.
self.variable_notifications: VariablesNotification = (
VariablesNotification(variables=[])
@@ -292,6 +297,16 @@ class SessionView:
connections=list(next_connections.values())
)
+ # Remove any external storage namespaces that are no longer in scope.
+ next_namespaces = [
+ ns
+ for ns in self.external_storage_namespaces.namespaces
+ if ns.name in variable_names
+ ]
+ self.external_storage_namespaces = StorageNamespacesNotification(
+ namespaces=next_namespaces
+ )
+
elif isinstance(notification, VariableValuesNotification):
for value in notification.variables:
self.variable_values[value.name] = value
@@ -327,6 +342,16 @@ class SessionView:
connections=list(connections.values())
)
+ elif isinstance(notification, StorageNamespacesNotification):
+ # Merge external storage namespaces, dedupe by name and keep the latest
+ prev_namespaces = self.external_storage_namespaces.namespaces
+ namespaces_by_name = {ns.name: ns for ns in prev_namespaces}
+ for ns in notification.namespaces:
+ namespaces_by_name[ns.name] = ns
+ self.external_storage_namespaces = StorageNamespacesNotification(
+ namespaces=list(namespaces_by_name.values())
+ )
+
elif isinstance(notification, SQLTablePreviewNotification):
sql_table_preview = notification
sql_metadata = sql_table_preview.metadata
@@ -517,6 +542,8 @@ class SessionView:
all_notifications.append(self.datasets)
if self.data_connectors.connections:
all_notifications.append(self.data_connectors)
+ if self.external_storage_namespaces.namespaces:
+ all_notifications.append(self.external_storage_namespaces)
# Model messages must come before cell notifications to ensure
# the model exists before the view tries to use it.
diff --git a/mkdocs.yml b/mkdocs.yml
index 56cf52ce17..7780f214f0 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -85,11 +85,12 @@ nav:
- Running cells: guides/reactivity.md
- Interactive elements: guides/interactivity.md
- Visualize outputs: guides/outputs.md
- - SQL, dataframes, and plots:
- - SQL, dataframes, and plots: guides/working_with_data/index.md
+ - Working with data:
+ - Working with data: guides/working_with_data/index.md
- DataFrames: guides/working_with_data/dataframes.md
- SQL: guides/working_with_data/sql.md
- Plotting: guides/working_with_data/plotting.md
+ - Remote Storage: guides/working_with_data/remote_storage.md
- Migrate from Jupyter: guides/coming_from/jupyter.md
- Expensive notebooks: guides/expensive_notebooks.md
- Understanding errors:
diff --git a/packages/openapi/api.yaml b/packages/openapi/api.yaml
index ae1ce5422b..21df7d7195 100644
--- a/packages/openapi/api.yaml
+++ b/packages/openapi/api.yaml
@@ -2343,6 +2343,8 @@ components:
- $ref: '#/components/schemas/ListSQLTablesCommand'
- $ref: '#/components/schemas/ValidateSQLCommand'
- $ref: '#/components/schemas/ListDataSourceConnectionCommand'
+ - $ref: '#/components/schemas/StorageListEntriesCommand'
+ - $ref: '#/components/schemas/StorageDownloadCommand'
- $ref: '#/components/schemas/ListSecretKeysCommand'
- $ref: '#/components/schemas/RefreshSecretsCommand'
- $ref: '#/components/schemas/ClearCacheCommand'
@@ -2370,6 +2372,8 @@ components:
refresh-secrets: '#/components/schemas/RefreshSecretsCommand'
rename-notebook: '#/components/schemas/RenameNotebookCommand'
stop-kernel: '#/components/schemas/StopKernelCommand'
+ storage-download: '#/components/schemas/StorageDownloadCommand'
+ storage-list-entries: '#/components/schemas/StorageListEntriesCommand'
sync-graph: '#/components/schemas/SyncGraphCommand'
update-cell-config: '#/components/schemas/UpdateCellConfigCommand'
update-ui-element: '#/components/schemas/UpdateUIElementCommand'
@@ -2449,6 +2453,8 @@ components:
- $ref: '#/components/schemas/DataSourceConnectionsNotification'
- $ref: '#/components/schemas/ValidateSQLResultNotification'
- $ref: '#/components/schemas/StorageNamespacesNotification'
+ - $ref: '#/components/schemas/StorageEntriesNotification'
+ - $ref: '#/components/schemas/StorageDownloadReadyNotification'
- $ref: '#/components/schemas/SecretKeysResultNotification'
- $ref: '#/components/schemas/CacheClearedNotification'
- $ref: '#/components/schemas/CacheInfoNotification'
@@ -2487,6 +2493,8 @@ components:
sql-table-list-preview: '#/components/schemas/SQLTableListPreviewNotification'
sql-table-preview: '#/components/schemas/SQLTablePreviewNotification'
startup-logs: '#/components/schemas/StartupLogsNotification'
+ storage-download-ready: '#/components/schemas/StorageDownloadReadyNotification'
+ storage-entries: '#/components/schemas/StorageEntriesNotification'
storage-namespaces: '#/components/schemas/StorageNamespacesNotification'
update-cell-codes: '#/components/schemas/UpdateCellCodesNotification'
update-cell-ids: '#/components/schemas/UpdateCellIdsNotification'
@@ -4248,6 +4256,116 @@ components:
- type
title: StopKernelCommand
type: object
+ StorageDownloadCommand:
+ description: "Download a storage entry.\n\n Downloads file bytes from storage\
+ \ and creates a virtual file\n so the frontend can fetch the contents.\n\
+ \n Attributes:\n request_id: Unique identifier for this request.\n\
+ \ namespace: Variable name identifying the storage backend.\n \
+ \ path: Full path of the entry to download."
+ properties:
+ namespace:
+ type: string
+ path:
+ type: string
+ requestId:
+ type: string
+ type:
+ enum:
+ - storage-download
+ required:
+ - type
+ - requestId
+ - namespace
+ - path
+ title: StorageDownloadCommand
+ type: object
+ StorageDownloadReadyNotification:
+ description: "Signals that a storage file download is ready as a virtual file.\n\
+ \n Attributes:\n request_id: Request ID this responds to.\n \
+ \ url: Virtual file URL to download from.\n filename: Suggested\
+ \ filename for the download.\n error: Error message if the download\
+ \ failed."
+ properties:
+ error:
+ anyOf:
+ - type: string
+ - type: 'null'
+ default: null
+ filename:
+ anyOf:
+ - type: string
+ - type: 'null'
+ default: null
+ op:
+ enum:
+ - storage-download-ready
+ request_id:
+ type: string
+ url:
+ anyOf:
+ - type: string
+ - type: 'null'
+ default: null
+ required:
+ - op
+ - request_id
+ title: StorageDownloadReadyNotification
+ type: object
+ StorageDownloadRequest:
+ properties:
+ namespace:
+ type: string
+ path:
+ type: string
+ requestId:
+ type: string
+ required:
+ - requestId
+ - namespace
+ - path
+ title: StorageDownloadRequest
+ type: object
+ StorageEntriesNotification:
+ description: "Result of a storage operation that returns entries.\n\n Attributes:\n\
+ \ request_id: Request ID this responds to.\n entries: Storage\
+ \ entries returned by the operation.\n namespace: Variable name of\
+ \ the storage backend.\n prefix: The prefix that was listed (set by\
+ \ list_entries).\n query: The search query that was used (set by search).\n\
+ \ error: Error message if the operation failed."
+ properties:
+ entries:
+ items:
+ $ref: '#/components/schemas/StorageEntry'
+ type: array
+ error:
+ anyOf:
+ - type: string
+ - type: 'null'
+ default: null
+ namespace:
+ type: string
+ op:
+ enum:
+ - storage-entries
+ prefix:
+ anyOf:
+ - type: string
+ - type: 'null'
+ default: null
+ query:
+ anyOf:
+ - type: string
+ - type: 'null'
+ default: null
+ request_id:
+ type: string
+ required:
+ - op
+ - request_id
+ - entries
+ - namespace
+ title: StorageEntriesNotification
+ type: object
StorageEntry:
description: "A storage entry is a file, directory, or object for external storage\
\ systems\n\n Attributes:\n path: The path of the storage entry.\n\
@@ -4278,6 +4396,54 @@ components:
- lastModified
title: StorageEntry
type: object
+ StorageListEntriesCommand:
+ description: "List storage entries at a prefix.\n\n Navigates storage like\
+ \ a folder tree using delimiter-based listing.\n Returns entries (files/objects)\
+ \ and virtual directories at one level.\n\n Attributes:\n request_id:\
+ \ Unique identifier for this request.\n namespace: Variable name identifying\
+ \ the storage backend.\n limit: Max entries to return.\n prefix:\
+ \ Path prefix to list (None = root)."
+ properties:
+ limit:
+ type: integer
+ namespace:
+ type: string
+ prefix:
+ anyOf:
+ - type: string
+ - type: 'null'
+ default: null
+ requestId:
+ type: string
+ type:
+ enum:
+ - storage-list-entries
+ required:
+ - type
+ - requestId
+ - namespace
+ - limit
+ title: StorageListEntriesCommand
+ type: object
+ StorageListEntriesRequest:
+ properties:
+ limit:
+ type: integer
+ namespace:
+ type: string
+ prefix:
+ anyOf:
+ - type: string
+ - type: 'null'
+ default: null
+ requestId:
+ type: string
+ required:
+ - requestId
+ - namespace
+ - limit
+ title: StorageListEntriesRequest
+ type: object
StorageNamespace:
description: "Represents external storage systems (filesystems and object storage)\n\
\n Attributes:\n name: The variable name of the storage namespace.\n\
@@ -4289,9 +4455,7 @@ components:
displayName:
type: string
name:
- anyOf:
- - type: string
- - type: 'null'
+ type: string
protocol:
type: string
rootPath:
@@ -6195,6 +6359,46 @@ paths:
type: integer
type: object
description: Get the number of active websocket connections
+ /api/storage/download:
+ post:
+ parameters:
+ - in: header
+ name: Marimo-Session-Id
+ required: true
+ schema:
+ type: string
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/StorageDownloadRequest'
+ responses:
+ 200:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/SuccessResponse'
+ description: Download a storage entry
+ /api/storage/list_entries:
+ post:
+ parameters:
+ - in: header
+ name: Marimo-Session-Id
+ required: true
+ schema:
+ type: string
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/StorageListEntriesRequest'
+ responses:
+ 200:
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/SuccessResponse'
+ description: List storage entries at a prefix
/api/usage:
get:
responses:
diff --git a/packages/openapi/src/api.ts b/packages/openapi/src/api.ts
index 00dfb9f93c..608f12cb54 100644
--- a/packages/openapi/src/api.ts
+++ b/packages/openapi/src/api.ts
@@ -2893,6 +2893,88 @@ export interface paths {
patch?: never;
trace?: never;
};
+ "/api/storage/download": {
+ parameters: {
+ query?: never;
+ header?: never;
+ path?: never;
+ cookie?: never;
+ };
+ get?: never;
+ put?: never;
+ post: {
+ parameters: {
+ query?: never;
+ header: {
+ "Marimo-Session-Id": string;
+ };
+ path?: never;
+ cookie?: never;
+ };
+ requestBody?: {
+ content: {
+ "application/json": components["schemas"]["StorageDownloadRequest"];
+ };
+ };
+ responses: {
+ /** @description Download a storage entry */
+ 200: {
+ headers: {
+ [name: string]: unknown;
+ };
+ content: {
+ "application/json": components["schemas"]["SuccessResponse"];
+ };
+ };
+ };
+ };
+ delete?: never;
+ options?: never;
+ head?: never;
+ patch?: never;
+ trace?: never;
+ };
+ "/api/storage/list_entries": {
+ parameters: {
+ query?: never;
+ header?: never;
+ path?: never;
+ cookie?: never;
+ };
+ get?: never;
+ put?: never;
+ post: {
+ parameters: {
+ query?: never;
+ header: {
+ "Marimo-Session-Id": string;
+ };
+ path?: never;
+ cookie?: never;
+ };
+ requestBody?: {
+ content: {
+ "application/json": components["schemas"]["StorageListEntriesRequest"];
+ };
+ };
+ responses: {
+ /** @description List storage entries at a prefix */
+ 200: {
+ headers: {
+ [name: string]: unknown;
+ };
+ content: {
+ "application/json": components["schemas"]["SuccessResponse"];
+ };
+ };
+ };
+ };
+ delete?: never;
+ options?: never;
+ head?: never;
+ patch?: never;
+ trace?: never;
+ };
"/api/usage": {
parameters: {
query?: never;
@@ -4491,6 +4573,8 @@ export interface components {
| components["schemas"]["ListSQLTablesCommand"]
| components["schemas"]["ValidateSQLCommand"]
| components["schemas"]["ListDataSourceConnectionCommand"]
+ | components["schemas"]["StorageListEntriesCommand"]
+ | components["schemas"]["StorageDownloadCommand"]
| components["schemas"]["ListSecretKeysCommand"]
| components["schemas"]["RefreshSecretsCommand"]
| components["schemas"]["ClearCacheCommand"]
@@ -4551,6 +4635,8 @@ export interface components {
| components["schemas"]["DataSourceConnectionsNotification"]
| components["schemas"]["ValidateSQLResultNotification"]
| components["schemas"]["StorageNamespacesNotification"]
+ | components["schemas"]["StorageEntriesNotification"]
+ | components["schemas"]["StorageDownloadReadyNotification"]
| components["schemas"]["SecretKeysResultNotification"]
| components["schemas"]["CacheClearedNotification"]
| components["schemas"]["CacheInfoNotification"]
@@ -5617,6 +5703,77 @@ export interface components {
/** @enum {unknown} */
type: "stop-kernel";
};
+ /**
+ * StorageDownloadCommand
+ * @description Download a storage entry.
+ *
+ * Downloads file bytes from storage and creates a virtual file
+ * so the frontend can fetch the contents.
+ *
+ * Attributes:
+ * request_id: Unique identifier for this request.
+ * namespace: Variable name identifying the storage backend.
+ * path: Full path of the entry to download.
+ */
+ StorageDownloadCommand: {
+ namespace: string;
+ path: string;
+ requestId: string;
+ /** @enum {unknown} */
+ type: "storage-download";
+ };
+ /**
+ * StorageDownloadReadyNotification
+ * @description Signals that a storage file download is ready as a virtual file.
+ *
+ * Attributes:
+ * request_id: Request ID this responds to.
+ * url: Virtual file URL to download from.
+ * filename: Suggested filename for the download.
+ * error: Error message if the download failed.
+ */
+ StorageDownloadReadyNotification: {
+ /** @default null */
+ error?: string | null;
+ /** @default null */
+ filename?: string | null;
+ /** @enum {unknown} */
+ op: "storage-download-ready";
+ request_id: string;
+ /** @default null */
+ url?: string | null;
+ };
+ /** StorageDownloadRequest */
+ StorageDownloadRequest: {
+ namespace: string;
+ path: string;
+ requestId: string;
+ };
+ /**
+ * StorageEntriesNotification
+ * @description Result of a storage operation that returns entries.
+ *
+ * Attributes:
+ * request_id: Request ID this responds to.
+ * entries: Storage entries returned by the operation.
+ * namespace: Variable name of the storage backend.
+ * prefix: The prefix that was listed (set by list_entries).
+ * query: The search query that was used (set by search).
+ * error: Error message if the operation failed.
+ */
+ StorageEntriesNotification: {
+ entries: components["schemas"]["StorageEntry"][];
+ /** @default null */
+ error?: string | null;
+ namespace: string;
+ /** @enum {unknown} */
+ op: "storage-entries";
+ /** @default null */
+ prefix?: string | null;
+ /** @default null */
+ query?: string | null;
+ request_id: string;
+ };
/**
* StorageEntry
* @description A storage entry is a file, directory, or object for external storage systems
@@ -5637,6 +5794,36 @@ export interface components {
path: string;
size: number;
};
+ /**
+ * StorageListEntriesCommand
+ * @description List storage entries at a prefix.
+ *
+ * Navigates storage like a folder tree using delimiter-based listing.
+ * Returns entries (files/objects) and virtual directories at one level.
+ *
+ * Attributes:
+ * request_id: Unique identifier for this request.
+ * namespace: Variable name identifying the storage backend.
+ * limit: Max entries to return.
+ * prefix: Path prefix to list (None = root).
+ */
+ StorageListEntriesCommand: {
+ limit: number;
+ namespace: string;
+ /** @default null */
+ prefix?: string | null;
+ requestId: string;
+ /** @enum {unknown} */
+ type: "storage-list-entries";
+ };
+ /** StorageListEntriesRequest */
+ StorageListEntriesRequest: {
+ limit: number;
+ namespace: string;
+ /** @default null */
+ prefix?: string | null;
+ requestId: string;
+ };
/**
* StorageNamespace
* @description Represents external storage systems (filesystems and object storage)
@@ -5650,7 +5837,7 @@ export interface components {
*/
StorageNamespace: {
displayName: string;
- name: string | null;
+ name: string;
protocol: string;
rootPath: string;
storageEntries: components["schemas"]["StorageEntry"][];
diff --git a/pyproject.toml b/pyproject.toml
index 8e45591c24..617ffbf97f 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -576,7 +576,6 @@ format_on_save = true
[tool.marimo.experimental]
multi_column = true
cache_panel = true
-storage_inspector = true
[tool.marimo.display]
dataframes = "rich"
diff --git a/tests/_data/_external_storage/test_get_storage.py b/tests/_data/_external_storage/test_get_storage.py
index 33d9941863..350a2e9735 100644
--- a/tests/_data/_external_storage/test_get_storage.py
+++ b/tests/_data/_external_storage/test_get_storage.py
@@ -102,6 +102,7 @@ class TestStorageBackendToStorageNamespace:
def test_converts_backend_to_namespace(self) -> None:
mock_backend: Any = MagicMock()
mock_backend.variable_name = VariableName("test_store")
+ mock_backend.display_name = "Amazon S3"
mock_backend.protocol = "s3"
mock_backend.root_path = "my-bucket"
@@ -110,7 +111,7 @@ class TestStorageBackendToStorageNamespace:
assert result == snapshot(
StorageNamespace(
name=VariableName("test_store"),
- display_name="test_store",
+ display_name="Amazon S3",
protocol="s3",
root_path="my-bucket",
storage_entries=[],
@@ -118,30 +119,12 @@ class TestStorageBackendToStorageNamespace:
)
mock_backend.list_entries.assert_not_called()
- def test_handles_none_variable_name(self) -> None:
- mock_backend: Any = MagicMock()
- mock_backend.variable_name = None
- mock_backend.protocol = "file"
- mock_backend.root_path = "/tmp"
- mock_backend.list_entries.return_value = []
-
- result = storage_backend_to_storage_namespace(mock_backend)
-
- assert result == snapshot(
- StorageNamespace(
- name=None,
- display_name="",
- protocol="file",
- root_path="/tmp",
- storage_entries=[],
- )
- )
-
def test_handles_none_root_path(self) -> None:
mock_backend: Any = MagicMock()
mock_backend.variable_name = VariableName("mem")
mock_backend.protocol = "in-memory"
mock_backend.root_path = None
+ mock_backend.display_name = "In-memory"
mock_backend.list_entries.return_value = []
result = storage_backend_to_storage_namespace(mock_backend)
@@ -149,7 +132,7 @@ class TestStorageBackendToStorageNamespace:
assert result == snapshot(
StorageNamespace(
name=VariableName("mem"),
- display_name="mem",
+ display_name="In-memory",
protocol="in-memory",
root_path="",
storage_entries=[],
@@ -169,7 +152,7 @@ class TestStorageBackendToStorageNamespace:
assert result == snapshot(
StorageNamespace(
name=VariableName("mem_fs"),
- display_name="mem_fs",
+ display_name="Memory",
protocol="memory",
root_path="/",
storage_entries=[],
@@ -188,7 +171,7 @@ class TestStorageBackendToStorageNamespace:
assert result == snapshot(
StorageNamespace(
name=VariableName("mem_store"),
- display_name="mem_store",
+ display_name="In-memory",
protocol="in-memory",
root_path="",
storage_entries=[],
diff --git a/tests/_data/_external_storage/test_storage_models.py b/tests/_data/_external_storage/test_storage_models.py
index 277f7326b1..75b7261147 100644
--- a/tests/_data/_external_storage/test_storage_models.py
+++ b/tests/_data/_external_storage/test_storage_models.py
@@ -9,7 +9,7 @@ import pytest
from dirty_equals import IsDatetime, IsPositiveFloat
from inline_snapshot import snapshot
-from marimo._data._external_storage.models import StorageEntry
+from marimo._data._external_storage.models import DownloadResult, StorageEntry
from marimo._data._external_storage.storage import (
FsspecFilesystem,
Obstore,
@@ -182,6 +182,101 @@ class TestObstore:
assert result == b"hello world"
mock_store.get_async.assert_called_once_with("some/path.txt")
+ async def test_download_file(self) -> None:
+ mock_store = MagicMock()
+ mock_bytes_result = MagicMock()
+ mock_bytes_result.bytes_async = MagicMock(
+ return_value=_async_return(b"file content")
+ )
+ mock_store.get_async = MagicMock(
+ return_value=_async_return(mock_bytes_result)
+ )
+
+ backend = self._make_backend(mock_store)
+ result = await backend.download_file("bucket/data/report.csv")
+
+ assert result == DownloadResult(
+ file_bytes=b"file content",
+ filename="report.csv",
+ ext="csv",
+ )
+
+ async def test_download_file_no_extension(self) -> None:
+ mock_store = MagicMock()
+ mock_bytes_result = MagicMock()
+ mock_bytes_result.bytes_async = MagicMock(
+ return_value=_async_return(b"data")
+ )
+ mock_store.get_async = MagicMock(
+ return_value=_async_return(mock_bytes_result)
+ )
+
+ backend = self._make_backend(mock_store)
+ result = await backend.download_file("bucket/noext")
+
+ assert result == DownloadResult(
+ file_bytes=b"data",
+ filename="noext",
+ ext="bin",
+ )
+
+ async def test_download_file_nested_path(self) -> None:
+ mock_store = MagicMock()
+ mock_bytes_result = MagicMock()
+ mock_bytes_result.bytes_async = MagicMock(
+ return_value=_async_return(b"nested")
+ )
+ mock_store.get_async = MagicMock(
+ return_value=_async_return(mock_bytes_result)
+ )
+
+ backend = self._make_backend(mock_store)
+ result = await backend.download_file("a/b/c/deep.tar.gz")
+
+ assert result == DownloadResult(
+ file_bytes=b"nested",
+ filename="deep.tar.gz",
+ ext="gz",
+ )
+
+ async def test_download_file_trailing_dot(self) -> None:
+ mock_store = MagicMock()
+ mock_bytes_result = MagicMock()
+ mock_bytes_result.bytes_async = MagicMock(
+ return_value=_async_return(b"data")
+ )
+ mock_store.get_async = MagicMock(
+ return_value=_async_return(mock_bytes_result)
+ )
+
+ backend = self._make_backend(mock_store)
+ result = await backend.download_file("bucket/file.")
+
+ assert result == DownloadResult(
+ file_bytes=b"data",
+ filename="file.",
+ ext="bin",
+ )
+
+ async def test_download_file_empty_path(self) -> None:
+ mock_store = MagicMock()
+ mock_bytes_result = MagicMock()
+ mock_bytes_result.bytes_async = MagicMock(
+ return_value=_async_return(b"data")
+ )
+ mock_store.get_async = MagicMock(
+ return_value=_async_return(mock_bytes_result)
+ )
+
+ backend = self._make_backend(mock_store)
+ result = await backend.download_file("")
+
+ assert result == DownloadResult(
+ file_bytes=b"data",
+ filename="download",
+ ext="bin",
+ )
+
def test_protocol_memory(self) -> None:
from obstore.store import MemoryStore
@@ -247,6 +342,13 @@ class TestObstore:
assert Obstore.is_compatible(42) is False
assert Obstore.is_compatible(None) is False
+ def test_display_name_known_protocol(self) -> None:
+ from obstore.store import MemoryStore
+
+ store = MemoryStore()
+ backend = self._make_backend(store)
+ assert backend.display_name == "In-memory"
+
@pytest.mark.skipif(not HAS_FSSPEC, reason="fsspec not installed")
class TestFsspecFilesystem:
@@ -516,6 +618,21 @@ class TestFsspecFilesystem:
result = await backend.download("path/to/file.txt")
assert result == b"text content"
+ async def test_download_file(self) -> None:
+ mock_store = MagicMock()
+ mock_file = MagicMock()
+ mock_file.read.return_value = b"csv data"
+ mock_store.open.return_value = mock_file
+
+ backend = self._make_backend(mock_store)
+ result = await backend.download_file("bucket/export.csv")
+
+ assert result == DownloadResult(
+ file_bytes=b"csv data",
+ filename="export.csv",
+ ext="csv",
+ )
+
def test_protocol_tuple(self) -> None:
mock_store = MagicMock()
mock_store.protocol = ("gcs", "gs")
@@ -554,6 +671,18 @@ class TestFsspecFilesystem:
fs = MemoryFileSystem()
assert FsspecFilesystem.is_compatible(fs) is True
+ def test_display_name_known_protocol(self) -> None:
+ mock_store = MagicMock()
+ mock_store.protocol = "s3"
+ backend = self._make_backend(mock_store)
+ assert backend.display_name == "Amazon S3"
+
+ def test_display_name_unknown_protocol(self) -> None:
+ mock_store = MagicMock()
+ mock_store.protocol = "custom-proto"
+ backend = self._make_backend(mock_store)
+ assert backend.display_name == "Custom-proto"
+
@pytest.mark.skipif(not HAS_FSSPEC, reason="fsspec not installed")
class TestFsspecFilesystemIntegration:
diff --git a/tests/_runtime/test_runtime_external_storage.py b/tests/_runtime/test_runtime_external_storage.py
new file mode 100644
index 0000000000..6de995da94
--- /dev/null
+++ b/tests/_runtime/test_runtime_external_storage.py
@@ -0,0 +1,541 @@
+# Copyright 2026 Marimo. All rights reserved.
+
+from __future__ import annotations
+
+import threading
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+from dirty_equals import IsPositiveFloat, IsStr
+
+from marimo._data._external_storage.models import StorageEntry
+from marimo._data._external_storage.storage import Obstore
+from marimo._dependencies.dependencies import DependencyManager
+from marimo._messaging.notification import (
+ StorageDownloadReadyNotification,
+ StorageEntriesNotification,
+)
+from marimo._runtime.commands import (
+ ExecuteCellCommand,
+ StorageDownloadCommand,
+ StorageListEntriesCommand,
+)
+from marimo._types.ids import CellId_t, RequestId, VariableName
+from tests.conftest import MockedKernel
+
+HAS_OBSTORE = DependencyManager.obstore.has()
+
+STORAGE_VAR = "my_store"
+
+
+class TestExternalStorageErrors:
+ """Error-handling tests that don't require a real storage backend."""
+
+ async def test_list_entries_variable_not_found(
+ self, mocked_kernel: MockedKernel
+ ) -> None:
+ k = mocked_kernel.k
+ stream = mocked_kernel.stream
+
+ request = StorageListEntriesCommand(
+ request_id=RequestId("req-1"),
+ namespace="nonexistent_var",
+ limit=100,
+ prefix=None,
+ )
+ await k.handle_message(request)
+
+ results = [
+ op
+ for op in stream.operations
+ if isinstance(op, StorageEntriesNotification)
+ ]
+ assert results == [
+ StorageEntriesNotification(
+ request_id=RequestId("req-1"),
+ entries=[],
+ namespace="nonexistent_var",
+ prefix=None,
+ error="Variable 'nonexistent_var' not found",
+ )
+ ]
+
+ async def test_list_entries_incompatible_backend(
+ self, mocked_kernel: MockedKernel
+ ) -> None:
+ k = mocked_kernel.k
+ stream = mocked_kernel.stream
+
+ await k.run(
+ [
+ ExecuteCellCommand(
+ cell_id=CellId_t("0"),
+ code='not_storage = "just a string"',
+ ),
+ ]
+ )
+
+ request = StorageListEntriesCommand(
+ request_id=RequestId("req-2"),
+ namespace="not_storage",
+ limit=100,
+ prefix=None,
+ )
+ await k.handle_message(request)
+
+ results = [
+ op
+ for op in stream.operations
+ if isinstance(op, StorageEntriesNotification)
+ ]
+ assert results == [
+ StorageEntriesNotification(
+ request_id=RequestId("req-2"),
+ entries=[],
+ namespace="not_storage",
+ prefix=None,
+ error=(
+ "Variable 'not_storage' is not a compatible "
+ "storage backend (expected obstore or fsspec)"
+ ),
+ )
+ ]
+
+ async def test_download_variable_not_found(
+ self, mocked_kernel: MockedKernel
+ ) -> None:
+ k = mocked_kernel.k
+ stream = mocked_kernel.stream
+
+ request = StorageDownloadCommand(
+ request_id=RequestId("req-3"),
+ namespace="nonexistent_var",
+ path="data/file.csv",
+ )
+ await k.handle_message(request)
+
+ results = [
+ op
+ for op in stream.operations
+ if isinstance(op, StorageDownloadReadyNotification)
+ ]
+ assert results == [
+ StorageDownloadReadyNotification(
+ request_id=RequestId("req-3"),
+ url=None,
+ filename=None,
+ error="Variable 'nonexistent_var' not found",
+ )
+ ]
+
+ async def test_download_incompatible_backend(
+ self, mocked_kernel: MockedKernel
+ ) -> None:
+ k = mocked_kernel.k
+ stream = mocked_kernel.stream
+
+ await k.run(
+ [
+ ExecuteCellCommand(
+ cell_id=CellId_t("0"),
+ code="not_storage = 42",
+ ),
+ ]
+ )
+
+ request = StorageDownloadCommand(
+ request_id=RequestId("req-4"),
+ namespace="not_storage",
+ path="data/file.csv",
+ )
+ await k.handle_message(request)
+
+ results = [
+ op
+ for op in stream.operations
+ if isinstance(op, StorageDownloadReadyNotification)
+ ]
+ assert results == [
+ StorageDownloadReadyNotification(
+ request_id=RequestId("req-4"),
+ url=None,
+ filename=None,
+ error=(
+ "Variable 'not_storage' is not a compatible "
+ "storage backend (expected obstore or fsspec)"
+ ),
+ )
+ ]
+
+
[email protected](not HAS_OBSTORE, reason="obstore not installed")
+class TestExternalStorageCallbacks:
+ """Integration tests using a real obstore MemoryStore."""
+
+ async def test_list_entries(self, mocked_kernel: MockedKernel) -> None:
+ k = mocked_kernel.k
+ stream = mocked_kernel.stream
+
+ await k.run(
+ [
+ ExecuteCellCommand(
+ cell_id=CellId_t("0"),
+ code=(
+ "from obstore.store import MemoryStore\n"
+ f"{STORAGE_VAR} = MemoryStore()"
+ ),
+ ),
+ ]
+ )
+
+ store = k.globals[VariableName(STORAGE_VAR)]
+ await store.put_async("data/file1.csv", b"a,b,c")
+ await store.put_async("data/file2.txt", b"hello")
+
+ request = StorageListEntriesCommand(
+ request_id=RequestId("req-10"),
+ namespace=STORAGE_VAR,
+ limit=100,
+ prefix="data/",
+ )
+ await k.handle_message(request)
+
+ results = [
+ op
+ for op in stream.operations
+ if isinstance(op, StorageEntriesNotification)
+ ]
+ assert results == [
+ StorageEntriesNotification(
+ request_id=RequestId("req-10"),
+ entries=[
+ StorageEntry(
+ path="data/file1.csv",
+ kind="object",
+ size=5,
+ last_modified=IsPositiveFloat(), # pyright: ignore[reportArgumentType]
+ metadata={"e_tag": IsStr()},
+ ),
+ StorageEntry(
+ path="data/file2.txt",
+ kind="object",
+ size=5,
+ last_modified=IsPositiveFloat(), # pyright: ignore[reportArgumentType]
+ metadata={"e_tag": IsStr()},
+ ),
+ ],
+ namespace=STORAGE_VAR,
+ prefix="data/",
+ )
+ ]
+
+ async def test_list_entries_with_limit(
+ self, mocked_kernel: MockedKernel
+ ) -> None:
+ k = mocked_kernel.k
+ stream = mocked_kernel.stream
+
+ await k.run(
+ [
+ ExecuteCellCommand(
+ cell_id=CellId_t("0"),
+ code=(
+ "from obstore.store import MemoryStore\n"
+ f"{STORAGE_VAR} = MemoryStore()"
+ ),
+ ),
+ ]
+ )
+
+ store = k.globals[VariableName(STORAGE_VAR)]
+ await store.put_async("a.txt", b"1")
+ await store.put_async("b.txt", b"2")
+ await store.put_async("c.txt", b"3")
+
+ request = StorageListEntriesCommand(
+ request_id=RequestId("req-11"),
+ namespace=STORAGE_VAR,
+ limit=2,
+ prefix=None,
+ )
+ await k.handle_message(request)
+
+ results = [
+ op
+ for op in stream.operations
+ if isinstance(op, StorageEntriesNotification)
+ ]
+ assert results == [
+ StorageEntriesNotification(
+ request_id=RequestId("req-11"),
+ entries=[
+ StorageEntry(
+ path="a.txt",
+ kind="object",
+ size=1,
+ last_modified=IsPositiveFloat(), # pyright: ignore[reportArgumentType]
+ metadata={"e_tag": IsStr()},
+ ),
+ StorageEntry(
+ path="b.txt",
+ kind="object",
+ size=1,
+ last_modified=IsPositiveFloat(), # pyright: ignore[reportArgumentType]
+ metadata={"e_tag": IsStr()},
+ ),
+ ],
+ namespace=STORAGE_VAR,
+ prefix=None,
+ )
+ ]
+
+ async def test_list_entries_empty(
+ self, mocked_kernel: MockedKernel
+ ) -> None:
+ k = mocked_kernel.k
+ stream = mocked_kernel.stream
+
+ await k.run(
+ [
+ ExecuteCellCommand(
+ cell_id=CellId_t("0"),
+ code=(
+ "from obstore.store import MemoryStore\n"
+ f"{STORAGE_VAR} = MemoryStore()"
+ ),
+ ),
+ ]
+ )
+
+ request = StorageListEntriesCommand(
+ request_id=RequestId("req-12"),
+ namespace=STORAGE_VAR,
+ limit=100,
+ prefix=None,
+ )
+ await k.handle_message(request)
+
+ results = [
+ op
+ for op in stream.operations
+ if isinstance(op, StorageEntriesNotification)
+ ]
+ assert results == [
+ StorageEntriesNotification(
+ request_id=RequestId("req-12"),
+ entries=[],
+ namespace=STORAGE_VAR,
+ prefix=None,
+ )
+ ]
+
+ async def test_list_entries_backend_exception(
+ self, mocked_kernel: MockedKernel
+ ) -> None:
+ k = mocked_kernel.k
+ stream = mocked_kernel.stream
+
+ await k.run(
+ [
+ ExecuteCellCommand(
+ cell_id=CellId_t("0"),
+ code=(
+ "from obstore.store import MemoryStore\n"
+ f"{STORAGE_VAR} = MemoryStore()"
+ ),
+ ),
+ ]
+ )
+
+ # Use a real Obstore backend with a broken list_entries
+ store = k.globals[VariableName(STORAGE_VAR)]
+ broken_backend = Obstore(store, VariableName(STORAGE_VAR))
+ broken_backend.list_entries = MagicMock( # type: ignore[method-assign]
+ side_effect=RuntimeError("connection timeout"),
+ )
+
+ with pytest.MonkeyPatch.context() as mp:
+ mp.setattr(
+ k.external_storage_callbacks,
+ "_get_storage_backend",
+ lambda _: (broken_backend, None),
+ )
+ request = StorageListEntriesCommand(
+ request_id=RequestId("req-13"),
+ namespace=STORAGE_VAR,
+ limit=100,
+ prefix=None,
+ )
+ await k.handle_message(request)
+
+ results = [
+ op
+ for op in stream.operations
+ if isinstance(op, StorageEntriesNotification)
+ ]
+ assert results == [
+ StorageEntriesNotification(
+ request_id=RequestId("req-13"),
+ entries=[],
+ namespace=STORAGE_VAR,
+ prefix=None,
+ error="Failed to list entries: connection timeout",
+ )
+ ]
+
+ async def test_list_entries_runs_in_background_thread(
+ self, mocked_kernel: MockedKernel
+ ) -> None:
+ """list_entries dispatches to a thread via asyncio.to_thread."""
+ k = mocked_kernel.k
+ stream = mocked_kernel.stream
+
+ await k.run(
+ [
+ ExecuteCellCommand(
+ cell_id=CellId_t("0"),
+ code=(
+ "from obstore.store import MemoryStore\n"
+ f"{STORAGE_VAR} = MemoryStore()"
+ ),
+ ),
+ ]
+ )
+
+ store = k.globals[VariableName(STORAGE_VAR)]
+ real_backend = Obstore(store, VariableName(STORAGE_VAR))
+
+ call_thread_name: str | None = None
+ original_list = real_backend.list_entries
+
+ def spy_list_entries(
+ prefix: str | None, *, limit: int = 100
+ ) -> list[StorageEntry]:
+ nonlocal call_thread_name
+ call_thread_name = threading.current_thread().name
+ return original_list(prefix, limit=limit)
+
+ real_backend.list_entries = spy_list_entries # type: ignore[method-assign]
+
+ with pytest.MonkeyPatch.context() as mp:
+ mp.setattr(
+ k.external_storage_callbacks,
+ "_get_storage_backend",
+ lambda _: (real_backend, None),
+ )
+ request = StorageListEntriesCommand(
+ request_id=RequestId("req-14"),
+ namespace=STORAGE_VAR,
+ limit=100,
+ prefix=None,
+ )
+ await k.handle_message(request)
+
+ # asyncio.to_thread runs the callable in a worker thread,
+ # not the main thread
+ assert call_thread_name is not None
+ assert call_thread_name != threading.main_thread().name
+
+ results = [
+ op
+ for op in stream.operations
+ if isinstance(op, StorageEntriesNotification)
+ ]
+ assert results == [
+ StorageEntriesNotification(
+ request_id=RequestId("req-14"),
+ entries=[],
+ namespace=STORAGE_VAR,
+ prefix=None,
+ )
+ ]
+
+ async def test_download(self, mocked_kernel: MockedKernel) -> None:
+ k = mocked_kernel.k
+ stream = mocked_kernel.stream
+
+ await k.run(
+ [
+ ExecuteCellCommand(
+ cell_id=CellId_t("0"),
+ code=(
+ "from obstore.store import MemoryStore\n"
+ f"{STORAGE_VAR} = MemoryStore()"
+ ),
+ ),
+ ]
+ )
+
+ store = k.globals[VariableName(STORAGE_VAR)]
+ await store.put_async("reports/data.csv", b"a,b,c\n1,2,3")
+
+ request = StorageDownloadCommand(
+ request_id=RequestId("req-20"),
+ namespace=STORAGE_VAR,
+ path="reports/data.csv",
+ )
+ await k.handle_message(request)
+
+ results = [
+ op
+ for op in stream.operations
+ if isinstance(op, StorageDownloadReadyNotification)
+ ]
+ assert results == [
+ StorageDownloadReadyNotification(
+ request_id=RequestId("req-20"),
+ url=IsStr(), # pyright: ignore[reportArgumentType]
+ filename="data.csv",
+ )
+ ]
+
+ async def test_download_backend_exception(
+ self, mocked_kernel: MockedKernel
+ ) -> None:
+ k = mocked_kernel.k
+ stream = mocked_kernel.stream
+
+ await k.run(
+ [
+ ExecuteCellCommand(
+ cell_id=CellId_t("0"),
+ code=(
+ "from obstore.store import MemoryStore\n"
+ f"{STORAGE_VAR} = MemoryStore()"
+ ),
+ ),
+ ]
+ )
+
+ store = k.globals[VariableName(STORAGE_VAR)]
+ broken_backend = Obstore(store, VariableName(STORAGE_VAR))
+ broken_backend.download_file = AsyncMock( # type: ignore[method-assign]
+ side_effect=PermissionError("access denied"),
+ )
+
+ with pytest.MonkeyPatch.context() as mp:
+ mp.setattr(
+ k.external_storage_callbacks,
+ "_get_storage_backend",
+ lambda _: (broken_backend, None),
+ )
+ request = StorageDownloadCommand(
+ request_id=RequestId("req-21"),
+ namespace=STORAGE_VAR,
+ path="secret/file.bin",
+ )
+ await k.handle_message(request)
+
+ results = [
+ op
+ for op in stream.operations
+ if isinstance(op, StorageDownloadReadyNotification)
+ ]
+ assert results == [
+ StorageDownloadReadyNotification(
+ request_id=RequestId("req-21"),
+ url=None,
+ filename=None,
+ error="Failed to download: access denied",
+ )
+ ]
diff --git a/tests/_runtime/test_virtual_file.py b/tests/_runtime/test_virtual_file.py
index 9de6fde346..837ef672b2 100644
--- a/tests/_runtime/test_virtual_file.py
+++ b/tests/_runtime/test_virtual_file.py
@@ -6,6 +6,7 @@ from marimo._runtime.context import get_context
from marimo._runtime.runtime import Kernel
from marimo._runtime.virtual_file.storage import InMemoryStorage
from marimo._runtime.virtual_file.virtual_file import (
+ VirtualFile,
VirtualFileLifecycleItem,
VirtualFileRegistry,
read_virtual_file,
@@ -326,3 +327,77 @@ def test_virtual_file_registry_shared_inmemory_storage(
# Ensure old file should still readable
assert read_virtual_file(vf.filename, 3) == b"abc"
+
+
+def test_create_and_register_with_context(
+ run_mode_kernel: MockedKernel, # noqa: ARG001
+) -> None:
+ ctx = get_context()
+ assert len(ctx.virtual_file_registry.registry) == 0
+
+ vfile = VirtualFile.create_and_register(b"hello world", "txt")
+
+ assert vfile.filename.endswith(".txt")
+ # Should be a relative file URL, not a data URL
+ assert vfile.url.startswith("./@file/")
+ assert "11-" in vfile.url # 11 bytes = len(b"hello world")
+ # Should be registered
+ assert len(ctx.virtual_file_registry.registry) == 1
+ assert read_virtual_file(vfile.filename, 11) == b"hello world"
+
+
+def test_create_and_register_without_context() -> None:
+ # No kernel context initialized — should fall back to data URL
+ vfile = VirtualFile.create_and_register(b"test data", "bin")
+
+ assert vfile.filename.endswith(".bin")
+ assert vfile.url.startswith("data:")
+
+
+def test_create_and_register_empty_buffer_uses_data_url(
+ run_mode_kernel: MockedKernel, # noqa: ARG001
+) -> None:
+ ctx = get_context()
+ registry_size_before = len(ctx.virtual_file_registry.registry)
+
+ vfile = VirtualFile.create_and_register(b"", "csv")
+
+ assert vfile.filename.endswith(".csv")
+ # Empty buffer should use a data URL, not a ./@file/ URL,
+ # because empty buffers can't be served via the file registry.
+ assert vfile.url.startswith("data:")
+ assert not vfile.url.startswith("./@file/")
+ # Registry should not have grown
+ assert len(ctx.virtual_file_registry.registry) == registry_size_before
+
+
+def test_create_and_register_empty_buffer_without_context() -> None:
+ # No kernel context — empty buffer should still produce a data URL
+ vfile = VirtualFile.create_and_register(b"", "bin")
+
+ assert vfile.filename.endswith(".bin")
+ assert vfile.url.startswith("data:")
+
+
+def test_create_and_register_virtual_files_not_supported(
+ run_mode_kernel: MockedKernel, # noqa: ARG001
+) -> None:
+ ctx = get_context()
+ ctx.virtual_files_supported = False
+ try:
+ vfile = VirtualFile.create_and_register(b"some data", "txt")
+
+ assert vfile.filename.endswith(".txt")
+ # Should fall back to data URL when virtual files aren't supported
+ assert vfile.url.startswith("data:")
+ assert len(ctx.virtual_file_registry.registry) == 0
+ finally:
+ ctx.virtual_files_supported = True
+
+
+def test_create_and_register_preserves_extension(
+ run_mode_kernel: MockedKernel, # noqa: ARG001
+) -> None:
+ for ext in ("pdf", "png", "csv"):
+ vfile = VirtualFile.create_and_register(b"content", ext)
+ assert vfile.filename.endswith(f".{ext}")
diff --git a/tests/_server/api/endpoints/test_storage_endpoints.py b/tests/_server/api/endpoints/test_storage_endpoints.py
new file mode 100644
index 0000000000..d0706f3ca0
--- /dev/null
+++ b/tests/_server/api/endpoints/test_storage_endpoints.py
@@ -0,0 +1,89 @@
+# Copyright 2026 Marimo. All rights reserved.
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+from tests._server.mocks import token_header, with_read_session, with_session
+
+if TYPE_CHECKING:
+ from starlette.testclient import TestClient
+
+SESSION_ID = "session-123"
+HEADERS = {
+ "Marimo-Session-Id": SESSION_ID,
+ **token_header("fake-token"),
+}
+
+
+@with_session(SESSION_ID)
+def test_list_entries(client: TestClient) -> None:
+ response = client.post(
+ "/api/storage/list_entries",
+ headers=HEADERS,
+ json={
+ "requestId": "test_request_id",
+ "namespace": "my_store",
+ "limit": 100,
+ },
+ )
+ assert response.status_code == 200, response.text
+ assert response.json()["success"] is True
+
+
+@with_session(SESSION_ID)
+def test_list_entries_with_prefix(client: TestClient) -> None:
+ response = client.post(
+ "/api/storage/list_entries",
+ headers=HEADERS,
+ json={
+ "requestId": "test_request_id",
+ "namespace": "my_store",
+ "limit": 50,
+ "prefix": "data/images/",
+ },
+ )
+ assert response.status_code == 200, response.text
+ assert response.json()["success"] is True
+
+
+@with_session(SESSION_ID)
+def test_download(client: TestClient) -> None:
+ response = client.post(
+ "/api/storage/download",
+ headers=HEADERS,
+ json={
+ "requestId": "test_request_id",
+ "namespace": "my_store",
+ "path": "data/file.csv",
+ },
+ )
+ assert response.status_code == 200, response.text
+ assert response.json()["success"] is True
+
+
+@with_read_session(SESSION_ID)
+def test_list_entries_fails_in_read_mode(client: TestClient) -> None:
+ response = client.post(
+ "/api/storage/list_entries",
+ headers=HEADERS,
+ json={
+ "requestId": "test_request_id",
+ "namespace": "my_store",
+ "limit": 100,
+ },
+ )
+ assert response.status_code == 401
+
+
+@with_read_session(SESSION_ID)
+def test_download_fails_in_read_mode(client: TestClient) -> None:
+ response = client.post(
+ "/api/storage/download",
+ headers=HEADERS,
+ json={
+ "requestId": "test_request_id",
+ "namespace": "my_store",
+ "path": "data/file.csv",
+ },
+ )
+ assert response.status_code == 401
diff --git a/tests/_session/state/test_session_external_storage.py b/tests/_session/state/test_session_external_storage.py
new file mode 100644
index 0000000000..716929b6f0
--- /dev/null
+++ b/tests/_session/state/test_session_external_storage.py
@@ -0,0 +1,245 @@
+# Copyright 2026 Marimo. All rights reserved.
+from __future__ import annotations
+
+from inline_snapshot import snapshot
+
+from marimo._data._external_storage.models import StorageNamespace
+from marimo._messaging.notification import (
+ StorageNamespacesNotification,
+ VariableDeclarationNotification,
+ VariablesNotification,
+)
+from marimo._messaging.serde import serialize_kernel_message
+from marimo._session.state.session_view import SessionView
+from marimo._types.ids import CellId_t, VariableName
+
+cell_id = CellId_t("cell_1")
+
+
+def test_add_storage_namespaces(session_view: SessionView) -> None:
+ # Add initial namespaces
+ session_view.add_raw_notification(
+ serialize_kernel_message(
+ StorageNamespacesNotification(
+ namespaces=[
+ StorageNamespace(
+ name=VariableName("s3_store"),
+ display_name="s3_store",
+ protocol="s3",
+ root_path="my-bucket",
+ storage_entries=[],
+ ),
+ StorageNamespace(
+ name=VariableName("local_fs"),
+ display_name="local_fs",
+ protocol="file",
+ root_path="/data",
+ storage_entries=[],
+ ),
+ ]
+ )
+ )
+ )
+
+ assert session_view.external_storage_namespaces.namespaces == snapshot(
+ [
+ StorageNamespace(
+ name=VariableName("s3_store"),
+ display_name="s3_store",
+ protocol="s3",
+ root_path="my-bucket",
+ storage_entries=[],
+ ),
+ StorageNamespace(
+ name=VariableName("local_fs"),
+ display_name="local_fs",
+ protocol="file",
+ root_path="/data",
+ storage_entries=[],
+ ),
+ ]
+ )
+
+ # Add a new namespace and update an existing one
+ session_view.add_raw_notification(
+ serialize_kernel_message(
+ StorageNamespacesNotification(
+ namespaces=[
+ StorageNamespace(
+ name=VariableName("s3_store"),
+ display_name="s3_store_updated",
+ protocol="s3",
+ root_path="new-bucket",
+ storage_entries=[],
+ ),
+ StorageNamespace(
+ name=VariableName("gcs_store"),
+ display_name="gcs_store",
+ protocol="gcs",
+ root_path="gcs-bucket",
+ storage_entries=[],
+ ),
+ ]
+ )
+ )
+ )
+
+ assert session_view.external_storage_namespaces.namespaces == snapshot(
+ [
+ StorageNamespace(
+ name=VariableName("s3_store"),
+ display_name="s3_store_updated",
+ protocol="s3",
+ root_path="new-bucket",
+ storage_entries=[],
+ ),
+ StorageNamespace(
+ name=VariableName("local_fs"),
+ display_name="local_fs",
+ protocol="file",
+ root_path="/data",
+ storage_entries=[],
+ ),
+ StorageNamespace(
+ name=VariableName("gcs_store"),
+ display_name="gcs_store",
+ protocol="gcs",
+ root_path="gcs-bucket",
+ storage_entries=[],
+ ),
+ ]
+ )
+ ns_by_name = {
+ ns.name: ns
+ for ns in session_view.external_storage_namespaces.namespaces
+ }
+ # Updated
+ assert (
+ ns_by_name[VariableName("s3_store")].display_name == "s3_store_updated"
+ )
+ assert ns_by_name[VariableName("s3_store")].root_path == "new-bucket"
+
+ # Check namespaces appear in notifications
+ assert (
+ session_view.external_storage_namespaces in session_view.notifications
+ )
+
+
+def test_storage_namespaces_filtered_by_variables(
+ session_view: SessionView,
+) -> None:
+ # Add storage namespaces
+ session_view.add_notification(
+ StorageNamespacesNotification(
+ namespaces=[
+ StorageNamespace(
+ name=VariableName("s3_store"),
+ display_name="s3_store",
+ protocol="s3",
+ root_path="bucket-1",
+ storage_entries=[],
+ ),
+ StorageNamespace(
+ name=VariableName("gcs_store"),
+ display_name="gcs_store",
+ protocol="gcs",
+ root_path="bucket-2",
+ storage_entries=[],
+ ),
+ ]
+ )
+ )
+ assert len(session_view.external_storage_namespaces.namespaces) == 2
+
+ # Declare both variables in scope
+ session_view.add_notification(
+ VariablesNotification(
+ variables=[
+ VariableDeclarationNotification(
+ name="s3_store", declared_by=[cell_id], used_by=[]
+ ),
+ VariableDeclarationNotification(
+ name="gcs_store", declared_by=[cell_id], used_by=[]
+ ),
+ ]
+ )
+ )
+ assert len(session_view.external_storage_namespaces.namespaces) == 2
+
+ # Remove gcs_store from variables => only s3_store remains
+ session_view.add_notification(
+ VariablesNotification(
+ variables=[
+ VariableDeclarationNotification(
+ name="s3_store", declared_by=[cell_id], used_by=[]
+ ),
+ ]
+ )
+ )
+ assert session_view.external_storage_namespaces.namespaces == snapshot(
+ [
+ StorageNamespace(
+ name=VariableName("s3_store"),
+ display_name="s3_store",
+ protocol="s3",
+ root_path="bucket-1",
+ storage_entries=[],
+ ),
+ ]
+ )
+
+ # Remove all variables => no namespaces remain
+ session_view.add_notification(VariablesNotification(variables=[]))
+ assert session_view.external_storage_namespaces.namespaces == []
+
+
+def test_storage_namespaces_empty_not_in_notifications(
+ session_view: SessionView,
+) -> None:
+ """Empty storage namespaces should not appear in notifications."""
+ assert session_view.external_storage_namespaces.namespaces == []
+ storage_ops = [
+ op
+ for op in session_view.notifications
+ if isinstance(op, StorageNamespacesNotification)
+ ]
+ assert len(storage_ops) == 0
+
+
+def test_storage_namespaces_in_notifications(
+ session_view: SessionView,
+) -> None:
+ """Non-empty storage namespaces should appear in notifications."""
+ session_view.add_notification(
+ StorageNamespacesNotification(
+ namespaces=[
+ StorageNamespace(
+ name=VariableName("store"),
+ display_name="store",
+ protocol="s3",
+ root_path="bucket",
+ storage_entries=[],
+ ),
+ ]
+ )
+ )
+ storage_ops = [
+ op
+ for op in session_view.notifications
+ if isinstance(op, StorageNamespacesNotification)
+ ]
+ assert storage_ops == snapshot(
+ [
+ StorageNamespacesNotification(
+ namespaces=[
+ StorageNamespace(
+ name=VariableName("store"),
+ display_name="store",
+ protocol="s3",
+ root_path="bucket",
+ storage_entries=[],
+ ),
+ ]
+ )
+ ]
+ )