Skip to content

Kepler.gl is a powerful open source geospatial analysis tool for large-scale data sets.

feat: extract in-memory rows from a drawn polygon (#3739)

Igor Dykhtae9551a975cfe
diff --git a/docs/user-guides/draw-on-map.md b/docs/user-guides/draw-on-map.md
index c633f92b5..9f38ac82d 100644
--- a/docs/user-guides/draw-on-map.md
+++ b/docs/user-guides/draw-on-map.md
@@ -1,13 +1,15 @@
# Draw on Map
<!-- TOC -->
- - [Open the drawing tools](#open-the-drawing-tools)
- - [Drawing tools](#drawing-tools)
- - [Select, move, and edit](#select-move-and-edit)
- - [Edit properties](#edit-properties)
- - [Filter layers](#filter-layers)
- - [Copy and convert to a layer](#copy-and-convert-to-a-layer)
- - [Disable sketch tools](#disable-sketch-tools)
+
+- [Open the drawing tools](#open-the-drawing-tools)
+- [Drawing tools](#drawing-tools)
+- [Select, move, and edit](#select-move-and-edit)
+- [Edit properties](#edit-properties)
+- [Filter layers](#filter-layers)
+- [Extract data](#extract-data)
+- [Copy and convert to a layer](#copy-and-convert-to-a-layer)
+- [Disable sketch tools](#disable-sketch-tools)
<!-- /TOC -->
Draw on Map lets you sketch points, lines, polygons, rectangles, and circles on the map. Sketches stay on the map until you delete them, convert them to a layer, or use a polygon, rectangle, or circle as a spatial filter.
@@ -20,14 +22,14 @@ Press **Escape** while a draw tool is active to return to **Select**.
## Drawing tools
-| Tool | How to draw | Notes |
-| --- | --- | --- |
-| **Select** | Click a sketch to select it | Move and edit existing sketches |
-| **Point** | Click the map | Stays in point mode so you can add several points |
-| **Line** | Click to add vertices, double-click or press **Enter** to finish | Stays in line mode so you can draw several lines. A length label appears at the cursor while you draw |
-| **Polygon** | Click to add vertices, click the first vertex (or double-click) to close | Sketch only until you apply it as a filter |
-| **Rectangle** | Click and drag, or click two opposite corners | Applied as a polygon filter when you finish drawing |
-| **Circle** | Click the center, then click or drag to set the radius | Tessellated polygon. A radius label appears at the rim while you draw. Sketch only until you apply it as a filter or convert it to a layer |
+| Tool | How to draw | Notes |
+| ------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ |
+| **Select** | Click a sketch to select it | Move and edit existing sketches |
+| **Point** | Click the map | Stays in point mode so you can add several points |
+| **Line** | Click to add vertices, double-click or press **Enter** to finish | Stays in line mode so you can draw several lines. A length label appears at the cursor while you draw |
+| **Polygon** | Click to add vertices, click the first vertex (or double-click) to close | Sketch only until you apply it as a filter |
+| **Rectangle** | Click and drag, or click two opposite corners | Applied as a polygon filter when you finish drawing |
+| **Circle** | Click the center, then click or drag to set the radius | Tessellated polygon. A radius label appears at the rim while you draw. Sketch only until you apply it as a filter or convert it to a layer |
## Select, move, and edit
@@ -36,7 +38,7 @@ Press **Escape** while a draw tool is active to return to **Select**.
3. Drag the **body of a line** to move the whole line. Drag a vertex to reshape it. Click the line to insert a vertex.
4. Drag the **interior of a polygon, rectangle, or circle** to move it. Drag a vertex to reshape it.
-Right-click a selected sketch for **Edit Properties**, **Copy Geometry**, and **Delete**. **Filter Layers** appears only for polygons, rectangles, and circles.
+Right-click a selected sketch for **Edit Properties**, **Copy Geometry**, and **Delete**. **Filter Layers** and **Extract data** appear only for polygons, rectangles, and circles.
## Edit properties
@@ -55,6 +57,20 @@ Points and lines cannot be used as filters.
If you turn off every filtered layer, the polygon returns to a sketch.
+## Extract data
+
+Polygons, rectangles, and circles can copy in-memory rows that fall inside the drawing into a **new dataset**. This is independent of **Convert to Layer**, which turns the drawing itself into GeoJSON.
+
+1. Draw a polygon, rectangle, or circle.
+2. Right-click the shape and choose **Extract data**.
+3. Pick the layer to extract from. Only layers that can be spatially clipped (the same types as **Filter Layers**, plus vector tiles) are listed.
+
+kepler.gl keeps currently filtered rows (range, select, time) and then clips them to the drawing. The new dataset is named `Extracted` plus the source dataset label, and a layer is created automatically.
+
+**Vector tiles:** extract copies GeoJSON features from tiles that are **already loaded in the current view** (current zoom). It is a snapshot, not a download of the whole tileset. Features that intersect the drawing are included, including lines that cross it and polygons that contain it. Features split across tiles are deduped when they share an id. Raster, WMS, and 3D tiles cannot be extracted.
+
+The original dataset and the drawing stay on the map.
+
## Copy and convert to a layer
The drawing menu also includes:
diff --git a/src/actions/src/action-types.ts b/src/actions/src/action-types.ts
index 13ce0c52c..662ac2d3b 100644
--- a/src/actions/src/action-types.ts
+++ b/src/actions/src/action-types.ts
@@ -222,6 +222,7 @@ export const ActionTypes = {
DELETE_FEATURE: `${ACTION_PREFIX}DELETE_FEATURE`,
TOGGLE_EDITOR_VISIBILITY: `${ACTION_PREFIX}TOGGLE_EDITOR_VISIBILITY`,
CONVERT_EDITOR_FEATURES_TO_LAYER: `${ACTION_PREFIX}CONVERT_EDITOR_FEATURES_TO_LAYER`,
+ EXTRACT_DATA_FROM_FEATURE: `${ACTION_PREFIX}EXTRACT_DATA_FROM_FEATURE`,
// storage
START_SAVE_STORAGE: `${ACTION_PREFIX}START_SAVE_STORAGE`
diff --git a/src/actions/src/vis-state-actions.ts b/src/actions/src/vis-state-actions.ts
index 80710fbfd..95f8ab22b 100644
--- a/src/actions/src/vis-state-actions.ts
+++ b/src/actions/src/vis-state-actions.ts
@@ -2069,6 +2069,35 @@ export function convertEditorFeaturesToLayer(): Merge<
};
}
+export type ExtractDataFromFeatureUpdaterAction = {
+ layerId: string;
+};
+/**
+ * Copy in-memory rows that fall inside the currently selected Draw on Map polygon
+ * into a new local dataset. Vector tile layers extract a snapshot of features
+ * already loaded in the current view. No-ops for raster/WMS/3D/bitmap tiles,
+ * unsupported layer types, and non-polygon drawings.
+ * @memberof visStateActions
+ * @param layerId Layer whose dataset should be extracted
+ * @return action
+ * @public
+ * @example
+ * import {extractDataFromFeature} from '@kepler.gl/actions';
+ *
+ * this.props.dispatch(extractDataFromFeature({layerId: 'point-layer-id'}));
+ */
+export function extractDataFromFeature({
+ layerId
+}: ExtractDataFromFeatureUpdaterAction): Merge<
+ ExtractDataFromFeatureUpdaterAction,
+ {type: typeof ActionTypes.EXTRACT_DATA_FROM_FEATURE}
+> {
+ return {
+ type: ActionTypes.EXTRACT_DATA_FROM_FEATURE,
+ layerId
+ };
+}
+
type FileContent = {
fileName: string;
header: string[];
diff --git a/src/components/src/editor/editor.tsx b/src/components/src/editor/editor.tsx
index 8fe324a3d..b3a165560 100644
--- a/src/components/src/editor/editor.tsx
+++ b/src/components/src/editor/editor.tsx
@@ -20,6 +20,7 @@ import {Layer, EditorLayerUtils} from '@kepler.gl/layers';
import {Filter, FeatureSelectionContext, Feature} from '@kepler.gl/types';
import {Feature as EditableFeature, Polygon} from '@deck.gl-community/editable-layers';
import {Datasets} from '@kepler.gl/table';
+import {isExtractableLayer} from '@kepler.gl/utils';
import {RootContext} from '../context';
@@ -46,6 +47,7 @@ interface EditorProps {
onDeleteFeature: (f: Feature) => any;
onSetFeatureProperties: (f: Feature, properties: Record<string, unknown>) => any;
onTogglePolygonFilter: (l: Layer, f: Feature) => any;
+ onExtractData?: (l: Layer) => any;
}
export type PortalEditorProps = FeatureActionPanelProps & {
@@ -63,11 +65,13 @@ export default function EditorFactory(
selectedFeature,
datasets,
layers,
+ extractLayers,
currentFilter,
onClose,
onDeleteFeature,
onSetFeatureProperties,
onToggleLayer,
+ onExtractData,
position
}) => {
return (
@@ -81,11 +85,13 @@ export default function EditorFactory(
selectedFeature={selectedFeature as EditableFeature<Polygon>}
datasets={datasets}
layers={layers}
+ extractLayers={extractLayers}
currentFilter={currentFilter}
onClose={onClose}
onDeleteFeature={onDeleteFeature}
onSetFeatureProperties={onSetFeatureProperties}
onToggleLayer={onToggleLayer}
+ onExtractData={onExtractData}
position={position || null}
/>
) : null}
@@ -113,6 +119,7 @@ export default function EditorFactory(
}
layerSelector = (props: EditorProps) => props.layers;
+ datasetsSelector = (props: EditorProps) => props.datasets;
filterSelector = (props: EditorProps) => props.filters;
selectedFeatureIdSelector = (props: EditorProps) =>
get(props, ['editor', 'selectedFeature', 'id']);
@@ -130,6 +137,18 @@ export default function EditorFactory(
.filter(layer => layer.config?.isVisible && layer.id !== GEOCODER_LAYER_ID)
);
+ extractableLayersSelector = createSelector(
+ this.layerSelector,
+ this.datasetsSelector,
+ (layers, datasets) =>
+ layers.filter(
+ layer =>
+ layer.config?.isVisible &&
+ layer.id !== GEOCODER_LAYER_ID &&
+ isExtractableLayer(layer, datasets)
+ )
+ );
+
allFeaturesSelector = createSelector(
this.filterSelector,
this.editorFeatureSelector,
@@ -202,6 +221,10 @@ export default function EditorFactory(
}
};
+ _onExtractData = (layer: Layer) => {
+ this.props.onExtractData?.(layer);
+ };
+
_onSetFeatureProperties = (feature: Feature, properties: Record<string, unknown>) => {
this.props.onSetFeatureProperties(feature, properties);
};
@@ -211,6 +234,7 @@ export default function EditorFactory(
const {selectedFeature, selectionContext} = editor;
const currentFilter = this.currentFilterSelector(this.props);
const availableLayers = this.availableLayersSelector(this.props);
+ const extractableLayers = this.extractableLayersSelector(this.props);
const {rightClick, position, mapIndex} = selectionContext || {};
@@ -220,6 +244,7 @@ export default function EditorFactory(
visiblePanel={Boolean(rightClick) && selectedFeature && index === mapIndex}
datasets={datasets}
layers={availableLayers}
+ extractLayers={extractableLayers}
currentFilter={currentFilter}
onClose={this._closeFeatureAction}
onDeleteFeature={this._onDeleteSelectedFeature}
@@ -227,6 +252,7 @@ export default function EditorFactory(
this._onSetFeatureProperties as FeatureActionPanelProps['onSetFeatureProperties']
}
onToggleLayer={this._togglePolygonFilter}
+ onExtractData={this._onExtractData}
position={position || null}
className={className}
style={style}
diff --git a/src/components/src/editor/feature-action-panel.tsx b/src/components/src/editor/feature-action-panel.tsx
index 0f102c015..d0383ef50 100644
--- a/src/components/src/editor/feature-action-panel.tsx
+++ b/src/components/src/editor/feature-action-panel.tsx
@@ -12,10 +12,10 @@ import {Layer} from '@kepler.gl/layers';
import {Filter} from '@kepler.gl/types';
import {Feature} from '@deck.gl-community/editable-layers';
import {Datasets} from '@kepler.gl/table';
-import {canApplyFeatureFilter, getApplicationConfig} from '@kepler.gl/utils';
+import {canApplyFeatureFilter, getApplicationConfig, isExtractableLayer} from '@kepler.gl/utils';
import ActionPanel, {ActionPanelItem} from '../common/action-panel';
-import {Trash, Layers, Copy, Checkmark, Edit} from '../common/icons';
+import {Trash, Layers, Copy, Checkmark, Edit, Files} from '../common/icons';
import FeaturePropertiesEditor from './feature-properties-editor';
const LAYOVER_OFFSET = 4;
@@ -29,13 +29,32 @@ const StyledActionsLayer = styled.div`
.layer-panel-item-disabled {
color: ${props => props.theme.textColor};
}
+ .editor-extract-list .nested-group {
+ max-width: none;
+ overflow: visible;
+ width: max-content;
+ }
+ .editor-extract-list:hover .nested-group {
+ display: flex;
+ flex-direction: column;
+ align-items: stretch;
+ }
+ .editor-extract-list .nested-group > .action-panel-item {
+ max-width: none;
+ width: auto;
+ }
+ .extract-layer-panel-item .label {
+ overflow: visible;
+ text-overflow: unset;
+ }
`;
const defaultActionIcons = {
remove: Trash,
layer: Layers,
copy: Copy,
copied: Checkmark,
- edit: Edit
+ edit: Edit,
+ extract: Files
};
PureFeatureActionPanelFactory.deps = [];
@@ -48,9 +67,11 @@ export interface FeatureActionPanelProps {
y: number;
} | null;
layers: Layer[];
+ extractLayers?: Layer[];
currentFilter?: Filter;
onToggleLayer: (layer: Layer) => void;
onDeleteFeature: () => void;
+ onExtractData?: (layer: Layer) => void;
onSetFeatureProperties?: (feature: Feature, properties: Record<string, unknown>) => void;
onClose?: () => void;
children?: React.ReactNode;
@@ -66,9 +87,11 @@ export function PureFeatureActionPanelFactory(): React.FC<FeatureActionPanelProp
selectedFeature,
position = null,
layers,
+ extractLayers,
currentFilter,
onToggleLayer,
onDeleteFeature,
+ onExtractData,
onSetFeatureProperties,
actionIcons = defaultActionIcons,
children,
@@ -106,6 +129,9 @@ export function PureFeatureActionPanelFactory(): React.FC<FeatureActionPanelProp
const canFilterLayers = canApplyFeatureFilter(selectedFeature as any);
const enableSketches = getApplicationConfig().enableDrawOnMapSketches;
+ const extractableLayers = canFilterLayers
+ ? (extractLayers ?? layers).filter(layer => isExtractableLayer(layer, datasets))
+ : [];
return (
<StyledActionsLayer
ref={refs.setFloating}
@@ -153,6 +179,52 @@ export function PureFeatureActionPanelFactory(): React.FC<FeatureActionPanelProp
)}
</ActionPanelItem>
) : null}
+ {canFilterLayers ? (
+ <ActionPanelItem
+ className="editor-extract-list"
+ label={intl.formatMessage({
+ id: 'editor.extractData',
+ defaultMessage: 'Extract data'
+ })}
+ Icon={actionIcons.extract}
+ >
+ {extractableLayers.length ? (
+ extractableLayers.map((layer, index) => {
+ const dataset = datasets[layer.config.dataId];
+ return (
+ <ActionPanelItem
+ key={layer.id || index}
+ label={intl.formatMessage(
+ {
+ id: 'editor.extractFromLayer',
+ defaultMessage: 'from {layerName} layer'
+ },
+ {layerName: layer.config.label}
+ )}
+ // @ts-ignore
+ color={dataset?.color}
+ onClick={() => {
+ onExtractData?.(layer);
+ onClose?.();
+ }}
+ className="extract-layer-panel-item"
+ />
+ );
+ })
+ ) : (
+ <ActionPanelItem
+ key={'no-layers-extract'}
+ label={intl.formatMessage({
+ id: 'editor.noLayersToExtract',
+ defaultMessage: 'No layers to extract'
+ })}
+ isSelection={false}
+ isActive={false}
+ className="layer-panel-item-disabled"
+ />
+ )}
+ </ActionPanelItem>
+ ) : null}
{enableSketches ? (
<ActionPanelItem
label={intl.formatMessage({
diff --git a/src/components/src/map-container.tsx b/src/components/src/map-container.tsx
index 876aacf2c..1207ff36a 100644
--- a/src/components/src/map-container.tsx
+++ b/src/components/src/map-container.tsx
@@ -1506,6 +1506,7 @@ export default function MapContainerFactory(
onSetFeatureProperties={visStateActions.setEditorFeatureProperties}
onSelect={visStateActions.setSelectedFeature}
onTogglePolygonFilter={visStateActions.setPolygonFilterLayer}
+ onExtractData={layer => visStateActions.extractDataFromFeature({layerId: layer.id})}
onSetEditorMode={visStateActions.setEditorMode}
style={{
pointerEvents: 'all',
diff --git a/src/localization/src/translations/ca.ts b/src/localization/src/translations/ca.ts
index f777ab1cb..1e22e9bbc 100644
--- a/src/localization/src/translations/ca.ts
+++ b/src/localization/src/translations/ca.ts
@@ -411,12 +411,15 @@ export default {
filterLayer: 'Filtra capes',
filterLayerDisabled: 'Les geometries no poligonals no es poden utilitzar per filtrar',
copyGeometry: 'Copia geometria',
+ extractData: 'Extreu dades',
+ extractFromLayer: 'de la capa {layerName}',
editProperties: 'Edita propietats',
propertyName: 'Propietat',
propertyValue: 'Valor',
removeProperty: 'Suprimeix la propietat',
polygonFilter: 'Filtre de polígon',
- noLayersToFilter: 'No hi ha capes per filtrar'
+ noLayersToFilter: 'No hi ha capes per filtrar',
+ noLayersToExtract: 'No hi ha capes per extreure'
},
exportVideoModal: {
animation: 'Animació',
diff --git a/src/localization/src/translations/cn.ts b/src/localization/src/translations/cn.ts
index 07ba0b962..58367acf9 100644
--- a/src/localization/src/translations/cn.ts
+++ b/src/localization/src/translations/cn.ts
@@ -396,12 +396,15 @@ export default {
filterLayer: '过滤图层',
filterLayerDisabled: '非多边形几何图形不能用于过滤',
copyGeometry: '复制几何图形',
+ extractData: '提取数据',
+ extractFromLayer: '从 {layerName} 图层',
editProperties: '编辑属性',
propertyName: '属性',
propertyValue: '值',
removeProperty: '删除属性',
polygonFilter: '多边形过滤器',
- noLayersToFilter: '没有可过滤的图层'
+ noLayersToFilter: '没有可过滤的图层',
+ noLayersToExtract: '没有可提取的图层'
},
exportVideoModal: {
animation: '动画',
diff --git a/src/localization/src/translations/en.ts b/src/localization/src/translations/en.ts
index 177c0fd1d..529c046c5 100644
--- a/src/localization/src/translations/en.ts
+++ b/src/localization/src/translations/en.ts
@@ -500,12 +500,15 @@ export default {
filterLayer: 'Filter Layers',
filterLayerDisabled: 'Non-polygon geometries cannot be used for filtering',
copyGeometry: 'Copy Geometry',
+ extractData: 'Extract data',
+ extractFromLayer: 'from {layerName} layer',
editProperties: 'Edit Properties',
propertyName: 'Property',
propertyValue: 'Value',
removeProperty: 'Remove property',
polygonFilter: 'Polygon filter',
- noLayersToFilter: 'No layers to filter'
+ noLayersToFilter: 'No layers to filter',
+ noLayersToExtract: 'No layers to extract'
},
exportVideoModal: {
diff --git a/src/localization/src/translations/es.ts b/src/localization/src/translations/es.ts
index 592e1cad6..dc1f9d1fc 100644
--- a/src/localization/src/translations/es.ts
+++ b/src/localization/src/translations/es.ts
@@ -412,12 +412,15 @@ export default {
filterLayer: 'Filtrar capas',
filterLayerDisabled: 'Las geometrías no poligonales no se pueden usar para filtrar',
copyGeometry: 'Copiar geometría',
+ extractData: 'Extraer datos',
+ extractFromLayer: 'de la capa {layerName}',
editProperties: 'Editar propiedades',
propertyName: 'Propiedad',
propertyValue: 'Valor',
removeProperty: 'Quitar propiedad',
polygonFilter: 'Filtro de polígono',
- noLayersToFilter: 'No hay capas para filtrar'
+ noLayersToFilter: 'No hay capas para filtrar',
+ noLayersToExtract: 'No hay capas para extraer'
},
exportVideoModal: {
animation: 'Animación',
diff --git a/src/localization/src/translations/fi.ts b/src/localization/src/translations/fi.ts
index 627c64e85..d307f7e70 100644
--- a/src/localization/src/translations/fi.ts
+++ b/src/localization/src/translations/fi.ts
@@ -410,12 +410,15 @@ export default {
filterLayer: 'Suodata tasoja',
filterLayerDisabled: 'Ei-monikulmiogeometrioita ei voi käyttää suodatukseen',
copyGeometry: 'Kopioi geometria',
+ extractData: 'Poimi tiedot',
+ extractFromLayer: 'tasosta {layerName}',
editProperties: 'Muokkaa ominaisuuksia',
propertyName: 'Ominaisuus',
propertyValue: 'Arvo',
removeProperty: 'Poista ominaisuus',
polygonFilter: 'Polygonisuodatin',
- noLayersToFilter: 'Ei tasoja suodatettavaksi'
+ noLayersToFilter: 'Ei tasoja suodatettavaksi',
+ noLayersToExtract: 'Ei tasoja poimittavaksi'
},
exportVideoModal: {
animation: 'Animaatio',
diff --git a/src/localization/src/translations/ja.ts b/src/localization/src/translations/ja.ts
index f45df252d..04a11dc63 100644
--- a/src/localization/src/translations/ja.ts
+++ b/src/localization/src/translations/ja.ts
@@ -408,12 +408,15 @@ export default {
filterLayer: 'レイヤをフィルタ',
filterLayerDisabled: 'ポリゴン以外のジオメトリはフィルタリングに使用できません',
copyGeometry: 'ジオメトリをコピー',
+ extractData: 'データを抽出',
+ extractFromLayer: '{layerName} レイヤーから',
editProperties: 'プロパティを編集',
propertyName: 'プロパティ',
propertyValue: '値',
removeProperty: 'プロパティを削除',
polygonFilter: 'ポリゴンフィルタ',
- noLayersToFilter: 'フィルタするレイヤがありません'
+ noLayersToFilter: 'フィルタするレイヤがありません',
+ noLayersToExtract: '抽出できるレイヤがありません'
},
exportVideoModal: {
animation: 'アニメーション',
diff --git a/src/localization/src/translations/pt.ts b/src/localization/src/translations/pt.ts
index bc3523ff6..5666ac47a 100644
--- a/src/localization/src/translations/pt.ts
+++ b/src/localization/src/translations/pt.ts
@@ -412,12 +412,15 @@ export default {
filterLayer: 'Filtrar camadas',
filterLayerDisabled: 'Geometrias não poligonais não podem ser usadas para filtragem',
copyGeometry: 'Copiar geometria',
+ extractData: 'Extrair dados',
+ extractFromLayer: 'da camada {layerName}',
editProperties: 'Editar propriedades',
propertyName: 'Propriedade',
propertyValue: 'Valor',
removeProperty: 'Remover propriedade',
polygonFilter: 'Filtro de polígono',
- noLayersToFilter: 'Sem camadas para filtrar'
+ noLayersToFilter: 'Sem camadas para filtrar',
+ noLayersToExtract: 'Sem camadas para extrair'
},
exportVideoModal: {
animation: 'Animação',
diff --git a/src/localization/src/translations/ru.ts b/src/localization/src/translations/ru.ts
index 94331bb6c..84ad5a72e 100644
--- a/src/localization/src/translations/ru.ts
+++ b/src/localization/src/translations/ru.ts
@@ -411,12 +411,15 @@ export default {
filterLayer: 'Слои фильтров',
filterLayerDisabled: 'Неполигональные геометрии нельзя использовать для фильтрации',
copyGeometry: 'Копировать геометрию',
+ extractData: 'Извлечь данные',
+ extractFromLayer: 'из слоя {layerName}',
editProperties: 'Изменить свойства',
propertyName: 'Свойство',
propertyValue: 'Значение',
removeProperty: 'Удалить свойство',
polygonFilter: 'Полигональный фильтр',
- noLayersToFilter: 'Нет слоев для фильтрации'
+ noLayersToFilter: 'Нет слоев для фильтрации',
+ noLayersToExtract: 'Нет слоев для извлечения'
},
exportVideoModal: {
animation: 'Анимация',
diff --git a/src/reducers/src/vis-state-updaters.ts b/src/reducers/src/vis-state-updaters.ts
index 6218eb86d..c8512b72f 100644
--- a/src/reducers/src/vis-state-updaters.ts
+++ b/src/reducers/src/vis-state-updaters.ts
@@ -68,6 +68,8 @@ import {
adjustValueToFilterDomain,
errorNotification,
editorFeaturesToFeatureCollection,
+ extractRowsInsideFeature,
+ isVectorTileExtractLayer,
mergeUserFeatureProperties,
toSketchFeature,
featureToFilterValue,
@@ -5210,6 +5212,111 @@ export function convertEditorFeaturesToLayerUpdater(
});
}
+/**
+ * Copy in-memory rows (or loaded vector-tile features) inside the selected
+ * Draw on Map polygon into a new dataset.
+ */
+export function extractDataFromFeatureUpdater(
+ state: VisState,
+ {layerId}: VisStateActions.ExtractDataFromFeatureUpdaterAction
+): VisState {
+ const feature = state.editor.selectedFeature;
+ const layer = state.layers.find(l => l.id === layerId);
+ const dataId = layer?.config.dataId;
+ let dataset = dataId ? state.datasets[dataId] : null;
+
+ if (!layer || !dataset || !dataId) {
+ return state;
+ }
+
+ // GPU range/time filters are not reflected in filteredIndex; evaluate them on CPU
+ // the same way export data does, then clip the result to the drawing.
+ if (!isVectorTileExtractLayer(layer)) {
+ state = filterDatasetCPU(state, dataId);
+ dataset = state.datasets[dataId];
+ if (!dataset) {
+ return state;
+ }
+ }
+
+ let extracted;
+ try {
+ extracted = extractRowsInsideFeature({layer, dataset, feature});
+ } catch (err) {
+ const message = err instanceof Error ? err.message : String(err);
+ return withTask(
+ state,
+ ACTION_TASK_ADD_NOTIFICATION().map(() =>
+ addNotification(
+ errorNotification({
+ message: `Failed to extract data: ${message}`,
+ id: 'extract-data-from-feature'
+ })
+ )
+ )
+ );
+ }
+
+ if (!extracted) {
+ return state;
+ }
+
+ if (!extracted.rowCount) {
+ return withTask(
+ state,
+ ACTION_TASK_ADD_NOTIFICATION().map(() =>
+ addNotification(
+ errorNotification({
+ message: isVectorTileExtractLayer(layer)
+ ? 'No loaded vector tile features found inside the selected drawing'
+ : 'No rows found inside the selected drawing',
+ id: 'extract-data-from-feature-empty'
+ })
+ )
+ )
+ );
+ }
+
+ let data;
+ try {
+ data =
+ extracted.kind === 'geojson'
+ ? processGeojson({
+ type: 'FeatureCollection',
+ features: extracted.features
+ })
+ : {fields: extracted.fields, rows: extracted.rows};
+ } catch (err) {
+ const message = err instanceof Error ? err.message : String(err);
+ return withTask(
+ state,
+ ACTION_TASK_ADD_NOTIFICATION().map(() =>
+ addNotification(
+ errorNotification({
+ message: `Failed to extract data: ${message}`,
+ id: 'extract-data-from-feature'
+ })
+ )
+ )
+ );
+ }
+
+ return updateVisDataUpdater(state, {
+ datasets: {
+ info: {
+ id: `extract-${generateHashId(6)}`,
+ label: `Extracted ${dataset.label}`
+ },
+ data
+ },
+ options: {
+ keepExistingConfig: true,
+ centerMap: false,
+ autoCreateLayers: true
+ }
+ });
+}
+
export function setFilterAnimationTimeConfigUpdater(
state: VisState,
{idx, config}: VisStateActions.SetFilterAnimationTimeConfigAction
diff --git a/src/reducers/src/vis-state.ts b/src/reducers/src/vis-state.ts
index 9cc07b9f8..09e824f8b 100644
--- a/src/reducers/src/vis-state.ts
+++ b/src/reducers/src/vis-state.ts
@@ -160,6 +160,8 @@ const actionHandler = {
[ActionTypes.CONVERT_EDITOR_FEATURES_TO_LAYER]:
visStateUpdaters.convertEditorFeaturesToLayerUpdater,
+ [ActionTypes.EXTRACT_DATA_FROM_FEATURE]: visStateUpdaters.extractDataFromFeatureUpdater,
+
[ActionTypes.TOGGLE_FILTER_FEATURE]: visStateUpdaters.toggleFilterFeatureUpdater,
[ActionTypes.APPLY_CPU_FILTER]: visStateUpdaters.applyCPUFilterUpdater,
diff --git a/src/utils/package.json b/src/utils/package.json
index 9b694fc91..96c041709 100644
--- a/src/utils/package.json
+++ b/src/utils/package.json
@@ -36,6 +36,7 @@
"@luma.gl/constants": "^9.3.2",
"@luma.gl/core": "^9.3.2",
"@mapbox/geo-viewport": "^0.4.1",
+ "@turf/boolean-intersects": "^7.2.0",
"@turf/boolean-within": "^7.2.0",
"@turf/helpers": "^7.2.0",
"@types/d3-array": "^2.8.0",
diff --git a/src/utils/src/extract-dataset-utils.ts b/src/utils/src/extract-dataset-utils.ts
new file mode 100644
index 000000000..38a094a73
--- /dev/null
+++ b/src/utils/src/extract-dataset-utils.ts
@@ -0,0 +1,281 @@
+// SPDX-License-Identifier: MIT
+// Copyright contributors to the kepler.gl project
+
+import {booleanIntersects} from '@turf/boolean-intersects';
+import {DatasetType, EDITOR_AVAILABLE_LAYERS, LAYER_TYPES} from '@kepler.gl/constants';
+import {Feature, ProtoDatasetField} from '@kepler.gl/types';
+import type {Geometry} from 'geojson';
+
+import {DataContainerInterface} from './data-container-interface';
+import {
+ canApplyFeatureFilter,
+ generatePolygonFilter,
+ getPolygonFilterFunctor
+} from './filter-utils';
+
+const NON_EXTRACTABLE_DATASET_TYPES = new Set<string>([
+ DatasetType.RASTER_TILE,
+ DatasetType.WMS_TILE,
+ DatasetType.TILE_3D,
+ DatasetType.BITMAP
+]);
+
+const VECTOR_TILE_ID_FIELDS = [
+ 'ufid',
+ 'UFID',
+ 'id',
+ 'ID',
+ 'fid',
+ 'FID',
+ 'objectid',
+ 'OBJECTID',
+ 'gid',
+ 'GID',
+ 'feature_id',
+ 'FEATURE_ID',
+ '_id'
+];
+
+export type ExtractableDataset = {
+ type?: string;
+ disableDataOperation?: boolean;
+ fields: Array<{
+ name: string;
+ type: string;
+ format?: string;
+ analyzerType?: string;
+ displayName?: string;
+ }>;
+ dataContainer: DataContainerInterface;
+ filteredIndex?: number[];
+ filteredIdxCPU?: number[];
+ allIndexes?: number[];
+};
+
+export type ExtractableLayer = {
+ type?: string | null;
+ config: {
+ dataId: string | null;
+ label?: string;
+ isVisible?: boolean;
+ uniqueIdField?: string;
+ };
+ tileDataset?: {
+ getTiles?: () => readonly any[];
+ };
+ [key: string]: any;
+};
+
+export type ExtractedTable = {
+ kind: 'table';
+ fields: ProtoDatasetField[];
+ rows: any[][];
+ rowCount: number;
+};
+
+export type ExtractedGeojson = {
+ kind: 'geojson';
+ features: Array<{
+ type: 'Feature';
+ id?: string | number;
+ geometry: any;
+ properties?: Record<string, any> | null;
+ }>;
+ rowCount: number;
+};
+
+export type ExtractResult = ExtractedTable | ExtractedGeojson;
+
+/**
+ * Tiled raster/WMS/3D sources have no copyable rows. Vector tiles are extractable
+ * from the layer's loaded tile cache, not from dataset.rows.
+ */
+export function isExtractableDataset(dataset?: ExtractableDataset | null): boolean {
+ if (!dataset || dataset.disableDataOperation) {
+ return false;
+ }
+ if (dataset.type === DatasetType.VECTOR_TILE) {
+ return false;
+ }
+ if (dataset.type && NON_EXTRACTABLE_DATASET_TYPES.has(dataset.type)) {
+ return false;
+ }
+ return Boolean(dataset.dataContainer?.numRows?.());
+}
+
+export function isVectorTileExtractLayer(layer?: ExtractableLayer | null): boolean {
+ return Boolean(layer && layer.type === LAYER_TYPES.vectorTile);
+}
+
+export function isExtractableLayer(
+ layer: ExtractableLayer,
+ datasets: Record<string, ExtractableDataset | undefined>
+): boolean {
+ if (isVectorTileExtractLayer(layer)) {
+ return Boolean(layer.config?.dataId);
+ }
+ // getPolygonFilterFunctor returns true for unsupported types, which would copy
+ // every filtered row. Only layers with a real spatial predicate can extract.
+ if (!EDITOR_AVAILABLE_LAYERS.includes(layer.type || '')) {
+ return false;
+ }
+ const dataId = layer.config.dataId;
+ return Boolean(dataId && isExtractableDataset(datasets[dataId]));
+}
+
+function cloneFieldDescriptors(fields: ExtractableDataset['fields']): ProtoDatasetField[] {
+ return fields.map(field => ({
+ name: field.name,
+ type: field.type,
+ format: field.format,
+ analyzerType: field.analyzerType,
+ displayName: field.displayName
+ }));
+}
+
+function getTileFeatures(tile: any): any[] {
+ const content = tile?.content;
+ if (!content) {
+ return [];
+ }
+ if (content.shape === 'geojson-table') {
+ return Array.isArray(content.features) ? content.features : [];
+ }
+ return Array.isArray(content) ? content : [];
+}
+
+function toGeojsonGeometry(feature: any): Geometry | null {
+ if (!feature) {
+ return null;
+ }
+ const geometry = feature.type === 'Feature' || feature.geometry ? feature.geometry : feature;
+ if (!geometry?.type) {
+ return null;
+ }
+ return geometry as Geometry;
+}
+
+function featureIntersectsDrawnPolygon(feature: any, polygon: Feature): boolean {
+ const geometry = toGeojsonGeometry(feature);
+ const drawnGeometry = polygon?.geometry as Geometry | undefined;
+ if (!geometry || !drawnGeometry) {
+ return false;
+ }
+ try {
+ return booleanIntersects(geometry, drawnGeometry);
+ } catch {
+ return false;
+ }
+}
+
+function vectorTileFeatureId(feature: any, uniqueIdField?: string): string | null {
+ const properties = feature?.properties || {};
+ if (uniqueIdField && properties[uniqueIdField] != null) {
+ return String(properties[uniqueIdField]);
+ }
+ if (feature?.id != null) {
+ return String(feature.id);
+ }
+ for (let i = 0; i < VECTOR_TILE_ID_FIELDS.length; i++) {
+ const key = VECTOR_TILE_ID_FIELDS[i];
+ if (properties[key] != null) {
+ return `${key}:${properties[key]}`;
+ }
+ }
+ return null;
+}
+
+function cloneGeojsonFeature(feature: any) {
+ return JSON.parse(JSON.stringify(feature));
+}
+
+/**
+ * Copy GeoJSON features from currently loaded vector tiles that intersect a drawing.
+ * Coverage is the current viewport/zoom tile cache, not the full tileset.
+ */
+export function extractVectorTileFeaturesInsideFeature({
+ layer,
+ feature
+}: {
+ layer: ExtractableLayer;
+ feature: Feature | null;
+}): ExtractedGeojson | null {
+ if (!feature || !canApplyFeatureFilter(feature) || !isVectorTileExtractLayer(layer)) {
+ return null;
+ }
+
+ const tiles = layer.tileDataset?.getTiles?.() || [];
+ const uniqueIdField = layer.config?.uniqueIdField;
+ const seen = new Set<string>();
+ const features: ExtractedGeojson['features'] = [];
+
+ for (let t = 0; t < tiles.length; t++) {
+ const tileFeatures = getTileFeatures(tiles[t]);
+ for (let i = 0; i < tileFeatures.length; i++) {
+ const tileFeature = tileFeatures[i];
+ if (!featureIntersectsDrawnPolygon(tileFeature, feature)) {
+ continue;
+ }
+ const id = vectorTileFeatureId(tileFeature, uniqueIdField);
+ if (id) {
+ if (seen.has(id)) {
+ continue;
+ }
+ seen.add(id);
+ }
+ features.push(cloneGeojsonFeature(tileFeature));
+ }
+ }
+
+ return {kind: 'geojson', features, rowCount: features.length};
+}
+
+/**
+ * Copy in-memory rows whose geometry falls inside a drawn polygon.
+ * Existing table filters (range/select/time), including GPU-backed ones via
+ * `filteredIdxCPU`, are applied; the polygon clip is extra.
+ * Vector tiles use currently loaded viewport tiles instead of dataset.rows.
+ */
+export function extractRowsInsideFeature({
+ layer,
+ dataset,
+ feature
+}: {
+ layer: ExtractableLayer;
+ dataset: ExtractableDataset;
+ feature: Feature | null;
+}): ExtractResult | null {
+ if (isVectorTileExtractLayer(layer)) {
+ return extractVectorTileFeaturesInsideFeature({layer, feature});
+ }
+
+ if (
+ !feature ||
+ !canApplyFeatureFilter(feature) ||
+ !isExtractableDataset(dataset) ||
+ !EDITOR_AVAILABLE_LAYERS.includes(layer.type || '')
+ ) {
+ return null;
+ }
+
+ const polygonFilter = generatePolygonFilter([layer as any], feature);
+ const isInside = getPolygonFilterFunctor(layer, polygonFilter, dataset.dataContainer);
+ // Prefer CPU-evaluated indexes so GPU-backed range/time filters are included.
+ const sourceIndexes = dataset.filteredIdxCPU ?? dataset.filteredIndex ?? dataset.allIndexes ?? [];
+
+ const rows: any[][] = [];
+ for (let i = 0; i < sourceIndexes.length; i++) {
+ const index = sourceIndexes[i];
+ if (!isInside({index})) {
+ continue;
+ }
+ rows.push(dataset.dataContainer.rowAsArray(index).slice());
+ }
+
+ return {
+ kind: 'table',
+ fields: cloneFieldDescriptors(dataset.fields),
+ rows,
+ rowCount: rows.length
+ };
+}
diff --git a/src/utils/src/index.ts b/src/utils/src/index.ts
index 132bd0822..3a956353c 100644
--- a/src/utils/src/index.ts
+++ b/src/utils/src/index.ts
@@ -164,6 +164,20 @@ export * from './filter-utils';
export type {FilterChanged, FilterResult, dataValueAccessor} from './filter-utils';
export * from './editor-feature-utils';
export type {EditorFeatureCollection} from './editor-feature-utils';
+export {
+ extractRowsInsideFeature,
+ extractVectorTileFeaturesInsideFeature,
+ isExtractableDataset,
+ isExtractableLayer,
+ isVectorTileExtractLayer
+} from './extract-dataset-utils';
+export type {
+ ExtractableDataset,
+ ExtractableLayer,
+ ExtractedGeojson,
+ ExtractedTable,
+ ExtractResult
+} from './extract-dataset-utils';
export {
colorMapToColorBreaks,
diff --git a/test/browser/components/editor/feature-action-panel-test.js b/test/browser/components/editor/feature-action-panel-test.js
index 046f22054..3ccca4f20 100644
--- a/test/browser/components/editor/feature-action-panel-test.js
+++ b/test/browser/components/editor/feature-action-panel-test.js
@@ -27,7 +27,8 @@ test('FeatureActionPanel -> display layers', t => {
const datasets = {
puppy: {
- color: [123, 123, 123]
+ color: [123, 123, 123],
+ dataContainer: {numRows: () => 4}
}
};
@@ -122,3 +123,116 @@ test('FeatureActionPanel -> edit properties', t => {
t.end();
});
+
+test('FeatureActionPanel -> extract data', t => {
+ const layers = [
+ {
+ id: 'layer-1',
+ type: 'point',
+ config: {
+ label: 'layer 1',
+ dataId: 'puppy'
+ }
+ }
+ ];
+ const datasets = {
+ puppy: {
+ label: 'puppy.csv',
+ color: [123, 123, 123],
+ dataContainer: {numRows: () => 4}
+ }
+ };
+ const selectedFeature = {type: 'Feature', geometry: {type: 'Polygon', coordinates: []}};
+ const onExtractData = sinon.spy();
+ const onClose = sinon.spy();
+
+ const wrapper = mountWithTheme(
+ <IntlWrapper>
+ <FeatureActionPanel
+ className="action-item-test"
+ layers={layers}
+ datasets={datasets}
+ selectedFeature={selectedFeature}
+ onToggleLayer={() => {}}
+ onDeleteFeature={() => {}}
+ onExtractData={onExtractData}
+ onClose={onClose}
+ position={{x: 0, y: 0}}
+ />
+ </IntlWrapper>
+ );
+
+ t.ok(wrapper.find('.editor-extract-list').length, 'Should show Extract data for polygons');
+ t.ok(
+ wrapper.find('.extract-layer-panel-item .label').text().includes('from layer 1 layer'),
+ 'Extract options should name the source layer'
+ );
+
+ wrapper.find('.extract-layer-panel-item').simulate('click');
+ t.ok(onExtractData.calledOnce, 'Clicking a layer should extract from that layer');
+ t.equal(onExtractData.firstCall.args[0].id, 'layer-1', 'Should pass the clicked layer');
+ t.ok(onClose.calledOnce, 'Extract should close the action panel');
+
+ const pointWrapper = mountWithTheme(
+ <IntlWrapper>
+ <FeatureActionPanel
+ layers={layers}
+ datasets={datasets}
+ selectedFeature={{type: 'Feature', geometry: {type: 'Point', coordinates: [0, 0]}}}
+ onToggleLayer={() => {}}
+ onDeleteFeature={() => {}}
+ onExtractData={onExtractData}
+ position={{x: 0, y: 0}}
+ />
+ </IntlWrapper>
+ );
+
+ t.equal(
+ pointWrapper.find('.editor-extract-list').length,
+ 0,
+ 'Should hide Extract data for non-polygon drawings'
+ );
+
+ t.end();
+});
+
+test('FeatureActionPanel -> extract vector tile dataset', t => {
+ const extractLayers = [
+ {
+ id: 'vt-1',
+ type: 'vectorTile',
+ config: {
+ label: 'Buildings',
+ dataId: 'tiles'
+ }
+ }
+ ];
+ const datasets = {
+ tiles: {
+ label: 'buildings.pmtiles',
+ color: [123, 123, 123],
+ disableDataOperation: true,
+ dataContainer: {numRows: () => 0}
+ }
+ };
+
+ const wrapper = mountWithTheme(
+ <IntlWrapper>
+ <FeatureActionPanel
+ layers={[]}
+ extractLayers={extractLayers}
+ datasets={datasets}
+ selectedFeature={{type: 'Feature', geometry: {type: 'Polygon', coordinates: []}}}
+ onToggleLayer={() => {}}
+ onDeleteFeature={() => {}}
+ position={{x: 0, y: 0}}
+ />
+ </IntlWrapper>
+ );
+
+ t.ok(
+ wrapper.find('.extract-layer-panel-item .label').text().includes('from Buildings layer'),
+ 'Vector tile layers should appear in Extract data'
+ );
+ t.end();
+});
diff --git a/test/node/reducers/vis-state-test.js b/test/node/reducers/vis-state-test.js
index c00989fb7..69e52c6af 100644
--- a/test/node/reducers/vis-state-test.js
+++ b/test/node/reducers/vis-state-test.js
@@ -101,6 +101,7 @@ import {
} from 'test/helpers/comparison-utils';
import {
applyActions,
+ applyExistingDatasetTasks,
StateWTripGeojson,
StateWSplitMaps,
StateWFilters,
@@ -6140,6 +6141,188 @@ test('#visStateReducer -> CONVERT_EDITOR_FEATURES_TO_LAYER', t => {
t.end();
});
+test('#visStateReducer -> EXTRACT_DATA_FROM_FEATURE', t => {
+ const datasets = [
+ {
+ data: {
+ fields: [
+ {name: 'start_point_lat', format: '', fieldIdx: 0, type: 'real', analyzerType: 'FLOAT'},
+ {name: 'start_point_lng', format: '', fieldIdx: 1, type: 'real', analyzerType: 'FLOAT'},
+ {name: 'end_point_lat', format: '', fieldIdx: 2, type: 'real', analyzerType: 'FLOAT'},
+ {name: 'end_point_lng', format: '', fieldIdx: 3, type: 'real', analyzerType: 'FLOAT'}
+ ],
+ rows: mockPolygonData.data
+ },
+ info: {
+ label: 'test.csv',
+ size: 144
+ }
+ }
+ ];
+
+ let state = applyActions(reducer, INITIAL_VIS_STATE, [
+ {
+ action: VisStateActions.updateVisData,
+ payload: [datasets, {centerMap: true, keepExistingConfig: false}, {}]
+ }
+ ]);
+
+ const pointLayer = state.layers.find(layer => layer.type === 'point');
+ t.ok(pointLayer, 'Should create a point layer from lat/lng columns');
+
+ state = reducer(state, VisStateActions.setFeatures([mockPolygonFeature]));
+ state = reducer(state, VisStateActions.setSelectedFeature(mockPolygonFeature));
+
+ const sourceDataId = pointLayer.config.dataId;
+ const sourceRowCount = state.datasets[sourceDataId].dataContainer.numRows();
+ const layerCount = state.layers.length;
+
+ state = applyExistingDatasetTasks(
+ reducer,
+ reducer(state, VisStateActions.extractDataFromFeature({layerId: pointLayer.id}))
+ );
+
+ const extractedIds = Object.keys(state.datasets).filter(id => id !== sourceDataId);
+ t.equal(extractedIds.length, 1, 'Should create one extracted dataset');
+
+ const extracted = state.datasets[extractedIds[0]];
+ t.equal(extracted.dataContainer.numRows(), 2, 'Should copy the two points inside the polygon');
+ t.ok(extracted.label.startsWith('Extracted '), 'Extracted dataset should use Extracted prefix');
+ t.equal(
+ state.datasets[sourceDataId].dataContainer.numRows(),
+ sourceRowCount,
+ 'Source dataset should stay unchanged'
+ );
+ t.equal(state.editor.features.length, 1, 'Should keep the drawing after extract');
+ t.ok(state.layers.length > layerCount, 'Should auto-create a layer for the extracted dataset');
+
+ const emptyState = reducer(
+ state,
+ VisStateActions.extractDataFromFeature({layerId: 'missing-layer'})
+ );
+ t.equal(emptyState, state, 'Should no-op when the layer id is unknown');
+
+ t.end();
+});
+
+test('#visStateReducer -> EXTRACT_DATA_FROM_FEATURE respects GPU range filter', t => {
+ const datasets = [
+ {
+ data: {
+ fields: [
+ {name: 'start_point_lat', format: '', fieldIdx: 0, type: 'real', analyzerType: 'FLOAT'},
+ {name: 'start_point_lng', format: '', fieldIdx: 1, type: 'real', analyzerType: 'FLOAT'},
+ {name: 'end_point_lat', format: '', fieldIdx: 2, type: 'real', analyzerType: 'FLOAT'},
+ {name: 'end_point_lng', format: '', fieldIdx: 3, type: 'real', analyzerType: 'FLOAT'}
+ ],
+ rows: mockPolygonData.data
+ },
+ info: {
+ label: 'test.csv'
+ }
+ }
+ ];
+
+ let state = applyActions(reducer, INITIAL_VIS_STATE, [
+ {
+ action: VisStateActions.updateVisData,
+ payload: [datasets, {centerMap: false, keepExistingConfig: false}, {}]
+ }
+ ]);
+
+ const pointLayer = state.layers.find(layer => layer.type === 'point');
+ t.ok(pointLayer, 'Should create a point layer from lat/lng columns');
+ const sourceDataId = pointLayer.config.dataId;
+
+ state = reducer(state, VisStateActions.addFilter(sourceDataId));
+ state = reducer(state, VisStateActions.setFilter(0, 'name', 'start_point_lat'));
+ state = reducer(state, VisStateActions.setFilter(0, 'value', [12, 12.5]));
+
+ t.equal(state.filters[0].gpu, true, 'Range filter should be GPU-backed');
+ t.equal(
+ state.datasets[sourceDataId].filteredIndex.length,
+ state.datasets[sourceDataId].dataContainer.numRows(),
+ 'GPU filter should not shrink filteredIndex'
+ );
+
+ state = reducer(state, VisStateActions.setFeatures([mockPolygonFeature]));
+ state = reducer(state, VisStateActions.setSelectedFeature(mockPolygonFeature));
+
+ state = applyExistingDatasetTasks(
+ reducer,
+ reducer(state, VisStateActions.extractDataFromFeature({layerId: pointLayer.id}))
+ );
+
+ const extractedIds = Object.keys(state.datasets).filter(id => id !== sourceDataId);
+ t.equal(extractedIds.length, 1, 'Should create one extracted dataset');
+ t.equal(
+ state.datasets[extractedIds[0]].dataContainer.numRows(),
+ 1,
+ 'Should keep only the in-polygon row that also passes the range filter'
+ );
+
+ t.end();
+});
+
+test('#visStateReducer -> EXTRACT_DATA_FROM_FEATURE empty polygon', t => {
+ const datasets = [
+ {
+ data: {
+ fields: [
+ {name: 'start_point_lat', format: '', fieldIdx: 0, type: 'real', analyzerType: 'FLOAT'},
+ {name: 'start_point_lng', format: '', fieldIdx: 1, type: 'real', analyzerType: 'FLOAT'},
+ {name: 'end_point_lat', format: '', fieldIdx: 2, type: 'real', analyzerType: 'FLOAT'},
+ {name: 'end_point_lng', format: '', fieldIdx: 3, type: 'real', analyzerType: 'FLOAT'}
+ ],
+ rows: mockPolygonData.data
+ },
+ info: {
+ label: 'test.csv'
+ }
+ }
+ ];
+
+ let state = applyActions(reducer, INITIAL_VIS_STATE, [
+ {
+ action: VisStateActions.updateVisData,
+ payload: [datasets, {centerMap: false, keepExistingConfig: false}, {}]
+ }
+ ]);
+
+ const farAwayPolygon = {
+ type: 'Feature',
+ id: 'far-away',
+ properties: {isClosed: true},
+ geometry: {
+ type: 'Polygon',
+ coordinates: [
+ [
+ [0, 0],
+ [1, 0],
+ [1, 1],
+ [0, 1],
+ [0, 0]
+ ]
+ ]
+ }
+ };
+
+ const pointLayer = state.layers.find(layer => layer.type === 'point');
+ state = reducer(state, VisStateActions.setFeatures([farAwayPolygon]));
+ state = reducer(state, VisStateActions.setSelectedFeature(farAwayPolygon));
+ const datasetCount = Object.keys(state.datasets).length;
+
+ state = reducer(state, VisStateActions.extractDataFromFeature({layerId: pointLayer.id}));
+ t.equal(
+ Object.keys(state.datasets).length,
+ datasetCount,
+ 'Should not create a dataset when no rows fall inside the drawing'
+ );
+ drainTasksForTesting();
+
+ t.end();
+});
+
test('#visStateReducer -> CONVERT_EDITOR_FEATURES_TO_LAYER disabled by config', t => {
initApplicationConfig({enableDrawOnMapSketches: false});
diff --git a/test/node/utils/extract-dataset-utils-test.js b/test/node/utils/extract-dataset-utils-test.js
new file mode 100644
index 000000000..28b830614
--- /dev/null
+++ b/test/node/utils/extract-dataset-utils-test.js
@@ -0,0 +1,217 @@
+// SPDX-License-Identifier: MIT
+// Copyright contributors to the kepler.gl project
+
+import test from 'tape-catch';
+
+import {DatasetType, LAYER_TYPES} from '@kepler.gl/constants';
+import {
+ createDataContainer,
+ extractRowsInsideFeature,
+ isExtractableDataset,
+ isExtractableLayer
+} from '@kepler.gl/utils';
+import {mockPolygonFeature} from 'test/fixtures/polygon';
+
+test('extract-dataset-utils -> isExtractableDataset', t => {
+ t.notOk(isExtractableDataset(null), 'null is not extractable');
+ t.notOk(
+ isExtractableDataset({
+ type: DatasetType.VECTOR_TILE,
+ fields: [],
+ dataContainer: createDataContainer([[1]])
+ }),
+ 'vector tiles are not extractable'
+ );
+ t.ok(
+ isExtractableDataset({
+ fields: [{name: 'lat', type: 'real'}],
+ dataContainer: createDataContainer([[1]])
+ }),
+ 'local tables with rows are extractable'
+ );
+ t.end();
+});
+
+test('extract-dataset-utils -> isExtractableLayer', t => {
+ const datasets = {
+ local: {
+ fields: [{name: 'lat', type: 'real'}],
+ dataContainer: createDataContainer([[1]])
+ }
+ };
+ t.ok(
+ isExtractableLayer({type: LAYER_TYPES.point, config: {dataId: 'local'}}, datasets),
+ 'layers on local datasets are extractable'
+ );
+ t.notOk(
+ isExtractableLayer({config: {dataId: 'missing'}}, datasets),
+ 'layers without a dataset are not extractable'
+ );
+ t.notOk(
+ isExtractableLayer({type: LAYER_TYPES.trip, config: {dataId: 'local'}}, datasets),
+ 'trip layers have no polygon clip and should not be extractable'
+ );
+ t.notOk(
+ isExtractableLayer({type: LAYER_TYPES.geohash, config: {dataId: 'local'}}, datasets),
+ 'geohash layers have no polygon clip and should not be extractable'
+ );
+ t.notOk(
+ isExtractableLayer({type: LAYER_TYPES.s2, config: {dataId: 'local'}}, datasets),
+ 's2 layers have no polygon clip and should not be extractable'
+ );
+ t.ok(
+ isExtractableLayer(
+ {
+ type: LAYER_TYPES.vectorTile,
+ config: {dataId: 'tiles'}
+ },
+ {
+ tiles: {
+ type: DatasetType.VECTOR_TILE,
+ disableDataOperation: true,
+ fields: [],
+ dataContainer: createDataContainer([])
+ }
+ }
+ ),
+ 'vector tile layers are extractable from the loaded tile cache'
+ );
+ t.end();
+});
+
+test('extract-dataset-utils -> extractRowsInsideFeature vector tiles', t => {
+ const layer = {
+ type: LAYER_TYPES.vectorTile,
+ config: {dataId: 'tiles', uniqueIdField: 'id'},
+ tileDataset: {
+ getTiles: () => [
+ {
+ content: [
+ {
+ type: 'Feature',
+ properties: {id: 1, name: 'inside'},
+ geometry: {type: 'Point', coordinates: [30.5, 12.25]}
+ },
+ {
+ type: 'Feature',
+ properties: {id: 1, name: 'duplicate-tile'},
+ geometry: {type: 'Point', coordinates: [30.5, 12.25]}
+ },
+ {
+ type: 'Feature',
+ properties: {id: 2, name: 'outside'},
+ geometry: {type: 'Point', coordinates: [35.5, 12.25]}
+ }
+ ]
+ }
+ ]
+ }
+ };
+
+ const extracted = extractRowsInsideFeature({
+ layer,
+ dataset: {
+ type: DatasetType.VECTOR_TILE,
+ disableDataOperation: true,
+ fields: [],
+ dataContainer: createDataContainer([])
+ },
+ feature: mockPolygonFeature
+ });
+
+ t.equal(extracted.kind, 'geojson', 'Vector tile extract should return GeoJSON features');
+ t.equal(
+ extracted.rowCount,
+ 1,
+ 'Should keep the point inside the polygon and drop tile duplicates'
+ );
+ t.equal(extracted.features[0].properties.name, 'inside');
+ t.end();
+});
+
+test('extract-dataset-utils -> extractRowsInsideFeature vector tiles intersection', t => {
+ const layer = {
+ type: LAYER_TYPES.vectorTile,
+ config: {dataId: 'tiles', uniqueIdField: 'id'},
+ tileDataset: {
+ getTiles: () => [
+ {
+ content: [
+ {
+ type: 'Feature',
+ properties: {id: 1, name: 'crossing-line'},
+ geometry: {
+ type: 'LineString',
+ coordinates: [
+ [27, 13],
+ [34, 13]
+ ]
+ }
+ },
+ {
+ type: 'Feature',
+ properties: {id: 2, name: 'containing-polygon'},
+ geometry: {
+ type: 'Polygon',
+ coordinates: [
+ [
+ [20, 5],
+ [40, 5],
+ [40, 20],
+ [20, 20],
+ [20, 5]
+ ]
+ ]
+ }
+ },
+ {
+ type: 'Feature',
+ properties: {id: 3, name: 'disjoint-line'},
+ geometry: {
+ type: 'LineString',
+ coordinates: [
+ [40, 20],
+ [41, 21]
+ ]
+ }
+ }
+ ]
+ }
+ ]
+ }
+ };
+
+ const extracted = extractRowsInsideFeature({
+ layer,
+ dataset: {
+ type: DatasetType.VECTOR_TILE,
+ disableDataOperation: true,
+ fields: [],
+ dataContainer: createDataContainer([])
+ },
+ feature: mockPolygonFeature
+ });
+
+ t.equal(extracted.kind, 'geojson', 'Vector tile extract should return GeoJSON features');
+ t.deepEqual(
+ extracted.features.map(feature => feature.properties.name).sort(),
+ ['containing-polygon', 'crossing-line'],
+ 'Should keep a crossing line and a polygon that contains the drawing'
+ );
+ t.end();
+});
+
+test('extract-dataset-utils -> extractRowsInsideFeature unsupported layer', t => {
+ const extracted = extractRowsInsideFeature({
+ layer: {type: LAYER_TYPES.trip, config: {dataId: 'local'}},
+ dataset: {
+ fields: [{name: 'lat', type: 'real'}],
+ dataContainer: createDataContainer([[12.25, 30.5]]),
+ allIndexes: [0],
+ filteredIndex: [0]
+ },
+ feature: mockPolygonFeature
+ });
+ t.equal(extracted, null, 'Unsupported layers should not copy every filtered row');
+ t.end();
+});
diff --git a/test/node/utils/index.js b/test/node/utils/index.js
index ac57e683c..1966dd9c7 100644
--- a/test/node/utils/index.js
+++ b/test/node/utils/index.js
@@ -25,6 +25,7 @@ import './a5-utils-test';
import './geohash-utils-test';
import './editor-utils-test';
import './editor-feature-utils-test';
+import './extract-dataset-utils-test';
import './kepler-gl-utils-test';
import './timeline-test';
import './timeline-zoom-test';
diff --git a/yarn.lock b/yarn.lock
index 5bdec3a02..594af2b65 100644
--- a/yarn.lock
+++ b/yarn.lock
@@ -4504,6 +4504,7 @@ __metadata:
"@luma.gl/constants": "npm:^9.3.2"
"@luma.gl/core": "npm:^9.3.2"
"@mapbox/geo-viewport": "npm:^0.4.1"
+ "@turf/boolean-intersects": "npm:^7.2.0"
"@turf/boolean-within": "npm:^7.2.0"
"@turf/helpers": "npm:^7.2.0"
"@types/d3-array": "npm:^2.8.0"
@@ -9144,6 +9145,32 @@ __metadata:
languageName: node
linkType: hard
+"@turf/boolean-disjoint@npm:7.4.0":
+ version: 7.4.0
+ resolution: "@turf/boolean-disjoint@npm:7.4.0"
+ dependencies:
+ "@turf/boolean-point-in-polygon": "npm:7.4.0"
+ "@turf/helpers": "npm:7.4.0"
+ "@turf/line-intersect": "npm:7.4.0"
+ "@turf/meta": "npm:7.4.0"
+ "@turf/polygon-to-line": "npm:7.4.0"
+ "@types/geojson": "npm:^7946.0.10"
+ tslib: "npm:^2.8.1"
+ checksum: 10c0/fe52efd592fa8756c5428c005a9a4e81f531f7b1b9014184db117e11b396f8da940424cdcf4cb4a771238263d14ea477c45543408c3e880d71b670e46036d68f
+ languageName: node
+ linkType: hard
+
+"@turf/boolean-intersects@npm:^7.2.0":
+ version: 7.4.0
+ resolution: "@turf/boolean-intersects@npm:7.4.0"
+ dependencies:
+ "@turf/boolean-disjoint": "npm:7.4.0"
+ "@types/geojson": "npm:^7946.0.10"
+ tslib: "npm:^2.8.1"
+ checksum: 10c0/699e54f4b397c8f5a6874ac84c98861483f163937cd9b24cff1dc12f30303f1caf5f6d7c3c21e82b008318079cb3bbe99e86fc07a11fd14c29686bb5b9b6852c
+ languageName: node
+ linkType: hard
+
"@turf/boolean-point-in-polygon@npm:7.3.4, @turf/boolean-point-in-polygon@npm:^7.2.0":
version: 7.3.4
resolution: "@turf/boolean-point-in-polygon@npm:7.3.4"
@@ -9157,6 +9184,19 @@ __metadata:
languageName: node
linkType: hard
+"@turf/boolean-point-in-polygon@npm:7.4.0":
+ version: 7.4.0
+ resolution: "@turf/boolean-point-in-polygon@npm:7.4.0"
+ dependencies:
+ "@turf/helpers": "npm:7.4.0"
+ "@turf/invariant": "npm:7.4.0"
+ "@types/geojson": "npm:^7946.0.10"
+ point-in-polygon-hao: "npm:^1.1.0"
+ tslib: "npm:^2.8.1"
+ checksum: 10c0/c4d92534d6e2de4602a6507bf35626a2fc825bb0edb7643908978d98069a14aee1e9cce41812a0575fc08039716b2c74def3bb6f530121f54d86fc2062964b84
+ languageName: node
+ linkType: hard
+
"@turf/boolean-point-on-line@npm:7.3.4":
version: 7.3.4
resolution: "@turf/boolean-point-on-line@npm:7.3.4"
@@ -9333,6 +9373,16 @@ __metadata:
languageName: node
linkType: hard
+"@turf/helpers@npm:7.4.0":
+ version: 7.4.0
+ resolution: "@turf/helpers@npm:7.4.0"
+ dependencies:
+ "@types/geojson": "npm:^7946.0.10"
+ tslib: "npm:^2.8.1"
+ checksum: 10c0/2fb850f41deb834260e655cae29bc3c0efd53614c8444737bd0bba64121cb183b8f641556b4e97d235c5ef5066e56bdb25b83b5b54ba563f60462dd21aac2273
+ languageName: node
+ linkType: hard
+
"@turf/helpers@npm:^5.1.5":
version: 5.1.5
resolution: "@turf/helpers@npm:5.1.5"
@@ -9374,6 +9424,17 @@ __metadata:
languageName: node
linkType: hard
+"@turf/invariant@npm:7.4.0":
+ version: 7.4.0
+ resolution: "@turf/invariant@npm:7.4.0"
+ dependencies:
+ "@turf/helpers": "npm:7.4.0"
+ "@types/geojson": "npm:^7946.0.10"
+ tslib: "npm:^2.8.1"
+ checksum: 10c0/d8ddc9883699471cf578a41fb269aa0538f91fc2425ad0f79c1627cdaa717d23b3f0f5b803ee7052a054e9de79a7e31a83476850f4ab4a78679af4ef6f17b7b3
+ languageName: node
+ linkType: hard
+
"@turf/invariant@npm:^5.1.5":
version: 5.2.0
resolution: "@turf/invariant@npm:5.2.0"
@@ -9415,6 +9476,19 @@ __metadata:
languageName: node
linkType: hard
+"@turf/line-intersect@npm:7.4.0":
+ version: 7.4.0
+ resolution: "@turf/line-intersect@npm:7.4.0"
+ dependencies:
+ "@turf/helpers": "npm:7.4.0"
+ "@types/geojson": "npm:^7946.0.10"
+ robust-predicates: "npm:^2.0.4"
+ tinyqueue: "npm:^2.0.3"
+ tslib: "npm:^2.8.1"
+ checksum: 10c0/02945aab639a8ae3b735f9d68dcf2a0b4885ceac95ec9dc6d34b5311e4771041ed133e2aeb0d4cf848b2839403a7059df639761a9f675315ae2fa190046912c3
+ languageName: node
+ linkType: hard
+
"@turf/line-segment@npm:7.3.4":
version: 7.3.4
resolution: "@turf/line-segment@npm:7.3.4"
@@ -9458,6 +9532,17 @@ __metadata:
languageName: node
linkType: hard
+"@turf/meta@npm:7.4.0":
+ version: 7.4.0
+ resolution: "@turf/meta@npm:7.4.0"
+ dependencies:
+ "@turf/helpers": "npm:7.4.0"
+ "@types/geojson": "npm:^7946.0.10"
+ tslib: "npm:^2.8.1"
+ checksum: 10c0/6d08c84c6573757d6f6df1272ea07d3f8daf4a81e83aa1fea2ed85cdc8047d984bcbc4ae3cc7708d374108c2cba92c78d6e885c475ff32a23ef54f77dc41e767
+ languageName: node
+ linkType: hard
+
"@turf/meta@npm:^5.1.5":
version: 5.2.0
resolution: "@turf/meta@npm:5.2.0"
@@ -9524,6 +9609,18 @@ __metadata:
languageName: node
linkType: hard
+"@turf/polygon-to-line@npm:7.4.0":
+ version: 7.4.0
+ resolution: "@turf/polygon-to-line@npm:7.4.0"
+ dependencies:
+ "@turf/helpers": "npm:7.4.0"
+ "@turf/invariant": "npm:7.4.0"
+ "@types/geojson": "npm:^7946.0.10"
+ tslib: "npm:^2.8.1"
+ checksum: 10c0/d5b46a30c2a87f3c0422280d04e6a39232774476cc6bca83b1e0a6caa6283bf96b58ca16118392b2dcf23affc26a0206cbffc526fd583eb4e18d88759d5d611e
+ languageName: node
+ linkType: hard
+
"@turf/polygon-to-line@npm:^7.2.0":
version: 7.3.4
resolution: "@turf/polygon-to-line@npm:7.3.4"
@@ -28934,6 +29031,13 @@ __metadata:
languageName: node
linkType: hard
+"robust-predicates@npm:^2.0.4":
+ version: 2.0.4
+ resolution: "robust-predicates@npm:2.0.4"
+ checksum: 10c0/3c801ff1ea5ce842c1818da62e2be0cc427ab507dc7a21f3ff2e6278f1d32b5c2f2fc740f8e9c188b4e7a2946228c0173607534432314940e89eae15ded79b5f
+ languageName: node
+ linkType: hard
+
"rrweb-cssom@npm:^0.6.0":
version: 0.6.0
resolution: "rrweb-cssom@npm:0.6.0"