Skip to content

armgit / ente

Public Imported from GitHub
0

πŸ’š End-to-end encrypted cloud for everything.

Make free-up-space deletion crash resilient

ashilknf730a3f01d1a
diff --git a/mobile/apps/photos/changes/crash-safe-free-space.md b/mobile/apps/photos/changes/crash-safe-free-space.md
new file mode 100644
index 0000000000..5f7e21d28d
--- /dev/null
+++ b/mobile/apps/photos/changes/crash-safe-free-space.md
@@ -0,0 +1 @@
+- Made freeing device space resilient to stale local media entries.
diff --git a/mobile/apps/photos/docs/README.md b/mobile/apps/photos/docs/README.md
index 47f83af2ec..1009aadf40 100644
--- a/mobile/apps/photos/docs/README.md
+++ b/mobile/apps/photos/docs/README.md
@@ -6,6 +6,10 @@ Documentation and notes about more advanced or infrequently needed details.
See [android-cli.md](android-cli.md) for working on the Android app without installing Android Studio.
+### Free up space deletion recovery
+
+See [free-up-space-deletion-recovery-plan.md](free-up-space-deletion-recovery-plan.md) for the failure and recovery plan used when deleting backed-up device files.
+
### VS Code
The [vscode](vscode) folder contains template launch configuration. Copy into a top-level `.vscode` to use them.
diff --git a/mobile/apps/photos/docs/free-up-space-deletion-recovery-plan.md b/mobile/apps/photos/docs/free-up-space-deletion-recovery-plan.md
new file mode 100644
index 0000000000..069e98e5ec
--- /dev/null
+++ b/mobile/apps/photos/docs/free-up-space-deletion-recovery-plan.md
@@ -0,0 +1,89 @@
+# Free up space deletion recovery plan
+
+## Goal
+
+Prevent a failed batch from turning into one platform deletion request per file, while keeping the existing recovery flow and UI.
+
+## Result model
+
+Batch deletion must return both the IDs deleted so far and one terminal status:
+
+- `completed`
+- `cancelled`
+- `failed`
+
+Success must not be inferred from `deletedIDs.isNotEmpty`. Already-missing or shared-media IDs must not hide a failure to delete the remaining platform assets.
+
+## Deletion rules
+
+- Remove the individual-ID fallback on Android and iOS.
+- Stop on the first batch exception or user cancellation; do not attempt later batches.
+- Remove `_recursivelyReduceBatchSizeAndRetryDeletion`.
+- Use one shared platform-deletion helper for every attempt, with a maximum batch size of 1,900 where applicable.
+- Preserve any intentional legacy Android batching and empty-result behavior.
+- Checkpoint only the IDs returned as deleted for each successful batch in `FilesDB` before starting the next batch.
+- Close progress dialogs in `finally` without relying on `dialogKey.currentContext!`.
+
+## Recovery flow
+
+1. **Normal deletion**
+ - Delete the requested platform assets in batches.
+ - If completed, finish successfully.
+ - If cancelled, stop without another automatic attempt.
+ - If failed, continue to stale-ID recovery.
+
+2. **Stale-ID recovery**
+ - Verify that Ente has full gallery permission before treating an unresolved asset as missing.
+ - Check which candidates still exist.
+ - Immediately checkpoint already-missing IDs.
+ - Delete the remaining valid assets through the same batched helper.
+ - If no valid assets remain, the operation is complete.
+ - If completed, finish successfully.
+ - If cancelled, stop.
+ - If failed on Android, continue to MediaStore recovery.
+ - If failed on iOS, stop.
+
+3. **MediaStore recovery β€” Android only**
+ - Run `removeAllNoExistsAsset()`.
+ - Run local sync and recalculate the freeable candidates, restricted to the
+ IDs included in the original confirmed operation.
+ - Perform one final batched deletion.
+ - This is the final automatic attempt regardless of its outcome.
+
+## Completion and consistency
+
+- Checkpoint successful platform batches, already-missing IDs, and successfully removed shared-media files independently.
+- If a database checkpoint fails, stop before deleting another batch.
+- A successful operation means all remaining valid candidates were handled, or every candidate was authoritatively found to be already missing.
+- A partially successful operation that later fails must remain failed, while retaining its completed checkpoints.
+- Add an operation lock so two free-space deletions cannot run concurrently.
+
+## Existing UI behavior
+
+No new UI is required for this fix:
+
+- First-attempt success: deletion progress, followed by the existing success UI.
+- Stale-ID recovery: deletion progress, `Loading...`, another deletion progress, then success.
+- MediaStore recovery: the previous steps, `Please wait, this will take a while.`, a final deletion progress, then success.
+- Final failure: remain on the Free Space page and show `Could not free up space`.
+- Cancellation must stop the flow instead of opening another deletion prompt.
+
+The existing success UI uses the original estimated size, and the existing failure UI does not describe partial progress.
+
+## Verification
+
+Cover at least:
+
+- A stale ID at the start, middle, and end of a batch.
+- A successful batch followed by failure or cancellation; the successful batch remains checkpointed.
+- All candidates already missing.
+- More than 1,900 valid candidates after filtering.
+- Missing IDs found while deletion of remaining valid IDs fails; this must not report success.
+- Limited or revoked gallery permission; invisible assets must not be classified as missing.
+- Shared-media success combined with platform deletion failure.
+- A database checkpoint failure; no later platform batch is attempted.
+- Concurrent free-space attempts.
+- Context disposal during a platform request.
+- Final Android recovery failure; no further deletion calls occur.
+
+Lifecycle-sync suppression and a richer native `photo_manager` result API are optional follow-up hardening, not required for this initial fix.
diff --git a/mobile/apps/photos/lib/services/free_space/deletion_batch_runner.dart b/mobile/apps/photos/lib/services/free_space/deletion_batch_runner.dart
new file mode 100644
index 0000000000..ebb9e4a266
--- /dev/null
+++ b/mobile/apps/photos/lib/services/free_space/deletion_batch_runner.dart
@@ -0,0 +1,147 @@
+import "dart:math";
+
+import "package:logging/logging.dart";
+
+final _logger = Logger("DeletionBatchRunner");
+
+enum LocalDeletionStatus { completed, cancelled, failed }
+
+class LocalDeletionResult {
+ final LocalDeletionStatus status;
+ final Set<String> deletedIDs;
+ final Set<String> alreadyMissingIDs;
+ final bool canAttemptRecovery;
+
+ const LocalDeletionResult({
+ required this.status,
+ this.deletedIDs = const <String>{},
+ this.alreadyMissingIDs = const <String>{},
+ this.canAttemptRecovery = false,
+ });
+
+ bool get isCompleted => status == LocalDeletionStatus.completed;
+ bool get isCancelled => status == LocalDeletionStatus.cancelled;
+}
+
+Set<String> retainOriginalDeletionCandidates({
+ required Iterable<String> refreshedLocalIDs,
+ required Iterable<String> originalLocalIDs,
+}) {
+ return refreshedLocalIDs.toSet().intersection(originalLocalIDs.toSet());
+}
+
+Future<LocalDeletionResult> executeDeletionBatches({
+ required List<String> localIDs,
+ required int batchSize,
+ required Future<List<String>> Function(List<String>) deleteBatch,
+ required Future<void> Function(Set<String>) checkpoint,
+ bool emptyResultMeansCompleted = false,
+ void Function(int completed, int total)? onProgress,
+}) async {
+ if (batchSize <= 0) {
+ throw ArgumentError.value(batchSize, "batchSize", "Must be positive");
+ }
+ final uniqueLocalIDs = localIDs.toSet().toList();
+ final deletedIDs = <String>{};
+ if (uniqueLocalIDs.isEmpty) {
+ return const LocalDeletionResult(status: LocalDeletionStatus.completed);
+ }
+
+ _logger.info("Batch size: $batchSize");
+ for (var index = 0; index < uniqueLocalIDs.length; index += batchSize) {
+ onProgress?.call(index, uniqueLocalIDs.length);
+ final batch = uniqueLocalIDs
+ .getRange(index, min(uniqueLocalIDs.length, index + batchSize))
+ .toList();
+ final attemptedIDs = batch.toSet();
+ _logger.info("Trying to delete ${batch.length} files");
+
+ late final List<String> returnedIDs;
+ try {
+ returnedIDs = await deleteBatch(batch);
+ } catch (e, s) {
+ _logger.severe("Could not delete batch of ${batch.length} files", e, s);
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
+ deletedIDs: deletedIDs,
+ canAttemptRecovery: true,
+ );
+ }
+
+ final removedIDs = returnedIDs.toSet().intersection(attemptedIDs);
+ if (removedIDs.isNotEmpty) {
+ try {
+ await checkpoint(removedIDs);
+ } catch (e, s) {
+ _logger.severe("Could not checkpoint deleted batch", e, s);
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
+ deletedIDs: deletedIDs..addAll(removedIDs),
+ );
+ }
+ deletedIDs.addAll(removedIDs);
+ }
+ _logger.info("Deleted ${removedIDs.length} of ${batch.length} files");
+
+ if (removedIDs.length != attemptedIDs.length) {
+ if (removedIDs.isEmpty && emptyResultMeansCompleted) {
+ continue;
+ }
+ _logger.info(
+ "Platform deletion returned fewer IDs than requested; stopping",
+ );
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.cancelled,
+ deletedIDs: deletedIDs,
+ );
+ }
+ }
+ onProgress?.call(uniqueLocalIDs.length, uniqueLocalIDs.length);
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.completed,
+ deletedIDs: deletedIDs,
+ );
+}
+
+LocalDeletionResult combineDeletionResults(
+ LocalDeletionResult sharedMediaResult,
+ LocalDeletionResult platformResult, {
+ Set<String> alreadyMissingIDs = const <String>{},
+}) {
+ final deletedIDs = <String>{
+ ...sharedMediaResult.deletedIDs,
+ ...platformResult.deletedIDs,
+ };
+ final missingIDs = <String>{
+ ...alreadyMissingIDs,
+ ...sharedMediaResult.alreadyMissingIDs,
+ ...platformResult.alreadyMissingIDs,
+ };
+ if (platformResult.isCancelled) {
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.cancelled,
+ deletedIDs: deletedIDs,
+ alreadyMissingIDs: missingIDs,
+ );
+ }
+ if (sharedMediaResult.status == LocalDeletionStatus.failed ||
+ platformResult.status == LocalDeletionStatus.failed) {
+ final sharedFailureCanRecover =
+ sharedMediaResult.status != LocalDeletionStatus.failed ||
+ sharedMediaResult.canAttemptRecovery;
+ final platformFailureCanRecover =
+ platformResult.status != LocalDeletionStatus.failed ||
+ platformResult.canAttemptRecovery;
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
+ deletedIDs: deletedIDs,
+ alreadyMissingIDs: missingIDs,
+ canAttemptRecovery: sharedFailureCanRecover && platformFailureCanRecover,
+ );
+ }
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.completed,
+ deletedIDs: deletedIDs,
+ alreadyMissingIDs: missingIDs,
+ );
+}
diff --git a/mobile/apps/photos/lib/ui/tools/free_space_page.dart b/mobile/apps/photos/lib/ui/tools/free_space_page.dart
index 8a31aac39f..66bf062098 100644
--- a/mobile/apps/photos/lib/ui/tools/free_space_page.dart
+++ b/mobile/apps/photos/lib/ui/tools/free_space_page.dart
@@ -7,9 +7,12 @@ import 'package:intl/intl.dart';
import 'package:logging/logging.dart';
import "package:photos/generated/l10n.dart";
import 'package:photos/models/freeable_space_info.dart';
+import "package:photos/services/free_space/deletion_batch_runner.dart";
import "package:photos/ui/notification/toast.dart";
import 'package:photos/utils/delete_file_util.dart';
+bool _isFreeSpaceDeletionInProgress = false;
+
class FreeSpacePage extends StatefulWidget {
final FreeableSpaceInfo status;
final bool clearSpaceForFolder;
@@ -25,6 +28,8 @@ class FreeSpacePage extends StatefulWidget {
}
class _FreeSpacePageState extends State<FreeSpacePage> {
+ bool _isFreeingSpace = false;
+
@override
Widget build(BuildContext context) {
return Scaffold(
@@ -90,6 +95,7 @@ class _FreeSpacePageState extends State<FreeSpacePage> {
const SizedBox(height: 32),
ButtonComponent(
shouldSurfaceExecutionStates: false,
+ isDisabled: _isFreeingSpace,
onTap: () async {
await _showConfirmFreeSpaceSheet(context, status);
},
@@ -102,29 +108,47 @@ class _FreeSpacePageState extends State<FreeSpacePage> {
}
Future<void> _freeStorage(FreeableSpaceInfo status) async {
- bool isSuccess = await deleteLocalFiles(context, status.localIDs);
-
- if (isSuccess == false) {
- if (!mounted) return;
- isSuccess = await deleteLocalFilesAfterRemovingAlreadyDeletedIDs(
- context,
- status.localIDs,
- );
+ if (_isFreeSpaceDeletionInProgress) {
+ return;
}
-
- if (isSuccess == false && Platform.isAndroid) {
- if (!mounted) return;
- isSuccess = await retryFreeUpSpaceAfterRemovingAssetsNonExistingInDisk(
- context,
- );
+ _isFreeSpaceDeletionInProgress = true;
+ if (mounted) {
+ setState(() => _isFreeingSpace = true);
}
- if (isSuccess) {
- if (!mounted) return;
- Navigator.of(context).pop(true);
- } else {
+ try {
+ var result = await deleteLocalFiles(context, status.localIDs);
+
+ if (result.status == LocalDeletionStatus.failed &&
+ result.canAttemptRecovery) {
+ if (!mounted) return;
+ result = await deleteLocalFilesAfterRemovingAlreadyDeletedIDs(
+ context,
+ status.localIDs,
+ );
+ }
+
+ if (result.status == LocalDeletionStatus.failed &&
+ result.canAttemptRecovery &&
+ Platform.isAndroid) {
+ if (!mounted) return;
+ result = await retryFreeUpSpaceAfterRemovingAssetsNonExistingInDisk(
+ context,
+ originalLocalIDs: status.localIDs,
+ );
+ }
+
if (!mounted) return;
- showToast(context, AppLocalizations.of(context).couldNotFreeUpSpace);
+ if (result.isCompleted) {
+ Navigator.of(context).pop(true);
+ } else if (!result.isCancelled) {
+ showToast(context, AppLocalizations.of(context).couldNotFreeUpSpace);
+ }
+ } finally {
+ _isFreeSpaceDeletionInProgress = false;
+ if (mounted) {
+ setState(() => _isFreeingSpace = false);
+ }
}
}
diff --git a/mobile/apps/photos/lib/utils/delete_file_util.dart b/mobile/apps/photos/lib/utils/delete_file_util.dart
index ff7bc29815..ee9763823f 100644
--- a/mobile/apps/photos/lib/utils/delete_file_util.dart
+++ b/mobile/apps/photos/lib/utils/delete_file_util.dart
@@ -24,6 +24,7 @@ import 'package:photos/models/selected_files.dart';
import 'package:photos/module/download/file.dart';
import "package:photos/service_locator.dart";
import "package:photos/services/files_service.dart";
+import "package:photos/services/free_space/deletion_batch_runner.dart";
import "package:photos/services/sync/local_sync_service.dart";
import 'package:photos/services/sync/remote_sync_service.dart';
import 'package:photos/services/sync/sync_service.dart';
@@ -370,92 +371,52 @@ Future<bool> emptyTrash(BuildContext context) async {
}
}
-Future<bool> deleteLocalFiles(
+Future<LocalDeletionResult> deleteLocalFiles(
BuildContext context,
List<String> localIDs,
) async {
- _logger.info("Trying to delete local files ");
- final List<String> deletedIDs = [];
- final List<String> localAssetIDs = [];
- final List<String> localSharedMediaIDs = [];
+ _logger.info("Trying to delete local files");
+ final localAssetIDs = <String>[];
+ final localSharedMediaIDs = <String>[];
- // Keep large platform asset deletes in smaller batches. Shared-media sandbox
- // IDs are deleted separately above, and only platform asset IDs are sent to
- // PhotoManager.editor.deleteWithIds. Android 11+ routes those IDs through
- // MediaStore.createDeleteRequest, where our target SDK has a 2000 URI cap:
- // https://developer.android.com/reference/android/provider/MediaStore#createDeleteRequest(android.content.ContentResolver,%20java.util.Collection%3Candroid.net.Uri%3E)
- // Smaller batches are also safer for large iOS Photos deletes.
- const largeCountThreshold = 1900;
try {
- for (String id in localIDs) {
+ for (final id in localIDs.toSet()) {
if (id.startsWith(sharedMediaIdentifier)) {
localSharedMediaIDs.add(id);
} else {
localAssetIDs.add(id);
}
}
- deletedIDs.addAll(await _tryDeleteSharedMediaFiles(localSharedMediaIDs));
- final tooManyAssets = localAssetIDs.length > largeCountThreshold;
- final bool shouldDeleteInBatches =
- await isAndroidSDKVersionLowerThan(android11SDKINT) || tooManyAssets;
+ final sharedMediaResult = await _deleteAndCheckpointSharedMediaFiles(
+ localSharedMediaIDs,
+ );
+ if (!sharedMediaResult.canAttemptRecovery &&
+ sharedMediaResult.status == LocalDeletionStatus.failed) {
+ return sharedMediaResult;
+ }
if (!context.mounted) {
_logger.info(
"Skipping platform asset deletion after the initiating page was disposed",
);
- } else if (shouldDeleteInBatches) {
- if (tooManyAssets) {
- _logger.info(
- "Too many assets (${localAssetIDs.length}) to delete in one shot, deleting in batches",
- );
- await _recursivelyReduceBatchSizeAndRetryDeletion(
- batchSize: largeCountThreshold,
- context: context,
- localIDs: localAssetIDs,
- deletedIDs: deletedIDs,
- );
- } else {
- _logger.info("Deleting in batches");
- deletedIDs.addAll(
- await deleteLocalFilesInBatches(context, localAssetIDs),
- );
- }
- } else {
- _logger.info("Deleting in one shot");
- deletedIDs.addAll(
- await _deleteLocalFilesInOneShot(context, localAssetIDs),
- );
- }
- if (deletedIDs.isEmpty && Platform.isIOS) {
- _logger.warning(
- "Deletion failed in deleteLocalFiles for ${localAssetIDs.length} files, on iOS",
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
+ deletedIDs: sharedMediaResult.deletedIDs,
);
}
- if (deletedIDs.isNotEmpty) {
- final deletedFiles = await FilesDB.instance.getLocalFiles(deletedIDs);
- await FilesDB.instance.deleteLocalFiles(deletedIDs);
- _logger.info(deletedFiles.length.toString() + " files deleted locally");
- Bus.instance.fire(
- LocalPhotosUpdatedEvent(deletedFiles, source: "deleteLocal"),
- );
- return true;
- } else {
- //On android 10, even if files were deleted, deletedIDs is empty.
- //This is a workaround so that users are not shown an error message on
- //android 10
- if (!await isAndroidSDKVersionLowerThan(android11SDKINT)) {
- return false;
- }
- return true;
- }
+ final platformResult = await _deletePlatformAssets(context, localAssetIDs);
+ return combineDeletionResults(sharedMediaResult, platformResult);
} catch (e, s) {
_logger.severe("Could not delete local files", e, s);
- return false;
+ return const LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
+ canAttemptRecovery: true,
+ );
}
}
-Future<bool> deleteLocalFilesAfterRemovingAlreadyDeletedIDs(
+Future<LocalDeletionResult> deleteLocalFilesAfterRemovingAlreadyDeletedIDs(
BuildContext context,
List<String> localIDs,
) async {
@@ -463,21 +424,39 @@ Future<bool> deleteLocalFilesAfterRemovingAlreadyDeletedIDs(
"Trying to delete local files after removing already deleted IDs",
);
- final List<String> deletedIDs = [];
- final List<String> localAssetIDs = [];
- final List<String> localSharedMediaIDs = [];
- final List<String> alreadyDeletedIDs = []; // to ignore already deleted files
+ final localAssetIDs = <String>[];
+ final localSharedMediaIDs = <String>[];
+ final alreadyDeletedIDs = <String>{};
+ late LocalDeletionResult sharedMediaResult;
final dialog = createProgressDialog(context, "Loading...");
await dialog.show();
try {
+ late final PermissionState permissionState;
+ try {
+ permissionState = await permissionService.getPermissionState();
+ } catch (e, s) {
+ _logger.severe(
+ "Could not verify gallery permission for stale-ID recovery",
+ e,
+ s,
+ );
+ return const LocalDeletionResult(status: LocalDeletionStatus.failed);
+ }
+ if (permissionState != PermissionState.authorized) {
+ _logger.warning(
+ "Skipping stale-ID recovery because gallery permission is "
+ "$permissionState",
+ );
+ return const LocalDeletionResult(status: LocalDeletionStatus.failed);
+ }
+
final files = await FilesDB.instance.getLocalFiles(
localIDs,
dedupeByLocalID: true,
);
for (final file in files) {
if (!(await _localFileExist(file))) {
- _logger.warning("Already deleted ${file.tag}");
alreadyDeletedIDs.add(file.localID!);
} else if (file.localID!.startsWith(sharedMediaIdentifier)) {
localSharedMediaIDs.add(file.localID!);
@@ -485,64 +464,67 @@ Future<bool> deleteLocalFilesAfterRemovingAlreadyDeletedIDs(
localAssetIDs.add(file.localID!);
}
}
- deletedIDs.addAll(alreadyDeletedIDs);
- deletedIDs.addAll(await _tryDeleteSharedMediaFiles(localSharedMediaIDs));
-
- await dialog.hide();
-
- final bool shouldDeleteInBatches = await isAndroidSDKVersionLowerThan(
- android11SDKINT,
+ _logger.info(
+ "Stale-ID recovery found ${alreadyDeletedIDs.length} missing files",
);
- if (!context.mounted) {
- _logger.info(
- "Skipping platform asset deletion after the initiating page was disposed",
- );
- } else if (shouldDeleteInBatches) {
- _logger.info("Deleting in batches");
- deletedIDs.addAll(
- await deleteLocalFilesInBatches(context, localAssetIDs),
- );
- } else {
- _logger.info("Deleting in one shot");
- deletedIDs.addAll(
- await _deleteLocalFilesInOneShot(context, localAssetIDs),
- );
- }
- if (deletedIDs.isEmpty && Platform.isIOS) {
- _logger.warning(
- "Deletion failed in deleteLocalFilesAfterRemovingAlreadyDeletedIDs for ${localAssetIDs.length} files, on iOS",
+ try {
+ await _checkpointRemovedLocalIDs(alreadyDeletedIDs);
+ } catch (e, s) {
+ _logger.severe("Could not checkpoint already-missing files", e, s);
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
+ alreadyMissingIDs: alreadyDeletedIDs,
);
}
- if (deletedIDs.isNotEmpty) {
- final deletedFiles = await FilesDB.instance.getLocalFiles(deletedIDs);
- await FilesDB.instance.deleteLocalFiles(deletedIDs);
- _logger.info(deletedFiles.length.toString() + " files deleted locally");
- Bus.instance.fire(
- LocalPhotosUpdatedEvent(deletedFiles, source: "deleteLocal"),
+ sharedMediaResult = await _deleteAndCheckpointSharedMediaFiles(
+ localSharedMediaIDs,
+ );
+ if (!sharedMediaResult.canAttemptRecovery &&
+ sharedMediaResult.status == LocalDeletionStatus.failed) {
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
+ deletedIDs: sharedMediaResult.deletedIDs,
+ alreadyMissingIDs: alreadyDeletedIDs,
);
- return true;
- } else {
- //On android 10, even if files were deleted, deletedIDs is empty.
- //This is a workaround so that users are not shown an error message on
- //android 10
- if (!await isAndroidSDKVersionLowerThan(android11SDKINT)) {
- return false;
- }
- return true;
}
} catch (e, s) {
_logger.severe("Could not delete local files", e, s);
- await dialog.hide();
- return false;
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
+ alreadyMissingIDs: alreadyDeletedIDs,
+ canAttemptRecovery: true,
+ );
+ } finally {
+ await _hideProgressDialog(dialog);
+ }
+
+ if (!context.mounted) {
+ _logger.info(
+ "Skipping platform asset deletion after the initiating page was disposed",
+ );
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
+ deletedIDs: sharedMediaResult.deletedIDs,
+ alreadyMissingIDs: alreadyDeletedIDs,
+ );
}
+
+ final platformResult = await _deletePlatformAssets(context, localAssetIDs);
+ return combineDeletionResults(
+ sharedMediaResult,
+ platformResult,
+ alreadyMissingIDs: alreadyDeletedIDs,
+ );
}
/// Only to be used on Android
-Future<bool> retryFreeUpSpaceAfterRemovingAssetsNonExistingInDisk(
- BuildContext context,
-) async {
+Future<LocalDeletionResult>
+retryFreeUpSpaceAfterRemovingAssetsNonExistingInDisk(
+ BuildContext context, {
+ required Iterable<String> originalLocalIDs,
+}) async {
_logger.info(
"Retrying free up space after removing assets non-existing in disk",
);
@@ -551,6 +533,10 @@ Future<bool> retryFreeUpSpaceAfterRemovingAssetsNonExistingInDisk(
context,
context.l10n.pleaseWaitThisWillTakeAWhile,
);
+ final localAssetIDs = <String>[];
+ final localSharedMediaIDs = <String>[];
+ late LocalDeletionResult sharedMediaResult;
+
await dialog.show();
try {
final stopwatch = Stopwatch()..start();
@@ -563,95 +549,132 @@ Future<bool> retryFreeUpSpaceAfterRemovingAssetsNonExistingInDisk(
);
await LocalSyncService.instance.sync();
- late final FreeableSpaceInfo status;
- final List<String> deletedIDs = [];
- final List<String> localAssetIDs = [];
- final List<String> localSharedMediaIDs = [];
- status = await FilesService.instance.getFreeableSpaceInfo();
+ final FreeableSpaceInfo status = await FilesService.instance
+ .getFreeableSpaceInfo();
- for (String localID in status.localIDs) {
+ final scopedLocalIDs = retainOriginalDeletionCandidates(
+ refreshedLocalIDs: status.localIDs,
+ originalLocalIDs: originalLocalIDs,
+ );
+ for (final localID in scopedLocalIDs) {
if (localID.startsWith(sharedMediaIdentifier)) {
localSharedMediaIDs.add(localID);
} else {
localAssetIDs.add(localID);
}
}
- deletedIDs.addAll(await _tryDeleteSharedMediaFiles(localSharedMediaIDs));
-
- await dialog.hide();
-
- final bool shouldDeleteInBatches = await isAndroidSDKVersionLowerThan(
- android11SDKINT,
+ sharedMediaResult = await _deleteAndCheckpointSharedMediaFiles(
+ localSharedMediaIDs,
);
- if (!context.mounted) {
- _logger.info(
- "Skipping platform asset deletion after the initiating page was disposed",
- );
- } else if (shouldDeleteInBatches) {
- _logger.info("Deleting in batches");
- deletedIDs.addAll(
- await deleteLocalFilesInBatches(context, localAssetIDs),
- );
- } else {
- _logger.info("Deleting in one shot");
- deletedIDs.addAll(
- await _deleteLocalFilesInOneShot(context, localAssetIDs),
+ if (!sharedMediaResult.canAttemptRecovery &&
+ sharedMediaResult.status == LocalDeletionStatus.failed) {
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
+ deletedIDs: sharedMediaResult.deletedIDs,
);
}
+ } catch (e, s) {
+ _logger.severe("Could not complete MediaStore recovery", e, s);
+ return const LocalDeletionResult(status: LocalDeletionStatus.failed);
+ } finally {
+ await _hideProgressDialog(dialog);
+ }
- if (deletedIDs.isNotEmpty) {
- final deletedFiles = await FilesDB.instance.getLocalFiles(deletedIDs);
- await FilesDB.instance.deleteLocalFiles(deletedIDs);
- _logger.info(deletedFiles.length.toString() + " files deleted locally");
- Bus.instance.fire(
- LocalPhotosUpdatedEvent(deletedFiles, source: "deleteLocal"),
- );
- return true;
- } else {
- //On android 10, even if files were deleted, deletedIDs is empty.
- //This is a workaround so that users are not shown an error message on
- //android 10
- if (!await isAndroidSDKVersionLowerThan(android11SDKINT)) {
- return false;
- }
- return true;
- }
- } catch (e) {
- await dialog.hide();
- return false;
+ if (!context.mounted) {
+ _logger.info(
+ "Skipping final platform deletion after the initiating page was disposed",
+ );
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
+ deletedIDs: sharedMediaResult.deletedIDs,
+ );
+ }
+
+ final platformResult = await _deletePlatformAssets(context, localAssetIDs);
+ final result = combineDeletionResults(sharedMediaResult, platformResult);
+ return LocalDeletionResult(
+ status: result.status,
+ deletedIDs: result.deletedIDs,
+ alreadyMissingIDs: result.alreadyMissingIDs,
+ );
+}
+
+Future<LocalDeletionResult> _deletePlatformAssets(
+ BuildContext context,
+ List<String> localIDs,
+) async {
+ final uniqueLocalIDs = localIDs.toSet().toList();
+ if (uniqueLocalIDs.isEmpty) {
+ return const LocalDeletionResult(status: LocalDeletionStatus.completed);
+ }
+ if (!context.mounted) {
+ return const LocalDeletionResult(status: LocalDeletionStatus.failed);
+ }
+
+ // Keep Android 11+ requests below MediaStore's 2000 URI limit. The same
+ // threshold also avoids oversized Photos requests on iOS.
+ const largeCountThreshold = 1900;
+ final isLegacyAndroid =
+ Platform.isAndroid && await isAndroidSDKVersionLowerThan(android11SDKINT);
+ if (!context.mounted) {
+ return const LocalDeletionResult(status: LocalDeletionStatus.failed);
+ }
+ if (isLegacyAndroid) {
+ _logger.info("Deleting platform assets in legacy Android batches");
+ return deleteLocalFilesInBatches(
+ context,
+ uniqueLocalIDs,
+ emptyResultMeansCompleted: true,
+ );
}
+ if (uniqueLocalIDs.length > largeCountThreshold) {
+ _logger.info(
+ "Deleting ${uniqueLocalIDs.length} platform assets in bounded batches",
+ );
+ return deleteLocalFilesInBatches(
+ context,
+ uniqueLocalIDs,
+ minimumParts: 1,
+ maximumBatchSize: largeCountThreshold,
+ );
+ }
+ return _deleteLocalFilesInOneShot(context, uniqueLocalIDs);
}
-Future<List<String>> _deleteLocalFilesInOneShot(
+Future<LocalDeletionResult> _deleteLocalFilesInOneShot(
BuildContext context,
List<String> localIDs,
) async {
_logger.info('starting _deleteLocalFilesInOneShot for ${localIDs.length}');
- final List<String> deletedIDs = [];
final dialog = createProgressDialog(
context,
"Deleting " + localIDs.length.toString() + " backed up files...",
);
await dialog.show();
try {
- deletedIDs.addAll(await PhotoManager.editor.deleteWithIds(localIDs));
- } catch (e, s) {
- _logger.severe("Could not delete files ", e, s);
+ final result = await executeDeletionBatches(
+ localIDs: localIDs,
+ batchSize: localIDs.length,
+ deleteBatch: PhotoManager.editor.deleteWithIds,
+ checkpoint: _checkpointRemovedLocalIDs,
+ );
+ _logger.info(
+ '_deleteLocalFilesInOneShot deleted ${result.deletedIDs.length} out '
+ 'of ${localIDs.length}',
+ );
+ return result;
+ } finally {
+ await _hideProgressDialog(dialog);
}
- _logger.info(
- '_deleteLocalFilesInOneShot deleted ${deletedIDs.length} out '
- 'of ${localIDs.length}',
- );
- await dialog.hide();
- return deletedIDs;
}
-Future<List<String>> deleteLocalFilesInBatches(
+Future<LocalDeletionResult> deleteLocalFilesInBatches(
BuildContext context,
List<String> localIDs, {
int minimumParts = 10,
int minimumBatchSize = 1,
int maximumBatchSize = 100,
+ bool emptyResultMeansCompleted = false,
}) async {
final dialogKey = GlobalKey<LinearProgressDialogState>();
final dialog = LinearProgressDialog(
@@ -667,82 +690,80 @@ Future<List<String>> deleteLocalFilesInBatches(
},
barrierColor: Colors.black.withValues(alpha: 0.85),
);
+ await WidgetsBinding.instance.endOfFrame;
final batchSize = min(
max(minimumBatchSize, (localIDs.length / minimumParts).round()),
maximumBatchSize,
);
- _logger.info("Batch size: $batchSize");
- final List<String> deletedIDs = [];
- for (int index = 0; index < localIDs.length; index += batchSize) {
- if (dialogKey.currentState != null) {
- dialogKey.currentState!.setProgress(index / localIDs.length);
- }
- final ids = localIDs
- .getRange(index, min(localIDs.length, index + batchSize))
- .toList();
- _logger.info("Trying to delete ${ids.length} files");
- final int countBefore = deletedIDs.length;
- try {
- deletedIDs.addAll(await PhotoManager.editor.deleteWithIds(ids));
- _logger.info(
- "Deleted ${deletedIDs.length - countBefore} of ${ids.length} files",
- );
- } catch (e, s) {
- _logger.severe("Could not delete batch of ${ids.length} files", e, s);
- for (final id in ids) {
- try {
- deletedIDs.addAll(await PhotoManager.editor.deleteWithIds([id]));
- _logger.info("Deleted " + id);
- } catch (e, s) {
- _logger.severe("Could not delete file " + id, e, s);
- }
- }
+ try {
+ return await executeDeletionBatches(
+ localIDs: localIDs,
+ batchSize: batchSize,
+ deleteBatch: PhotoManager.editor.deleteWithIds,
+ checkpoint: _checkpointRemovedLocalIDs,
+ emptyResultMeansCompleted: emptyResultMeansCompleted,
+ onProgress: (completed, total) {
+ dialogKey.currentState?.setProgress(completed / total);
+ },
+ );
+ } finally {
+ final dialogContext = dialogKey.currentContext;
+ if (dialogContext != null && dialogContext.mounted) {
+ Navigator.of(dialogContext).pop('dialog');
}
}
- Navigator.of(dialogKey.currentContext!).pop('dialog');
- return deletedIDs;
}
-Future<void> _recursivelyReduceBatchSizeAndRetryDeletion({
- required int batchSize,
- required BuildContext context,
- required List<String> localIDs,
- required List<String> deletedIDs,
- int minimumBatchSizeThresholdToStopRetry = 1900,
-}) async {
- // TODO: Revisit whether this recursive retry is still needed. The batch
- // helper already falls back to single-ID deletes when a batch fails.
- if (batchSize < minimumBatchSizeThresholdToStopRetry) {
- _logger.warning(
- "Batch size is too small ($batchSize), stopping further retries.",
- );
- throw Exception(
- "Batch size is too small ($batchSize), stopping further retries.",
+Future<void> _checkpointRemovedLocalIDs(Iterable<String> localIDs) async {
+ final uniqueLocalIDs = localIDs.toSet();
+ if (uniqueLocalIDs.isEmpty) {
+ return;
+ }
+ final ids = uniqueLocalIDs.toList();
+ final deletedFiles = await FilesDB.instance.getLocalFiles(ids);
+ await FilesDB.instance.deleteLocalFiles(ids);
+ _logger.info("${deletedFiles.length} files deleted locally");
+ if (deletedFiles.isNotEmpty) {
+ Bus.instance.fire(
+ LocalPhotosUpdatedEvent(deletedFiles, source: "deleteLocal"),
);
}
+}
+
+Future<LocalDeletionResult> _deleteAndCheckpointSharedMediaFiles(
+ List<String> localIDs,
+) async {
+ final requestedIDs = localIDs.toSet();
+ if (requestedIDs.isEmpty) {
+ return const LocalDeletionResult(status: LocalDeletionStatus.completed);
+ }
+ final deletedIDs = (await _tryDeleteSharedMediaFiles(
+ requestedIDs.toList(),
+ )).toSet().intersection(requestedIDs);
try {
- deletedIDs.addAll(
- await deleteLocalFilesInBatches(
- context,
- localIDs,
- minimumBatchSize: 1,
- maximumBatchSize: batchSize,
- minimumParts: 1,
- ),
- );
- } catch (e) {
- _logger.warning(
- "Failed to delete local files in batches of $batchSize. Reducing batch size and retrying.",
- e,
- );
- if (!context.mounted) return;
- await _recursivelyReduceBatchSizeAndRetryDeletion(
- batchSize: (batchSize / 2).floor(),
- context: context,
- localIDs: localIDs,
+ await _checkpointRemovedLocalIDs(deletedIDs);
+ } catch (e, s) {
+ _logger.severe("Could not checkpoint deleted shared-media files", e, s);
+ return LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
deletedIDs: deletedIDs,
);
}
+ return LocalDeletionResult(
+ status: deletedIDs.length == requestedIDs.length
+ ? LocalDeletionStatus.completed
+ : LocalDeletionStatus.failed,
+ deletedIDs: deletedIDs,
+ canAttemptRecovery: deletedIDs.length != requestedIDs.length,
+ );
+}
+
+Future<void> _hideProgressDialog(dynamic dialog) async {
+ try {
+ await dialog.hide();
+ } catch (e, s) {
+ _logger.warning("Could not hide deletion progress dialog", e, s);
+ }
}
Future<bool> _localFileExist(EnteFile file) {
diff --git a/mobile/apps/photos/test/services/free_space/deletion_batch_runner_test.dart b/mobile/apps/photos/test/services/free_space/deletion_batch_runner_test.dart
new file mode 100644
index 0000000000..aeb1678160
--- /dev/null
+++ b/mobile/apps/photos/test/services/free_space/deletion_batch_runner_test.dart
@@ -0,0 +1,151 @@
+import "package:flutter_test/flutter_test.dart";
+import "package:photos/services/free_space/deletion_batch_runner.dart";
+
+void main() {
+ test("stops after the first platform exception", () async {
+ final attemptedBatches = <List<String>>[];
+
+ final result = await executeDeletionBatches(
+ localIDs: ["1", "2", "3", "4"],
+ batchSize: 2,
+ deleteBatch: (ids) async {
+ attemptedBatches.add(List.of(ids));
+ throw StateError("failed batch");
+ },
+ checkpoint: (_) async {},
+ );
+
+ expect(result.status, LocalDeletionStatus.failed);
+ expect(result.canAttemptRecovery, isTrue);
+ expect(attemptedBatches, [
+ ["1", "2"],
+ ]);
+ });
+
+ test("checkpoints an earlier batch before a later failure", () async {
+ final attemptedBatches = <List<String>>[];
+ final checkpoints = <Set<String>>[];
+
+ final result = await executeDeletionBatches(
+ localIDs: ["1", "2", "3", "4", "5"],
+ batchSize: 2,
+ deleteBatch: (ids) async {
+ attemptedBatches.add(List.of(ids));
+ if (attemptedBatches.length == 2) {
+ throw StateError("failed batch");
+ }
+ return ids;
+ },
+ checkpoint: (ids) async => checkpoints.add(Set.of(ids)),
+ );
+
+ expect(result.status, LocalDeletionStatus.failed);
+ expect(result.deletedIDs, {"1", "2"});
+ expect(checkpoints, [
+ {"1", "2"},
+ ]);
+ expect(attemptedBatches, [
+ ["1", "2"],
+ ["3", "4"],
+ ]);
+ });
+
+ test("an empty result stops later batches as cancellation", () async {
+ final attemptedBatches = <List<String>>[];
+
+ final result = await executeDeletionBatches(
+ localIDs: ["1", "2", "3"],
+ batchSize: 2,
+ deleteBatch: (ids) async {
+ attemptedBatches.add(List.of(ids));
+ return [];
+ },
+ checkpoint: (_) async {},
+ );
+
+ expect(result.status, LocalDeletionStatus.cancelled);
+ expect(attemptedBatches, [
+ ["1", "2"],
+ ]);
+ });
+
+ test("a checkpoint failure prevents another platform batch", () async {
+ final attemptedBatches = <List<String>>[];
+
+ final result = await executeDeletionBatches(
+ localIDs: ["1", "2", "3"],
+ batchSize: 2,
+ deleteBatch: (ids) async {
+ attemptedBatches.add(List.of(ids));
+ return ids;
+ },
+ checkpoint: (_) async => throw StateError("checkpoint failed"),
+ );
+
+ expect(result.status, LocalDeletionStatus.failed);
+ expect(result.canAttemptRecovery, isFalse);
+ expect(attemptedBatches, [
+ ["1", "2"],
+ ]);
+ });
+
+ test("preserves legacy Android empty-result completion", () async {
+ final attemptedBatches = <List<String>>[];
+
+ final result = await executeDeletionBatches(
+ localIDs: ["1", "2", "3", "4", "5"],
+ batchSize: 2,
+ emptyResultMeansCompleted: true,
+ deleteBatch: (ids) async {
+ attemptedBatches.add(List.of(ids));
+ return [];
+ },
+ checkpoint: (_) async {},
+ );
+
+ expect(result.status, LocalDeletionStatus.completed);
+ expect(attemptedBatches.map((ids) => ids.length), [2, 2, 1]);
+ });
+
+ test("does not exceed the configured batch size", () async {
+ final attemptedBatchSizes = <int>[];
+ final ids = List.generate(1901, (index) => "$index");
+
+ final result = await executeDeletionBatches(
+ localIDs: ids,
+ batchSize: 1900,
+ deleteBatch: (ids) async {
+ attemptedBatchSizes.add(ids.length);
+ return ids;
+ },
+ checkpoint: (_) async {},
+ );
+
+ expect(result.status, LocalDeletionStatus.completed);
+ expect(attemptedBatchSizes, [1900, 1]);
+ });
+
+ test("reconciled stale IDs do not hide a platform failure", () {
+ final result = combineDeletionResults(
+ const LocalDeletionResult(status: LocalDeletionStatus.completed),
+ const LocalDeletionResult(
+ status: LocalDeletionStatus.failed,
+ canAttemptRecovery: true,
+ ),
+ alreadyMissingIDs: {"stale-id"},
+ );
+
+ expect(result.status, LocalDeletionStatus.failed);
+ expect(result.canAttemptRecovery, isTrue);
+ expect(result.alreadyMissingIDs, {"stale-id"});
+ });
+
+ test("final recovery retains only originally confirmed candidates", () {
+ final candidates = retainOriginalDeletionCandidates(
+ refreshedLocalIDs: ["folder-file", "unrelated-file"],
+ originalLocalIDs: ["folder-file", "already-handled-file"],
+ );
+
+ expect(candidates, {"folder-file"});
+ });
+}