Skip to content

Open Source Wealth Management Software. Angular + NestJS + Prisma + Nx + TypeScript 🤍

Task/improve throughput of market data gathering queue by applying rate limit per data source (#7128)

Thomas Kaul95b0318f8e73
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 04a4d896aa..df2f6ba9ef 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -15,6 +15,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Extended the asset profile details dialog of the admin control panel with a copy-to-clipboard button for the ISIN number
- Extended the asset profile details dialog of the admin control panel with a copy-to-clipboard button for the symbol
+### Changed
+
+- Improved the throughput of the market data gathering queue by applying the rate limit per data source
+
### Fixed
- Fixed an issue with hourly market data updates not refreshing prices for asset profiles with `MANUAL` data source
diff --git a/apps/api/src/services/queues/data-gathering/data-gathering.module.ts b/apps/api/src/services/queues/data-gathering/data-gathering.module.ts
index 5ac6c40c0c..7258c034c2 100644
--- a/apps/api/src/services/queues/data-gathering/data-gathering.module.ts
+++ b/apps/api/src/services/queues/data-gathering/data-gathering.module.ts
@@ -29,6 +29,7 @@ import { DataGatheringProcessor } from './data-gathering.processor';
BullModule.registerQueue({
limiter: {
duration: ms('4 seconds'),
+ groupKey: 'dataSource',
max: 1
},
name: DATA_GATHERING_QUEUE