All Questions
Tagged with message-queue architecture
55 questions
1
vote
1
answer
149
views
Background thread processing vs queue based processing for relatively short tasks (max 30-40 seconds)
I need suggestion / recommendation on the design approaches mentioned below.
UseCase: I have a usecase where a client uses my system to generate some recommendations. Now, these recommendations are ...
- 19
3
votes
3
answers
212
views
Handling rate limits / delays in consumers without affecting performance of other operations
I have a producer that generates a batch job that consists of multiple operations (approx. 100 - 10000). These operations can be processed in any order, ideally as fast as possible.
The processing of ...
- 141
0
votes
3
answers
2k
views
Backend to client communication: WebSocket or Message Queue?
I have a hobby project, for simplicity imagine a poker game, consisting of a backend (Spring) and a client application (Flutter) where the client can join rooms in which he can take a seat. The client ...
- 9
3
votes
2
answers
420
views
Can multiple producers cause events with the same key to be enqueued out of order within a specific partition of a topic?
I'm aiming to create a simple event-driven system where each microservice operates with its own database. The idea is to share database changes across microservices through events. To ensure proper ...
0
votes
1
answer
159
views
Generic message broker based on WCF
I apologize in advance for a long-read question, but i'm hardly stuck here and don`t how to ask for help using fewer words.
I need to implement a middleware service which should be placed between two ...
1
vote
1
answer
88
views
Handling a single event as broadcast and round-robin depending on service
I have Service A that publishes an event to RabbitMQ. I have two instances of Service B that will use the event to write to its database. I have two instances of Service C that will send the event ...
- 136
0
votes
0
answers
115
views
How to use an external event queue with inversion of control (IoC)
I'm working on an AspNetCore application, with a requirement to raise and handle certain events asynchronously using an external event queue. I'll use AWS services as examples here because that's what ...
- 135
0
votes
2
answers
244
views
Complicated task scheduling architecture
So I want to make an application in which a user will hit an endpoint to save a job model to storage that includes some metadata to perform a long computation against which will be offloaded to a ...
- 101
0
votes
2
answers
281
views
How can I track the status of multiple messages in asynchronous processes?
Let's say I've a sendMessage endpoint. This endpoint looks at configured message destinations and then puts 1..N messages into a queue (e.g. {userId: 3, type: SMS, trackingID: X} and {userId: 3, type: ...
user374484
-1
votes
3
answers
376
views
How can i handle really long-term running tasks?
Im not dealing with a x minute task that can use an message queue and some workers to be done. Lets say that i have an cctv camera that capture 24/7, but can be stopped or started again via Web API ...
- 1
1
vote
2
answers
3k
views
Implement message queue in API Gateway
Recently we have this client who asks for a API Gateway solution, in this case it’s Kong. Currently, they have 10 services (200 APIs) that are running on really legacy stuff (built with C++ and ...
- 119
0
votes
0
answers
54
views
How to handle maintenance to your queuing server (e.g. rabbitMQ)
I'm new to queuing and looking into options that could be done operationally. I'm sorry if I have any conceptual misunderstandings related to queuing.
I watched this video where the presenter made a ...
- 311
1
vote
0
answers
419
views
Do we really need a message brocker in WebSocket system?
I'm engineering architecture of a new web based software.
I've never worked on high-scale softwares before and I'm reading a lot about it.
To increase client-side speed and reduce load on servers, and ...
- 19
2
votes
2
answers
1k
views
Is Kafka a good idea for connecting a REST API with Worker which runs large computations per job in a queue?
We currently have a REST API service (let's call it as A) written in Python which triggers various heavy calculation jobs for Worker (W) written in Python as well. Both services are interconnected ...
- 83
3
votes
0
answers
760
views
System Design: Chat application in a distributed system context
I am currently working on a side project so I can learn new technologies and understand how the components fit together. For this project, I decided that I wanted to work on Chat Applications in the ...
