AWS SQS
Amazon SQS is a fully managed message queuing service that decouples the components of a distributed application. A producer sends a message to a queue; a consumer receives and processes it independently. We design SQS-based messaging architectures that handle failures gracefully, scale automatically, and make the relationship between producers and consumers explicit.
Capabilities
What we build with SQS
Producer / consumer
Decoupled Processing Pipelines
Separating the component that creates work from the component that processes it so that neither side is blocked by the other's availability or throughput. The queue absorbs bursts and smooths processing load.
Managed consumer
Lambda Event Source Mapping
Lambda functions triggered by SQS messages using event source mapping, with configurable batch sizes, concurrency limits, and bisect-on-error to isolate failed messages.
Failure handling
Dead-Letter Queue Configuration
DLQ configuration with appropriate maxReceiveCount thresholds, CloudWatch alarms on DLQ depth, and a process for inspecting and redriving failed messages without losing them.
Ordering · Deduplication
FIFO Queue Design
FIFO queues for workloads that require strict message ordering within a group or exactly-once processing semantics. Message group ID strategy and deduplication ID design.
Pub/sub + queuing
SNS-to-SQS Fan-out
Combining SNS topics with SQS subscriptions for fan-out patterns: one SNS publish delivers to multiple SQS queues, each with its own processing logic and DLQ.
Queue depth · Alarms
Backpressure and Rate Limiting
CloudWatch alarms on queue depth and age of oldest message, auto-scaling policies for consumer Lambda concurrency, and design patterns for applying backpressure to producers.
Our approach
Standard versus FIFO
SQS Standard queues are higher throughput and lower cost but deliver messages at-least-once in approximate order. FIFO queues deliver exactly-once in strict order within a message group but have lower throughput limits. Most workloads use Standard queues with idempotent consumers; FIFO is reserved for cases where ordering or exactly-once processing is a genuine requirement.
Every queue needs a DLQ
A queue without a dead-letter queue silently drops or reprocesses messages that consistently fail. We configure a DLQ for every production queue, set appropriate maxReceiveCount values, and alarm on DLQ depth so that poison messages surface immediately rather than disappearing.
Lambda consumers need careful batch sizing
When Lambda consumes from SQS, the batch size determines how many messages a single Lambda invocation processes. A large batch with one failing message can cause all messages in the batch to be retried. We configure batchSize, bisectBatchOnFunctionError, and report batch item failures to handle partial failures correctly.
Visibility timeout must exceed processing time
The visibility timeout is the time SQS hides a message from other consumers after it is received. If processing takes longer than the visibility timeout, SQS re-delivers the message to another consumer, causing duplicate processing. We set visibility timeouts to comfortably exceed the maximum expected processing time.
All engineering work is done by US-based engineers. We do not offshore any development or architecture work.
Part of our serverless practice
FAQ
Common questions
Virginia · United States
Designing an SQS architecture?
Reach out and we will discuss queue design, consumer strategy, and failure handling before any work begins.